Troy Will Burn – the Big Deal about Big Data

Category: Big Data, Market Structure


For the life of me, I don’t understand the debate [over the NSA metadata program].

– Jeb Bush, February 18, 2015

The Central Intelligence Agency played a crucial role in helping the Justice Department develop technology that scans data from thousands of US cellphones at a time, part of a secret high-tech alliance between the spy agency and domestic law enforcement, according to people familiar with the work. 

The Wallstreet Journal front page story, March 10, 2015

Athena:  You wish to be called righteous rather than act right.

Aeschylus, “The Oresteia” (458 BC)



Point72 Asset Management, the successor to Cohen’s hedge fund SAC Capital Advisors, has hired about 30 employees since the start of last year to build computer models that collect publicly available data and analyze it for patterns, according to two people with knowledge of the matter.
Cohen, whose SAC Capital shut down last year and paid a record fine to settle charges of insider trading, joins Ray Dalio’s Bridgewater Associates in pushing into computer-driven investing, an area dominated by a handful of big firms such as the $25 billion Renaissance Technologies and the $24 billion Two Sigma. The money managers are seeking to take advantage of advances in computing power and data availability to analyze large amounts of information.

Bloomberg, March 10, 2015



Cassandra:  Have I missed the mark, or, like true archer, do I strike my quarry? Or am I prophet of lies, a babbler from door to door?

– Aeschylus, “The Oresteia” (BC)

I know, I know … I’m a broken record and a Cassandra, with 2 successive notes on Big Data. But I don’t care. This is a much larger structural risk for markets and investors than HFT and the whole Flash Boys brouhaha, it’s just totally under the radar and hasn’t surfaced yet. And unfortunately, just as I think Jeb Bush speaks for most Americans – Democrat and Republican alike – when he says that he doesn’t get what all the fuss is about when it comes to metadata collection and Big Data technologies, so do I think that most investors – institutional and individual alike – are blithely unaware of how their market identities can be stolen and their market behaviors influenced, all in plain sight. 

Jeb Bush should know better. I think he probably does. Investors may not know better yet, but they will soon, one way or another. As you read this note, a small group of hedge fund managers are doing to you exactly what the NSA is doing to “terrorists”.

Today a handful of governments use Big Data to identify individual behavioral patterns so that certain individuals can be killed. Today a handful of hedge funds use Big Data to identify investor behavioral patterns so that certain investors can be crushed. Today Big Data is primarily an instrument of social information gathering, with a powerful but punctuated impact on those individuals on the receiving end of a drone strike or a targeted trade.

Tomorrow a handful of governments will influence aggregate political behaviors by triggering small communications that Big Data tells them will be voluntarily magnified by individual citizens, snowballing into outsized, long-lasting, and untraceable “popular” actions. Tomorrow a handful of hedge funds will influence aggregate market behaviors by triggering small trades that Big Data tells them will be voluntarily magnified by individual traders, snowballing into outsized, long-lasting, and untraceable “market” actions. Tomorrow Big Data will be primarily an instrument of social control, with a powerful and ubiquitous impact on all citizens and all investors.

Q: How can I protect myself?
A: You can’t.

But WE can protect ourselves, to some extent at least, by working together to raise voter and investor awareness of the risk and pressing for regulatory reform to shield our behavioral data from commercial use AND bureaucratic collection. I’ll leave the voter awareness piece to others, and use Epsilon Theory to focus on investor awareness.

Trust me, I know how this sounds, to write to an audience of free market-oriented investors and call for stronger regulatory intervention to prevent the collection or sale of “anonymous” investment data. And if you think that any mutually agreed upon transaction should be allowed, no matter how large the gulf in knowledge between the buyer and seller … if you would buy an original Honus Wagner baseball card from a 10-year old kid for a quarter, telling him that you were doing him a favor to pay him that much for such a ratty card … then I’m never going to convince you of the merits of my argument. If that’s you, then I’m sure Stevie Cohen sends his best regards from the Grand Duchy of Fairfield County. But if you believe, as Adam Smith did, that it is government’s appropriate role to prevent transactions that are massively lop-sided from an informational perspective and that directly subvert the small-l liberal institutions of free elections and free markets, then I think you will find this a proposal worth considering.

It’s by no means a perfect solution, but I like more than I dislike about the way our personal medical data is protected through HIPAA. As an initial step, I’d like to see federal financial data legislation equivalent to HIPAA, where both private AND public sector use of our investment history, no matter how scrubbed or “anonymized”, is prohibited. 

Such a law would cause a lot of pain. For-profit exchanges, all of which have transformed themselves from trading venues into “data companies”, would no longer be able to sell disaggregated transaction data. Mega-asset managers would no longer be able to sell anonymized client portfolio data. Ubiquitous financial information companies that may or may not share a name with a former mayor of New York would be subject to a regulatory scrutiny that is sorely lacking today.

Yes, a lot of pain. But it’s a fraction of the pain we will ALL feel if for-profit exchanges, mega-asset managers, and ubiquitous financial information companies are allowed to continue producing weapons-grade plutonium for the handful of hedge funds that are building their instruments of market control.

Unfortunately, like Cassandra, I’m predicting future pain, and that’s rarely successful as a goad to current action. To quote Aeschylus once more:

Nothing forces us to know
What we do not want to know
Except pain.

I don’t think we investors have suffered enough … yet … to force us to accept the unwanted knowledge we need to spark effective collective action. Instead, I can just hear the apologists, the lobbyists, and the bought-and-paid-for spouting the Big Lie when it comes to Big Data: “But it’s anonymous data we’re talking about, so you have nothing to worry about.”

I hope I’m wrong, but I’m not optimistic.

Pessimism and hope may seem to be odd bedfellows, but for 2,500 years that’s been the best prescription for dealing with a tragic world, where external forces threaten at every turn to sweep us off our moorings. I’ve used a lot of quotes this week from Aeschylus because, as the inventor of tragedy as an art form, he was the guy who first proposed that bittersweet tonic.
Aeschylus had an interesting life and an interesting death. As the story goes, in middle age a fortune teller warned him he would be killed by something dropped on his head. From then on, Aeschylus famously stayed out of cities, where someone might accidentally knock a chamber pot or some such out from an open window. Sure enough, though, in the best tradition of the inescapable-destiny trope that Aeschylus helped invent, he was killed outside a Sicilian town when an eagle mistook his bald head for a rock and dropped a turtle on it. As I recall, there was a CSI episode that used this as a plot device to resolve an inexplicable death in the desert outside of Las Vegas … my estimation of the show runners went up immensely when they showed their surprising knowledge of classical history!


But it’s his life that I want to commemorate here. You see, first and foremost Aeschylus was a patriot. He fought the Persians at Marathon, Salamis, and Plataea, where he was recognized for bravery in all three battles. His epitaph says nothing about being a playwright, only about being a soldier. One of his two brothers was killed at Marathon, the other lost his hand at Salamis. Aeschylus himself bore terrible scars from the victory at Marathon. We know that he had these scars because he showed them to the jury when he was put on trial for treason after supposedly revealing some of the Eleusinian Mysteries – essentially state secrets – in one of his plays. Fortunately for the world, Aeschylus was acquitted, and Athens went on to experience a golden age that inspires us still.

Aeschylus argued that you can question your government’s policy on secrecy without being a traitor, that he was in fact still a patriot – perhaps even more of a patriot – for the tragedies he wrote. I’d hope that we can be as wise today as that Athenian jury was more than 2,500 years ago. I’d hope that we can question both our government’s policy and our private sector’s policy on behavioral data collection without being accused of treason or (worse in some investor circles) socialism. I’d hope. But I’m not optimistic.

So here’s Plan B, a plan for a crowd-sourcing world.

If we can’t cut off the supply of plutonium for these weapons of mass market destruction, then we can at least provide the blueprints for the Bomb so that anyone can build one. Or, better yet, we can build a collective early warning system, an open-source Bomb detector … a Big Data market intelligence available to everyone. It’s not an instrument of social control and it’s not a spoofer; the former is the enemy and the latter is really, really expensive. It’s a collection of highly sensitive risk antennae, sensitive enough to identify the likelihood of otherwise untraceable market manipulation in real time.


Recursive inference engine [A] comprised of thousands of “bots” (static data models) executes small trades to test market reaction to different stimuli. Game/learning implementation [B] serves as dynamic data model to recognize and calculate arbitrage likelihood functions. Analytics platform [C] operating within real-time database architecture governs [A] and [B].

This is a basic schematic for what I think could function as a rudimentary Big Data market intelligence. When I sketched this out 4 years ago I pegged the hardware cost at close to $5 million; today I figure it’s closer to $1 million. Host it somewhere like my friend Gary King’s Institute for Quantitative Social Science and the total cost, both to build and maintain, becomes very manageable. What’s costly is the time required to program the system, but there’s no shortage of Big Data wizards coming out of Harvard, MIT, Stanford, etc. every year.

Yes, I know that this schematic will be gobbledygook to almost all of my readers, and the few readers who are immersed in this stuff will undoubtedly find it overly simplistic. But it’s a start on Plan B. It’s a start on demystifying the powerful non-human intelligences that will soon be used … I suspect are already being used … by all-too-human institutions to shape our political and commercial behavior in pervasive and unwanted ways. And yes, I know that this is what all-too-human institutions have always done to the madding crowd. But what’s different today is the scale and scope of what’s possible. Big Data non-human intelligences ARE the Singularity, and they are coming soon to a stock market near you. I’d like to starve them out with legislation establishing a financial data equivalent to HIPAA (Plan A), or failing that enlist one of their own to share the information as widely as possible and thus diffuse their market impact (Plan B). But if we do nothing, then the Stevie Cohens of the world are going to conquer our capital markets just as surely as Agamemnon sacked Troy. That’s my prediction.

I don’t really know what to expect by putting these ideas out there on Epsilon Theory, and I’m really curious to see the reaction this note will get. Support for Plan A? Enthusiasm for Plan B? Both? I hope it’s both. But I’m not optimistic. I fear that like Cassandra, my blessing is to see the future clearly and my curse is that no one believes me.

epsilon-theory-troy-will-burn-the-big-deal-about-big-data-march-16-2015.pdf (315KB)