Information Bottlenecks, Fake News and Boredom

Information bottleneck

A new idea called the “information bottleneck” is helping to explain the puzzling success of today’s artificial-intelligence algorithms — and might also explain how human brains learn:

“Claude Shannon, the founder of information theory, in a sense liberated the study of information starting in the 1940s by allowing it to be considered in the abstract — as 1s and 0s with purely mathematical meaning. Shannon took the view that, as Tishby put it, “information is not about semantics.” But, Tishby argued, this isn’t true. Using information theory, he realized, “you can define ‘relevant’ in a precise sense.”

Quantum computers need smart software

Nature reports “The world is about to have its first (useful) quantum computers … The problem is how best to program these devices. The stakes are high — get this wrong and we will have experiments that nobody can use instead of technology that can change the world.” Related to this, I’m excited to spend some time in a couple of weeks with Scott Aaronson of QCWare who “develop hardware-agnostic enterprise software solutions running on quantum computers”.

In other “the quantum age is nigh” news:

A pair of researchers from the University of Tokyo have developed what they’re calling the “ultimate” quantum computing method. Unlike today’s systems, which can currently only handle dozens of qubits, the pair believes their model will be able to process more than a million.

Australian researchers have designed a new type of qubit — the building block of quantum computers – that they say will finally make it possible to manufacture a true, large-scale quantum computer.

Microsoft now has 8,000 AI researchers

Apparently, Microsoft now has 8,000 AI researchers. That’s a veritable army. Presumably a big chunk of the 8,000 are datamungers, infrastructure engineers etc., just as on aircraft carrier like the USS Nimitz (pictured below) where there are, order of magnitude, the same number of personnel but most are cooks, logistics managers, medics etc. rather than fighter pilots. But still: Eight thousand!!!

And in other “that’s a lot of engineers” news: Amazon now has 5,000 people working on the Echo / Alexa.

As I’ve noted before, in my view it is utter conceit that it is possible to do something ‘AI’ which is truly and sustainably novel, scaled and production-ready in a high stakes environment (such as trading) without a decent sized team focused on a narrowly defined problem.

Fake news and botnets

Fascinating interview with Researcher Emilio Ferrara on fake news and botnets:

“We found that bots can be used to run interventions on social media that trigger or foster good behaviors,” said Ferrara. “This milestone shatters a long-held belief that ideas spread like an infectious disease, or contagion, with each exposure resulting in the same probability of infection. Now we have seen empirically that when you are exposed to a given piece of information multiple times, your chances of adopting this information increase every time.”

Representational universality

It has been at least a month since we have had a Hofstadter quote, and this week’s Rabbit Hole column feels light on existential theory, so here’s a classic:

“In the world of living things, the magic threshold of representational universality is crossed whenever a system’s repertoire of symbols becomes extensible without any obvious limit.”

Boredom

And finally, in general I have quite a bit of reticence about sharing TED Talk links as, to quote the low-agreeability Benjamin Bratton, they can be kinda “Middlebrow Megachurch Infotainment.” Having said that, here’s a link to a terrific TED Talk on why boredom is important.

Best quote:

“As one UX designer told me, the only people who refer to their customers as “users” are drug dealers and technologists.”

epsilon-theory-info-bottlenecks-fake-news-and-boredom.pdf (506 KB)