My Xmas Dream

Xmas is a time of reflection, of meditation on how the world faring. But the situation for many is dire. In the United States one out of zeven people live in poverty, getting their food from foodbanks whose budgets are now being cut by the government. In Syria, hundreds of thousands if not millions live in tents, with hardly enough blankets and clothes to keep them warm. In Central Africa rebel bands kill and rape in ways beyond our worst imaginations. In the meantime, a few fortunate bankers, traders, investors and CEOs of mega-companies make millions or even billions. This unequal distribution of wealth is perhaps the most disturbing fact of all.

At Xmas time we buy off our conscious by giving to charity. But most big charities are run as full scale companies these days. For instance, Salil Shetty, the Secretary-General of Amnesty International earns $305,000 a year. I wonder, how do you justify to Joe the carpenter who earns a tenth of that amount per year to give generously? This world is deeply twisted.

So then here is my Xmas dream. Let's build a company with the brightest minds in the world. Everyone who joins can not earn more than a modest salary (or if you do this on the side, you will earn nothing at all). We will join forces to develop the most amazingly intelligent algorithms to suck the financial markets dry. (Our secret algorithms will of course based on deep learning.) We will wreck the large trading firms into bankruptcy. With the billions of dollars that we earn with trading we will refrain from buying ourselves that Ferrari, but instead we will save the rainforest, educate the world, provide food and shelter for those who need it.

That is my Xmas dream. Who wants to join?

The Role of Corporate Involvement at NIPS

I was conference chair for NIPS 2013 together with Zoubin Ghahramani (but I only speak for myself in what I write below). Mark Zuckerberg visited NIPS and created quite stir. And discussion about the role of companies at NIPS. Here is my perspective on the issue.

First, I think it's great to have this discussion, because it's clear that the field is rapidly changing. I view corporate involvement of this magnitude as an interesting experiment that we have to work through and reevaluate every year. As with everything there are advantages and disadvantages to corporate involvement. Here are a few I can see:

pro's
-Industry is an integrated part part of our ecosystem from which we hugely benefit. Our students have good jobs (with higher salaries than mine). I have never heard of anyone being worried that their students could not find a job. That's a huge blessing.

-Industry helps us with funding our fundamental research and our conferences. There are many faculty grants that support fundamental research that only needs to be tangentially related to the main mission of the company. Industry also pays for a good chunk of our registration fees, meals, student awards etc.

-Some companies, in particular Microsoft, operate as an op research lab, allowing their employees to serve the community as reviewers, area chairs, workshop organizers and even program chairs (see last years program chairs at NIPS 2012). Their research papers are just as interesting as papers coming out of academia.

-Industry labs may accelerate the pace at which our field develops. We may actually need their resources (data and computational infrastructure) to make the next leap in building truly intelligent AI systems. These successes may generate enthusiasm among our new students entering the field, further accelerating progress. 

con's
- Reproducibility. There is a certain danger that more datasets and algorithm details will be locked behind the company's firewall. It's not good if talks have classified elements, or certain questions can not be answered. Mark Zuckerberg announced that Facebook has developed the best ever face recognition system in just under three months. That's super exciting, but will we have access to it (either to the parameters of the deep net, or to the dataset and algorithm used to train it)? In other words, can we reproduce their results?

-The Google question (soon the be renamed as the Facebook question). I am often reviewed by a mix of astronomers, mathematicians and computer scientists (The Netherlands is a small country..). I have been confronted more often than I like with what has become known as "the Google question": if this research can be done at Google with many more resources, why should we grant you this money? So here our close connection to industry fires back because it is not considered as "fundamental research" by many who don't understand how we operate. 

-Is it desirable that big companies lure our best students away to improve ad-placement, where they could also have contributed to curing some of the horrible diseases that plague mankind? Perhaps not, but people have the right to make their own decisions.

And as to Mark Zuckerberg's visit. I found it super exciting! I think it's a small price to pay that certain workshops were temporarily under-attended. But I agree that NIPS should not be turned into a publicity and recruitment event for big companies. We should acknowledge that our field is a complex ecosystem that involves both academia and industry. Our challenge is to find the right balance.

No-X Theorems

In physics there a few no-X theorems that seem rather suspicious in the sense that they point to cracks in our current understanding of nature. One of them is Penrose's cosmic censorship hypothesis that translates to a "no-naked-singularity" conjecture. The most famous examples are at the center of black holes where there are supposedly singularities. But we will never know about them because they are shielded by black hole horizons (unless you are willing to kill yourself and fly inside the black hole, however you will not be able to report back to us). That to me sounds a bit suspicious.

Here is another one. The no-communication theorem in quantum mechanics. When two particles are entangled they become correlated with each other over arbitrarily large distances. This implies that if Alice measures the spin of one of the particle here at earth, it collapses the joint wave function, say in a spin-up state, then *instantaneously* Bob's particle at the other end of the universe collapses into a spin-down state. Special relatively forbids anything moving faster then the speed of light, because if it does for some observers a signal can arrive before it was sent! Similarly here: for certain observers Bob first checks his spin before Alice has measured hers and therefore this observer concludes that Bob collapses the wave function and not Alice!

The only way out is that these two interpretations are actually coherent and this means that no information can be send between Alice and Bob. Alice can not force the particle in a up state (or a down state for that matter) so that's no use. But perhaps she can transmit information by simply collapsing a superposition into one of the two pure states (irrespective whether it's up or down)? Bob only has to determine if the wave function is in state A+B or in one of the pure states A or B. Alas, Bob can not do this with a single measurement because his measurement will collapse the wave function into either A or B. Now what if he could make N copies of his wave function? Then by measuring all N copies he finds either that all of them are in state A or B or he finds some of them in A and some in B indicating that the original wave function was still mixed. Unfortunately for Bob, the no-cloning theorem comes to quantum mechanic's rescue which says that you can not make copies of wave functions.

Feeling uneasy? To me this seems like a theory that is trying to rescue itself. Not really the most concise explanation. We need multiple no-X theorems to wiggle ourselves out of difficult questions. What this points to in my opinion is that the current theory (quantum mechanics) is an unnatural (but still accurate) theory of nature. We reach the right conclusion but through weird complicated reasonings. Very similar to Ptolemy's model of the universe that made the correct predictions but was complicated and difficult to interpret. The new theory replacing quantum mechanics will hopefully act as Ocam's razor and bring natural explanations for quantum weirdness.

Discrimination = Overgeneralization

Why is it that we get slightly nervous when we see a man with a long beard clothed in a dress sitting next to us in the plane? We overgeneralize. We associate terrorism with Islam and Islam with bearded people in dresses. But clearly, the number of peaceful, well willing muslims far exceeds the number of muslim terrorists.

If tomorrow a horrific terrorist attack from white Caucasians would take place, we would not suddenly get nervous with every white Caucasian sitting next to us in our plane. Perhaps the somewhat obvious reason could be that while we tend to ignore all the peaceful muslims in our society, we do have one important counter example to the hypothesis that all Caucasians are terrorists: ourself. Any viable hypothesis should not implicate ourself, so you start looking for more subtle attributes: does the person look like a slob? Or perhaps do they grow long hair etc.?

Our tendency to overgeneralize directly causes us to discriminate. Studies reveal that we (white people) are more afraid to be mugged by black people than by white people, even though we think we do not think so ourselves! In machine learning we call this "underfitting". Our theories about the world are too simple, our prediction about the world are poor and we draw conclusions too fast.

What can we do about this? Firstly, being aware of these inborn tendencies can help, at least at a conscious level. But it's not enough. My recommendation: build appreciation of other cultures by organizing multi-cultural parties in your community. Experience many positive counter examples to counteract your prejudices. Don't let those unfounded hypotheses based on a single negative example cloud your judgement.