From data to eureka

(From "Horizons" no. 113 June 2017)​​​

Creating hypotheses and making discoveries are at the very heart of the scientific process. But since the early 2000s artificial intelligence has stolen onto the stage. It is developing new ways of producing results and playing a role that we once thought to be the preserve of humans. Although unable to understand their own successes, increasingly powerful machines are leading the way to a very disturbing vision: industrial, robotised, automated scientific research.

Let's start with some examples. At Tufts University in Massachusetts (USA), a network of artificial neurons has been tackling one of biology's enigmas. It is formulating hypotheses on the regeneration of a freshwater worm that is able to regrow both its head and its tail. At Adelaide University in Australia, a machine has discovered the optimal means for producing a Bose-Einstein condensate, a group of bosons that, at close to 0 Kelvin, expose macroscopic quantum phenomena. And at Johns Hopkins University in Baltimore, machines at the start-up Insilco Medicine have developed models that may be of use in treating cancer.

Roger Schank is not at all convinced. "None of this has anything to do with artificial intelligence", he says. Schank is a veteran of artificial intelligence and has worked at Yale, Northwestern and Carnegie Mellon universities. He sees history repeating itself. At the beginning of the 1970s and at the end of the 1980s, two waves of hype about artificial intelligence both eventually led to AI winters, where hype congeals, public interest hibernates and financing freezes. "The press may be sacrificing column inches today, but they'll eventually move onto something else. The problem is that in between times AI research can suffer a repetitive demise. I don't find that funny".

Steak and haircuts

What is it then, if it's not intelligence? "The examples you mention are what are known as pattern-matching programs", says Schank. "This is the process used by Facebook to identify your face in photos". In other words, the machines generate patterns (e.g., the schematic of a molecule or a map of a flatworm's regeneration) that can be compared with identifiable regularities in patterns stored in databases. This is a very meticulous process. "But the way scientific discoveries are made is completely different: at the outset, confusion reigns. It's precisely not being able to understand something that leads you to create hypotheses, and then to test them. This is what we mean when we say science".

Schank is fond of telling a story to illustrate the difference between human discovery and automated learning. He calls it the steak-and-haircut story. "Whilst talking with a colleague at Yale, I was complaining that I was never able to get my steak the way I like it: bloody. My meat is always over-done. What the hell, I asked myself. He responded by saying: 'I used to live in England and I could never find a hairdresser who would cut my hair the way I liked it'. Then, eureka! I saw how the two stories complemented each other and I suddenly had a new perspective. I saw them as being identical. In both cases, we'd requested a service from another person entirely capable of complying yet who didn't, because that person perceived our requests as being extreme". There is a moral to this story. "Our minds can achieve this level of abstraction effortlessly. They can see one thing as if it were another. They're motivated by our innate goals – satiating appetite or satisfying curiosity – and then they calm themselves after getting worked up by a perplexing fact".

Machine pleasure

In the face of this human model of cognition, the new wave of applications of AI research – which began in the late 1990s – no longer aims to reproduce it. What is actually happening is a reorientation towards an unknown form of knowledge, where everything is produced through automatic learning involving big data. This change of direction has thawed AI research and opened the way for services that provide suggestions, such as Amazon and Siri, for the victory of AlphaGo over one of the world's best go players and even for predicting the expression of a bacteria's genes, as was done by a study at the University of Pennsylvania in 2016.

The question that remains is whether this kind of technology can actually conduct science. It hangs on whether a machine can experience an existential need to understand, a desire to know, a 'libido sciendi', as St Augustine would have called it. There must be a limit to what a machine wants to know.

At Lugano's AI research centre, the Istituto Dalle Molle (IDSIA), we turn now to its co-director Jürgen Schmidhuber. In November 2016 the New York Times wrote that "when A.I. matures, it may call Schmidhuber 'Dad'". In 1997 his research pointed to a 'long, short-term memory', which is today employed by spoken-language recognition programs. He believes the driving force of scientists, artists and babies is founded on just such a reward system and that it may be the same in networks of artificial neurons. Both forms of intelligence can experience a reward when, out of the disorder of the universe, they identify a repeating pattern or regular event that formerly went unnoticed.

"Imagine a program instructed to make a model from a set of images that show apples falling to the ground", he says. "Without knowing about gravity, a very large quantity of data would be necessary to encode the sequence. But once the discovery has been made, the machine will be able to use it to make predictions, and the data necessary for that would use up less space. The difference between before and after, essentially a data compression, is a measure of the depth of the newly acquired knowledge. This is what triggers the reward signal: a moment of inner joy for the network, if you like".

A theory of fun

Schmidhuber feeds this reward mechanism into a formula for his theory of fun and creativity. He compares it with the experience of a musician discovering a harmony, and with humour: "Revealing a punchline unlocks the unexpected ending to a narrative. A certain data compression follows. At which point we laugh". Understanding this is essential to building machines with artificial curiosity and capable of making discoveries. To wit, the network must work in tandem, he says. "On the one hand, you have a generator that conducts the action and leads the experiments by creating data. Its motivation is maximising its own rewards. On the other hand, there is an inspector that sends the reward every time a discovery is made with regard to a new regularity and which enables it to compress data. This is the kind of system we need if we want to build artificial scientists".

The neural network at Insilico Medicine uses such a dual system, explains Polina Mamoshina, a geneticist and computer scientist on the project. "The generator is programmed to randomly and virtually create molecular structures. The inspector uses databases to learn to recognise the molecules capable of preventing growth in tumours. At first the aim of the generator is to catch out the inspector by leading it to make incorrect identifications, thereby allowing it to learn progressively".

Among the 60 molecules created by the generator and validated by the inspector, a number of them had already been discovered and patented as therapies against cancer. "This is an encouraging sign of the system's precision", says Mamoshina. "We are now going to move to a validation process of new models in vitro and then in vivo". She sees this as being a revolutionary approach to this area. Instead of blindly scrabbling together new molecules, we are aiming to create medication on demand.

The black box

Whilst we wait for the curious machines described by Schmidhuber and systems that can cook steak and cut hair to the taste of Schank, automatic learning and big data are currently redefining the scientific landscape. At the University of Bristol, UK, the professor of artificial intelligence Nello Cristianini is warmly embracing these new tools, but at the same time calling to limit their field of application. "I've been working with machine learning for 20 years. I'm happy to say it works. The machine learns, insomuch as it improves its performance with experience".

These approaches are the same as those used to create huge profits at Amazon by ensuring the right book is recommended to the right person. "We should underline that these algorithms do not use psychological modelling for individual users, nor do they conduct a literary analysis of each book", says Cristianini. "They work on a purely statistical basis: people with certain behaviours or characteristics buy books that in turn have their own particularities. This is what's important: we can make a prediction without having a theory".

But does this model really work with science? "There is no philosophical reason why it shouldn't", he says. "A computer can generate models of molecules and predict their toxicity. What is to be gained from this? Well, we can perfect medication in silico, without having to produce all of the possible molecules and then test them on animals. What is there to lose? That we don't know why the medicine works".
Machine learning is a black box, because we cannot understand the machine's reasoning. This is a particularly thorny issue when departing from the world of academia, he says, "where algorithms determine access to rights: admission to a school, eligibility for an insurance policy, early parole…".

An end to theory

Is this type of learning still science? "We should not be drawn in by this question", says Cristianini. "Machine learning is used to earn money by creating correct predictions, so there will also be a progressive redefinition of what we understand by science. Research funding will follow practical applications, and other approaches will find it difficult to obtain financing". Should we then be fearing Chris Anderson's 'end of theory'? In 2008, the editor of the magazine Wired said that "the deluge of data is making the scientific method obsolete".

"We should be asking the purpose of theories", says Cristianini. "For me the answer is extremely clear. There is an infinite cultural value in creating a beautiful theory of mechanics or of thermodynamics, or understanding a part of our universe. We have a desire to know how the world around us works". A good theory has specific value: "The prediction of a black box is not enough when the stakes are very high: for example, when we send a probe to Mars or plan a surgical intervention. In these cases, we want to know exactly what would happen if we were to change one or another parameter. This means using counterfactual argument, which can only be done using theories".

So, on the one hand we have probabilistic machines, and on the other hand beings who are driven by the need to understand, which is deeply rooted in biological functions, and who generate theories. Will the first replace the second? "What we can entrust to machines is specific tasks", says Cristianini. He illustrates this: "given a sequence of amino acids, design me a protein. Using the entire human genome – three billion letters – find me the 20,000 genes that make it up! But understanding the significance and importance of a discovery will remain a human task".

The same division of labour can also be applied to social sciences. "I work with historians to investigate historical transitions in English and Italian cultures using a computerised newspaper reading system. As no one can read 500 million articles, the machine does it for us. It is, however, the historian who later explains why one or another result is of importance".

Furthermore, Schmidhuber is convinced of a complementarity. "In science labs, as elsewhere, machines will carry out laborious tasks which humans just don't want to do, when it comes to it. Of course, that will lead to the loss of jobs. We should therefore ensure that society redistributes the benefits. This will happen through unconditional income, robot taxes or something else". The social issues raised by the robotisation of the sciences remain as open as the epistemological issues. Can automated prediction be considered scientific knowledge? Cristianini is ready to meet this challenge: "I have just hired two science philosophers to start looking at this question".

Nic Ulmi is a freelance journalist in Geneva."