In the twinkling of an eye

(From "Horizons" no. 111 December 2016)​​​

Pedestrians often try to find their way about using their smartphones. The computer scientist Peter Kiefer and the geomatics expert Martin Raubal are at work together trying to make things easier for them. They work at the GeoGazeLab at ETH Zurich and are trying to refine smartphone maps so that pedestrians will find their way perfectly in any new environment. To this end they are developing special systems that involve attaching an eye-tracking module to one's head. These modules comprise different cameras that are variously focussed on the eyes of the user and on the user's field of vision. By means of eye tracking, Kiefer and Raubal can determine which landmarks pedestrians use to orient themselves. Their findings are interesting. "People ignore some elements on the map completely", says Raubal. In order not to confuse people, he suggests that these elements – railway tracks, for example – should be left off such maps altogether.

This is just one of many examples illustrating the remarkable progress made by so-called 'eye tracking' – the process of automatically tracking the direction of your gaze. The importance of this technology shouldn't surprise us, because people's gaze can tell us exactly what's the object of their attention, and also how they feel. Many areas of science and business use this technology today, from cognitive research and sociology to the car industry.

Stressed pilots in front of the camera

Kiefer and Raubal are also busy with another, especially ambitious project, this time in the field of air transportation. They are engaged in a collaboration with the airline Swiss, using eye tracking to monitor the training of pilots in flight simulators. In order not to hinder the pilot, eye-tracking cameras are not installed on his head, but in the cockpit itself. Raubal and Kiefer want to use the trainees' eye movements to recognise what kind of situations place them under stress. Swiss hopes that this method will offer new information to help them further refine their flight training programme.

You can also use eye tracking to help optimise your office space. This is the area of research of Mandana Sarey Khanie, a civil engineer at the Interdisciplinary Laboratory of Performance-Integrated Design (LIPID) at EPFL. People who sit for eight hours a day in front of their computer often complain of sore eyes, tiredness and headaches. This can be because of brightness contrasts in their environment. People usually work more productively if they're in an office with pleasant lighting. Sarey Khanie is investigating how the intelligent use of light can be applied when designing workspaces. Her focus is on offices that are lit up by natural light.

Sarey Khanie's project uses an eye-tracking system that comprises three cameras mounted on a person's head. Two look in the person's eyes while a third records the orientation of their head. Together, they serve to determine the person's viewing direction. Eye tracking enables Sarey Khanie to recognise when a person reacts to light in a systematic way. "In one experiment we observed that people like to look out of the window, and only avoid doing so when the incoming sunlight creates stark brightness contrasts", she says. You could carry out a survey instead, to try and find out if people feel they're being blinded by light at the workplace. But such a method would be too imprecise, explains Sarey Khanie. Together with Marilyne Andersen, the Director of LIPID, Sarey Khanie wants to develop software tools to enable architects to carry out simulations that meet three requirements of construction planning: maximising the use of daylight and visual contact with the outside world; avoiding the glare of bright light; and keeping energy use low.

Looking at nothing

Eye tracking is also used in pure research. Psychologists in particular are fond of the technology, because it enables them to observe human behaviour in an uncompromised manner. "Your eye movements aren't something you can really control", explains Agnes Scholz, a psychologist at the University of Zurich.

Scholz uses eye tracking in order to investigate fundamental thought processes. When people make decisions they can orient themselves on abstract rules, or base their decisions on examples taken from recent memory. Scholz carried out an experiment to see if she could observe differences between these two approaches. Test subjects were asked to assess several people whose profiles were presented to them on a computer screen. In order to check whether recent memory played a role in their assessment, the test subjects were presented with example cases on the monitor before they came to make their own assessment.

When the test subjects were observed by means of eye tracking, it revealed a fundamental difference in their direction of vision. The assessment ran differently if the test subjects remembered the examples they had seen. While they were making their decision, these test subjects looked at specific areas of the monitor – the empty spaces where the example cases had been shown just before. Psychologists call this behavioural phenomenon 'looking at nothing'. The other test subjects – those who based their assessment on abstract rules – did not engage in this 'looking at nothing'. In future, Scholz wants to find out more precisely when this specific viewing behaviour occurs, and what role it plays in decision-making.

Scholz used a special camera for her eye tracking. It is directed at the eyes of the test subjects and also uses infrared light to measure the geometric characteristics of their pupils. Such systems have been honed more and more in recent years, and now function very precisely. However, they often lack flexibility, especially in cases where people move about a lot without keeping anything firmly in their gaze.

Eye tracking at the conference table

At the Idiap Research Institute in Martigny, Kenneth Funes Mora and Jean-Marc Odobez are developing systems that use relatively inexpensive cameras without high resolution. They register both colours and distances. Sophisticated algorithms enable a computer to use the pictures from the cameras to determine the direction of one's gaze at all times. The variable angles of the head and eye movements are captured and then converted into data that describes the changes in a person's direction of vision. The researchers can place these camera systems inconspicuously on a conference table in order to study negotiation techniques.

Funes Mora and Odobez patented their new eye-tracking method a while ago now. Funes Mora is currently researching at the Institute only on a 50% post, because in the rest of his time he has to look after their spin-off company, 'Eyeware'.

The two researchers believe that such an eye-tracking system can have many different areas of possible application. Their newly developed camera is especially suited to investigating people's visual attention, and to supporting the interaction between people and computers. It could be used by a robot, for example, to advise customers in a shopping mall. Applications in the medical field would also be possible – such as in diagnosing disorders like autism, which can be recognised by tracking eye movements.

And this will hardly be the last of their ideas for applying their eye tracking system. "The eyes simply tell you a lot about people", says Funes Mora.

Sven Titz is a freelance science journalist.