Frequency-encoded eye tracking smart contact lens for human–machine interaction
Nature Communications volume 15, Article number: 3588 (2024 ) Cite this article
Eye tracking techniques enable high-efficient, natural, and effortless human-machine interaction by detecting users’ eye movements and decoding their attention and intentions. Here, a miniature, imperceptible, and biocompatible smart contact lens is proposed for in situ eye tracking and wireless eye-machine interaction. Employing the frequency encoding strategy, the chip-free and battery-free lens successes in detecting eye movement and closure. Using a time-sequential eye tracking algorithm, the lens has a great angular accuracy of <0.5°, which is even less than the vision range of central fovea. Multiple eye-machine interaction applications, such as eye-drawing, Gluttonous Snake game, web interaction, pan-tilt-zoom camera control, and robot vehicle control, are demonstrated on the eye movement model and in vivo rabbit. Furthermore, comprehensive biocompatibility tests are implemented, demonstrating low cytotoxicity and low eye irritation. Thus, the contact lens is expected to enrich approaches of eye tracking techniques and promote the development of human-machine interaction technology.
Recently the rise of wearable flexible devices triggers a revolution in human-machine interaction (HMI)1,2,3,4,5,6. Due to skin-like mechanical property and miniaturized lightweight feature, a seamless and fully connected bridge is built between smart devices and the human body, realizing HMI functions like haptic sensing7,8,9,10,11,12,13, speech recognition14,15,16, gesture recognition17,18,19, and motion capture20,21. Vision accounts for 83% of the information provided when human perceives the outside world22. Eye tracking technology, such as the newly released spatial computing device Apple Vision Pro, can analyze intention and cognition through detecting user’s attention23,24,25, thus enabling high-efficient, natural, and effortless eye-machine interaction26,27,28,29,30. Existing eye tracking devices mostly rely on pupil center corneal reflection technique, which is assisted by a near-infrared light31,32,33,34. However, this technology is severely limited because of its susceptibility to environmental light interference, the awkward positioning of cameras and light sources, and obstruction caused by user’s eyelids and eyelashes, resulting in its poor universality in daily consumer scenarios. Eye tracking technology based on electrooculography (EOG) uses skin electrodes to collect the potential signals of the eye dipole with a positive cornea and negative retina18,27,35,36, but it is susceptible to interference from muscle electrical signals and has low accuracy. Additionally, it poses a risk to the skin due to the nature of the electrode material. The obvious electrodes also make social interactions awkward. Nowadays, there is an urgent requirement for a wearable and imperceptible eye tracking device to promote the development and application of eye tracking technology in diverse fields, including interactions for individuals with degenerative diseases37, brain medical diagnosis38, cognitive science research39, product human-factor design40, consumer experience research41, and driver fatigue detection42.
