Home » Google, Technology » Could you put a camera in a contact lens?
May
26

camera-contact-1

A diagram from Google’s Patent No. US20140098226. The circular items labeled 210 in the diagram are described in the patent application as image capture components, i.e. cameras.

People have been working on putting computing hardware into contact lensesfor years now. One breakthrough happened in 2009 when a group at the University of Washington in Seattle successfully tested a prototype that incorporated an integrated circuit, an antenna, a radio receiver and an LED into a contact lens that could receive power via RF from an external battery to light the LED. Two of the same researchers worked with Google X labs to create a prototype for a glucose-detecting smart contact lens, a project that was officially announced by the company in January 2014. So integrating other minuscule hardware — such as a tiny camera — isn’t all that far-fetched.

And it’s already in the works, at least in the patent-sphere. Several patents filed by Google in 2012 having to do with integrating computing components into contact lenses were released by the U.S. Patent and Trademark Office in early 2014. At least one of them, titled “Image Capture Component On Active Contact Lens,” involves embedding tiny camera hardware.

One of the inventors listed on the patent, Babak A. Parviz, worked on the both of the formerly mentioned contact projects, as well as Google Glass, which integrates a smartphone with image capture and other capabilities into a glasses form factor. In a way, smart contacts are the next logical step after Google Glass.

Since the camera-laden contact lens is only at the patent level publicly — although it could be in the works in the lab at this point — there’s no telling if or when it will actually come out as a viable consumer product. But if it does, it is sure to be a game changer.

Hardware the Contacts Would Likely Contain

An image-capturing contact lens would need to integrate tiny, thin chips, wires, antennae and other miniature hardware, either bonded on top of or embedded within contact lens material. According toGoogle’s patent, each one would at the very least include a control circuit, a sensor and a camera (referred to in the patent as an image capture component), although more components may also be incorporated. The internal components might be connected using embedded wires, wirelessly, or a combination of the two.

The image capturing component would need or a sensor that can take in light and convert it into digital data. This could be achieved with a complementary metal-oxide-semiconductor (CMOS) image sensor, a charge-coupled device (CCD) image sensor.

The other embedded sensors listed in the patent include:

  • photodiodes (which sense light)
  • pressure sensors
  • conductivity sensors
  • temperature sensors
  • electrical field sensors
  • micro-mechanical switches

A control circuit would be required to incorporate the various components to make the image capturing contact lenses work. This circuit would have a processor that would likely be made up of an imaging control component to instruct the camera hardware how and when to capture image data; an analysis component for processing the raw image data and generating metadata about things detected in the captured images; and an interface component to exchange instructions and data with remote devices.

Another necessary circuit control component is power. The lenses would have to include some way to receive (or generate), store and distribute electric power to the internal hardware without being wired directly to an outside power source. (We can all agree that power wires attached to contact lenses would not be practical.) Energy can be transferred from the outside via RF (radio frequency) waves, or the sensors might even be able to generate energy themselves and send it to the power component. The patent lists the following possible power components:

  • battery
  • capacitor
  • solar power source
  • radio frequency (RF) power source
  • electrochemical power source
  • temperature power source
  • mechanically derived power source

The circuit control would also incorporate transceivers (components that can transmit and receive data) to facilitate the exchange of images, commands or other information between internal components and external devices. The transceiver could include an RF antenna, according to the patent. Another likely component is a data store for saving data or instructions from the internal components or remote devices.

The smart contact lenses may also include a display and possibly light emitting diodes (LEDs) to provide output to the wearer. For a display to work, the lenses would need to include various sorts of micro-lenses (possibly refractive, diffractive or hybrid lenses) to focus the images and make them appear to be suspended at some distance in front of the user. One possibility mentioned in the patent is a liquid crystal lens, which is a variable-focus refractive lens that can be altered with application of an electrical signal. Another mentioned is a Fresnel lens, which is a diffractive lens that doesn’t quite have the image quality of a refractive lens but can be made much thinner.

The system could work with one contact lens alone, or two contact lenses with identical or different distributions of components. With just one contact lens, the user would be able to capture images within view just fine since our two eyes move in conjunction with each other for the most part. With two contacts, the user could potentially capture 3-D images.

camera-contact-2

This patent application diagram shows the interaction of the contact lenses with an exterior remote device.

What Camera Contacts Could Do

The obvious function of contacts with embedded cameras is to capture images within the view of the wearer, but contacts with built-in computing components could do much more. If the contacts incorporate a display, the user might be able to see highlighted objects, expanded peripheral views, zoomed visuals or even text and other information superimposed on reality — in other words, augmented reality.

Images could be processed and interpreted by the internal contact lens components, but the interface and transceiver components could be used to communicate wirelessly with nearby remote devices such as smartphones, tablets, personal computers,Google Glass or other wearable or non-wearable devices. These external devices would have much greater processing ability, and could therefore be used to quickly provide valuable information to the user based on visual data received by the contacts.

The patent lists many other remote device possibilities, including televisions, gaming systems, set top boxes and even car stereos and navigation systems. Many devices these days are connected to the Internet, so with their help, the contacts could even possibly display information from or send images to the ‘Net.

The user might be able to blink input command patterns or send commands to the contact lenses via a remote device. It may also be possible to set parameters for automatic image capture, such as certain time intervals or anytime enough power is available, among other criteria. A lot of the functionality will be determined by what software is developed for the contacts, as well as whatever hardware ends up being incorporated.

The patent states that the image data can be processed to detect light, color, color patterns, objects, faces and movement, among anything else that can be determined by analyzing an image or multiple images, and the document even includes some example uses. For instance, images could be analyzed to look for danger, such as a car approaching an upcoming intersection, and the wearer could be warned via an audio or other non-visual cue from a remote device (which could be helpful for a visually impaired pedestrian), or some sort of alert could be displayed on the contacts themselves, such as a flashing LED or a highlighted view of the dangerous object. The contacts could incorporate multiple cameras at various positions and angles that provide a wider view than the human eye can manage, allowing the display to expand the user’s peripheral view to provide greater knowledge of the wearer’s surroundings. With the right micro-lenses, it could even allow the user to zoom in on things.

The technology could also be used for other things that require a camera and processing power, like facial recognition, another feature that might be useful for the visually impaired, but one that brings potential privacy issues to mind.

Possible Issues

Both the image capturing and facial recognition aspects of the technology are likely to cause privacy concerns. Already Google Glass wearers have had confrontations with strangers who aren’t too keen on having their every move tracked or filmed. Google has issued a list of suggested dos and don’ts for Glass users, such as always asking permission before filming or taking photos of people, turning them off in any situation where a phone wouldn’t be allowed and otherwise not using the devices in rude ways.

Depending upon how the final product looks and how tiny or transparent the components are, embedded camera contacts could allow users to take photos on the sly with much greater ease than they could with asmartphone, camera or even Google Glass. Anyone’s eye could, in effect, be wearing a hidden camera.

There’s also the concern that the user data generated by the contact components could get out into the wild. The patent does address the privacy of the wearers, stating that users would be able to opt in or out of providing demographic, location or other personal or sensitive data, and mentions that the device might be able to anonymize any data it collects, receives or transmits. But the privacy of others would, as always, be dependent on the etiquette of the users.

Comfort and safety are other concerns. To avoid obstructing the user’s vision, the tiny hardware components will either be transparent or positioned around the contact so that they won’t get in the way of the pupil. They also need to be as thin as possible to avoid making them substantially thicker than conventional contact lenses. The contacts would likely be weighted on the bottom to keep them aligned in a certain position on the eye.

Some computing components (such as LEDs) are made out of somewhat toxic materials, so they would have to be coated or embedded in such a way as to shield your eye from exposure. The RF emissions would also have to be kept at or below safe exposure levels.

If they’re anything like Google’s existing glucose-detecting prototype, the chips that the image-capturing lenses use will be embedded in soft contact material, which should keep users from being able to feel the hardware. People who already wear contacts might find them more comfortable than newbies to contact lenses, of course.

camera-contact-3

Could a full heads-up display in a contact be far off?

Coming soon to an eyeball near you?

As of early summer 2014, there aren’t any smart contacts on the market, although some products are coming close. Google is apparently in talks with the Food and Drug Administration (FDA) on its glucose-sensing contacts, which could be a stepping stone for this project, as well as a boon to people with diabetes.

Another company, Innovega, demonstrated contact assisted augmented reality at CES 2014. Their special iOptik contacts provide different optic paths to work in conjunction with a pair of display glasses, but in that case, the actual computing hardware is in the glasses rather than the contacts.

If Google’s image capturing contact lenses do come out, there are lots of possible applications. Google Glass has already been used by medical professionals and other emergency personnel to speedily access life-saving information, so one can imagine similar applications for this much less obtrusive tech [sources:MakarechiKelly]. The ability to blink commands, communicate with nearby devices and display information could make it a great hands-free way to get and send info while doing everyday things (hopefully not driving, of course).

Plus, anything that can do eye tracking, image interpretation and augmented reality display is bound to have fun gaming applications. That’s something we can all get behind.

 

Lots More Information

Author’s Note: Can you put a camera in a contact lens?

As far as comfort, I already wear contacts on a daily basis in lieu of glasses, so as long as the danger of scratching or leaching of harmful chemicals is minimal, I wouldn’t mind trying these out when and if they come to market. If prescription versions are available, they could become my eyewear of choice, especially if I can go from activity to activity without having to swap out devices and headgear.

This is yet another technological advance that might turn yesterday’s sci-fi virtual heads-up display into reality in the near future. We just might have to get even more used to the idea that anyone could be filming us at any time. So if they come into being, just don’t do anything in public that you wouldn’t want your mom to see on YouTube.

Sources

  • Anthony, Sebastian. “Google invents smart contact lens with built-in camera: Superhuman Terminator-like vision here we come.” ExtremeTech. April 15, 2014. (May 12, 2014) http://www.extremetech.com/extreme/180571-google-invents-smart-contact-lens-with-built-in-camera-superhuman-terminator-like-vision-here-we-come
  • Anthony, Sebastian. “Wireless contact lens display now a reality.” ExtremeTech. November 22, 2011. (May 12, 2014) http://www.extremetech.com/extreme/106263-wireless-contact-lens-display-now-a-reality
  • Baverman, Laura. “Meet the engineers behind Google contact lenses.” Upstart Business Journal. January 17, 2014. (May 13, 2014) http://upstart.bizjournals.com/companies/innovation/2014/01/17/researchers-behind-google-contact-lenses.html?page=all
  • Bereznak, Alyssa. “Google Patents Contact Lenses That Contain Microcameras.” Yahoo! Tech. April 14, 2014. (May 12, 2014) https://www.yahoo.com/tech/google-patents-contact-lenses-that-contain-microcameras-82730529423.html
  • Coldewey, Devin. “Google Applies for Patent on Contact Lens Camera.” NBC News. April 15, 2014. (May 12, 2014) http://www.nbcnews.com/tech/innovation/google-applies-patent-contact-lens-camera-n81231
  • Etherington, Darrell. “Google Patents Tiny Cameras Embedded In Contact Lenses.” Techcrunch. April 14, 2014. (May 11, 2014) http://techcrunch.com/2014/04/14/google-patents-tiny-cameras-embedded-in-contact-lenses/
  • Etzkorn, James, inventor; Google Inc, assignee. “Contact Lens Having a Chip Integrated Into a Polymer Substrate and Method of Manufacture.” U.S. Patent 20140085601, filed September 25, 2012, and issued March 27, 2014. http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220140085601%22.PGNR.&OS=DN/20140085601&RS=DN/20140085601
  • Flacy, Mike. “Google Developing Smart Contact Lens That Monitors Blood Sugar Levels.” Digital Trends. January 16, 2014. (May 11, 2014) http://www.digitaltrends.com/mobile/google-developing-smart-contact-lens-monitors-blood-sugar-levels/#!MxdF3
  • Gross, Doug. “Google Glass targeted as symbol by anti-tech crowd.” CNN. April 15, 2014. (May 12, 2014) http://www.cnn.com/2014/04/14/tech/mobile/google-glass-attack/index.html
  • Gross, Doug. “Google: How not to be a ‘Glasshole.'” CNN. February 24, 2014. (May 13, 2014) http://www.cnn.com/2014/02/19/tech/mobile/google-glasshole/
  • Guynn, Jessica. “Google developing contact lenses for diabetics to monitor glucose.” Los Angeles Times. January 16, 2014. (May 11, 2014) http://articles.latimes.com/2014/jan/16/business/la-fi-tn-google-develops-contact-lenses-for-diabetics-to-monitor-glucose-20140116
  • Hearn, Mark. “Google patent application details micro camera system for contact lenses of the future.” 9to5 Google. April 14, 2014. (May 11, 2014) http://9to5google.com/2014/04/14/google-patent-application-details-micro-camera-system-for-contact-lenses-of-the-future/
  • Hewitt, John. “How wireless charging works.” ExtremeTech. October 15, 2012. (May 11, 2014) http://www.extremetech.com/electronics/137729-how-wireless-charging-works
  • Honan, Mat. “I, Glasshole: My Year With Google Glass.” Wired. December 30, 2013. (May 12, 2014) http://www.wired.com/2013/12/glasshole/
  • iHealthBeat. “Google Files for Patent for Contact Lens Camera, Sensor.” April 24, 2014. (May 12, 2014) http://www.ihealthbeat.org/articles/2014/4/24/google-applies-for-patent-for-contact-lens-camera-sensor
  • Innovega. “Innovega – The Future of Personal Media.” (May 13, 2014) http://innovega-inc.com/
  • Kelly, Heather. “Fighting fires with the help of Google Glass.” CNN. February 23, 2014. (May 13, 2014) http://www.cnn.com/2014/01/21/tech/innovation/google-glass-firefighter/
  • Lanxon, Nate. “Google developing ‘smart’ contact lens to monitor blood sugar levels in diabetics.” Wired. January 17, 2014. (May 11, 2014) http://www.wired.co.uk/news/archive/2014-01/17/google-contact-lens
  • Larson, Selena. “Google X Marks The Spot On ‘Smart’ Contact Lenses.” Readwrite. January 16, 2014. (May 12, 2014) http://readwrite.com/2014/01/16/google-x-smart-contact-lenses-diabetes#awesm=~oE59LUuSFPwTPd
  • Lee, Adriana. “Google Eyes A Creepier Glass–A Camera-Bearing Contact Lens.” Readwrite. April 17, 2014. (May 12, 2014) http://readwrite.com/2014/04/17/google-patent-contact-lens-camera#awesm=~oE585rOVRS3kqY
  • Makarechi, Kia. “Google Glass Saved a Man’s Life.” Vanity Fair. April 9, 2014. (May 12, 2014) http://www.vanityfair.com/online/daily/2014/04/google-glass-saved-a-man-s-life
  • Makarechi, Kia. “Move Over, Google Glass; Here Come Google Contact Lenses.” Vanity Fair. April 22, 2014. (May 11, 2014) http://www.vanityfair.com/online/daily/2014/04/google-contact-lenses
  • Mogg, Trevor. “Google Considering Tiny Cameras for Contact Lenses, Patent Application Shows.” Digital Trends. April 15, 2014. (May 11, 2014) http://www.digitaltrends.com/mobile/google-considers-tiny-cameras-for-contact-lenses/#!Mw5dK
  • Otis, Brian and Babak Parviz. “Introducing our smart contact lens project.” Google – Official Blog. January 16, 2014. (May 11, 2014) http://googleblog.blogspot.com/2014/01/introducing-our-smart-contact-lens.html
  • Otis, Brian, Yu-Te Liao, Babak Amirparviz, and Huanfen Yao, inventors; University of Washington through its Center for Commercialization, assignee. “Wireless powered contact lens with biosensor.” U.S. Patent 8,608,310, filed February 21, 2012, and issued December 17, 2013. http://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=%2Fnetahtml%2FPTO%2Fsearch-adv.htm&r=2&f=G&l=50&d=PTXT&p=1&p=1&S1=((%22University+of+Washington%22.ASNM.)+AND+lens.ABTX.)&OS=AN/%22University+of+Washington%22+and+ABST/lens&RS=(AN/%22University+of+Washington%22+AND+ABST/lens)
  • Parviz, Babak A. “Augmented Reality in a Contact Lens.” IEEE Spectrum. September 1, 2009. (May 13, 2014) http://spectrum.ieee.org/biomedical/bionics/augmented-reality-in-a-contact-lens/0
  • Patent Bolt. “An Avalanche of new Google Contact Lens Patents Come to Light.” April 14, 2014. (May 11, 2014) http://www.patentbolt.com/2014/04/an-avalanche-of-new-google-contact-lens-patents-come-to-light.html
  • Patent Bolt. “Google Invents Micro Camera System for Future Contact Lenses.” April 13, 2014. (May 10, 2014) http://www.patentbolt.com/2014/04/google-invents-micro-camera-system-for-future-contact-lenses.html
  • Patent Bolt. “Google Takes their Glass Vision to Smart Contact Lenses.” March 21, 2014. (May 11, 2014) http://www.patentbolt.com/2014/03/google-takes-their-google-glass-vision-to-smart-contact-lenses.html
  • Pletcher, Nathan, Babak Amirparviz and Olivia Hatalsky, inventors; Google Inc, assignee. “Image Capture Component on Active Contact Lens.” U.S. Patent 20140098226, filed October 8, 2012, and issued April 10, 2014. http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&f=G&l=50&s1=%2220140098226%22.PGNR.&OS=DN/20140098226&RS=DN/20140098226
  • Rodriguez, Salvador. “Google wants to fit an entire camera into a contact lens.” Los Angeles Times. April 15, 2014. (May 11, 2014) http://www.latimes.com/business/technology/la-fi-tn-google-camera-contact-lens-20140415-story.html
  • Shankland, Stephen. “Google’s smart contact lenses: Could have been Microsoft.” CNET. January 17, 2014. (May 13, 2014) http://www.cnet.com/news/googles-smart-contact-lenses-could-have-been-microsoft/
  • Solon, Olivia. “Google embeds camera in smart contact lens.” Wired. April 15, 2014. (May 11, 2014) http://www.wired.co.uk/news/archive/2014-04/15/google-contact-lenses-cameras
  • Statt, Nick. “Augmented-reality contact lenses to be human-ready at CES.” CNET. January 3, 2014. (May 12, 2014) http://www.cnet.com/news/augmented-reality-contact-lenses-to-be-human-ready-at-ces/
  • Time. “Google’s Microcamera Contact Lens Is Coming to an Eyeball Near You.” April 15, 2014. (May 12, 2014) http://time.com/63604/google-contact-lens-patent-application/
  • Trenholm, Rich. “After Google Glass, Google developing contact lens camera.” CNET. April 15, 2014. (May 12, 2014) http://www.cnet.com/news/after-google-glass-google-developing-contact-lens-camera/
  • Week. “Google Glass contact lenses? Tech firm patents lens camera.” April 15, 2014. (May 11, 2014) http://www.theweek.co.uk/technology/58149/google-glass-contact-lenses-tech-firm-patents-lens-camera.

, , , , , , , , , , , , , , , , , , , , ,

Add reply

*