Sixth Sense Technology

‘Sixth Sense’ is a gestural interface device comprising a neck worn pendant that contains both a data projector and camera. Head worn versions were also built at MIT Media Lab in 1997.

Sixth Sense Technology


‘Sixth Sense’ is a wearable gesture interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.

‘Sixth Sense’ is a gestural interface device comprising a neck worn pendant that contains both a data projector and camera. Head worn versions were also built at MIT Media Lab in 1997 that combined cameras and illumination systems for interactive photographic art, and also included gestures recognition.

Sixth Sense is a name for extra information supplied by a wearable computer, such as the device  called “WuW–Wear your World” by Pranav Mistry , building on the concept of telepointer, a neck worn projector and camera combination first proposed and reduced to practice by MIT Media Lab student Steve Mann.

We’ve evolved over millions of years to sense the world around us. When we encounter something, someone or some place, we use our five natural senses to perceive information about it; that information helps us make decisions and chose the right actions to take. But arguably the most useful information that can help us make the right decision is not naturally perceivable with our five senses, namely the data, information and knowledge that mankind has accumulated about everything and which is increasingly available. Although the miniaturization of computing devices allows us to carry computers in our pockets, keeping us continually connected to the digital world, there is no between our digital devices and our interactions with the physical world. Information is confined traditionally on paper or digitally on a screen. Sixth Sense bridges this gap, bringing intangible, digital information out into the tangible world, and allowing us to interact with this information via natural hand gestures.

‘Sixth Sense’ frees information from its confines by seamlessly integrating it with reality, and thus making the entire world your computer.



There are many components required for Sixth Sense Technology. To use Sixth Sense Technology, you require some wearable components that will enable your movements or gestures to access that feature. Some of the devices that are required for the movements are:

  • A Mirror.
  • A Projector.
  • A Camera.
  • Colored Caps.
  • A Smart Phone.
  • A Microphone.


How they Work???

The Sixth Sense technology contains a pocket projector, a mirror and a camera contained in a head–mounted, handheld or pendant–like, wearable device. Both the projector and the camera are connected to a mobile computing device in the user’s pocket. The projector projects visual information enabling surfaces, walls and physical objects around us to be used as interfaces; while the camera recognizes and tracks users' hand gestures and physical objects using computer–vision based techniques. The software program processes the video stream data captured by the camera and tracks the locations of the colored markers (visual tracking fiducials) at the tips of the user’s fingers. The movements and arrangements of these fiducials are interpreted into gestures that act as interaction instructions for the projected application interfaces. Sixth Sense supports multi–touch and multi–user interaction.


Applications of Sixth Sense Technology



The drawing application, lets the user draw on any surface by tracking the fingertip movement of the user’s index finger and thumb.

Four colored cursor are controlled by four fingers wearing different colored markers in real time. The projector displays video feedback to the user on a vertical circle.


Create Multimedia Reading


Sixth Sense can be programmed to project related videos onto newspaper articles you are reading.

To check the Time

Draw a circle on your wrist to get a virtual watch that gives correct time.

Make a Call


You can use Sixth Sense to project a keypad onto your hand, and then use that virtual keypad to make a call.


Take Pictures

 If you stretch your index fingers and thumb into a square or frame, the system will snap a photo. After taking the desired no. of photos, we can project them onto a surface, use gestures to sort, organize and resize them.


Meeting a Person


If you are going to meet any stranger or any person of your day to day life, it will project all their details (i.e., name, qualification, age, work, etc.).


Other Applications

  • The projector displays a map on the wall, and the user controlling it using zoom and pan gestures.
  • The user can pick up a product in supermarket (e.g., a package of paper towels), and the system could display related information (e.g., the amount of bleach used) back on the product itself.
  • The system can recognize any book picked up by the user and display Amazon rating on the book cover.
  • As the user opens a book, the system can display additional information such as reader's comments.
  • The system is able to recognize individual pages of a book and display annotation by the user's friend. This demo also suggested the system would be able to handle tilted surface.
  • The system is able to recognize people by their appearances and project a word cloud of related information retrieved from the internet on the person's body.
  • The system is able to recognize a boarding pass and display related information such as flight delay and gate change.


What's Your Reaction?