Been some time that I came across an interesting patent application but here’s an interesting one. Microsoft engineers have applied for two patents detailing the Architecture to control the PC using hand gestures. Earlier Microsoft showed controlling devices by muscle movements and has applied to patent a pressure sensitive mouse.
The basic architectural diagram shows a user can use voice, gesture, keyboard or mouse as inputs to a computer. Microsoft engineers talk about using both voice and gesture together to control various actions on the PC. A basic image shows the two cameras and a mic are placed in front of the user:
To determine when to consider movement as input the system relies on the volume area around it. The person who plans to interact with the computer has be within the engagement volume before the system kicks in and starts considering the movement as input. Here’s a flowchart that explains how the whole system is intended to work:
The patent gives examples of how voice and gesture input would work together. When you make a gesture the screen will recognize it and show you options on the screen:
The following is what you have to do and what the system will do:
- You make a hand gesture
- The cameras track your movement and show 4 options on the screen
- You say ‘move’
- The mic recognizes the word and depending on the direction you now move your hand the window will be moved.
That’s the basic gist of it. There are some more commands such as ‘Flick’ that’ll allow you to move a window from one monitor to another as shown in the image:
One of the uses of this technology as shown in the patent application is while doing boardroom presentations. A user can flick through slides or demonstrate something using gestures:
An image in the application is a what Natal can do. Two users interacting with their PCs that are connected over a network and there’s a third broadcasting the two screens:
Since Project Natal offers pretty much all of the same capabilities, Microsoft engineers bringing this to the PC would be quite interesting. According to them, this might help people who’re not comfortable using a pointing device.
