How does a multitouch monitor send the touches to a PC?

Question

I'm fairly new to touch screens. I'm having a look on Amazon at multi touch monitors like an HP 2310ti and it seems everybody just accepts that Windows 7 can handle it out of the box.

How does it work? I can't find any resources explaining how the PC receives the input from the touch screen.

Do I need to connect the PC with the monitor via USB, or is there wireless/bluetooth technology embedded in the monitor like in a wireless keyboard/mouse? How does Linux handle such a monitor out of the box? Does it recognize something like 2 connected mouse devices? I read about a few projects trying to implement this stuff in Xorg/GTK, but it seems to be pretty experimental at current state (because there is little information, if any, on this topic).

EDIT: now i bought an Acer T231 in combination with foxconn's nT330i nettop. First thing to mention is that the monitor is connected via USB to the PC, the "touch device" (e.g. the monitor via usb) is detected as a HID-USB Device by the kernel. For Linux i think the kernel detects the touch device and Xorg/Gnome is able to use it.

I tried Ubuntu Lucid running a 2.6.32 kernel which detected a HID device but gnome didnt react to touches. Now im using Ubuntu Natty Narwhal which out of the box works with single touches. To get gestures running it also ships out of the box a daemon called 'Ginn' whose config is located in /etc/ginn/wishes.xml - have a look at 'man ginn'. After getting used to all the stuff i would suggest that one has to do the following things to get a multi touch running:

1. get at least a 2.6.33 kernel - natty narwhal uses 2.6.38 which works pretty good
2. add /usr/bin/ginn as a gnome startup application - besides, im using gnome not unity, maybe unity works out of the box for complex getures
3. modify /etc/ginn/wishes.xml so it fit's your needs - i think i have modify the settings cause my monitor has problems with more than 2 fingers, so not all gestures work and i also had a few compiz issues which i also try to get running with gestures

interessting resources:

https://wiki.ubuntu.com/Multitouch/Ginn

http://www.youtube.com/watch?v=VnLdzXe8DeQ

Answer 1

I'm going to give you a quick answer because I don't have much time today, but basicly touch monitors communicates with the desktop with different communication protocol. The one used by windows 7 is called WM_TOUCH which is then interpreted by the OS : either the applicatin can use multitouch gesture, in which case the action is transmitted directly to the activ window, or the application can't handle touch event, in which case the OS automaticly translate it in a mouse language (pinching for zoom would be translated into ctrl+scroll for example). There are other languages like TUIO (most common used because cross plateform) or XML based messages...

From the hardware point of view, those monitor usually use USB to communicate with the base...

Answer 2

Your question is ultimately about the vision system, i.e. the underlying hardware/software that generates and 'sees' a touch point. In my experience there are three basic approaches to touch hardware: Capacitive, Reactive, and Infrared.

1. Capacitive relies on electric impulses that create a very small distortion on the screen. Because the human body acts as a conductor, when we touch such a screen we can create the distortion. The iPhone is an example of a capacitive touchscreen. You can also purchase multitouch overlays that are secured to your regular monitor. These overlays plug into your computer via USB and come with their own driver.

2. Resistive screens consist of multiple layers that are seperated by a thin space. When the user presses down a contact point is created where the screens touch.

3. IR multitouch systems do not necessarily need any special type of screen to work. It uses a combination of IR emitters and IR camera(s) to see touch points. The IR emitters are configured to provide an even distribution of IR light across the bottom of the screen. When the user touches the screen the IR light is reflected back down where it is 'seen' by the IR camera. The camera sends this to special software (such as CCV) which registers the touch. A driver is then needed to take the touch point information from the software and translate it into a touch/mouse event for the Operating System.

Each solution requires both a hardware component and software.

Answer 3

I have no idea how linux handles this, so I'll give a Windows perspective.

Multitouch is simplly another set of APIs, or new set of commands the Monitor drivers send and applications respond to. Just like your mouse sends a KeyPressDown message that applications respond to, your monitor sends messages that Windows then interprets as TOUCH, GESTURE, etc etc. The Windows MultiTouch SDK and the Microsoft Surface SDK are good places to start reading. It really isn't all that different from how your mouse interacts with the OS.

The hard part comes in writing the application. The multitouch applications I've seen tend to be too ambitious, trying to do too much. The best multitouch applications I've seen are on the Microsoft Surface. A friend's company has one and they write all sorts of stuff for it.

When their company gives tours, they always fire up this game on the Microsoft surface. You can have like 12 players play this thing. Everyone is a different color and you get cannonballs that are queued to you with different colors. You use a quick swipe to fire the cannon at the matching player.

It was exciting seeing twelve people huddled around this glass table swiping away and yelling.

Also fun was the Kinect camera they had going. As people walked by, gave your height and guess if you were male or female, with some amusing results!! To me, the Kinect has more potential. . .

Answer 4

Until recently I worked for a touchscreen manufacturer. Some of our products cut costs on the hardware by having a smaller processor that just sent the raw data up the USB to the PC, then a Win7 driver interpreted the data and sent out multitouch events to Windows.

Because the driver code contains proprietary IP there isn't an open-source version available so these touchscreens won't work on Linux.