One of the fun things about working in high tech is that you'll occasionally get a look at something new that you know is going to have a big impact in the future. I got that feeling when I first saw a development version of HyperCard at Apple (back when they called it WildCard).
Today I had that same feeling when I saw a video called Multi-Touch Interaction Research. It was produced by researchers at NYU who are exploring what you can do with a tabletop video touchscreen that can track separate inputs from every finger that touches the screen.
Some of the demos are just fun geeky stuff, such as an interactive lava lamp where you can move blobs around and merge them. But several other demos show people manipulating 2D and 3D images and maps faster and more fluidly than you could possibly do with a conventional graphical interface. I could picture someone using a system like this for page layout, architecture, or industrial design. It would also be fascinating to see if you could lay out a relational database visually and use finger touches to navigate it quickly.
Be sure you watch the whole video; it gets more interesting as it goes on. The lead researcher has done some other interesting stuff as well, which you can read about here. Check out the touch-sensitive LEDs.
Although the table top screen in the video is cool, I'd settle for having this interface on a tablet computer.
The multi-touch interface was the biggest part of the recent spate of Apple patents that most interested me. The pictures published throughout the web suggested some sort of gesture interface, which got me happily pondering new user interface paradigms.
ReplyDeleteI haven't yet had time to go review the patents in detail, but it seems like everybody cites the same major categories:
- support more natural grabbing of objects
- zoom/resize with a two-finger gesture
- support artists better
- multiple users for large displays (I'd love to see a conference table ...)
But I keep thinking that there's more. As long as some inputs remain serial - such as a keyboard - there is a notion of focus. That, mixed with multiple "pointers" should prove interesting.
Hi, Barbara.
ReplyDeleteThanks for the comment! I hadn't seen the Apple patent reports, but went and looked at them tonight. You're right, they're very interesting, and very similar to some of the ideas in the NYU video.
I don't know who has precedence here. Maybe they'll both owe royalties to Steven Spielberg for the movie Minority Report, which showed some touch-driven UI concepts ;-)
Here's an interesting article with some pictures from the Apple patent filings.
I've had a FingerWorks TouchStream keyboard for a couple years. It works with the same multi-touch concept than the sensing screen demoed in the video: it supports finger chords, gestures and bi-manual operations (complete guides here. They have (had) a pretty complete firmware that was very well tuned to recognizing which finger(s) were being pressed, and which gesture was being executed.
ReplyDeleteThe only drawback when using these keyboards for typing is the time it takes to get your typing speed back. Zero-force is really disconcerting at first...
Unfortunately, Fingerworks ceased operations last year. Their docs and info are still available on the website. I have no idea whether their technology made it in other products, but it looks like they definitely had a good lead in this technology. A screen is no different from a keyboard -- all the value is in the recognition software.
It turns out that Fingerworks has been bought by Apple last year. People have traced the main Fingerworks developers and found they now work for Apple.
ReplyDeleteThanks for the info, Florent. This just gets more and more interesting.
ReplyDeleteI agree with you about the disconcerting feel of zero-touch keyboards. A mobile peripherals company a couple of years back made an input device that projected the image of a keyboard on a tabletop. An optical sensor detected your keypresses. The idea was brilliant, but typing felt like you were drumming your fingers on a table.
Chording is very interesting. I remeber seeing video years ago of Doug Engelbart demonstrating a chord keyboard. As I recall, it looked a little like a chunk of a piano keyboard. You could type very quickly by using combinations of multiple fingers, but it looked wickedly difficult to learn.
Maybe a lot easier if I were any good at playing the piano...
There's a nice article on good ol' Wikipedia about chord keyboards.
>This song was heavily used in the Animatrix
ReplyDeleteAhhhh, cool. It's at moments like this that I really love the Internet.
Thanks for the tidbit, Michael.