Tiny Startup Completely Reinvents How We Use Touchscreens (original) (raw)

If you want a reminder of how far technology has come in the last decade, check out the YouTube clip of Steve Jobs unveiling the first iPhone to a room full of people back in January 2007. In it, Jobs teaches his audience, step by step, how to swipe and scroll through photos. He shows them how to rotate the device so that they can see an image in landscape. And he demonstrates what he calls "the pinch," pressing his fingers together and moving them farther apart to zoom in on an image. After the pinch, the crowd lets out a collective "Whoa."

Yes, in hindsight, it's kind of adorable. But what's important to note about the moment is that when Jobs revealed the iPhone to the crowd, he was not only introducing the audience to a new device. He was teaching them a new language, a new way of interacting with a computer.

>This tiny startup has developed a new technology that can detect the difference between a fingertip, a knuckle, a fingernail, and a stylus.

Now, seven years later, Qeexo is hoping to emulate Steve Jobs. Backed by $2.3 million, the San Jose, California-based startup has developed a new touchscreen technology that can detect the difference between a fingertip, a knuckle, a fingernail, and a stylus. By assigning different parts of the finger to different actions, this technology--known as FingerSense--could reduce tasks that currently require multiple steps to just one. "You can imagine it'd be like having different buttons in your hand," explain's Sang Won Lee, the company's co-founder and CEO.

The iPhone, and indeed the entire smartphone industry, have evolved dramatically since that day in 2007. And yet, for all the features that have been tweaked and perfected over the years, the language Jobs taught us has remained unchanged. We still use a single input--a fingertip--to operate the device. And that limits the way we use our phones. On a desktop, there's a mouse, a right click button, a shift button, and many other inputs that serve different functions. But on the smartphone, because there's just the fingertip, a common task like copying and pasting becomes a tedious process of tapping, holding, dragging, and selecting. Qeexo wants to finally put the smartphone on par with the desktop.

Image: Qeexo

Good Vibrations

The technology was first invented by Chris Harrison and Julia Schwarz, the two other Qeexo co-founders, who were pursuing their Ph.Ds at the Human-Computer Interaction Institute at Carnegie Mellon. They, like many in their field, were trying to come up with a solution to this "multi-touch" problem when they began experimenting with ways to identify objects by their different vibration patterns.

At the time, Lee was working for electronics maker HTC, scouting new user experience technology for smartphones. Having worked in the product planning industry since 2003 when he got his start at Samsung, Lee was familiar with the industry's desire to come up with a new way for people to interact with their screens. Then, in 2012, he found Harrison's blog post about the prototype he and Schwarz had developed. "I thought this could really solve that problem the industry was having," Lee says. So he left HTC, and in September of that year, the three co-founders launched Qeexo.

>FingerSense uses the standard accelerometer in a mobile device to pick up on the vibration patterns different parts of the hand produce.

Today, FingerSense uses the standard accelerometer in a mobile device to pick up on the vibration patterns different parts of the hand produce when they come in contact with a touchscreen. The FingerSense machine-learning engine understands what part of the finger is touching the screen and triggers a correlated action. For instance, tapping a block of text with your knuckle could trigger the copy and paste menu to appear. To select text, you'd simply drag your knuckle down the text, much like you would a desktop mouse.

Lee says this technology could have implications beyond the world of smartphones and tablets. Touchscreens in cars could be greatly improved, for instance, if drivers could simply knock on a screen with their knuckles instead of taking their eyes off the road to press multiple buttons in a row. But what might really make Qeexo one of those rare technologies that can make a room full of people say "Whoa" is that touchscreens are just a start. The company is also working on products like FingerSense on Walls, which would allow you to set your device on a table and control it by tapping on the table. FingerSense on Body, another product that's in its conceptual phase, would make it easier to control wearable devices with tiny screens, because it would enable people to tap on their arm or hand instead.

Lee says much more work needs to be done to develop sensors sensitive enough to pick up on vibrations through something like skin. Still, according to Mark Rolston, the former chief creative officer of frog design and founder of the user experience design firm argodesign, the progress Qeexo has made on smartphones alone is a major accomplishment. For years, he says, people within the industry have been researching new input methods, but few have found a way to actually tie those new inputs (the knuckle, in our example) to specific functions (text selection). "It's not necessarily intuitive," he says of FingerSense, "but neither were right click or the shift button, and those things have been adopted, and they help us. Something like this is one of the better ideas to come along."

'Dangerous Territory'

That doesn't mean it won't be without its obstacles. Rolston says Qeexo is entering into "dangerous territory," because now, it has to convince all the major device manufacturers, from Apple to Samsung, to adopt its technology at once. Not only that, Qeexo also has to convince these ruthless competitors to come to an agreement as to how the technology should be used. A knuckle tap, in other words, can't perform a different function on a Samsung phone and than it does on an iPhone.

"It's all or nothing with methods like this. They're useful when everyone expects them to be around, and therefore, application developers adopt it," Rolston says. "So we can love it, but it's really adoption that matters."

And then, of course, there's an even bigger risk, and that is the fact that these device manufacturers, with their swollen patent portfolios, could develop their own version of FingerSense themselves. Lee, for one, does not underestimate his competition. "Whoever wants to grab this market and become the next generation de facto standard is going to be our competitor and become one of the challenges we need to overcome."