Speech recognition: Almost ready for mobile prime time

I've always wanted to see speech recognition incorporated into mobile devices. Since you don't have a big keyboard when you're on the go, you ought to be able to just talk to your phone and tell it what to do, or dictate memos to it and have it convert them into e-mails or SMS messages. In addition to being incredibly convenient, this would increase the safety of a lot of drivers. It's a spooky fact, but in surveys I've done more than 10 percent of the US population admitted to sometimes sending text messages while driving.

Not smart, not safe.

So, is voice recognition good enough to let you just talk to your mobile device and then send the converted text as a message?

I first asked myself that question a couple of years ago when I bought a copy of Dragon NaturallySpeaking and a small voice recorder. I tried recording weblog posts and other documents while driving, and then brought the recorded sound back to my computer to convert it into text. The result was a disaster. Dragon was unable to keep pace with the recorded sound in the files, and started dropping sentences, paragraphs, and eventually entire pages of spoken text. I was so disgusted, and so disappointed, that I gave up and went back to listening to sports talk radio while I drove.

Recently a newly appointed product manager at Nuance (publisher of Dragon) sent out a survey asking for feedback on the product. Unlike most product managers, she signed the survey form with her own name and with her own e-mail address. Most product managers wouldn't do that because they don't want to be overwhelmed with feedback. I don't know how much feedback she got in general, or how overwhelming it was, but she got a note back from me describing my problems with the product and explaining why I really wasn't satisfied with it.

I didn't expect to get any reply from the company; Nuance has a remarkably restrictive policy on providing technical support unless you pay extra for it. Usually, companies that do that aren't interested in getting any sort of conversation going with their customers. But to my surprise, I got a note from the product manager not only sympathizing with my problems but offering to send me a copy of the latest version of the software and a voice recorder that she said would work well with the software. I wish my weblog address hadn't been in my signature, so I would know if they do this sort of thing for every frustrated user. But anyway I took her up on the offer.

You can see the results here. I dictated this weblog post using the voice recorder, synced it onto my computer for recognition, and then corrected the (few) errors by hand. There are pluses and minuses to the dictation system. The good news is that the program can now keep up with my dictated speech. I no longer lose sentences or paragraphs of text. I'm also surprised with the way the product recognizes trade names, so for instance when I say Home Depot or McDonald's or Nike or Apple or IKEA or Lowes, Dragon gets the names correct and properly capitalized (I didn't have to fix anything in that sentence).

On the other hand it does make mistakes -- the packaging claims about 99% accuracy, which means that you should expect one word in every hundred to be incorrect. My guess is that I'm getting somewhere between 97 and 99% accuracy. That's not bad. In fact, it's pretty darned impressive. But in practice it still means you have to go back and do a lot of corrections.

The training is close to torture: reading aloud a 20-minute excerpt from a Dilbert book while trying to pronounce every word correctly. Later I tried setting up the program without any training, and it worked exactly the same. So my advice is to skip the training.

The software is not great at understanding where punctuation should be placed in the text. I have learned that I have to give grammatical guidance by saying things like "comma," "period," and "new paragraph" in order to make sure that the text will be reasonably well formatted.

If I just speak naturally the text will come out like this making it very difficult for anyone else to read and even making it hard for me to edit without punctuation inserted it is very hard to get tell where a sentence was supposed to end and another one start add in a few wreck cognition errors by the soft wear and the text is not something you would want to send to someone uncorrected

Speaking with punctuation is unnatural, and could be somewhat distracting while driving. I have to think carefully about the text that I'm dictating, and I believe for some people that could cause them not to pay enough attention to what's happening on the road. I think I can do it safely or I wouldn't do it, but it definitely is an issue to consider.

Overall, I think this approach will make me a bit more productive, so I should be able to produce a little bit more weblog content and maybe get some other sorts of things done as well.

So it's nice for me, and I finally feel like I got my money's worth from Dragon. But is the technology ready for broad deployment in mobile devices?

I think the answer is technically yes, but practically no. Mobile devices are casual-use; tasks that require too much commitment or effort just don't get used. Without careful attention to spoken punctuation, the software produces errors and the sort of run-on text you saw above. Even in a short message, I think it's likely that you'd get more mistakes than you'd find acceptable. Correcting those errors on a small screen with no mouse would be tedious at best (it's an annoying task even on a PC).

More importantly, the software is very sensitive to the quality of the sound file coming into it. I believe most phone microphones and headsets wouldn't produce the required quality. You'd probably get better results with a service that just records your speech and has someone in India retype it (such services exist today).

So, the news from the world of voice recognition is hopeful for mobile users but not yet wonderful. The technology is good enough that you can definitely use it as a substitute for typing if you have physical problems. It's also a useful PC productivity tool for someone who generates a lot of text for a living.

However, I think we're not yet quite at the point where you can just talk to your phone and have it reliably transform all of your speech into text. It's getting better, but it's not all the way there yet. For a mobile device, the dream of just talking is still a dream. But I do think it's a dream that's getting closer to reality.

===========

PS: I'd also like to compliment Kristen Wylie, the product manager at Nuance who responded to my message. Take notes, folks, this is the right way to communicate with customers online -- sign your real name, use an address they can respond to rather than a no-replies mailbox, and when someone has a problem help them solve it.

News flash: Microsoft lied

"I'm shocked, shocked to find that gambling is going on in here!" --Claude Rains, Casablanca

I'm a little late in getting to this item, but I recently came across James Plamondon's online confession that he lied while working as a technology evangelist for Microsoft (link). Actually, "lied" is probably the wrong word. James systematically misled and manipulated software developers, and enthusiastically taught others at Microsoft how to do the same. Some samples from his work there:

Working behind the scenes to orchestrate "independent" praise of our technology, and damnation of the enemy's, is a key evangelism function... "Independent" analyst's report should be issued, praising your technology and damning the competitors (or ignoring them). "Independent" consultants should write columns and articles, give conference presentations and moderate stacked panels, all on our behalf (and setting them up as experts in the new technology, available for just $200/hour). "Independent" academic sources should be cultivated and quoted (and research money granted). "Independent" courseware providers should start profiting from their early involvement in our technology....

Analysts sell out - that's their business model. But they are very concerned that they never look like they are selling out, so that makes them very prickly to work with....

The key to stacking a panel is being able to choose the moderator. Most conference organizers allow the moderator to select the panel, so if you can pick the moderator, you win. Since you can't expect representatives of our competitors to speak on your behalf, you have to get the moderator to agree to having only "independent ISVs" on the panel....Sounds marvelously independent doesn't it? In fact, it allows us to stack the panel with ISVs that back our cause....

Get a well-known consultant on your side early, but don't let him publish anything blatantly pro-Microsoft. Then, get him to propose himself to the conference organizers as a moderator, whenever a panel opportunity comes up. Since he's well-known, but apparently independent, he'll be accepted....

A Jihad is a road trip. in which an evangelist visits a large number of ISVs one-on-one to convince them to take some specific action. The classic Jihad is one focused on getting Tier A ISVs to commit to supporting a given technology by signing the technology's Letter of Agreement...As in sales, the purpose of the exercise is to close – to get the mark the ISV to sign on the dotted line, in pen, irrevocably.

The Role of ISVs
* Pawns in the struggle....
* Valuable pawns
o We can't win without 'em
o Must take good care of them
* Can't let 'em feel like pawns
o Treat them with respect (as you use them)

Developer Conferences....
* Subvert independent conferences
o Love them to death

Developer Magazines
* Same as developer conferences
* Infiltrate and subvert

You can see the details here. Be sure to skim the comments at the bottom. They're a hoot.

Speaking as someone who led the competitive teams at Apple and Palm for many years, I guess I ought to get worked up about this stuff. But mostly I think it's just old and tired. The whole "Microsoft is evil" theme is kind of pathetic these days, like the trial of an 85-year-old mobster. Yeah, I know, they deserve everything that Google's doing to them. Let's move on.

It's also not really news that a lot of analysts and conferences are on the take. For the record, you should always understand who's paying the bills when any "authority" talks.

I'm usually not moved by someone who apologizes only after being exposed in court and abandoned by his employer (link). But I'll take James at his word that he's trying to make amends.

Apology accepted, James.

That doesn't mean, though, that I agree with his prescription on what the industry should do about the situation. James says the best way to prevent a recurrence of Microsoft's misdeeds is to set professional standards for evangelism:

Microsoft...is this week launching its first public volley in the Mother of All Standards Battles, to control the de facto standards of cloud computing. For Microsoft, this is a life-or-death struggle. When Microsoft's back is to the wall, can it reasonably be expected to refrain from using the TE tactics that it KNOWS will help it win, if its use of those tactics is unrestrained?.... This problem can only be treated, I believe, by professionalizing TE, and thereby inoculating platform vendors against unethical TE practices. That's why I felt compelled to come forward now. Only now have I realized how wrong I was, and by coming forward now, in the opening skirmishes of the Cloud Computing Wars, I can begin to make amends for my past wrong-doing.

James is even writing a book on what he thinks those professional standards should be. That ought to be an interesting read. Maybe Amazon could do a two-for-one offer with the Martha Stewart Guide to Ethical Investing.

But I think the real problem isn't missing standards, it's missing morality. I believe James was able to thrive at Microsoft because the company's hypercompetitive culture condoned dishonesty, as long as you didn't get caught in public. Everybody in the industry believed they worked that way. I think the problem wasn't just in Microsoft's evangelism, it also included the company's marketing, business development, and so on.

Unfortunately, they're not the only company in the tech industry that thinks that way.

The real cure here is not to read a book or professionalize anything. Just take a course in business ethics. Or better yet, save your money and memorize this rule:

Never mislead a customer or partner.

And if an appeal to morality isn't enough to move you, keep in mind that even the cleverest liars eventually get outed. Just ask James Plamondon.

(By the way, James, if you really want to make amends, how about sending the royalties from your book to the stockholders of the companies you damaged or destroyed?)

==============

Thanks to Andrew Shebanow for pointing out this issue, in a cool little essay here.

The Palm Pre: Think Similar

Palm died. Palm OS died. Get over it.

Now let's talk about this new company, and product, that happens to be named Palm. I don't know if they'll survive or not, but they have a chance, and they're definitely interesting.

That was my overall impression after visiting Palm at CES 2009. The differences started with the meeting room itself. Rather than shelling out for a (very expensive) booth, Palm had an upstairs display room off the show floor. That in itself is not unusual; companies low on money often take a display room at CES so they can have some sort of presence at the show. Usually they get very little traffic, because you have to make an effort to find them.

But there was a short line outside Palm's room. A friend and I got into line, and the Palm folks asked us for our business cards. They went away for about 30 seconds, came back, and pulled us both out of line. "You can go right in."

I'm not sure why they did it, since neither of us are VIPs. But somebody was screening the cards and pulling out anyone whose name they recognized. That was the first sign that I was dealing with a different company -- although the old Palm was pretty well organized, that level of attention to detail would have been unusual.

The second difference came when we entered the room itself. A display room at CES usually is an empty space about 40 feet (13m) or more on a side, with one big presentation screen, some chairs, and a couple of demo stations along the walls. You can take in the whole thing in 30 seconds. Instead, Palm had divided its space into almost a maze, with little meeting rooms (lined with couches) and corridors, all set off by gauze curtains. Along the "corridors" were abundant food carts (with servers, another unusual touch), and small stations where employees were giving continuous Pre demos to groups of up to about a dozen people. You could get very close and intimate with the device, although no touching was allowed.

It felt like a technology harem.

I don't think the old Palm would have decorated quite like that, let alone shell out that much money for exhibit space in a time of layoffs and financial stress.

The presenters were extremely well briefed and disciplined, although they didn't feel robotic. They showed the features they wanted us to see, and wouldn't be baited into going further. The overall impression of the space and staff was extreme design consciousness, a bit of opulence, and intense discipline.

Very un-Palm-like. More like boutique Apple without the rock star CEO.


Think Similar

The theme continued in the product. The Pre does not look like the Treo or any previous Palm product. If anything, it looks like an iPhone with some of its limitations fixed. The design of the hardware, graphics, the fonts, the way things move on screen, and the touchscreen gestures are all elegant, and reminded me intensely of the iPhone. You can even do a pinch gesture to shrink and expand things, which I thought was patented by Apple (this shows why I'm not a lawyer).

Unlike an iPhone, you can run multiple applications at the same time and switch between them. There's a thumb keyboard built in. The battery can be replaced. The APIs are supposedly based on web standards, so many people should be able to program the Pre without learning a new OS. Palm says it will have a software store built in, but the app approval process won't be as restrictive as Apple's. Palm will also apparently allow companies to port other platforms, like Adobe Flash, to the Pre, which addresses another iPhone drawback. (There's a comparison table between the iPhone and Pre here, but it focuses mostly on hardware specs.)

In contrast to all the iPhone references, it's very hard to spot any Palm legacy in the Pre (other than the company logo). The calendar still compresses unused hours, which was one of my favorite Palm features. But literally that was the main similarity that I noticed.

The device won't run current Palm OS apps, although I think Palm is open to someone porting a Palm OS emulator to the device if they want to. But I don't know how you'd operate those apps without a stylus. The browser is based on Webkit, so no more Blazer (yay).

The design of the interface looks nothing like Palm OS. Palm's old design ethic was all about sacrificing beauty in order to produce maximum utility. The result was often extremely efficient but plain (okay, ugly). The new Palm treats aesthetics like Apple does -- the device has to be useful, playful, and beautiful. That's incredibly hard to design, but apparently Palm has imported enough Apple talent to pull it off (or at least to make the demos look good).


Will Palm survive?

Prior to CES, it was fashionable for a lot of people online to predict Palm's imminent demise. That was a misreading of how the world works -- we technology insiders lose interest in a brand long before the public does. Palm still has a strong name, and it will get a good hearing in the market.

So the real question is, is the Pre good enough to make Palm profitable? I think it's too early to answer.

For one thing, we can't touch the product yet. The canned demos were incomplete -- I didn't see the dialer or the software store, for example, and I don't know details of how the product will sync. The SDK hasn't been released, so we don't know what it will be like to create apps for the device.

But my biggest concern is about the strategy, not the product. I'm not sure who the customer is for the Pre. Dr. Rob Enderle took time off from diagnosing Steve Jobs' medical condition (link) to tell a San Jose radio station that the Pre is a better e-mail device than the iPhone and a better consumer device than a Blackberry. Which is probably true, but misses the point -- it's probably a worse entertainment device than the iPhone (because it doesn't have iTunes) and probably a worse e-mail device than RIM (because it doesn't have RIM's server infrastructure). So who exactly is it best for?

Mobile devices that sell well usually have a well-defined market of people who look at them and say, "that one's perfect for me." The Pre is intensely elegant, which intrigues aficionados like me, but there aren't enough of us to make a lasting market. Beyond that, it's apparently perfect for people who want a compromise between a Blackberry and an iPhone, but don't need the best of either. Who are those people? And are there enough of them to make a business for Palm? I honestly don't know.

I guess the old Palm installed base might be a first source of customers, but many of them have moved on. Although there's a lot of enthusiasm on the Palm discussion forums (for a wonderfully detailed article, check here), longtime Palm users don't appear to have a lot of compelling ties holding them to the new device. Their old apps won't work, and they'll have to learn a new interface. Usually when a company makes a transition like this without backward compatibility, the user base reads it as an invitation to consider alternatives. Palm cannot take them for granted -- and even if it could, they alone are not enough to sustain the company.


What it means for the industry

Regardless of whether Palm survives, I think the Pre does some important things to the industry. It's the first smartphone that matches the iPhone on overall UI aesthetics, and it fixes many of the drawbacks of the iPhone. Other smartphone companies will be under pressure to match the Pre's features. Mobile companies like Samsung and Motorola, which lack software expertise, look increasingly vulnerable to gradual share erosion.

I'm very hopeful about the application development model for the Pre. By basing its development model on web standards, Palm apparently will empower the world's vast base of web app developers to quickly create Pre applications. If Palm implements the APIs right, that is a very smart move. It aligns Palm with the forces of the web, and might even make Pre the preferred mobile development platform of the web app community.

I don't know if that alone can make the Pre a success -- mobile devices usually build a base first with a particular function and then branch into apps. But it gives Palm a much better shot than it would have had if it tried to create yet another proprietary platform. The brass ring in the mobile app world is getting the attention of the web app community, and Palm now has a shot at it.

Google, are you listening?


What to do if you're a user

Wait.

We'll learn tons more about the Pre as it gets closer to shipping. Apple's undoubtedly working on new iPhone products (I'm betting on a smaller device, like a Nano version of the iPhone), RIM's getting the Storm debugged, Nokia is finishing the N97, and there are rumored to be more Android devices coming.* If you're thinking about getting a smartphone, you're going to have a great selection later this year. Hold out until you understand more about your choices.

__________

*There are probably some more Windows Mobile products coming too, but does anyone care any more?

Welcome to the real world

The most memorable quote I heard at this year's Consumer Electronics Show came from a business contact who works with a lot of different consumer companies. "No one knows what's going to happen," he said. "No one has ever been through an economic situation like this, and they honestly don't know what to do. They just don't know what's going to happen."

The reality is that we never know what's going to happen, but most of the time we kid ourselves about it. We think we see patterns and we expect events to follow those patterns, which they always do -- except when they don't.

Recent economic events have rubbed our noses in the basic unpredictability of the world. But don't let that paralyze you with fear. The situation has always been that way; you just had some comforting illusions before. Make the best guesses you can, stay nimble, and when the weather is nice be sure to enjoy the sunshine.

Will the mobile phone really eat the PC?

It's long been an article of faith for many mobile enthusiasts that mobile phones are going to become the dominant means by which the human race deals with the internet. Lately that idea has been echoed by more mainstream voices:

"There are over one billion people in emerging markets who will never access the Internet using a PC."
-- Adobe CEO Shantanu Narayen, on stage at CTIA, September 2008

"Most new internet users will be in developing countries and will use mobile phones."
--The Economist, September 2008 (link)

"Mobile phones are the future of computing, and they are ideally suited for accessing Web services....(The mobile phone will be:)
-the device most likely to subsume the PC's computing and informational dominance
-the most functional and accessible device for conducting Web search
-the natural gateway to Web 2.0 platform applications and services"
--E-Week, October 2008 (link)

I have a lot of respect for those sources, but in their enthusiasm for mobile technology, I think they have made two big mistakes:

--They've assumed the internet is a thing, and
--They have forgotten about Moore's law.


The internet is not a thing

It's a collection of standards for data transport, storage, presentation and so on. People do an incredible range of tasks that take advantage of the internet, some of them well suited to a mobile phone and some of them not. Creating documents and graphics that you want to share online, browsing content, and writing online comments all are moderately to enormously easier when you're using a PC with a keyboard, mouse, and larger screen. They also benefit from having large amounts of local storage, which you don't have on a mobile phone.

The idea that people in the developing world won't want or need the benefits of a larger screen and keyboard is patronizing. It assumes that they'll be content to be second class citizens for many Internet services permanently.

If they can't afford a PC, of course they'll do what they can with a mobile phone. But that brings us to the second assumption -- who says they'll never be able to afford a PC?


Moore's Law lives

As I've written in the past, some people in Silicon Valley worry that we're starting to run up against physical limits on the growth of computing performance (link). Although that may or may not turn out to be an issue at the high end of the market, at the low end no such barrier exists.

I'm told by friends on the manufacturing side of things that the production cost of a fully-equipped ultra-mobile PC or netbook (what we used to call a mini-notebook) is now around $200. The street price for basic models is $282-299, a drop of about 17% in the last four months (see here, and here). Many of the key components in UMPCs, such as the screens and optical drives, are also used in DVD players, which means they're being manufactured in large volumes, driving rapid price erosion.

UMPCs aren't perfect -- the keyboards are very cramped, and the screens display text so small that they can be uncomfortable to read. But they are far, far better than a smartphone for many computing and internet tasks.

I am not saying that PCs will become affordable for the world's poorest people anytime soon. But let Moore's Law continue to chew on the UMPC, and I think a PC will soon be within the reach of a working-class family in much of the developing world.


The most likely outcome is that most people who can afford a mobile phone and service plan will also be able to afford a small PC if they want one. My guess is that they'll use both devices in the same way you and I do -- the phone will be better for some tasks, the PC for others. The idea of using one to take over for the other will seem silly, kind of like using a hair dryer to cook dinner.

Someday in the distant future, of course, we'll have smartphones with flexible screens and fold-out keyboards that can fulfill all of the functions of a PC. At that point, the line between your PC and your phone will blur, and you'll be able to say that your phone has taken over your PC (you'll also be able to say that your PC has taken over your phone). But after watching the lethargic pace of change in mobiles over the last ten years, I think we've still got a long wait for the merger of phone and PC. Besides, integrating more features generally raises prices, so the merger of phone and PC will happen first at the high end of the smartphone market. PCs will be dirt cheap long before they merge physically with phones.