Apple-Newton Handwriting Recognition (original) (raw)

Despite the abysmal recognition accuracy in the first generation Newton, most Newton afficianados or people interested in handwriting recognition will tell you that the second generation, "Print Recognizer" in Newton OS 2.x was a vast improvement, offering fast and surprisingly accurate recognition. Unlike the first generation software, this second generation recognition engine was developed in-house at Apple, in the Advanced Technology Group (ATG), later (and briefly) renamed the Apple Research Laboratories (ARL). I served as Technical Lead for the project, and together with a core team of three Apple engineers and two contractors, plus a host of other contributors (most of whom are listed in the slides mentioned below), we managed to produce what many have called the first genuinely usable handwriting recognition system. The technical papers, articles, and slides below document a lot of the key technological hurdles that were overcome and the innovations that were made in order to make this possible.

The core recognition technology from the Newton has gained a new lease on life in the Jaguar release of Mac OS X (10.2). Together with a different team of engineers I have helped integrate handwriting recognition into Mac OS X in such a way that it just works with all existing apps; i.e., applications are not required to rev in order to support ink and the routine input of text by a pen and graphics tablet. This technology has been dubbed "Inkwell". (Partly it just seemed like a good name, plus I have a long-standing fondness for the Fleischer Brothers' animations, including their "Out of the Inkwell" series.) The Apple Computer page on Inkwell is here: http://www.apple.com/macosx/jaguar/inkwell.html

For a silly paragraph concocted entirely out of words in the original Newton's limited dictionaries (for benchmarking our recognizer against the old one), check out Test Drive1.

For detailed technical info, please refer to:

[Note: there was an extended abstract for the IWFHR5 workshop here for a while, but it was removed when we withdrew our paper from that conference.]

If you have problems printing the PostScript Type 2, Binary format (in which the MicroNeuro '96 paper is provided), email me about obtaining a Type 1, ASCII format.

I've now added PDF format versions of all but the Machines that Learn slides, so most of the papers can be read online if desired (or downloaded in the nice, compact PDF format). I may get around to puting a PDF version of the slides up, but if anyone wants that in a hurry, email me.

Though many people contributed to this effort, the core group consists of: