CellWriter: Open source handwriting recognition for Linux

Handwriting recognition, like its cousins speech recognition and optical character recognition, is a domain still dominated by proprietary products. Where there are Linux solutions, such as the one in Nokia’s Maemo Internet tablets, they are often closed source plugins protected by patent claims. Thus I was pleasantly surprised to find CellWriter, a small, straightforward handwriting recognition tool that integrates easily with modern Linux desktops.

CellWriter provides a small, grid-like window into which you write with normal pen strokes. Thus it works best with a pen interface, such as a Wacom tablet, but that isn’t strictly required.

The grid window hovers in always-on-top mode above your other applications, so that the app you are working in retains focus. When you satisfied with the text you have scrawled in CellWriter, you click the Enter button beside the grid, and your words are inserted into the active app just as if you had typed them on the keyboard. It’s brain-dead simple.

Basic writing

You can download .DEB packages or source code from the project’s Web site. The application requires GTK+, libgnome, and libxtst, all of which are standard issue on modern Linux distributions.

CellWriter works by learning the writing style of each individual user, so training is a must. If CellWriter doesn’t find a ~/.cellwriter file at run time, it will open in training mode so it can learn your character style. If at any time you experience a lot of trouble with a particular character, you can click the Train button and provide more samples — the app continues to learn from repeated experience.

Train mode sports a matrix of characters, each in its own cell. All you do is draw the appropriate character in its cell, and CellWriter commits it to memory. Trained and un-trained cells are marked in different colors so you can keep track of your work.

If you mess up on a character, you can reset it. As you write in normal text-entry mode, CellWriter converts your strokes to characters as you move the pen from one grid cell to the next. If it interprets incorrectly, you can right-click on the cell and choose the correct letter from a drop-down list of close matches. Whenever CellWriter in unsure of a letter, it highlights that cell in a different color to alert you, but doesn’t interrupt your writing.

Options

In CellWriter’s setup dialog, you can configure the size and color of the cells, and tweak some of the character recognition algorithm’s settings — such as its sensitivity to stroke direction.

Better still, you can tell CellWriter what alphabet you are using, courtesy of Unicode. You can enable and disable different Unicode blocks individually, and set the language direction to left-to-right or right-to-left. Satisfied with Basic Latin? Do nothing. Need Cyrillic, Thai, or International Phonetic Alphabet extensions? Just click the respective checkboxes.

The blocks you select are added to Train mode, but CellWriter only recognizes those characters on which it is trained. So if you only need a few particular Greek letters for an equation, you can just train the ones you need, and not worry about CellWriter sending an epsilon where you meant to write an E.

Project past, present, and future

CellWriter is the product of University of Minnesota undergraduate computer science student Michael Levin. Levin had previously authored OneStroke, a simpler character recognition utility, but wrote CellWriter essentially from the ground up rather than evolving the old codebase.

OneStroke was a hobby project, but Levin got the opportunity to spend considerably more time developing CellWriter through the university’s Undergraduate Research Opportunity Program (UROP). UROP provides grants to University of Minnesota students to undertake summer research on projects that they devise individually.

Levin’s interest in handwriting recognition stems from scratching his own personal “itch” with a Toshiba Portégé M200 tablet PC. “It was priced at nearly twice the cost of any other laptop, but I was mesmerized by the tablet interface. I had a naïve desire to do research in artificial intelligence and saw handwriting recognition as the most approachable problem. I ended up buying the Portégé solely because I wanted to work on the problem.”

The artificial intelligence community regards handwriting recognition as a “solved problem,” Levin says, so there is scarce ongoing research into new methods. “But after I’d been using the Windows [Tablet Input Panel] for several years and it continued to infuriate me by consistently refusing to recognize my “o” (drawn clockwise from the bottom), I knew there was room for improvement.”

Levin says he developed CellWriter’s algorithms himself, through personal trial and error. Basing the character matching on per-user training sidesteps an important problem: no two people share the same handwriting, so general-purpose recognition has to assume lowest-common-denominator penmanship.

Sticking to a grid-based, one-character-at-a-time approach simplifies the recognition task (i.e., you only need to consider one glyph at a time), but it also permits some sophisticated tricks. Unlike some handwriting recognition systems, CellWriter can understand multi-stroke characters, and it can recognize each character regardless of the order in which you make the strokes.

As a full-time student, Levin is unable to devote much time to CellWriter during the non-summer months, but he says he intends to keep working on it. “There is a great deal of localization support that still needs to be done. Currently, CellWriter is atrociously difficult for Asian language users. I was hoping to find a way to simplify Kanji training or generate it automatically somehow.”

The outlook for other projects

There is no magic bullet for handwriting recognition; it is a task even human beings struggle with. As an undergraduate math major I was always intrigued by it; working out proofs on paper forced us to adopt distinctive forms to demarcate similar-looking Latin, Greek, and specialty symbols. But even in your own handwriting, it can occasionally be difficult to distinguish between “w” and “É” (omega), or “∃” (there exists) and “∋” (contains).

The keyboard as an input device is not going anywhere, but we are seeing an explosion of touchscreen devices running Linux — from Nokia tablets to OpenMoko phones. We will need handwriting recognition soon, and non-free software dominates that space right now.

CellWriter is a young project, but it shows a lot of promise both in its character recognition abilities and, more importantly, in its ease of use.