Weekend project: Zap Your Coworkers’ Minds with Multi-Pointer X

975

Care to rethink your desktop user experience? It may be simpler than you think. Chances are you own more than one pointing device, but for years you’ve never been able to take full advantage of that hardware, because whenever you plugged in a second hardware mouse, it simply shared control of the same cursor. But with X, it doesn’t have to: you can have any many distinct, independent cursors on screen as you have devices to control them. So grab a spare from the parts box in the closet, and get ready for the ol’ one-two punch.

What, you don’t have a spare mouse? There a good chance you do have a second pointing device, though — even if it’s a mouse and the built-in trackpad on your laptop. But this process will work for any pointing hardware recognized by X, including a trackball and a mouse, or even a mouse and a pen tablet. To X, they are all the same. You will need to ensure that you are running the X.org X server 1.7 or later to use the feature in questions, Multi-Pointer X (MPX). You almost certainly already are — X.org 1.7 was finalized in 2009, and all mainstream distributions follow X.org closely. Bring up your distribution’s package manager and install the libxi and libxcursor development packages as well; they will come in handy later.

What is MPX?

Demo of Vinput demo appSo why don’t we use MPX all the time? For starters, most of the time one cursor on screen is all that we need, because there are very few applications that benefit from multiple pointers. But there are several. The simplest use case is for multi-head setups, where two or more separate login sessions run on the same machine, using different video outputs and separate keyboard/mouse combinations. You often see this type of configuration in classrooms and computer labs.

But the more unusual and intriguing alternative is running multiple pointers in a single session, which can simplify tasks that involve lots of manipulating on-screen objects. If you paint with a pressure-sensitive tablet using Krita or MyPaint, for example, a second cursor might allow you to keep your tablet pen on the canvas and alter paint settings or brush dynamics without hopping back-and-forth constantly. The same goes for other creative jobs like animation and 3D modeling: the more control, the easier it gets. Plus, let’s be frank, there’s always the wow factor — experimenting and showing off what Linux can do that lesser OSes cannot.

The basics

MPX is implemented in the XInput2 extension, which handles keyboards and pointing devices. The package is a core requirement for all desktop systems, but most users don’t know that it comes with a handy command-line tool to inspect and alter the X session’s input settings. The tool is named xinput, and if you run xinput list, it will print out a nested list of all of your system’s active input devices. Mine looks like this:

 

⎡ Virtual core pointer          	id=2	[master pointer (3)]
⎜  ↳ Virtual core XTEST pointer       	id=4	[slave pointer (2)]
⎜  ↳ Microsoft Microsoft Trackball Explorer®	id=9	[slave pointer (2)]
⎣ Virtual core keyboard          	id=3	[master keyboard (2)]
  ↳ Virtual core XTEST keyboard       	id=5	[slave keyboard (3)]
  ↳ Power Button              	id=6	[slave keyboard (3)]
  ↳ Power Button              	id=7	[slave keyboard (3)]
  ↳ Mitsumi Electric Goldtouch USB Keyboard 	id=8	[slave keyboard (3)]

 

As you can see, I have one “virtual” core pointer and a corresponding “virtual” core keyboard, both of which are “master” devices to which the real hardware (the trackball and USB keyboard) are attached as slaves. Normally, whenever you attach a new input device, X attaches it as another slave to the existing (and only) master, so that all of your mice control the same pointer. In order to use them separately, we need to create a second virtual pointer device, then attach our new hardware and assign it as a slave to the second virtual pointer.

From a console, run xinput create-master Secondary to create another virtual device. The command as written will name it “Secondary” but you can choose any name you wish. Now plug in your second pointing device (unless it is already plugged in, of course), and run xinput list again. I plugged in my Wacom tablet, which added three lines to the list:

 

⎜  ↳ Wacom Graphire3 6x8 eraser       	id=16	[slave pointer (2)]
⎜  ↳ Wacom Graphire3 6x8 cursor       	id=17	[slave pointer (2)]
⎜  ↳ Wacom Graphire3 6x8 stylus       	id=18	[slave pointer (2)]

 

The new virtual device also shows up, as a matched pointer/keyboard pair:

⎡ Secondary pointer              	id=10	[master pointer (11)]
⎜  ↳ Secondary XTEST pointer        	id=12	[slave pointer (10)]
⎣ Secondary keyboard              	id=11	[master keyboard (10)]
  ↳ Secondary XTEST keyboard         	id=15	[slave keyboard (11)]

 

That’s because so far, X does not know whether we’re interested in the keyboard or the pointer. You’ll also note that every entry has a unique id, including the new hardware and the new virtual devices. The “cursor” entry for the Wacom tablet is the only one of the three we care about, and its id is 17. Running xinput reattach 17 "Secondary pointer" assigns it as a slave to the Secondary virtual pointer device, and voila — a second cursor appears on screen immediately.

You can start to use both pointers immediately (provided that your hand-eye coordination is up to the task). Grab two icons on the desktop at once, rearrange your windows with both hands, click on menus in two running applications at once.

Cursors, Foiled Again

What you'll soon find out, though, is that life on the bleeding edge comes with some complications. First of all, your mouse cursors look exactly the same, which can be confusing. Secondly, although GTK+3 itself is fully aware of XInput2 and can cope with MPX, individual applications may not be. That means you can uncover a lot of bugs simply by using both pointers at once.

You can tackle the indistinguishable-cursors issue by assigning a different X cursor theme to the second pointer. The normal desktop environments don't support this in their GUI preferences tools, but developer Antonio Ospite wrote a quick utility called xicursorset that fills in the gap. Compiling it requires the libxi and libxcursor development packages mentioned earlier, but it does not require any heavy dependencies, nor do you even need to install it as root — it runs quite happily in your home directory.

To assign a theme to your new cursor, simply run ./xicursorset 10 left_ptr some-cursor-theme-name. In this call, 10 is the id of our "Secondary pointer" entry from above, and left_ptr is the default cursor shape (X changes the cursor to a text-insertion point, resize-tool, and "wait" symbol, among other options, as needed). X.org cursor theme packages are provided by every Linux distribution; the "DMZ" theme is GNOME's default and uses a white arrow.

The simplest option might be to use the DMZ-Black variant: ./xicursorset 10 left_ptr DMZ-Black, but there are far more, including multi-color options. I personally grabbed the DMZ-Red theme from gnome-look.org's X cursor section, to make the auxiliary cursor stand out more.

One thing you will notice about these instructions is that they do not persist between reboots or login sessions. Currently there is no way to configure X.org to remember your wild and crazy cursor setups. I asked MPX author Peter Hutterer what the reasoning was, and he said it simply hadn't been asked for yet. Because MPX usage is limited to a few applications and usage scenarios, most people only use it in a dynamic sense — plugging in a second device when they need it for a particular task.

On the other hand, one of Ospite's readers (an MPX user named Vicente) wrote a small Bash script to automate the process; if you use MPX a lot, it could save you some keystrokes.

Multi-Tasking

But, as Hutterer said, apps with explicit support for MPX are pretty limited at this stage. There is a multi-cursor painting application included with Vinput, and the popular multiplayer game World Of Goo can use MPX to enable simultaneous play. There was a dedicated standalone painting application called mpx-paint hosted at GitHub, but the developer's account appears to have shut down.

Mainstream applications have been slower to pick up on MPX support. Inkscape is a likely candidate, where manipulating two or more control points at once would open up new possibilities. Developer Jon Cruz says it is "on the short list" alongside Inkscape's growing support for extended input devices. Whenever the application completes its port to GTK+3, the MPX support will follow.

Blender has also discussed MPX, but so far the project's main focus has been on six-axis 3D controllers (which are understandably a higher priority). However, for broader adoption we may have to wait. Hutterer himself worked on bringing MPX support to the Compiz window manager, but said that the real sticking point is tackling the "unsolved questions" about window management — which is something the desktop environment will need to tackle. Luckily, the latest X.org release includes multi-touch support — a related technology useful for gestures. As Linux on tablets continues to grow, gestures become an increasingly in-demand option. As applications are adapted to support multi-touch gestures, MPX support should grow as well.

The implications are interesting for a number of tasks, including the gaming and creative applications already discussed. When you throw in the possibility of multiple keyboards, the possibilities get even broader. After all, thanks to network-aware text editors like Gobby and Etherpad, we are getting more comfortable with multiple edit points in our document editors. Imagine what you could do with an IDE that allowed you to edit the code in one window while you interacted with the runtime simultaneously. In can be hard to grasp the implications — but you don't have to dream them all up; with MPX you can experiment today.