Where Apple’s reinvention of the keyboard may go next

Keyboards are important. While speech-to-text technology has come a long way, I suspect there will always be value in writing one letter after another with your hands. Apple clearly thinks to, too, because in recent weeks there have been reports about several new Apple patents regarding keyboard technology. From those patents to the controversial innovations that drove the latest generation of MacBook keyboards, Apple’s continues to push the boundaries of text input.

So what do Apple’s current technology directions suggest about where the company might be innovating in the future when it comes to keyboards?

A reliable tool

I’ll admit that as a writer, I’m especially opinionated when it comes to keyboards. As a writer friend of mine told me a very long time ago, keyboards are to us what guitars are to rock musicians—they’re the tool of our trade. But even if you’re inputting data into a spreadsheet or writing an email to a friend, most people need keyboards. (Even smartphones and tablets will slide up a software keyboard when you need to tap something out.)

Gordon Mah Ung

When it comes to traditional hardware keyboards, reliability is a key factor, as it must be for any hardware. While Apple has received a lot of criticism for the feel of its new-generation MacBook keyboards, the more serious problem has been a general sense that the design is unreliable because any stray crumb or fleck of dust can cause keys to stop functioning.

It’s interesting, then, that one of the recently disclosed Apple keyboard patents is specifically about preventing keyboards from getting jammed with crumbs or soda or actual jam. Most Apple patents never make their way into real products, but the idea that Apple might be considering what it would take to make a truly junk-proof keyboard—maybe even one that uses the typing motion to blow debris out of the keyboard!—is a sign that the company is still trying to build a better, and more reliable, keyboard.

In its quest to make all its devices thinner, Apple has pushed its keyboard designs thinner as well. This has led to reduced “travel,” which is keyboard-speak for the amount of space a key can depress. But while the travel on the MacBook and MacBook Pro keyboards is quite small, Apple has tried to compensate with added audio and tactile effects. The new keyboards provide a crunchy feedback that can be quite satisfying, hiding the reduction in physical movement. I wouldn’t be surprised if there’s more innovation on this front in the future—as well as improvements to reliability.

But what about touchscreens?

When Steve Jobs introduced the iPhone, he spent a long time explaining why Apple had chosen to replace the physical hardware keyboard common on phones at the time with a touchscreen-based keyboard. When your device is all touchscreen, the space that would’ve been used for keys can be used for keys, or more space for apps, or even for alternate forms of input like number pads or (as we commonly do today) emoji input spaces.

Would Apple consider one day replacing the physical keyboard with a touchscreen? I realize that the mere suggestion of this will send a chill down the spine of anyone who loves their physical keyboard. But when you look at another recent patent filing, it’s clear that Apple has at least investigated the possibility of turning the traditional laptop into a two-screen device.

Adam Patrick Murray

There are plenty of reasons why this is a terrible idea, at first glance. Consider the Touch Bar, where Apple has dipped its toe into the waters by replacing a row of traditional keys with a touchscreen. The Touch Bar has all the advantages of the iPhone screen or any other touchscreen, in that it can be programmed to display different information in different contexts.

The Touch Bar falls short because you have to look down in order to see where you’re meant to be touching, rather than being able to orient by feel as you can with physical keys. (This is the same reason that all-touchscreen interfaces for cars and TV remotes are also failures; in many contexts you need to operate by touch, not sight, because you need to keep your eyes on the road or the screen.) And if you even lightly touch to orient your fingers, you’re touching a touchscreen… which means you’ve just typed something accidentally.

But all is not lost. If you look at a few other areas of Apple tech, you can start to see that Apple may be moving slowly toward building a much better touchscreen keyboard—maybe even one that dinosaurs like me would be happy to use.

Let’s start with the issue of not being able to navigate by feel. How do you provide touch-based feedback on a smooth glass touchscreen surface? The answer lies in Apple’s Taptic Engine, which is embedded in iPhones and trackpads. In compatible apps, my fingers get direct feedback (in the form of small taps and vibrations) as they move the cursor around on the screen. Taptic feedback could certainly provide indications about finger positioning.

But still, once your fingers are on the touchscreen, they’re already providing input, right? Not necessarily. Consider 3D Touch on the iPhone and Force Touch on the Mac; maybe a future touch-based input device will couple haptics with pressure sensitivity to guard against accidental presses. Imagine running your finger over a touchscreen, getting slight buzzes and taps to direct your hands to the proper position, then pressing slightly harder to actually provide input. You know, like how physical keyboards work today.

Apple

Finally, let’s take this one step further: Imagine using Apple’s advanced image-recognition hardware, like those found in the iPhone X’s TrueDepth sensors, FaceTime cameras, and the A11 Bionic processor, to sense the exact position of your hands over the touchscreen input area on a future laptop. What if you didn’t need to look down or use feel to orient your hands? What if wherever you placed your hands on the touch area was automatically the correct area, with the keyboard shifted to match your exact hand position?

I know it sounds a bit like science fiction, but today you can buy an iPhone that maps your face and unlocks itself when it recognizes you.

There are a lot of clear benefits to replacing physical keyboards with touchscreen input areas, and they’re largely the same ones that Steve Jobs described back in 2007. When you’re not typing, that area can be used for other gestures, to display alternate information, or even become a drawing surface for pen input. To be sure, if Apple replaced the MacBook keyboard today with an overgrown Touch Bar, the result would be disastrous. But with the clever application of new technology—something Apple’s awfully good at—a future evolution of the Touch Bar really could do the job that the physical keyboard does today.