Apple is working on multiple methods to use a keyboard with a pair of augmented reality glasses or other head-mounted device, including virtual keyboards projected onto surfaces.
The Cupertino tech giant is thought to be developing at least one head-mounted device. It’s likely to be an Apple AR headset or AR glasses like the so-called “Apple Glass,” but there are also indications that a virtual reality device is also in the works.
In a patent application published Oct. 15 and titled “Keyboard Operation with Head-mounted Device,” Apple details several different ways that a user can use a keyboard with a pair of AR glasses or a similar product.
The patent gives some background on what the head-mounted device would entail. It could, for example, be a headset that sports lenses in front of the eyes and is equipped with several cameras. The display lenses, which could vision-correcting optical properties, could exclude a physical environment or overlay information on top of it.
When needing to type text, users could use this head-mounted device with either an actual physical keyboard or a “representation thereof through the display.” Text that is typed on the keyboard could appear in the user’s field of vision through the display lenses.
Most interestingly, however, are the alternative methods for text input that Apple explores here. They could include a virtual keyboard projected onto a touch-sensitive surface, or a keyboard projected onto any flat surface like a desk or tabletop.
Along with a tracking device, an AR headset could allow for use of any surface as a virtual keyboard. Credit: Apple
“A head-mounted device can provide a virtual keyboard based on the arrangement of the user’s hands with respect to a surface,” the patent reads, adding that the device could detect where the keyboard should be by the position of the hands on the surface.
In any of these cases, Apple notes that a tracking device placed in close proximity to a user’s hands could submit text to the AR device by using hand-tracking systems, inertial measurement units, or force detection sensors.
“It will be understood that combinations of the above can be provided such that operation of the keyboard is detected based on multiple detections,” the patent reads.
Furthermore, these tracking sensors could allow a keyboard to function without needing to rely on signals. In other words, the typing would be detected by the headset, which could allow for keyboards with virtually no electronics within them, or virtual keyboards projected onto surfaces.
For projected keyboards, Apple notes that a user’s hands could be made transparent to facilitate viewing the actual keys when typing. When it comes to editing text, hand gestures or other inputs could be used to select text and make changes.
An illustration of the hand transparency effect, which could allow users to better see the keyboard. Credit: Apple
Additionally, the patent suggests that users could also use an Apple Pencil or another stylus to draw or annotate on flat surfaces.
Apple files numerous patent applications weekly, and there’s no guarantee that the technology described in them will ever make it to market. Similarly, patent applications don’t give any indication when such features could arrive.