Apple Patents Next-Gen Apple Pencil That Works on Any Surface
Apple has been granted a new US patent that reveals major innovations in its Apple Pencil line, taking the stylus far beyond the iPad. The next-gen Apple Pencil may soon be able to draw on almost any surface, or even in mid-air, without touching a screen.
Optical Sensors for Spatial Tracking
Titled “Input Device With Optical Sensors,” the patent outlines a stylus that can track movement, orientation, and position in 3D space using advanced optical sensors. Unlike current Apple Pencil models, which rely on capacitive touchscreen interaction, this new version would detect motion independently.
Two types of optical systems are described:
- Optical flow sensors: Similar to those used in optical mice, they analyze changes in light between frames to detect motion.
- Laser speckle flow sensors: These track laser light patterns reflected off a surface to determine movement, even up to 100mm (3.94 inches) above it.
Trackball Mechanism and Sensor Fusion
One intriguing implementation includes a trackball tip, which rolls across a surface. Rather than traditional mechanical encoders, Apple would use internal optical sensors to track the ball’s movement. This design could allow the Pencil to work on non-touch materials like wood or paper.
In addition, the patent suggests incorporating inertial measurement units, force sensors, and machine vision to support handwriting recognition, gesture input, and 3D drawing capabilities.
Use Cases Across Apple Ecosystem
Beyond drawing and writing, Apple sees broader potential for the Pencil. The stylus could be used as a spatial controller for system-level tasks such as:
- Navigating files
- Controlling audio
- Initiating phone calls
- Manipulating cursors on devices like the iPhone, MacBook Pro, and Apple Watch
Inspiration from Vision Pro and Logitech Muse
At WWDC 2025, Apple showcased the Logitech Muse, a stylus designed for spatial interaction with the Vision Pro headset. It lets users draw in mid-air, relying on precise motion tracking. Apple’s own patent echoes similar ambitions, hinting at internal development of native spatial input devices.
While a patent does not confirm immediate product plans, it’s a clear sign that Apple is actively exploring new directions for the Apple Pencil, and potentially redefining how we interact with devices in the era of spatial computing.