If you know me, you know my penchant for discussing, often to the point of oversharing, topics that intrigue me. They are in majority tech related. One of them is the future of spatial computing. Before we proceed, I want to clarify that this article is purely speculative, a product of my technophile imagination. I do not claim VR headsets/glasses to be “the next thing”, rather it might be a fleeting trend that could disappear in a near future.
The Problem: Information Lost in the Ether
We've all been there: You encounter a riveting post or a captivating image, and a few days, weeks or months later, when you want to retrieve it, it's like finding a needle in a haystack. Despite our technological advancements, it's baffling that there's no application that can effortlessly scan, vectorize, categorize, and retrieve any content you've previously encountered.
In addition, I always found the process of toggling between windows to input a prompt, retrieve results, and then paste these results back into the original context somewhat unnatural. Even in IDE, I find the integration too shallow (no tools can simply go through all your code and comment everything red for a simple run test, or transform a whole repository into a Turborepo/Monorepo, or attach your design library full documentation to produce accurate code, although I don't think it would be a problem for the LLM), yearned for a world where a language model would be always listening and ready to respond to my prompt, vocally, delivering what I needed instantly, right where I needed it.
The Dream: An Intuitive Future
One night, I found myself dreaming of a potential future of computers usage. My dream took the shape of Virtual Reality glasses connected to a laptop. Equipped with eye-tracking sensors and a microphone, these glasses would allow users to select and click using just their eyes or voice, eliminating the need for keyboards or mice. A bit like this:
The selected context can then serve as a context prompt to study/search/reshape or anything we want in a LLM.
The Inspiration: Notion AI
My dream was probably also influenced by Notion AI, a technology that is redefining the way we interact with our content, as detailed in my previous post. By enabling us to select text or images using our eyes, it removes the need for additional actions like mouse clicks. This shift towards a keyboard and mouse-less future also seem to be the direction taken by platforms like GitHub Audio, GitHub Copilot Workspace, and ChatGPT macOS.
OpenAI also seems to take that direction with their macOS Desktop app
In an ideal world, our computers would be able to read our thoughts and transcribe them as swiftly as they are conceived. Although we are still far from this reality*, eye-tracking technology, which allows your computer to understand your focus area, has made significant strides.
An application that caught my attention was the Apple Vision Pro, which allows you to type on your keyboard by merely looking at a key and touching your thumb with your index finger. My concrete test didn't leave any doubt on it.
The Obstacle: Precision and Size
I initially had reservations about the precision of eye-tracking as a mouse substitute. However, eye-tracking technology has proven its accuracy over time. A potential issue is the varying size of objects we might want to “select” as context, but thankfully, eye-gaze technology has been developed to accommodate this and, combined with AI, potentially has strong potential.
The remaining challenge is the need for verbal commands to specify the desired action, as it's currently not possible for a computer to directly read your thoughts and know exactly the action you want to achieve.
Last words
Things are getting even more interesting since my recent purchase, the Visor 4K has eye-tracking feature, Bluetooth microphone, and an opened SDK I’m excited to dig into on my free time.
In my dream, the actual prototype was using a Visor 4K (maybe edition 2), and because of the AI super assistant, the whole thing was actually called AdVisor 4K !
Anyway, I am really curious to see how things will evolve, are you too?
Cheers 🍻
