Augmented life
How Apple's Vision Pro and spatial computing can transform everyday experiences
Welcome back to the Brand Next newsletter. As a reminder, we are here for 3 reasons.
Educate people and brands about how emerging technologies can benefit businesses, marketers, people, communities, and the world.
Highlight use cases likely to have a positive impact while also driving scale and adoption.
Spotlight the innovators who are leading and building in emerging spaces.
If you are reading this and haven’t subscribed, please do. It’s a great place to keep up with the changes that are impacting us today, and which will impact us tomorrow. 🚀
You can also subscribe to my Podcast on Spotify, Apple Podcasts, or YouTube to hear interviews with the builders, innovators, and marketing pioneers shaping what’s new and what’s Brand Next.
It been about six weeks since Apple’s Vision Pro launch. We’ve now had time for the impact of the launch to sync in and overall, it’s been positive. Product return rates are roughly the same as for the iPhone so that is a positive sign, but more important for it’s success is the fact that it has reasonably high sales conversion rates after people demo the product (I have read numbers between 10% and 15%). And while there are some negative reviews, I have seen a lot of the naysayers who poopoo’ed VR, AR, XR and the Metaverse getting excited about where “spatial computing” will take us showing just how quickly the tides can turn.
These aren’t the naysayers, but let’s start with an overview of what some of Tech’s biggest reviewers said:
Casey Neistat says “Vision Pro isn't just great, it's the single greatest piece of tech I’ve ever used.” So he is clearly a fan.
Marquest Brownlee talks about how quickly you get used to having windows and information all around you - “I also had a moment where I was using the Vision Pro for a while and I had my Mac and some other monitors around me, and then I took it off and then I went and did something, and then I came back and before I put the headset back on I looked up at the wall to where I thought a window was going to be.”
Kara Swisher love the experience, and says that beyond the demo unit she got from Apple, she will buy one.
On the whole, people are still raving about this as an innovative product that pushes the category forward and creates a much more complete experience than has been possible so far. But isn’t that what we would all come to expect from Apple. So no real surprises there.
Beyond the hype that surrounds any new Apple product release, the question to ask is what is the use case that has everyone excited? What is the use case that would make this category of product a must have for everyone?
Is it for movies and entertainment? Despite Netflix and YouTube not having native apps (yet) for the Vision Pro, this is an amazing product for immersing yourself in a movie. Is it a replacement my big screen TV? I’d say NO because I enjoy watching TV with my kids, and wouldn’t want a device to come between us. That said, this is a much better way to watch a movie on a plane.
Is it for gaming? The potential here is huge. My son just got a PlayStation and I am playing The Last of Us and can absolutely see how this would make that game better not only by making the experience more immersive, but also by simplifying the controls to hand movements and gestures instead of a joy stick.
Is it for video calls and connections? Apple is getting a lot of praise for the new Facetime experience. Not only can you see multiple people, but spatial sound provides a sense of where people are located within your virtual perspective.
Is it about visual passthrough for AR type experiences? There is where I see a huge number of opportunities ranging from supply chain, to commerce, to product development.
But these are just the tip of the iceberg in terms of what devices such as this will be used for, and yet what has me most excited is something different. What has me excited is the idea of Persistent Presence. This is the idea that people can be connected to the internet in a visual way that doesn’t require them to isolate themselves from what is happening in the world around them.
Persistent Presence might not sound new, after all, we are all constantly connected to the internet through our phones today, but this hardly qualifies. Instead, phones isolate people and remove them from the physical world. In the version we have today, people go to dinner together but don’t speak as they just stare at their phones, people bump into stationary objects as they walk down the street because they are not paying attention, or worse, people crash their cars as they to balance online + driving.
Don’t get me wrong, the Vision Pro does not solve these problems … yet, but the product is an indicator of future direction and potential. It’s a direction that is being shaped buy a new class of devices that together with Meta’s Rayban glasses, the Humane AI pin, and to a limited degree the Rabbit R1 allow people to operate at the intersection of the digital and the physical world’s simultaneously.
To realize the full potential of these devices, we need innovations in the experience design that are specific to this visual and interactive medium. Today most of the apps on the Vision Pro are built on traditional interaction models that rely on Windows and Apps with the main difference being that they are free floating in space. And while some of the gesture control hints at what will be possible, it should be noted that it took multiple generations of iPhone and iPad development before we got services that felt unique to those devices.
It will take time before we can capture the real potential of Persistent Presense, but for now, here are four ideas that play on the idea in different ways. These are just some first thoughts and they range from the truly silly to the more serious.
IDEA 1: Liar Liar, Hair on Fire
INSIGHT: Computer vision can detect 88% of lies a person tells by analyzing eye and facial movements.
PROBLEMS: Lies are an ever-present problem, but here are a few fun scenarios.
As kids grow up they often start to rebel and parents don’t always get the true story or the whole story.
Roughly 40% of people under the age of 30, and 30% of people overall use dating apps. Can you trust everything they tell you about themselves?
When negotiating a business deal, is the counterparty accurately representing their business? (this problem might actually be a bit too serious for this solution, but let’s go with it)
IDEA
Introducing Liar Liar, Hair on Fire.
Play’s on the expression “Liar Liar Pants on Fire”, this idea uses realtime computer vision and AI to detect when people are lying and then playfully lights their hair on fire so you catch them in the act. So when my daughter comes home two hours late, and proceeds to give all kinds of semi-plausible excuses, well, I know right away if she is telling the truth. Liar Liar, Hair on Fire.
Or, given that this expression is English, we could make this concept more inclusive of language differences by adding a Pinnochio setting that makes a persons nose grow longer when they are suspected to be lying.
Imagine using Tinder with this service to help you screen people and discover what is an exaggeration. It would definitely make online dating funnier.
And whatever your political beliefs, I can promise that watching political debates during an election cycle with Pinnochio turned ON would be both fun and funny.
The challenge with this idea for now is that while Computer Vision can currently detect 88% of lies, it does so with only 60% precision. So it’s not perfect yet, but with more data, and more training around facial expressions as people lie, I think this could soon be realistic.
But let’s look at another idea
IDEA 2: The Parent Coach
INSIGHT: All parents want to support their kids passions, but sometimes their kids are interested in things the parent knows nothing about.
PROBLEM: I have never been the best athlete, but my son is obsessed with soccer (football for most of the world). I love to play with him, but I am not great at coaching him beyond the basics.
IDEA
Introducing the Parent Coach. It’s an app that allows you to play sports with your kids, or just watch them, but as you do, the app does AI analysis of what they should do to get better.
Advice can be tailored for age skill set and age group.
Advice is provided by kids sports hero’s so even though you are giving the advice, the AI training and insights come straight from the best - from Messi, to Jordan, to Tiger, to Serena, it doesn’t matter what the sport is. With this, as a parent you can up your parental coaching game by getting insights advice from the best.
BBH worked with Nike on a project similar to this in the Philippines a few years ago. It required multiple camera’s and specially painted courts. I’ve included the video case study for Hypercourt just below, but it’s easy to see how that coaching capability could be further democratized and put into the hands of every parent. I know my son, and may other kids would love it.
IDEA 3: Keep Memories Alive
Now a more serious topic. What about an App to help Alzheimers Patients. Quite a few members of my family have had Alzheimers so this is a subject that’s important to me.
INSIGHT: In the early stages, the impacts of Alzheimers are more pronounced and noticeable through short-term memory loss. This can result in confusion, and while it’s not a direct cause of any anger or aggression that some Alzheimers patients experience, it can contribute.
PROBLEM: Digital technology is obviously not a solution to Alzheimers. That said, there are some problems where it could potentially help.
Families of patients often ask them question like “what did you do today?” as a way to trying to exercise their memory. This focus on short-term can cause more frustration.
Older people are often filled with stories and family history that no one else knows. It gets lost and forgotten as they pass.
IDEA
Create a memory palace app. This would be content that could initially be created and curated by the patient themselves (pre-disease, or at the onset / diagnosis)while a patient still has their longer-term memories, they could upload them into the Vision Pro app where Generative AI could create immersive environments. The patient could narrate each environment, and memory to add context to the photos allowing them to revisit their memories and hear their own voice talking to them and reminding them of what they did and who they are.
While this would not halt the progression of the disease, memory therapies have in some cases been seen to help calm and provide comfort to patients. It can also help the caregivers and people around the patient.
This idea of bringing photos to life and making them more immersive and real is something that people have already highlighted as a key use case for the Vision Pro. This idea is simply to add further functionality to better support the specific needs of Alzheimers patients and caregivers.
Idea 4: How-to-fix-it
This is just very practical. Maybe it’s a boring way to end this list of ideas, but it’s real.
INSIGHT: 53% of people watch an informational or instructional video two or more times per week. These can relate to fixing things around the house or any sort of instructional help.
PROBLEM: Watching a video doesn’t mean you know how to fix something. It gives you the confidence to start, but often people wind up just watching one part over and over as they try to figure it out.
IDEA
Very simple. How-to-fix-it Augmented Reality. Imagine going to change your car tire, and having a virtual overlay that tells you where the Jack is, and then shows you how to jack up the car so you do it correctly and safely the first time. Or a virtual overlay that shows you how to change a fuse, or even how to julienne vegetables.
There are tens of thousands, if not hundreds of thousands of fix it videos on the internet. They exist in all languages. The last one I used was a video in how to change the battery in my car key and I wound up snaps a small plastic piece because the instructions weren’t clear. While not exciting, I wish I could just use Vision Pro for fix-it content going forward.
The opportunity for people and brands
The above are four thoughts are just examples that were quickly put together to demonstrate the diversity of what will be possible as new devices and form factors make Persistent Presence a reality and allow people to operate seamlessly at the intersection of the Digital and Physical world. This being the first generation of these devices, it isn’t all possible yet and nor should anyone expect that it would be. Instead, what’s important is to consider where this new paradigm is going and, as the Great One Wayne Gretzky says, “skate to where the puck is going to be”.
Three principles stand out be seen on where this will take us:
Add value through real-time interactive AI - Both “Liar, Liar” and “The Parent Coach” are illustrations of how real-time AI that connects with people through both sensory experience and valuable information can be both playful and helpful. As AI becomes an increasingly dominant part of communications, this is a window into how it will also become part of culture and everyday life.
Build empathy and trust through emotion - Again “The Parent Coach”, but also “Keep Memories Alive” show off how this technology can be used in ways that are deeply personal and emotional. These are tools designed to demonstrate empathy with an audience, and in many ways to allow others to experience the world through their eyes.
Be helpful everyday - This one hardly needs to be said because it should be the golden rule of service design. But what’s clear through the “How-to-fix-it” idea is just how simple it can be to add an additional layer of visual / audio information which reflects a persons exact need or context of use. This means no more trying to re-interpret instructions for your need, everything should be personalized to the person and the moment in time.
So whether you are a fan of the Vision Pro or not, what seems clear to me is that this device, along with others like the Meta Ray-Bans represent the start of a new paradigm in interaction design and that Persistent Presence will reshape how we work, learn, and connect with one another, all while opening up new avenues for creativity, productivity, personal fulfillment, and brand engagement.
If you enjoy the Newsletter and want to get more, please subscribe to make sure you never miss an issue.
If you already subscribe but want to do more to help grow our readership, you can share the article with your friends. One share from you makes a world of difference to me.
If you want more content from Brand N3xt, you can also subscribe to the Podcast or follow and connect with me on social channels through the links below:
Subscribe to the Podcast on Spotify, Apple Podcasts, or even on YouTube
Follow me on X at @justinkpeyton
Follow me on LinkedIn
Nothing in this newsletter is intended as financial advice. This newsletter is for educational and entertainment purposes.