Is Your Next Reality Going to be Augmented?

John Lennon reportedly once said “Reality leaves a lot to the imagination…” and now we have the technology to do something about making our reality a lot more imaginative. Unless you have been living under a rock (and there is nothing wrong with that – I just haven’t found the right rock myself) there is a LOT going on these days in the virtual reality / augmented reality (VR / AR) markets. There is some confusion about the differences between virtual reality and augmented reality and which market is bigger and better. So first, let’s talk about the differences.

Augmented Reality (AR) is the overlaying or projecting of graphics content into our view of the “real” world. While the real world might simply be what we see through with our own eyes (i.e. wearing AR glasses) it could just as easily be the real world as seen by a camera (like your phone camera – ala Pokemon Go) or even previously recorded videos of the real world.

Virtual Reality (VR) is typically a completely computer generated world that you are an active participant in with little or no “real world” content. Ok – yeah – kind of like “The Matrix”. Generally speaking, the “killer” app for VR is video games – but there are a whole host of other uses as well from education, architecture, healthcare, etc. Eventually, the graphics content of VR will become so sophisticated that it might be hard to differentiate it from the real world – and then we might have to redefine the word “real”.

There is another concept called “mixed reality” out there – which, in my opinion, is just a significantly higher level of AR where extremely complex digital objects are inserted into your real world view. In the future, this higher level of digital object creation / insertion might become so sophisticated that you will possibly have a difficult time distinguishing it from actual reality.

I personally believe that AR is going to be a HUUGE market (surpassing VR by a pretty wide margin) and here’s why. AR is just so darn practicaland can be made affordableto the point of almost being like those silly 3D glasses you wear at the movie theater.

I believe that there is a “sophistication hierarchy” for AR and that it goes something like the following – with each higher level incorporating the feature(s) of the levels below it.

AR Level 0 – 1 – simple text + basic icon overlay – very inexpensive hardware – simple to build.

AR Level 2 – 2D graphics overlay – – perhaps still images like photos or even movies. No “real time” generated 3D interactive content. Slightly more complex – perfect for places like museums or tourist attractions…

AR Level 3 –– interactive 3D content graphics overlay – here things are starting to get VERY interesting..ala Microsoft HoloLens.

AR Level 4 – object recognition -yep- Terminator style.

AR Level 5 – real world object wrapping / overlay – way cool…I think this is the the highest technology use level of AR – Tony Stark style.

From a basic electronics standpointLevels 0 and 1 require a simple projection unit and a very simple display controller. Something like our DCNanodisplay controller! Very simple, very easy. The markets for this are ENORMOUS all by itself. Think about simple turn by turn navigation or object labeling based on simple GPS data combined with compass data.

Level 2 requires the addition of a 2D GPU – either a simple raster engine or something more sophisticated like a 2D vector graphics processor. The level of sophistication that can be added here is quite impressive.

Level 3requires the addition of a 3D GPU – now things are getting more serious. With a 3D GPU – you can project full 3D objects into your reality that you can interact with at some level. Again – think something like Pokemon Go except at a useful, practical level. OK – STOP IT! Pokemon Go is NOT a practical use of AR – although it is addictive and fun…

Level 4 requires the addition of a vision image processor (VIP). While you could handle this with a big enough GPU – a VPU will be MUCH lower power and you will still need a GPU to project 3D graphics into your world.

Level 5requires the addition of a VIP that has deep learning capabilities (like CNN / DNN features – see above!). So why do you want machine learning in this case? You need to be able to recognize an object to the level of sophistication to overlay and wrap graphics around the object without having to send data into the cloud for processing. The best way to do this is to have a “trainable” system so you don’t end up with an AR system that only recognizes cows until it can call home.

So – what are the killer use cases for all of this AR goodness? How vast is your imagination? I was recently at the Roman ruins in Glanum, France. It was very hard to imagine how incredible that sight must have been (and it is a magnificent set of ruins). Imagine having AR glasses on that could project -in real time and from the vantage point I was standing at- the fully reconstructed ruins. Imagine walking into the market and seeing the stalls and maybe even people buying and selling. Imagine walking into the temple and seeing the magnificent statues and artwork that might have been there. Imagine just seeing the architecture in all its glory.

Another use case? How about something simple like looking at a sign in a foreign language and having it instantly translated for you? Or how about translating a menu for you from a foreign language?

More use cases? How about looking at something that needs to be serviced or repaired and having the step by step instructions projected, walking you through every step? How about combining IoT and having the device tell you what component needs replacing and showing you where it is and what to do to replace it – or even ordering it if you don’t have it with a simple gesture.

I could go on and on and on – but the bottom line is that AR done right will be extraordinarily affordable and will change our world significantly. I think that even John Lennon would have appreciated the ability to sprinkle imagination all over our reality…