AR vs MR vs VR – same devices, different market value.
Augmented reality and mixed reality (also known as extended reality) have already reached usability phase thanks to the recent advances in computer vision, sensor fusion, and new-age display technologies. Their business applicability has been proven repeatedly, and now the community has moved on to researching, thoroughly, the security and privacy implications of AR’s adoption (which, turns out, are quite substantial.)
A person wearing a head-mounted device (HMD) captures lots of information about surrounding physical objects and locations and thus exposes, sometimes unwittingly, massive volumes of data to third-party applications that are meant to process the data and deliver relevant outputs. If these apps happen to be corrupted, sensitive (and potentially damaging) info can be leaked and used with malicious intent.
In this post, we’ll try to answer the question ‘what is XR?’, describe the conceptual differences between the types of immersive technologies (since there’s still a great deal of confusion surrounding augmented reality vs virtual reality comparisons) and discuss how AR’s key security concerns can be addressed through the implementation of the latest protection approaches.
AR vs MR vs VR – the three types of XR technology
The term Virtual Reality refers to software-generated simulations of real experiences. VR completely blocks out a person’s view of the world (typically through an HDM device) and immerses them into an artificial environment.
Augmented Reality, however, is about bringing digital information into a person’s view of the real world. Instead of shutting down reality, it superimposes computer-generated content onto the physical environment so that both can be experienced together.
Mixed Reality has a hybrid definition that blends aspects of both VR and AR. It is essentially an extension of AR that, on top of overlaying information, enables meaningful interaction between synthetic and physical objects to deliver rich, immersive experiences.
Though tech media loves lumping the two together, AR (MR) and VR are two distinct technologies that each have their own path. AR (MR) market is predicted to reach over $149b by 2025, while the global VR market is expected to grow to $33.8b in the same timeframe.
AR (MR) Data Threats
There are several vulnerable spots in any AR environment due to the complexity of the data flow running through it; there are input, data storage/aggregation/processing, output, and interaction elements in such systems and it’s crucial that they are all sufficiently protected.
What we input into an augmented reality environment might be sensitive and affect not only the immediate user but also bystanders who happen to be in the camera’s shot; the principles of confidentiality, unobservability and untraceability must, therefore, be accounted for when designing any type of an AR/MR solution.
After being sensed, the data winds up in storage (database) of some sort from which processing applications will retrieve it to deliver user-consumable outputs. The problem is, once the information has been accessed by apps, it is no longer in the user’s control; a malicious third-party can potentially intrude and corrupt the app to cause a data breach.
Another issue, regarding outputs, stems from the fact that one application in such a complex environment may intercept, inadvertently or not, another app’s data. A malicious app, therefore, might render the AR system’s outputs unreliable.
Finally, in the context mixed-presence collaboration systems, AR (MR) devices create certain transparent boundaries (between physical and virtual objects) in shared spaces and the directionality of these boundaries can profoundly influence the mutuality and privacy of collaborating users; it’s vital that there are strict policies in place regarding non-repudiation as well as robust authentication & authorization schemes. Otherwise, an adversarial agent can sneak into an unprotected space and obtain private data (while blocking legitimate users from entering using dos attacks, etc.)
So How Do We Establish Security in Mixed Reality and Augmented Reality?
To secure latent (accidental) inputs and information that’s been purposely put forth, maintain the integrity of data during aggregation/storage/processing, and make tamper-proof the anchored and unanchored outputs (and outputs of external displays), an intermediary protection layer between AR devices and application resources, which serves as a controller of the in- and out-flow of data, has to be inserted. That’s the general rule; the exact features of the layer depend solely on a specific environment’s needs.
Passive or latent inputs can be protected through various input sanitization techniques implemented between sensors’ interfaces and applications. Their goal, generally, is to remove potentially sensitive info from the input data stream and facilitate tight access control.
When handling active inputs, we typically utilize a least privilege mechanism through technologies such as Prepose. It obscures the raw data feed (gesture feed, etc.) and enables disclosing the minimum amount of information to a certain application – only the data objects an app needs to run.
To protect data, we use privacy models such k-identity anonymization which means we ensure that a particular user’s data is indistinguishable from k-1 individuals whose records also occur in a given data release. To achieve this, we leverage a range of data perturbation and manipulation techniques. If there’s scaling issues with this method (which can arise due to massive volumes of data incorporated) we can either switch to using differential privacy algorithms (introducing randomness to data to achieve plausible deniability and unidentifiability) or combine the two approaches.
The incorruptibility of outputs that are displayed to a user by AR/MR devices is reached via proper management of rendering priority (synthetic object transparency, occlusion, etc.) and access control. A popular design framework for policy (regarding handling outputs from third parties) specification and enforcement was introduced by Lebeck et al. in 2017. It builds on the principles outlined in Hololens Developer Guidelines as well as policies for user safety in automobile-installed AR released by the U.S. Department of Transportation’s National Highway Traffic Safety Administration (NHTSA).
Lastly, to establish interaction security in collaborative spaces, we need mechanisms for enforcement of user-defined privacy policies. The software framework that’s consistently proved effective in this context is SecSpace; it allows designing advanced security schemes for mixed-presence environments, has a variety of interface mechanisms for interactions between virtual and physical network’s elements and provides a high degree of feature decoupling (which in turn permits a variety of development strategies). The system enables users to take privacy-oriented actions in either virtual or physical space and have them reflected immediately in both spaces.
Summing Up
The promise of Augmented Reality lies in enhancing the human ability. In the business context, AR’s value is in granting deeper insight into products and processes, allowing experts to educate and instruct efficiently new employees remotely, etc.
IT titans, such as Google, Intel, and Microsoft have already thrown their multi-million rings into the AR/MR circle to cement their leadership in the space; but as businesses become more acculturated to utilizing it, as it gets progressively more sophisticated, and as AR hardware becomes less cumbersome and expensive, we expect mid-sized and small firms to follow suit. AR/MR technology is likely to reshape, in a meaningful way, business processes across various industries.
At the point being, the technology is nascent and certain challenges, regarding privacy, are still to be addressed before it can be adopted widely. The security implications of augmented and mixed reality are being actively researched, however, and a variety of innovative protection approaches for AR/MR environments have already been proposed.
Still have questions regarding ar vs mr vs vr taxonomy? Want to learn how your company can leverage AR(MR) environments? Contact our expert right now for a free consultation.
