Current Augmented Reality (AR) frameworks lack standardisation and platform independence in how applications specify their spatial and system-level requirements, complicating cross-platform development and limiting the coexistence of multiple AR applications. We introduce AURA --- the \textbf{A}ugmented Reality \textbf{U}nified \textbf{R}epresentation \textbf{A}rchitecture ---, which defines a manifest format that allows applications to specify their spatial components, interaction patterns, participating agents, and required system resources. By assigning applications to scoped containers and managing shared physical surfaces access, AURA enables concurrent execution while preserving spatial coherence. It also supports dynamic behaviour through event-driven triggers and system-mediated data exchanges at runtime.
a) An AURA-based system scans the room, and independently created and launched applications submit their spatial and behavioural requirements through AURA manifests.
b) The AURA-powered Head-Mounted Display (HMD) identifies suitable components in the environment and, with user input, assigns them to each application by creating dedicated ambients (e.g., $\alpha_1$, $\alpha_2$).}
Augmented Reality (AR) is emerging as a key paradigm for immersive computing, blending digital content with the physical world in real time. Currently, and in spite of rapid advances in hardware and toolkits, developing AR applications today remains a fragmented endeavour. Different platforms (e.g., mobile AR toolkits, smart-glasses) and engines (e.g., Unity, Unreal) impose distinct content representations and siloed runtime contexts, making it difficult to coordinate multiple AR experiences. In practice, users are often confined to one AR application at a time, and there is no straightforward way to allow multiple AR apps to augment the environment concurrently \cite{huynh2022layerable}.
Most existing AR development platforms, such as Unity's XR Interaction Toolkit and Unreal Engine's AR APIs, are geared toward building single-application experiences. These tools provide abstractions for tracking, rendering, and input (e.g., ARKit provides world tracking and scene understanding \cite{ARKitDocumentation}, ARCore offers similar capabilities \cite{ARCoreDocumentation}), but they lack a standardized model for representing application structure, coordinating access to physical components, or supporting interaction logic across applications. This single-app focus limits scalability, complicates cross-platform deployment, and hinders the development of multi-application AR environments.
Current AR experiences are typically built in isolation: there is no standard way for one AR application to expose or to incorporate content from others, limiting content reuse and preventing multi-application scenarios (e.g., a navigation app and a social AR app cannot easily co-exist and share the view \cite{lebeck2019multiple}). Prior efforts to standardize AR content (such as AR markup languages or model-based UI design tools like ARML~\cite{OGC_ARML2_Adoption} or SSIML/AR~\cite{vitzthum2006ssimlar}) have not been widely adopted by today's AR platforms, which favour imperative and engine-specific approaches.
There is a growing need for cross-application interoperability and consistency, akin to how web browsers unified content on the Internet. As illustrated in \cref{fig:teaser}, AURA addresses the standardisation problems by defining a unified representation layer that sits between AR applications and the underlying AR runtime or operating system. At its core, AURA introduces a manifest format, expressed in JSON-LD, that allows applications to formally declare their spatial and behavioural requirements, including entities, components, agents, and interaction logic. The AURA-based system, in turn, guides the runtime context definition, including available physical components, spatial subdivisions (\textit{ambients} and \textit{worlds}), and mappings between devices and applications. AURA does not enforce a single global spatial model; instead, each application is bound to an ambient-scoped view, enabling coexistence through a layered abstraction of space. The \textit{``unified''} nature of AURA refers to its consistent runtime contract: all applications interface with the system using a common schema, while the system tracks general interaction and coordinates access to components without requiring direct inter-application communication.
Execution in AURA begins when a user launches one or more applications, each submitting its manifest to the AURA-based system. At runtime, the system monitors spatial triggers, schedules component updates, routes input and output, and manages state transitions, all according to the declarative contracts. This process explicitly ensures negotiated component use, avoids runtime conflicts (e.g., overlapping visual output or sensor contention) upfront, and facilitates a consistent understanding of spatial layout between the system and applications.
AURA provides a structured and extensible runtime model for AR systems by decoupling application behaviour from system-level resource management. AURA's contract is defined by two JSON-LD-based manifests: the static application manifest and the dynamic system manifest. The application manifest, authored by the developer, defines an AR application's structure, component requirements, entities, and behaviour. It specifies abstract requirements such as minimum spatial dimensions and expected coordinate systems, as well as roles for agents (e.g., tracked users with hand or position tracking services) and required physical characteristics and I/O capabilities for components (e.g., minimum dimensions for a flat surface, or visual output capabilities). The system manifest, generated at setup time or by an administrator, defines the physical AR environment, available concrete components (e.g., a specific ``table top'' with spatial attributes and capabilities), agents, ambients, and their grouping into worlds. Ambients define spatially and logically bounded contexts, grouping components and enforcing isolation. Ambients can be grouped into worlds, which are higher-level constructs enabling shared spatial and semantic contexts and defining global coordination rules such as data visibility and access policies. These manifests collectively define the logical, spatial, and operational interface between AR applications and their runtime environment.
\paragraph{Ambient/World Scope and Security Guarantees} Shared components must not be assigned to overlapping ambients, and applications can only observe and write to components within their assigned ambient. Outputs (e.g., visual rendering or actuator control) must target components currently available in the ambient. Cross-ambient observation requires explicit world-scoped coordination. These constraints mitigate several threats outlined by Ruth et al.~\cite{Ruth2019ARsecurity}, including attempts by one user to (1) share unwanted or undesirable content, (2) see private content belonging to another user, or (3) manipulate another user's AR content. By scoping visibility and interaction to well-defined ambients, AURA provides strong default boundaries.
\paragraph{Component-Based Memory Sharing} Components in AURA expose structured data stores that act as shared memory spaces. This memory is accessible to the application assigned to the ambient the component belongs to, and may be readable across ambients if the component is part of a world, though writes are ambient-scoped. AURA supports data exchange between applications by attaching shared memory to components, enabling coordination without direct application dependencies. Applications may subscribe to updates on component storage (event-driven access) or query current/historical changes (request-based access). The system is responsible for producing consistent snapshots of component memory to avoid inconsistent histories.
AURA distinguishes between system-managed components and application-defined virtual elements. Virtual elements --- such as photos, tools, avatars, or icons --- are not explicitly declared in manifests, but their behavior can be grounded and made interoperable by binding them to component-based shared memory. This model allows for spatial anchoring, inter-application observability, and indirect coordination across loosely coupled applications.
AURA enables modular development, scalable coordination, and conflict-free coexistence across applications. Currently, AURA assumes a centralized runtime and lacks global synchronization guarantees, which may lead to diverging histories when applications observe shared state independently. Future work includes implementing a reference runtime with arbitration and event routing, formal validation of manifests and ambient compatibility constraints, and distributed orchestration across heterogeneous AR platforms. We are particularly interested in probabilistic extensions to heuristic evaluation for improving tracking and rendering on moving objects, as well as developing an ambient calculus for structured data exchange between components.
