Researchers at WIoT at Northeastern University study whether today’s ultra-fast 5G mmWave networks can support latency-critical multi-user AR applications.

An in-depth measurement study done with a popular commercial AR application, Cloud Anchor, revealed that 5G-mmWave does not provide substantial performance benefits over LTE networks in enabling such multi-user applications. However, in a follow-up work with another popular application, Just a Line, researchers reported that the underlying wireless network has minimal impact, and the application design plays a very important role in optimizing the overall end-to-end user performance.

They found that Just a Line, to localize the virtual objects among all users, employs periodic asynchronous SLAM (Simultaneous Localization And Mapping), unlike Cloud Anchor, which does SLAM before each virtual object placement and resolution by users. Hence, there is an interesting interplay between the underlying wireless network as well as the application design, which leads to higher QoE in such applications.

Multi-user AR, unlike single-user AR, critically relies on the cellular connection and a cloud-based server to support user interactions. 5G mmWave, with its ultra-low latency and high bandwidth, promises to unleash the capabilities of such applications, which was not possible with LTE.

The below figure shows Cloud Anchor’s breakdown of latency components using 5G mmWave and LTE.

5G mmWave offers little of an edge in terms of performance when compared to LTE. The reason for that is as follows:

1. TCP does not exit slow start, which does not allow 5G mmWave to unleash its full potential.
2. The researchers observed an extra delay called delay-2x (not observed in prior literature), probably caused by middleboxes, which have nothing to do with the underlying wireless network.
3. Cloud processing incurs a substantial amount of delay.

In contrast to Cloud Anchor, Just a Line has an order of magnitude lower end-to-end latency (Figure (a) below). The reason for such low latency is due to the fundamental architectural difference between the two applications.

This difference is related to the way the two apps perform SLAM (Simultaneous Localization and Mapping), which serves as the foundation for localization in every mobile multi-user AR app. Cloud Anchor performs SLAM synchronously, i.e., before every virtual object placement. This coupling of the two operations – SLAM and virtual object placement — results in high end-to-end latency, as a successful SLAM update must always be completed before the resolver can render a virtual object placed by the host.

In contrast, Just a Line decouples SLAM from the virtual object placement. SLAM is performed asynchronously in a periodic manner, and the app always uses the results from the most recent SLAM update to render a virtual object. The decoupling of the two operations allows Just a

Line to achieve a much lower end-to-end latency compared to Cloud Anchor.

Can 5G mmWave support Multi-User AR-3

However, Just a Line, requires a dedicated synchronization phase in the beginning to exchange SLAM-related data with the cloud server. This initial synchronization phase can last anytime between 16 to 300 seconds, as shown in Figure (b) above. The best-case initial synchronization latency for Just a Line is much longer than the worst-case end-to-end latency of Cloud Anchor. Hence, there is a design trade-off on whether it is better to incur a higher delay before an AR session starts so that there is no further latency. At the same time, the user plays the AR game or does the in-game delay in performing the synchronized SLAM with every virtual object placement suffice for the high initial synchronization delay?