A Scalable Mixed Reality Platform for Remote Collaborative LEGO Design

Mixed reality (MR) is a new paradigm that merges both real and virtual worlds to create new environments and visualizations. This together with the rapid growth of wireless virtual/augmented reality devices (such as smartphones and Microsoft HoloLens) spurs collaborative MR applications that provide an interactive and immersive experience for a group of people. In this demo, we develop a scalable MR-based platform for remote collaborative LEGO design. To provide the best immersive experience, the system should provide 1) high-speed and high-resolution image rendering: the rendering should achieve screen resolution of the mobile device and at least 60 frames-per-second; 2) extremely low delay guarantees: the motion-to-display latency of each user should be below 20ms; 3) synchronization: the synchronization latency should be small enough to enable the smooth collaboration; 4) scalability: the number of users should not have a significant impact on the system performance. To achieve all these goals, we introduce a central server to facilitate user synchronization via exchanging small messages. Each user reports to the server with its LEGO design progress, which is then distributed to all other users by the server; all other users render the corresponding virtual LEGO models in its own design space. We demonstrate via real-world implementations and evaluations that: 1) our system performance (e.g., synchronization delay, frame rate) does not degrade with the increase of the number of users; 2) our developed system not only yields a motion-to-display delay of 11 ms (i.e., 90 frames per second) but also achieves a screen resolution of each user's mobile device (e.g., 2400 × 1080 pixels for Google Pixel 6).