WebXR Performance Testing with React: Revolutionizing Mixed Reality Development

Software EngineeringDevelopmentOpen Source

Overview

This case study highlights exploring the capabilities of WebXR for developing cross-platform Mixed Reality (MR) applications with a focus on battery life, performance optimization for user comfort, and user experience, the project aimed to determine the effectiveness of code sharing and leveraging open standards for MR development.

Through the creation of an interactive MVP featuring advanced data visualizations, the collaboration showcased the seamless integration of React, ThreeJS, React Three Fiber (R3F), and Vite technologies. The project achieved impressive results, including 100% code sharing across multiple devices, efficient rendering techniques, innovative features like drag-and-drop functionality and text highlighting, and CI-based code testing.

This collaborative endeavor demonstrated the potential of WebXR and showcased the power of leveraging community support and expertise in the open-source ecosystem.

Services Provided

To gauge the viability of the WebXR ecosystem, our innovative engineers joined forces with Rebecker Specialties. The client's ambitious objectives included achieving 90% code sharing across Oculus Go, Microsoft HoloLens 2, Meta Quest 2, and desktop browsers, all while maintaining a minimum of 60fps performance. Additionally, they sought seamless 1:1 interactions with both controllers and hands (where applicable).

Through meticulous efforts, the functional goals were not only met but surpassed, resulting in the exceptional accomplishment of 100% code sharing across all targeted platforms. Our team of experts delved into dependencies, diligently profiling and optimizing at every level, while also contributing back to the thriving open-source WebXR community.

Moreover, this collaboration empowered us to pioneer innovative techniques for unit and integration testing, obviating the need for costly devices or specialized hardware setups. This exemplified the economic scalability of our cross-platform WebXR development approach, solidifying our position as pioneers in this cutting-edge field.

Tech-stack

React: We opted to use React as our UI library because it is a familiar and well-established choice that our team has experience working with. We are confident in our ability to use React effectively and efficiently.
ThreeJS: For our 3D needs, we wanted to select a library that was both reliable and user-friendly. ThreeJS stood out as the most popular 3D library with a supportive community and many examples available. Its ease of use and versatility made it a safe choice for our project.
React Three Fiber (R3F): Since we were already using React, we wanted to find a way to integrate ThreeJS into our workflow seamlessly. R3F offers a convenient solution for rendering ThreeJS within a React environment. It provides all the functionality of ThreeJS with the added benefit of being able to take advantage of any new features or updates as soon as they become available.
Vite: We chose Vite as our frontend tooling because it is currently one of the most popular options available. We were interested in experimenting with it, and exploring its potential for speeding up our development process.

This Dot Labs is a development consultancy focused on providing staff augmentation, architectural guidance, and consulting to companies.

We help implement and teach modern web best practices with technologies such as React, Angular, Vue, Web Components, GraphQL, Node, and more.

Client

Rebecker Specialties creates cross-platform Mixed Reality applications for its clients. It is led by its founder, Matt Hargett, who is a 25-year veteran of Silicon Valley and has led teams at BlueJeans, PlayStation, and Roblox.

Promoting Performance Across Platforms

To provide a smooth and enjoyable experience for all users, regardless of their VR device's specifications, our team emphasized performance optimization. We wanted to see at least 30 frames per second (fps) on low-end devices, and if possible, 60 fps on high-end devices such as the Quest 2.

VR view

Text Elements

Although showing text is one of the key features of the app, we found that loading text elements was having a significant impact on the application's FPS. This posed a challenge as we expected the application to sometimes need to render hundreds of text elements at the same time.However, we discovered that each Text element was triggering a new draw call, which we addressed using a third-party library, called troika-three-text. This library features atlas texture packing, which allows multiple characters to be stored in a single texture and reduces the number of draw calls. It also includes signed distance field rendering, which produces smooth, high-quality text with minimal aliasing.

Cubes

Our team used Three.js to create cubes with text on them in a node graph within a 3D space. We started with a Mesh object, the digital representation of an object's shape which allows it to be rendered and manipulated, but the application struggled to display a larger number cubes, causing a poor user experience and even the risk of motion sickness.To address this, we used the InstancedMesh object, which allows us to efficiently render many identical or similar objects in a scene, create a single "template" Mesh object, and then render it multiple times in the scene with varying positions, rotations, and scales. This means that, instead of rendering hundreds of individual objects, the computer can render a single object and simply transform it multiple times, improving the performance!

Promoting User Experience

Dragging Cubes

We built and delivered the ability to drag the cubes since it allows users to organize the cubes and create a visually appealing space using React Three Fiber along with three-forcegraph. This allows the user to drag the cubes in any direction and, if desired, drag along all of the cube's children (since the cubes were all connected in a node graph) by pointing and pressing the squeeze buttons on the controller.

VR view

Text Highlighting Feature

The text highlighting feature was a crucial addition, as it enabled users to easily locate the texts they were searching for. We had already used the troika-three-text library to create a 3D text mesh, so we took advantage of what this library had to offer. Using the getSelectionRects method, we were able to return a list of rectangles covering all the characters within a given character range to obtain the text's bounding box. With this information, we generated a 3D box mesh that could be used to highlight the text, and made this information available to other cubes using the React context API.

VR view

Ease of Use

However, it wasn't enough to simply implement this feature; we had to ensure that the user experience was optimal. To help, we added functionality that enabled the user to highlight the text they were looking for by pointing at it and pressing the trigger button on the controller. Additionally, we incorporated various colors into the highlighter, enabling the user to highlight multiple texts at once. We also made sure to highlight the cube containing the text, making it easier to differentiate from other cubes in the distance. The outcome was a visually pleasing and user-friendly feature that simplified the process of locating specific texts.

Summary

Our collaboration with Rebecker Specialties on exploring WebXR for cross-platform Mixed Reality (MR) application development has yielded remarkable results. Through our expertise in performance optimization and user experience, we successfully created an MVP that showcases the potential of WebXR in delivering immersive experiences.Our achievement of 100% code sharing across multiple platforms, along with the efficient rendering techniques and innovative features we implemented, highlights our commitment to pushing the boundaries of WebXR development. By leveraging technologies like React, ThreeJS, React Three Fiber (R3F), and Vite, we demonstrated the seamless integration and powerful capabilities of these tools.Furthermore, our collaboration with renowned organizations and contributions to the open-source community underscore our position as pioneers in the WebXR space. We invite organizations seeking optimal code sharing, exceptional performance, and immersive MR experiences to partner with us and harness the full potential of WebXR.Contact us today to embark on your journey into the future of cross-platform MR development and leverage our expertise to transform your ideas into reality.

Let's see how This Dot can help you

Are you looking for a development consulting firm that can provide staff augmentation, architectural guidance, and consulting to your company? Look no further than This Dot Labs. Our team of experienced developers and managers can help you implement and teach modern web best practices, and deliver your projects on time and within budget. Contact us today to learn more about our services and how we can help you achieve your development goals.

Get in touch