Multiplayer multiplatform XR through Fynd CORE – a technical breakdown
Fynd CORE is the result of a multi-year effort spanning multiple projects and sectors, and has given the team the chance to experience the progress of XR development. In this article, we will outline some of the technological advancements that has made Fynd CORE possible.
Running on any device
From the beginning, Fynd CORE was designed to be run both in VR and on desktop. This was an important choice for us, because it lowers the bar for taking XR tools in use, as the step into a headset can be big for many. By offering a way into the experience through a familiar interface, we can show that XR technology can give major benefits without forcing everyone into a headset.
This philosophy has since evolved into the development of Fynd CORE for Mac and mobile devices. A mobile app is the platform with the lowest barrier to entry, and we see this as an important step for both distribution and adoption.
When approaching multi-platform development, the benefits of working with an established game engine becomes obvious. Unity works continuously with platform updates and security, meaning we can focus on the content of our app. During the development of Fynd CORE Unity has improved its tools for XR development, kept up to date with iOS and Android, and in 2023 played a role in Apple’s launch of the Vision Pro.
We can’t always promise to run on every platform, but with Unity as our foundation we are happy to be pushing for PC, Mac, iOS, Android, Magic Leap, and standalone VR headsets, which have grown significantly in popularity over the last couple of years.
Device agnostic VR
Development for VR headsets has been chaotic since the launch of the Oculus Rift and HTC Vive around ten years ago. The VR industry pushed for standardization, and managed to form the OpenXR coalition, launching its cross-industry standard in 2019. This was a major step in unifying the developer and user experience, but it was still some time before the standard was ironed out and implemented to a satisfying degree in Unity.
In short, OpenXR is an abstraction layer that all hardware providers can plug into, with a single interface for developers to work with. This means we no longer have to implement platform-specific SDKs like what was necessary in the past.
The adoption of OpenXR has made it possible for a company like ours to target a variety of different headsets, which at launch includes Meta Quest, Pico Neo, HTC Vive Focus, and all supported PCVR headsets. There are still challenges ahead, like controllers with more functionality than others, and hand tracking will be getting more focus in the years to come. OpenXR allows us to focus on prioritizing user experience and ease of access rather than programming technicalities.
Cross-continent multiplayer
Working with Unity has the benefit of gaining access to the Asset Store, with a broad selection of content and technical tools. There are various networking solutions offered for Unity, including their own framework (Netcode for Gameobjects) which launched in 2022. At Fynd Reality, we decided early on to go for a network solution with full control over both the engine code and the server side.
This has allowed our network programmer and backend programmer to implement a highly customized Azure networking backend, designed for security, scaling and functionality across continents, which is a limitation for many of the popular Unity networking solutions. In partnership with Microsoft, we are continuously working on improvements to our networking infrastructure and making sure we always adhere to the latest security standards.
Breaking free of UI bottlenecks
While working with Unity brings plenty of benefits, it also has its limits. After several years of wrestling with the engine’s UI tools, our programmers and designer decided in unison that we wanted a more iterative and flexible solution as Fynd CORE grows. Unity have realized this themselves too, but unfortunately their work with the web-like UI Toolkit has been a slow push and was never prioritized to work in VR.
After a prototype and a few months of work, our first working version of a HTML/CSS/JavaScript based UI was running in Unity. Developing with a web-based workflow unlocked a lot of opportunities for us, particularly gaining access to plugins and frameworks and improving iteration times. A big benefit is that our UI codebase is completely separate from the Unity project, meaning we can edit the interface while the engine is running, or even in a built version.
Our UI designer has been working incredibly hard to gain the most from this transition, and has gone into detail to this process in her own article.
The road ahead
We have spent a lot of time during the development of Fynd CORE to build a foundation that will allow us to perform better in the long term, and that’s coming to fruition now. Our scenario builder is going to be expanded piece by piece, our VR UI is getting an overhaul, and our content delivery pipeline will give our partners real power to deliver custom-made solutions to their customers.
The XR horizon is looking bright, and the arrival of Apple’s Vision Pro might usher in a new age of spatial programs. In particular the normalization of controller-less XR will be presenting some challenges for those of us developing more complex software. It’s an exciting time in our field, and we are exactly where we want to be.
