[The Forefront of VR Technology] A Night Exploring the Possibilities of Photogrammetry and Resonite

Several users gathered in a VR space to discuss the latest 3D technologies and the deep features of various platforms. The conversation centered on "photogrammetry," a technology that recreates the real world with startling realism, and "Resonite," a VR platform that boasts high customizability. In this article, we explore the present and future of VR technology through their conversation.

Click here if you would like to watch the video

(Due to a settings error, the recorder's audio sounds strange. Sorry about that! 💦)

London Natural History Museum recreated via photogrammetry

The first place the group visited that day was a world in a VR space that recreated the Natural History Museum in London using photogrammetry technology. Photogrammetry is a technique for reconstructing 3D shape data from multiple photographs.

When the question "What kind of place is this?" was answered with "It's a world created by photogrammetry of the London Natural History Museum," exclamations of "Wow, that's amazing!" rose from the participants. The high ceilings of the hall, the giant whale skeleton specimen, and even the details of the majestic architecture were brilliantly recreated, and the participants were overwhelmed by the high quality.

During the conversation, technical explanations were also exchanged, such as "This is photogrammetry, not Gaussian Splatting, so it has a mesh." Because it is composed of a mesh (a network of polygons), it exists as a solid 3D model, and technical surprise was expressed that such beautiful textures could be reproduced on top of it.

The impact and potential of Gaussian Splatting

Next, the topic shifted from photogrammetry to a more advanced technology: "Gaussian Splatting." This is a new method that represents a 3D space using a large number of points (point clouds).

One of the participants brought in data captured with an iPhone at the real-world venue of the "Virtual Market (Vket)" event and deployed a 3D model using Gaussian Splatting into the space on the spot.

Differences between photogrammetry and Gaussian Splatting

• Photogrammetry (Conventional): Creates a collection of polygons (mesh) based on photos and applies textures to them.

• Gaussian Splatting: Generates a collection of colored points (point cloud) based on captured video.

A model generated by Gaussian Splatting shows a realism that can be mistaken for a photograph when viewed from a specific angle, but it has the characteristic of revealing itself as a point cloud when the angle is changed. This is because it faithfully reproduces the "appearance from the position where it was captured." Participants were heard saying, "This one is way more beautiful!" and they could not hide their surprise at its expressive power.

This technology has the great advantage that anyone can easily generate a 3D space as long as they have a few minutes of video shot with a smartphone. While creating higher-quality data requires the effort of importing the video into a PC for processing, its ease of use and realism have the potential to revolutionize future VR content production.

Amazing customizability! The VR platform "Resonite"

In the latter half of the conversation, the advanced features of the VR platform "Resonite" that they use were introduced. Of particular note was the "Facets" feature, which allows users to freely customize the UI (user interface).

Normally, the menu screen in a VR space is fixed, but in Resonite, it is possible to add features created by users or delete existing ones. In the demonstration, they showed how to add buttons for instantly moving between multiple worlds (sessions) and a mirror feature to see oneself to the menu screen using intuitive operations like drag-and-drop.

Surprisingly, this customization work cannot be done in desktop mode and can only be executed while wearing a VR headset, making it a truly "VR-native" specification.

Furthermore, a feature that links with Google Calendar to automatically display registered event information on a calendar within the VR space was also introduced. This allows users to check the community's event schedule without leaving VR. However, the danger of personal private appointments being accidentally shared was also pointed out, and the importance of balancing convenience with risk management was discussed.

Summary

This gathering was more than just a chat in a VR space; it was a valuable opportunity to experience the latest technology firsthand and delve deeply into its possibilities. Photogrammetry and Gaussian Splatting give us a glimpse of a future where anyone can easily create realistic 3D spaces. Furthermore, platforms like Resonite empower users to reshape the world of VR, making it more accessible and richer. As technology evolves, our VR experiences will undoubtedly become increasingly interesting and profound.