This chapter is from the book
This chapter is from the book
This chapter is from the book
Summary
In this chapter, you learned about some of the most useful tools for visionOS: scene reconstruction and occlusion. Using scene reconstruction, you can rebuild your entire environment using the Apple Vision Pro sensors and compute power. Successfully combining the real and virtual is the lynchpin of creating compelling experiences. Although Apple hasn’t made this process as easy as it could be, it is still simple enough to include in everyday projects with the help of the reusable SceneReconstructor class.
You also explored advanced hand-tracking with the HandTracker class. This code takes the complexities of working with the ARKit hand skeleton and, again, turns it into a reusable piece of code that makes entities available for every single joint in both of a user’s hands.
While there is still more ahead, you have what you need to build some fun and functional applications. I’ll round out your primary toolkit over the next two chapters, then show you how you can prepare your creations to reach as wide an audience as possible via the App Store.
Go Further
I highly recommend downloading and exploring the source code for Apple’s scene reconstruction example: https://developer.apple.com/documentation/visionos/incorporating-real-world-surroundings-in-an-immersive-experience. It may give you some good ideas of how to manipulate and place objects differently from what we’ve done in these examples.
It would also be good practice to go back to the Chapter 7 plane detection example and add scene reconstruction for more precise placement of the objects within the environment. Plane detection is a much less resource-intensive operation than scene reconstruction, so don’t disregard it entirely, but scene reconstruction does a significantly better job of enabling your physical environment to accommodate virtual objects.
With hand-tracking, you now have access to all the data that visionOS can provide. Experiment with ways that hands can be involved in natural direct and indirect gestures. An important goal for any AR or VR developer is to make the actions the user performs feel as natural as possible. The more you can make your virtual world feel real, the better. Just adding the ability to flick an object if you want to move it feels incredibly satisfying and can make you forget you’re staring at a piece of glass and metal.