There may be differences between your version of AR Studio and this tutorial because the product is currently in beta and we update it regularly.
In this tutorial you'll learn how to separate the people in your scene from the background.
You'll do this by using segmentation. When you combine segmentation with a feature called texture extraction, you can create textures based on parts of the scene. Using these features you can replace the background behind the person.
Download the sample content to follow along. If you open the finished project, you'll see that you're going to build a simple background replacement effect.
To get started, open AR Studio and create a new project. First:
You should now see segmentation listed in the Scene tab. You can't see it in the Viewport, but segmentation has created an outline around the person in the scene.
Next, you'll need to create a texture using the contents of the outline. Make sure that segmentation is selected in the Scene tab, go to the Inspector panel and then click + next to Texture Extraction. Your new segmentation texture should now be listed in the Assets panel.
Once you've done that, select Camera in the Scene tab, go to Texture Extraction in the Inspector panel and click the +. You should see a camera texture listed in the Assets panel as well.
If you select the segmentation texture in the Assets panel, you'll see that it has 2 properties. You can use:
Now that you have the textures you need, you'll need to insert a surface to apply them to. So click Insert, select Rectangle and then click Insert.
You should see a canvas with a rectangle nested below it in the Scene tab and a 2D object in the scene.
For this effect, the rectangle needs to fill the screen. If you look at the Viewport, you'll see a large green rectangle. This indicates the size of the device screen, as does the Simulator. To get these dimensions, go to the Scene tab, click Device and note down the Screen Size values.
For me, that's 375 and 667, because I have the Aspect Ratio in the Toolbar set to iPhone 8 and the parent canvas takes the size of the selected device in screen space.
Now, go back to the rectangle's Inspector panel:
The rectangle should now be the only thing you can see in the Simulator.
Next, you'll need to create the material to apply to your rectangle. To do this:
Once you've done that, go to Diffuse, click the arrow next to Texture and select the camera texture from the menu. This renders the camera feed onto the rectangle.
Then, go to Alpha, check the box, click the arrow next to Texture and select segmentation texture from the menu. This cuts away the rest of the rectangle and lets you mask the person in the scene.
Now you can add your custom background to the scene. The first thing you'll need is another rectangle, so click Insert, select Rectangle and click Insert.
Change the size of the rectangle to match the first one. The rectangle should now fill the Simulator. You'll also need to create the background material, so click the plus sign next to Material and select Create New Material from the menu. Double-click on the material you've created to inspect its properties.
We're going to use a basic color texture for this material, but you can use another texture if you'd prefer. To create the color texture, click in the box next to Color to open the color wheel and choose the color you want to use. We've chosen blue. The background should now be covering the Simulator.
To make the person visible and place the background behind the face, you'll need to use layers to adjust the render order. Go to the Layers tab and click the plus sign to create a new one. Move the new layer to the bottom of the tab, so that it is rendered first.
Then, go to the Scene tab, select the second rectangle, go to the Inspector panel and change the layer to the new one. You should now see the custom background behind the person in the scene.