Use segmentation to create an effect that separates someone from their background. You can then transform the environment behind them using textures and objects.
Segmentation can only identify people from the chest upwards, in the immediate foreground.
In this guide, you'll learn about:
To learn about different ways of using segmentation in effects, take a look at the Floating Particles sample effect.
It's worth noting that segmentation only works on newer devices. iOS devices must be iPhone 6S or later. Android devices must be Samsung Galaxy S6, Song Xperia Z2 or equivalent, or later.
To create a simple effect with segmentation and a 2D background, you'll:
To start, select the Camera in the Scene tab.
In the Inspector panel, click + next to Segmentation.
You'll now see a Segmentation Mask Texture in the Assets panel.
Next, insert a Rectangle - it'll automatically be inserted as a child of a Canvas.
You'll see a small rectangle in the middle of the scene. For a segmentation effect, we want the rectangle to fill the screen and automatically rescale to fit whichever device is using this effect. To do this, select Fill Parent in the Inspector panel.
The rectangle should now cover your scene.
Next create a material. At the bottom of the Inspector panel, click + next to Material, and select Create New Material.
Select the material in the Assets panel. In the Inspector panel:
You'll now have a complete effect using segmentation!
You'll need to use 3 different layers to make an effect with segmentation and 3D objects in the background:
Follow all the same steps as you would when creating a 2D segmentation effect.
This time, instead of selecting Invert under Alpha in the Inspector panel, apply the CameraTexture you created under Diffuse.
In the Layers tab, rename this layer Segmentation.
Create another rectangle, and set it to Fill Parent in the Inspector Panel. Create a Material for this rectangle, and choose a color or texture for this the material.
Apply this to a new layer, and label the layer Background.
Next to Render Mode select Transparent, so the 3D objects are visible through the background.
Import your 3D object under the Canvas.
Create and apply this object to a new layer. It's worth naming the layer something like Objects, to keep track.
This layer should be in the middle of the list of layers, to make sure it renders correctly.
The Layers tab should look like this, with the segmentation layer first and the background last in the list:
You'll now have a segmentation effect with 3D objects in the background!