The Face Tracker Texture and Face Swap Effects

You can use the video of someone's face when they're using the effect as a texture in your scene, by extracting the face tracker texture.

You can then apply the texture to a material and object - for example, to create a face swap effect.

Learn about:

  1. Creating a texture from the face tracker.
  2. Using the texture in an effect.
  3. Creating a face swap effect.

To use the video captured of the entire scene, use the camera texture instead.

Creating a texture from the face tracker

  1. Add a face tracker to your scene.
  2. Select the face tracker in the Scene Panel.
  3. In the Inspector panel, click + next to Texture Extraction.

You'll see a texture called faceTracker0 in the Assets Panel:

Using the texture in an effect

The texture can applied to any material channel. Or, drag it in to the Patch Editor to combine with other visual shader patches.

The texture can be used in 2 different ways:

  1. 3D Direct - the texture will have the 2D layout of the face mesh. Use this option to create a 3D representation of the user's face.
  2. 2D Bounding box - to create a flat cut out of the user's face.

To select either of these options:

  1. Select the face tracker texture in the Assets Panel.
  2. In the Inspector, choose an option next to Copy Mode.

3D Direct

Setting Copy Mode to 3D Direct lets you create a 3D representation of the user's face. You'd generally use this mode when applying the texture to a face mesh, for example in a face swap effect.

To do this:

  1. Add a face mesh to your scene.
  2. Create a material for the face mesh.
  3. Set the Shader Type to Flat - because the texture will already have realistic lighting.
  4. Next to Diffuse, select the face tracker texture.

2D Bounding Box

In this example below, we've set the Copy Mode to 2D Bounding Box. We've then applied it to a material, and applied the material to a plane:

You could use the 2D Bounding Box mode to create 2D animations featuring the user's face.

Creating a face swap effect

Start by adding the face trackers:

  1. Add two face trackers to your scene.
  2. To help keep track, rename the face trackers - name the first face_1, and the second face_2.

Then, edit the second face tracker so that it responds to the second face to appear in the scene:

  1. Select face_2 in the Scene tab.
  2. In the Inspector panel, set Tracked Face to Face 2.

Next, create the face mesh, textures and materials:

  1. Extract the face tracker texture for each face tracker.
  2. Create 2 face meshes, each as a child of one of the face trackers.
  3. Create a material for each face mesh, and set the Shader Type to Flat.
  4. Apply each face texture to the material applied to the opposite face tracker in the scene.

The face mesh will have hard edges. To blend these with the face, use the face mesh mask included in the face reference assets.