There may be differences between your version of Spark AR Studio and this tutorial because the product is currently in beta and we update it regularly.

Tutorial: Creating a Face Tracking Effect

Something Went Wrong
We're having trouble playing this video.

Hello! In this guide, you'll learn how to make an effect that responds to someone's face - using a face tracker, face mesh, material and texture.

We'll talk about:

You can download the sample content to follow along with this tutorial.

Setting up

To get started, open the unfinished effect in the sample content folder.

The space in the middle of the screen is called the Viewport. This is where you can see your effect as you're building it.

The Simulator will show how your effect will look on a device.

We've already imported the texture you'll need for this effect, it's called Moustache, and listed in the Assets Panel. You'll combine this texture with a material to create the effect of a moustache drawn across a person's face.

The face tracker

The first thing you need to do is add a face tracker to your scene. The face tracker detects the position and orientation of the face of the person using your effect. It's the foundation of any effect that responds to someone's face.

To add any object to you scene, just click the Add Object button. You'll see a list of all the objects available in Spark AR Studio. For this tutorial, select the face tracker.

When the face tracker is selected in the Scene Panel, you'll see axes moving with the face in the Viewport.

Adding a face mesh

You've already added a face tracker, so now you can insert a face mesh. The face mesh is a 3D model that responds to facial expressions tracked by the face tracker. You'll apply materials to the face mesh, to change how it looks - creating the mask effect.

This time, instead of clicking Add Object like you did before:

  1. Right-click on the face tracker in the Scene Panel.
  2. Select Add.
  3. Choose the Face Mesh from the list.

What we've done here is add the face mesh as a child of a the face tracker.

When an object is the child of another object, it will take on any changes made to the object it's a child of - the parent object. It can also respond to the parent object. In this case, the face mesh responds to the movement detected by the face tracker.

You can tell if an object is a child of another by clicking the arrow next to the parent object in the Scene Panel. This will hide any of its children.

You can also drag and drop objects on to one another to create child-parent relationships.

Creating a material

To create a material for the face mesh, make sure it's selected in the Scene Panel.

On the right of the interface is the Inspector. This is where you can make changes to the object or asset you've selected in the Scene Panel - including creating materials for objects.

So, click the + next to Material to create a material for the face mesh.

You should see the material you created in the Assets Panel. It'll be called defaultMaterial0.

You can rename objects and assets in Spark AR Studio. It's a good idea to do this, to help stay organized as you build your project. All you need to do is right-click, and select rename. For this tutorial, rename the material mask_material.

Setting the Shader Type

Now go back over to the Inspector to edit the material.

First let's take a look at this option at the top - Shader Type. You can use this option to choose from several different built-in shaders in Spark AR Studio, to define the appearance of materials in your effects.

The mask that we're going to make will show the user's face behind it, so we're going to choose the Face Paint shader. So:

  1. Click the dropdown menu next to Shader Type.
  2. select Face Paint.

The face paint shader is the best option for mask effects where a texture is painted on top of the face. This is because the face paint shader preserves the luminance of the face, but removes the color. This means your own texture can be multiplied over it.

Applying the texture

Next let's apply the texture. Next to Texture:

  1. Select Choose File....
  2. Select moustache.

Alright, we've finished the mask! Now let's take a look at how to add the option to use an effect with a friend - by adding a second face tracker.

Adding a second face

To try this out, you can duplicate the face tracker you've just made. This will copy its children, too. Simply:

  1. Select the Face Tracker in the Scene Panel.
  2. Hold down control and v on your keyboard, then control and c.

Another face tracker and face mesh will be listed in the Scene Panel.

You'll need to make one edit in the Inspector panel. Make sure the second face tracker, faceTracker1, is selected in the Scene tab. Over in the Inspector panel, you'll see a property called Tracked face. Use this to assign a particular face to a particular face tracker.

It's set to Face 1 by default. Change it to Face 2, so it responds to the second face that appears in the effect. And there we go, an effect that can be used by two people. You might want to add a different texture to the material applied to the second face mesh, to add more variety to your effect.

Congratulations! You've finished an effect with face tracking!

Learn more

In this tutorial, you've learned to create a simple mask effect. Next, learn about: