There may be differences between your version of AR Studio and this tutorial because the product is currently in beta and we update it regularly.

Using the Patch Editor: Eyebrows Raised

Something Went Wrong
We're having trouble playing this video.

In previous tutorials you learned how to use the Patch Editor, our visual programming environment, to add interactivity, logic and animation to your project without the use of JavaScript. In this tutorial, you'll learn how to use more advanced patches, like Eyebrows Raised, Transition, Pulse and Animation, to build an effect that responds to facial movements.

Please download the sample content to follow along.

If you open the finished effect in the sample content folder, you'll see you're going to build an effect with bug eyes that spin when you raise your eyebrows.

Opening the Patch Editor

To get started, open the unfinished version in the sample content folder. You should already see the eyes in the Viewport and Simulator, and listed in the Scene tab. If you click Run in the Toolbar, you'll see that nothing happens. To make the eyes respond to facial movements, you'll need to animate them.

First, open the Patch Editor by clicking View in the menu bar and selecting Show Patch Editor. You should now see it below the Viewport.

Setting Up the Trigger

For this project, we want the eyes to move when your eyebrows change position. To detect this change, you'll need to use the face tracker. To add this patch, select the face tracker in the Scene tab and drag it into the Patch Editor. As you can see, the face tracker patch is a group with three patches inside. For this effect, we need the facetracker1 patch and the Face port.

Now that you have the face tracker information, you can start to build a graph that triggers the eye rotation animation when eyebrows are raised. To do this, double-click anywhere in the Patch Editor, select Eyebrows Raised from the menu and click Insert. Eyebrows Raised lets you capture data about specific parts of the face. You could also use other patches to do this, like the Mouth Open patch. These patches must be connected to a face tracker patch in order to work properly, so click the port next to Face on the facetracker0 patch and drag it to the Face port on the Eyebrows Raised patch. You should now see a connection between the two patches.

The next step is build the trigger for the animation and for this you need the Pulse patch. So, double-click in the Patch Editor, select Pulse from the menu and click Insert. Then, connect the unnamed port on the Eyebrows Raised patch to the On/Off port on the Pulse patch.

We want an animation to start when eyebrows are raised, so double-click in the Patch Editor and select Animation, then click Insert. Connect the Turned On port on the Pulse patch to the Play port on the Animation patch. This tells the effect to start the animation when the input is on, in this case, when the eyebrows are raised. Once you've done that, connect the Turned On port on the Pulse patch to the Reset port on the Animation patch to reset the animation every time the eyebrows are raised.

Making the Eyes Rotate

Now that you've set up the trigger, you can start to build the eye rotation animations. You'll need to do this separately, starting with the right eye. First, insert a Transition patch, then change the value next to End Z to 180 and the Curve to Quadratic In-Out. Once you've done that, connect the Progress port on the Animation patch to the Progress port on the Transition patch. This will make the eye spin around in a circle.

To connect it to the eye, you'll need to make the values from the patches you've already inserted affect the rotation of the eye. To do this, select right_eyeA0 in the Scene tab and click the circle next to Rotation in the Inspector panel. You should now see a right_eyeA0 rotation patch in the Patch Editor. Now, connect the Value port on the Transition patch to the right_eyeA0 patch you just inserted.

To test what you've built, click Run in the Toolbar and raise your eyebrows. When your eyebrows are raised, you should see the right eye rotate. Once you've seen it, click Stop in the Toolbar.

Next, you're going to do the same to the left eye, so insert another Transition patch, change the value next to End Z to 180 and the Curve to Quadratic In-Out. Then, connect the Animation patch to the Transition patch. Now, select left_eyeA0 in the Scene tab, and insert a rotation patch. Make sure you connect the Transition patch to the patch you inserted.

Your effect should now be ready! To check that it works, click Run in the Toolbar and raise your eyebrows. You should now see both eyes rotate.

Congratulations, you created an effect that responds to facial movements!

Recap

To recap, in this video you learned how to trigger animation based on facial movements, using face landmark patches.