Visual Programming

Visual programming lets you add animation, logic and interactivity to your project. In visual programming, parts of your scene are represented as patches and you can connect them to build a graph. You do this in the Patch Editor. Once connected, this graph visualizes the animations and interactions in your project.

If you don't know much about scripting, you can use visual programming to create interactive effects similar to those built by somebody who knows Javascript well.

Patch Editor

You can open the Patch Editor from the menu bar.

To do this:

  1. Click View
  2. Select Show Patch Editor

The Patch Editor will also open automatically when you insert patches from the Actions section in the Inspector panel.


Patches are building blocks. Each one has a specific function and can pass and receive information to and from other patches.

There are 3 kinds of patches in AR Studio, as demonstrated by this example:


In the example above, the face tracker is a producer patch. Producers are always purple and they represent a scene object that produces values, like this face tracker.


The plane visibility patch is a consumer and consumer patches are always blue. This patch will drive the visibility of that plane, meaning that you can use it to determine when the patch should be shown in the scene and when it should be hidden.

List of producer and consumer patches

Name Description

Device Info

Access information about the mobile device.

Camera Info

Access information about the mobile device camera.


The Mouth Open patch takes the information from the producer patch, manipulates it and sends the results to the consumer patch. Patches like this are always gray.

List of intermediary patches

Name Description

Screen Tap

Capture a tap anywhere on the screen.

Object Tap

Capture a tap on an object in the scene.

Head Rotation

Capture head rotation, using data from a face tracker in the scene.

Head Nod

Capture a head nod, using data from a face tracker in the scene.

Head Shake

Capture a head shake, using data from a face tracker in the scene.

Mouth Open

Capture an opened mouth, using data from a face tracker in the scene.


Capture blinked eyes, using data from a face tracker in the scene.

Eyebrows Raised

Capture raised eyebrows, using data from a face tracker in the scene.

Eyebrows Lowered

Capture lowered eyebrows, using data from a face tracker in the scene.

Left Eye Closed

Determine when the left eye is closed, using data from a face tracker in the scene.

Right Eye Closed

Determine when the right eye is closed, using data from a face tracker in the scene.


Detect a smile, using data from a face tracker in the scene.


Animate an element in your scene.

Loop Animation

Animate an element in your scene repeatedly.


Convert a progress value to a value between a new range defined by the start and end values.

Keyframe Transition

Convert a progress value to a value between a new range defined by the start and end values. Progress output is used to chain multiple transitions.

Value Progress

Map scalar values to progress values.


Flip a boolean from true to false, or from false to true.


Check if 2 booleans are true together.


Check if either one of 2 booleans are true.


Check if 2 numbers are equal.

Greater Than or Equal To

Check if one input value is greater than or equal to another.

Greater Than

Check if one value is greater than another.

Less Than or Equal To

Check if one value is less than or equal to another.

Less Than

Check if one value is less than another.


Add 2 values together.


Subtract one value from another value.


Multiply 2 values.


Divide one value by another value.


Calculate the remainder when 2 values are divided.


Calculate the angle of a triangle given its X and Y dimensions.

Point Pack

Create a point with 3 dimensions (X, Y and Z).

Point Unpack

Turn a point into its individual X, Y and Z values.


Track the number of seconds that have elapsed since the effect started to run.

2D Point Unpack

Turn a 2D point into its individual values. For example, convert Size to Width and Height, or a 2D point to X and Y.


Add comments and notes to a group of patches to organize them or describe their purpose. Select the patches and right-click one to add a comment that fits around the patches. Double-click the name or press return on your keyboard to edit the comment. Right-click the patch to change the color. Drag the bottom right corner to resize the comment.


Trigger a pulse whenever a state changes from on to off or vice versa.


Toggle between boolean outputs for every input event.

Option Switch

Control up to 5 states using an index (0, 1, 2...). This is useful for tracking a state that is one of many options. By default, the output is 0.

Option Picker

Pick one of the options to output based on the option selected. Often used with Option Switch to control which option is picked.


Create a counter that starts at 0, where the value can be increased or decreased gradually, or set to a specific number.

Insets Unpack

Turn insets into individual top, bottom, left and right values.

Inserting Patches

Inserting producers

To insert a producer patch:

  1. Go to the Scene tab and select your object
  2. Click + next to Actions in the Inspector panel
  3. Select the patch or combination of patches you want from the menu

You can also drag the object from the Scene tab and drop it in the Patch Editor directly.

Inserting consumers

To insert a consumer patch:

  1. Go to the Scene tab and select the object you want to affect
  2. Go to Properties in the Inspector panel
  3. Click the dot next to the property you want to drive

If you click on a particular property in the Inspector panel, it will focus on the relevant consumer patch in the graph.

Inserting intermediaries

To insert an intermediary patch:

  1. Double-click anywhere in the Patch Editor
  2. Find the patch you want to insert from the menu
  3. Click Insert

Tip: You can rename a patch by double-clicking its name. This can help keep you organized in more complicated work.

Connecting Patches

To connect 2 or more patches together, you need to use the ports. The ports are the small dots next to some of the properties on each patch.

Connections made between patches are read from left to right. You would always connect an output which is on the right side of a patch to an input which is on the left side of another patch. An output can have multiple outgoing connections, but an input can only have one incoming connection. This means that the result of an output can be propagated to one or more patches.

To connect 2 patches together:

  1. Click and hold the first port
  2. Drag the connector across to the second port

You should see a blue line appear, which will turn green once it has found a connection. If you let go, you should see a connection made between your patches.

Data Types

Boolean signal

At every point in time, a boolean signal is either true or false. For example, your mouth is either open or closed. It can't be both at the same time and it is always one or the other.


A pulse is an event that occurs at some point in time. It could happen multiple times, but it's a discrete event, as opposed to a boolean signal, which is continuous. For example, a head shake or a head nod.


A point is a three-dimensional signal that can represent a rotation or scale or position.


A scalar is a one-dimensional, continuous value that changes over time. For example, how open your mouth is.


Progress is a scalar signal that's restricted to the 0-1 range. It could be used to drive an animation.


This data type represents objects like face trackers and scene objects.

Testing and Debugging

To test the patch graph you've created, click Run in the Toolbar. If you receive an error message, it could be for one of these reasons:

  • Some ports won't work with others. Please make sure that you connect ports with the same data type. For example, ports with small empty squares next to them can be connected to each other and represent boolean data types. Ports with small squares with circles in can be connected to each other and represent pulse data types.
  • The patch graph becomes the source of truth for object transformations. Once you insert a consumer patch for a particular property, won't be able to adjust that property in the Inspector panel anymore.


Moving scene objects with the face tracker

The simplest thing you can do is bind the movement of a scene object to a face tracker, so that the object moves with the face. To do this, insert a face tracker patch and then go to your object's properties and click the dot next to Rotation. You should now see a face tracker and object patch in the Patch Editor.

Click the Rotation port on the face tracker patch and drag to the port on the object patch. Once the patches are connected, click Run to see it in action. Whenever your head moves, your object should move with it.

Using the face tracker to control animation

You can set an animation to begin and end when triggered by specific actions on a face. In the example below, the object is set to appear if someone opens their mouth. Then, if they lean their head to the right, the object changes position. If they close their mouth, the object disappears.

By connecting the Face Tracker, Head Rotation and Mouth Open patches via the Tracking Data ports, we're telling AR Studio that it should be using the information from the face tracker to look for an open mouth or a leaned head.

Using boolean signals to start animations

Both Mouth Open and Head Rotation are boolean signals, which means they it can either be happening or not happening. Patches like Mouth Openness have scalar signals.

If you want to use a boolean signal to start an animation, you'll need to use the Pulse patch to transform the signal into a discrete event.

Using logic

You can use logic to make your animation react to a specific set of conditions. For example, you can create a patch graph that makes a ball drop from top to bottom if the head is tilted in either direction. To do this, us the Or patch to indicate that the animation should occur if either or the other action happens.

Here, the Or patch is placed after Head Rotation so that it can take both the inputs from the face tracker and trigger the movement if the head is leaned left or right.

Recreating a tutorial

In the Basics of Scripting tutorial, we explain how to make pizza fly into your mouth when you open it. You can create this effect with visual programming.