Visual programming lets you add animation, logic and interactivity to your project. In visual programming, parts of your scene are represented as patches and you can connect them to build a graph. You do this in the Patch Editor. Once connected, this graph visualizes the animations and interactions in your project.
You can open the Patch Editor from the menu bar.
To do this:
The Patch Editor will also open automatically when you insert patches from the Actions section in the Inspector panel.
Patches are building blocks. Each one has a specific function and can pass and receive information to and from other patches.
There are 3 kinds of patches in AR Studio, as demonstrated by this example:
In the example above, the face tracker is a producer patch. Producers are always purple and they represent a scene object that produces values, like this face tracker.
The plane visibility patch is a consumer and consumer patches are always blue. This patch will drive the visibility of that plane, meaning that you can use it to determine when the patch should be shown in the scene and when it should be hidden.
Access information about the mobile device.
Access information about the mobile device camera.
The Mouth Open patch takes the information from the producer patch, manipulates it and sends the results to the consumer patch. Patches like this are always gray.
Capture a tap anywhere on the screen.
Capture a tap on an object in the scene.
Capture head rotation, using data from a face tracker in the scene.
Capture a head nod, using data from a face tracker in the scene.
Capture a head shake, using data from a face tracker in the scene.
Capture an opened mouth, using data from a face tracker in the scene.
Capture blinked eyes, using data from a face tracker in the scene.
Capture raised eyebrows, using data from a face tracker in the scene.
Capture lowered eyebrows, using data from a face tracker in the scene.
Left Eye Closed
Determine when the left eye is closed, using data from a face tracker in the scene.
Right Eye Closed
Determine when the right eye is closed, using data from a face tracker in the scene.
Detect a smile, using data from a face tracker in the scene.
Animate an element in your scene.
Animate an element in your scene repeatedly.
Convert a progress value to a value between a new range defined by the start and end values.
Convert a progress value to a value between a new range defined by the start and end values. Progress output is used to chain multiple transitions.
Map scalar values to progress values.
Flip a boolean from true to false, or from false to true.
Check if 2 booleans are true together.
Check if either one of 2 booleans are true.
Check if 2 numbers are equal.
Greater Than or Equal To
Check if one input value is greater than or equal to another.
Check if one value is greater than another.
Less Than or Equal To
Check if one value is less than or equal to another.
Check if one value is less than another.
Add 2 values together.
Subtract one value from another value.
Multiply 2 values.
Divide one value by another value.
Calculate the remainder when 2 values are divided.
Calculate the angle of a triangle given its X and Y dimensions.
Create a point with 3 dimensions (X, Y and Z).
Turn a point into its individual X, Y and Z values.
Track the number of seconds that have elapsed since the effect started to run.
2D Point Unpack
Turn a 2D point into its individual values. For example, convert Size to Width and Height, or a 2D point to X and Y.
Add comments and notes to a group of patches to organize them or describe their purpose. Select the patches and right-click one to add a comment that fits around the patches. Double-click the name or press return on your keyboard to edit the comment. Right-click the patch to change the color. Drag the bottom right corner to resize the comment.
Trigger a pulse whenever a state changes from on to off or vice versa.
Toggle between boolean outputs for every input event.
Control up to 5 states using an index (0, 1, 2...). This is useful for tracking a state that is one of many options. By default, the output is 0.
Pick one of the options to output based on the option selected. Often used with Option Switch to control which option is picked.
Create a counter that starts at 0, where the value can be increased or decreased gradually, or set to a specific number.
Turn insets into individual top, bottom, left and right values.
To insert a producer patch:
You can also drag the object from the Scene tab and drop it in the Patch Editor directly.
To insert a consumer patch:
If you click on a particular property in the Inspector panel, it will focus on the relevant consumer patch in the graph.
To insert an intermediary patch:
Tip: You can rename a patch by double-clicking its name. This can help keep you organized in more complicated work.
To connect 2 or more patches together, you need to use the ports. The ports are the small dots next to some of the properties on each patch.
Connections made between patches are read from left to right. You would always connect an output which is on the right side of a patch to an input which is on the left side of another patch. An output can have multiple outgoing connections, but an input can only have one incoming connection. This means that the result of an output can be propagated to one or more patches.
To connect 2 patches together:
You should see a blue line appear, which will turn green once it has found a connection. If you let go, you should see a connection made between your patches.
At every point in time, a boolean signal is either true or false. For example, your mouth is either open or closed. It can't be both at the same time and it is always one or the other.
A pulse is an event that occurs at some point in time. It could happen multiple times, but it's a discrete event, as opposed to a boolean signal, which is continuous. For example, a head shake or a head nod.
A point is a three-dimensional signal that can represent a rotation or scale or position.
A scalar is a one-dimensional, continuous value that changes over time. For example, how open your mouth is.
Progress is a scalar signal that's restricted to the 0-1 range. It could be used to drive an animation.
This data type represents objects like face trackers and scene objects.
To test the patch graph you've created, click Run in the Toolbar. If you receive an error message, it could be for one of these reasons:
The simplest thing you can do is bind the movement of a scene object to a face tracker, so that the object moves with the face. To do this, insert a face tracker patch and then go to your object's properties and click the dot next to Rotation. You should now see a face tracker and object patch in the Patch Editor.
Click the Rotation port on the face tracker patch and drag to the port on the object patch. Once the patches are connected, click Run to see it in action. Whenever your head moves, your object should move with it.
You can set an animation to begin and end when triggered by specific actions on a face. In the example below, the object is set to appear if someone opens their mouth. Then, if they lean their head to the right, the object changes position. If they close their mouth, the object disappears.
By connecting the Face Tracker, Head Rotation and Mouth Open patches via the Tracking Data ports, we're telling AR Studio that it should be using the information from the face tracker to look for an open mouth or a leaned head.
Both Mouth Open and Head Rotation are boolean signals, which means they it can either be happening or not happening. Patches like Mouth Openness have scalar signals.
If you want to use a boolean signal to start an animation, you'll need to use the Pulse patch to transform the signal into a discrete event.
You can use logic to make your animation react to a specific set of conditions. For example, you can create a patch graph that makes a ball drop from top to bottom if the head is tilted in either direction. To do this, us the Or patch to indicate that the animation should occur if either or the other action happens.
Here, the Or patch is placed after Head Rotation so that it can take both the inputs from the face tracker and trigger the movement if the head is leaned left or right.
In the Basics of Scripting tutorial, we explain how to make pizza fly into your mouth when you open it. You can create this effect with visual programming.