Reactive Programming

Spark AR Studio uses reactive programming, a declarative programming model that uses asynchronous data streams. This guide will cover the benefits and use of reactive programming within Spark AR Studio.


Benefits of Reactive Programming

Spark AR Studio's implementation of reactive programming allows you to create relationships between objects, assets and values. This means that the engine doesn't have to execute JavaScript code every frame when performing common tasks such as animating content, looking for user input, or realigning a mask to a face.

Reactive programming is also compatible with visual programming, reducing the frequency of calls made into the scripting engine.


Transitioning from Imperative to Reactive Programming

Theory

Imperative programming is a model of programming that uses a sequence of statements to determine how to reach a certain goal. If you've scripted in an environment with the need for an update loop to update values, it's likely you've been using imperative programming.

Transitioning from imperative to reactive programming requires you to shift the way that you think, because of how values are propagated throughout your program.

For example, consider this code:

 y = 2x + 4; 

With imperative programming, the value of y is set when the expression is executed. If the value of x changes later in the program, y will remain the same, only updating if the expression is re-executed.

With reactive programming, we bind the value of y to the result of the equation. The value of y will then update automatically as the value of x changes, without having to re-execute the statement.

Practical Use

We can use a practical example to explore this difference further, updating the position of an object based on the rotation of a person's face.

In imperative programming we would need to write JavaScript to modify the object's values every frame.

// Imperative Pseudocode - THIS CODE WILL NOT COMPILE
// Function called every frame
function update() {
  myObject.x = face.rotation.y.currentValue;
}

However in reactive programming we bind the value to a signal and the object will update automatically when the face moves.

// Reactive Pseudocode - THIS CODE WILL NOT COMPILE
// The y rotation of the user's face is bound to the x position of the object once
myObject.x = face.rotation.y;

Here the updating happens in native code, as opposed to JavaScript, so it has minimal impact on application performance.


Signal Binding

One of the key ways in which reactive programming helps to improve performance is that it lets you treat values as signals. A signal is a special object containing a value that changes over time.

When signals are bound to variables or another object's properties, changes to the values are propagated in native code, eliminating the need for a context switch into the scripting engine.

The example below shows you how you can bind the signal from a source object - the user's face, to move a target object - a plane.

// Load in the required modules
const FaceTracking = require('FaceTracking');
const Scene = require('Scene');

// Locate a plane we've added to our scene
const myPlane = Scene.root.find('plane0');

// Bind the users's face rotation in the Y-axis to the X-position of our plane.
myPlane.transform.x = FaceTracking.face(0).cameraTransform.rotationY;

When the object at the left hand side of an assignment (the lvalue) is a Reactive object, this statement is not a simple assignment like in standard JavaScript, but rather a binding.

This means:

  • The rvalue should be a signal such as a ScalarSignal or StringSignal, rather than a simple scalar or string value.
  • Rather than assigning a value, you're binding the signal to that property. When the value underlying the signal is updated, so is the value of the property.

So rather than programming in the usual imperative style, where you use conditional logic to control the flow of data throughout a program and perform an assignment every time you need it, you're programming in a declarative style.

We can see this in action in the following example:

// Load in the required modules
const FaceTracking = require('FaceTracking');
const Scene = require('Scene');

// Locate a plane we've added to our scene
const plane = Scene.root.find('plane0');

// Set the plane to hidden if the mouth is open more than 50%
plane.hidden = FaceTracking.face(0).mouth.openness.gt(0.5);

The gt() method belongs to the ScalarSignal class and is used the same as the greater-than (>) sign.

We declare a signal path, binding a signal - the mouth openness, to a property - the hidden value of a plane, and let the framework do the work of shuttling data around for us.

We still have conditionals for flow control within these signal paths, but they are part of the signal path, not statements surrounding the signal path (as in the case of an imperative if-else or switch statement).

The example above would look something like this in imperative:

// Imperative Pseudocode - THIS CODE WILL NOT COMPILE
// Function called every frame
function update() {
  if(mouth.openness > 0.5) {
    plane.hidden = true;
  }
  else {
    plane.hidden = false;
  }
}

Signal Operators

As signals are special objects containing values that change over time, standard JavaScript operators such as +, * and - are not supported. Instead the Reactive module exposes methods for reactive programming that perform the same functionality.

// Load in the required modules
const FaceTracking = require('FaceTracking');
const Reactive = require('Reactive');

// Store a reference to the mouth openness of a detected face which returns a ScalarSignal
const mouthOpenness = FaceTracking.face(0).mouth.openness;

// Add 1 to the signal using the scalarSignal add method
const mouthOpennessPlusOne = mouthOpenness.add(1);

// Multiply the signal by 2 using the reactive mul method
const doubleMouthOpenness = Reactive.mul(mouthOpenness,2);

Some methods can be applied directly to the signal as mouthOpennessPlusOne does, or by using the Reactive module as doubleMouthOpenness does.

The StringSignal and BoolSignal pages cover the methods that can be applied to them for operators such as & and !.


Watching Signals

You can use the Diagnostics.watch() method to add specific signals to the watch view in Spark AR Studio, allowing you to see how their values change over time.

// Load in the required modules
const Diagnostics = require('Diagnostics');
const FaceTracking = require('FaceTracking');

// Add the mouth openness signal to the watch view
Diagnostics.watch("Mouth Openness - ", FaceTracking.face(0).mouth.openness);

The watch view appears in the top right of the console.




Converting Values to Signals

Numbers, strings and booleans are implicitly converted to constant signals when passed as function or property-setter arguments.

// Load in the required modules
const FaceTracking = require('FaceTracking');

// Numerical value of 0.1 converted to a ScalarSignal automatically
const mouthOpen = FaceTracking.face(0).mouth.openness.gt(0.1);

The Reactive.val() method, however, can be used to explicitly convert a primitive type to ScalarSignal, StringSignal or BoolSignal.

// Load in the required modules
const FaceTracking = require('FaceTracking');
const Reactive = require('Reactive');

// Convert the numerical value 0.1 to a ScalarSignal
const convertedValue = Reactive.val(0.1);

// Use the converted value
const mouthOpen = FaceTracking.face(0).mouth.openness.gt(convertedValue);

Converting Signals to Values

Obtaining the value of a StringSignal, ScalarSignal or BoolSignal at a specific point within a script can be done using the pinLastValue() method.

// Load in the required modules
const Diagnostics = require('Diagnostics');
const FaceTracking = require('FaceTracking');

// Get the value of mouth openness when this line of code is executed
const mouthOpennessValue = FaceTracking.face(0).mouth.openness.pinLastValue();

// Log the value
Diagnostics.log(mouthOpennessValue);

The same values can also be obtained using the subscribeWithSnapshot() method if needed during a callback function.

// Load in the required modules
const Diagnostics = require('Diagnostics');
const FaceTracking = require('FaceTracking');
const TouchGestures = require('TouchGestures');

// Subscribe to tap gestures
TouchGestures.onTap().subscribeWithSnapshot( {
    // Get the value of mouth openness when the tap gesture is detected
    'mouthOpennessValue' : FaceTracking.face(0).mouth.openness
}, function (gesture, snapshot) {
    // Log the value from the snapshot
    Diagnostics.log(snapshot.mouthOpennessValue);
});

EventSource Subscriptions

Some methods return an EventSource that needs to be subscribed to in order to execute code when a particular event occurs. The methods provided by the TouchGestures module are a typical example of this.

// Load in the required modules
const Diagnostics = require('Diagnostics');
const TouchGestures = require('TouchGestures');

// Subscribe to tap gestures
TouchGestures.onTap().subscribe(function (gesture) {
    // Log a message to the console when a tap is detected
    Diagnostics.log('tap gesture detected');
});

StringSignals, ScalarSignals and BoolSignals will return an EventSource when the monitor() method is called on them. This, however, is discouraged as it increases the number of scripting engine callbacks and can often be avoided with signal binding.

// Load in the required modules
const FaceTracking = require('FaceTracking');
const Scene = require('Scene');
const TouchGestures = require('TouchGestures');

// Locate a plane we've added to our scene
const plane = Scene.root.find('plane0');

//==============================================================================
// Hide the plane when the mouth oppenness value is greater than 0.5
//==============================================================================

// Avoidable use of subscribe 
FaceTracking.face(0).mouth.openness.monitor().subscribe(function(event) {
  if(event.newValue > 0.5) {
    plane.hidden = true;
  } else {
    plane.hidden = false;
  }
});

// Good use of signal binding
plane.hidden = FaceTracking.face(0).mouth.openness.gt(0.5);

You can unsubscribe from an EventSource by storing the subscription in a variable and then calling the unsubscribe() method on it.

// Load in the required modules
const Diagnostics = require('Diagnostics');
const TouchGestures = require('TouchGestures');

// Store the subscription in a variable
var subscription = TouchGestures.onTap().subscribe(function (gesture) {
    Diagnostics.log('tap gesture detected');
});

// Unsubscribe from tap gestures
subscription.unsubscribe();

Next Steps

Now you've learned about reactive programming its time to see it in action with our tutorial on how to create an effect with Scripting.