Audio

With AR Studio's audio tools, you can add ambient and scripted sounds to your projects.

Ambient sounds are looping audio files that run constantly through the lifetime of an effect. You might use these to create a specific mood, setting or vibe.

Scripted sounds allow you to control when your audio plays. You can either script these sounds or create them using visual programming. You can trigger sounds based on:

  • Time elapsed
  • Interaction with the scene
  • Specific animations

Prepping Assets

AR Studio allows you to do some sound mixing, but for the most part, audio clips should be digitally mastered before you import them. This helps to ensure that none of the imported sounds dominate the experience.

Supported file formats

AR Studio supports mono M4A files. We require that you only use files with a 44.1kHz sample rate and 16-bit-depth resolution in order to constrain the size of an effect and preserve its audio quality. You should be able to achieve this conversion through a DAW, but you can also use a program such as iTunes to accomplish this task.

If you are converting ambient or music beds from stereo tracks to mono, it's best to convert to summed mono, rather than just dropping one of the tracks. If you do the latter, you're likely to lose elements of the audio file. In general, try converting your files to the lowest kbps possible, while checking that they still sound good through the device. We don't recommend going lower than 64kbps when converting to m4as.

Creating ambient audio assets

When creating your ambient audio assets, it's important to consider:

  • What you want people to use them for. Do you want the viewer to dance along with their friends? Are you using your sounds to create a story?
  • That your effect will be experienced on a mobile device, so the assets you design will most likely be played back through speaker sets on Android and iOS devices. Knowing the general range of frequencies that these devices can reproduce will help determine the levels and types of sound that would perform best in these spaces. For example, most mobile devices can't reproduce low frequencies well, so a booming bass track that sounds great through your computer speakers probably won't have the same impact through an iPhone.
  • Whether you want the file to loop seamlessly or not. We recommend a small, looping ambient bed. If you choose not to use a seamless looping ambient bed, make sure that your ambient bed runs the max capture length, which for the Facebook camera is 20 seconds.

Creating small, seamless, looping ambient audio assets

If you create a smaller, seamlessly looping ambient sound asset, you'll need to edit the file in a digital audio workstation, such as Pro Tools, and make the edit cut on the zero crossing at the sample level. This edit is essential to creating a seamless loop, so make sure you cut it so that the wavelength at the end of the file and the wavelength at the beginning of the file flow continuously into each other, without a fade applied to either end.

Below are two screen shots: on the left, the wavelength has been cut correctly and no pop will occur, on the right, the wavelength has been cut incorrectly and this will result in a pop when your ambient bed ends and begins again.

When mixing ambient beds, we recommend that you record audio from a couple of mobile devices and import those files to mix against in your digital audio workstation. An effective ambient bed, much like environmental sounds or a score in a film, should sit well below the production audio (or a person's voice in a capture video), so creating some dummy audio captures will help you achieve the proper levels. You'll be able to make adjustments in AR Studio (only in lowering dB levels, currently), so testing on a mobile device and mixing levels before implementation will save you a lot of time on the implementation stage.

Creating scripted audio assets

Adding scripted sounds to effects is a good way to add depth to interactive moments. If you had an effect that made the hat on your head fall off and a bird fly out when you opened your mouth, the effect would be more of an immersive experience with the sound of the wind and of the bird flying away.

With scripted audio assets, it's important to consider:

  • The mobile device the person will use to view your effect and environment they will be in.
  • The frequency response.
  • The audio content captured by the microphone on a mobile device.
  • Repetitive tolerance. Audio adds depth, but walks a fine line between effective and destructive. Trigger sounds with caution.

You can use a one shot - a trigger that plays only once all the way through - to trigger a scripted sound.

If you plan to couple ambient beds with scripted effects, make sure to mix both assets together and against the production audio references that you capture from your test mobile device. You'll be able to adjust scripted audio assets in AR Studio just as you can adjust ambient beds, but it's best to get your levels set before importing.

Audio Source

The Audio Source is a speaker in the scene to which you can connect audio clips. In its default state it won't render any sound in your project, but you can configure it to represent audio effects. You can have multiple audio sources in a scene.

Properties

Enabled

Uncheck this box to stop the audio source from being rendered in the scene.

Audio Source

Add an audio clip from your computer, select one from the Assets panel or select the Microphone.

Volume

Adjust the volume level of your audio source in the scene.

Ambient

This appears when an audio clip is added in Audio Source. Check Ambient to make your audio source loop continuously.

Effect

This appears when Microphone is selected in Audio Source. Use built in Digital Signal Processing (DSP), more commonly known as 'voice changing effects' to configure a voice-changer effect.

Transformations

Edit the position of the audio source in your scene.

Enable For

Choose the camera or cameras on a mobile device in which the audio should play.

Audio Capability

This capability is added automatically when you insert an audio source into your scene. It lets you add audio to your project.

Using Audio

To add audio to your project:

  1. Click Insert > Audio Source > Insert
  2. Select Audio Source in the Scene tab and go to its Properties
  3. Click next to Audio Source and select Add Audio Clip from the menu
  4. Open the audio file from your computer
  5. Check the box next to Ambient if you want the sound to loop continuously

If you'd rather have a scripted sound, you can use the Audio module.

Testing

During design

It's always a good idea to test effects through a device during the creation process. There are mirroring programs you can leverage to hear the audio through a connected device, or you can always use cloud storage to test mixes on device. Testing often in the design stage will make the refining process at the implementation stage much easier!

After building

Once you've added audio to your project, make sure you test it. Mobile devices can vary widely in their speaker sets and microphones, which can vastly impact the performance of audio effects. Due to the range of mobile devices and environments that your effect could be used in, it's important to try to experience as many different variations of the effect as possible before uploading it to the Facebook camera.

When testing, you'll need to check:

  • If you can hear the sounds you've added
  • Whether the ambient sound plays during preview and playback or not
  • Whether your scripted sound plays when it's supposed to or not
  • Whether there's a pop sound or a gap when the sound loops or not
  • If the ambient sound sits below your voice in the mix when you capture some dialog
  • If the scripted sound is startling or repetitive

Make sure you test your audio in various environments, like outside, inside, with people and without, and on multiple mobile devices.

Expected Audio Behavior

Pre-Capture

  • Audio plays as intended: ambient loops and triggered SFX are both audible
  • DSP effects cannot be previewed because these require microphone input during capture

Capture

  • Audio elements are muted during recording

Device microphones will pick up anything played from the phone's speakers, so playing audio over the device's speaker during recording results in drastically lower quality audio. Doing so would re-record any audio you've added to an effect and would potentially overtake the microphone input, leading to a poor quality mix with doubled and out-of-sync audio.

This would also interfere with any use of DSP effects, since any audio picked up by the device's microphone would be processed and modulated unintentionally.

Muting audio elements during recording allows for DSP use of microphone input to be processed independently of any additional effect audio, flexible, high quality processing options and creates audio quality on the same level as standard HD videos.

Post-Capture

  • Audio elements are re-added to the final video and can be heard in full. This allows full spectrum, high-quality audio to accompany a clean microphone input recording
  • Ambient loops will restart and play from the beginning of the recorded capture
  • Only audio that is triggered during recording will be added to the final video. DSP effects will be applied to the microphone input capture and will be audible for the user to review