Create and run an experiment to measure your Facebook campaign's efficiency. Determine what ads strategy drives the most business impact. See Ad Study, Reference.
When you create a lift study, you create a randomized test group of people that see your ads and control group who don't see your ads.
You can securely share conversion data from your ad campaign with Facebook using Facebook pixels, or App Events. Facebook determines the increased conversions generated from your campaign. We compare the number of conversions, people converting, and available sales revenue between test and control groups.
Set up a study with one or more groups, called cells. When you set up your study, Facebook randomizes the audience for your ads and assigns people to either the test or control group. After you run a study, Facebook calculates the difference between the test groups and control groups so that you evaluate the impact of your Facebook ads towards business goals.
To set up a study, make a POST
call:
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
You can set up a study with a single test group to see how Facebook ads lead to additional business. You can also set up a study with multiple test groups, which lets you determine what advertising approach works best for your audience.
Example - Set up a lift study with one test group
curl \
-F 'name="new study"' \
-F 'description="description of my study"' \
-F 'start_time=1435622400' \
-F 'end_time=1436918400' \
-F 'cooldown_start_time=1433116800' \
-F 'observation_end_time=1438300800' \
-F 'viewers=[<USER_ID1>, <USER_ID2>]' \
-F 'type=LIFT' \
-F 'cells=[{name:"test group",description:"description of my test group",treatment_percentage:90,control_percentage:10,adaccounts:[<ACCOUNT_ID1>,<ACCOUNT_ID2>]}]' \
-F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]' \
-F 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
To create a new study, provide the following:
Parameter | Description |
---|---|
| Name of study. |
| Brief description of the study's purpose. |
| Deprecated. Facebook still delivers during the time between |
| Start time of campaign active period. Study start time must be in the future. |
| End time of campaign active period. |
| End of your post-campaign quiet period which started when the study period ended. We attribute conversions during this period to your study. We still deliver your ads, but our systems stop creating new study audiences. If you don't need a quiet period for your study, set this to |
| Cells in study that define test and control groups. |
| Objectives of the study. See Defining Study Objective. |
| Share this study to a list of Facebook user IDs. |
| For Conversion Lift, the type should be |
RESTRICTIONS -
Once the study starts, you cannot update start_time
and treatment_percentage
of the cells. You also cannot remove the associated objects, such as adaccounts
or campaigns
, of the test groups. You can still update the end_time
and observation_end_time
to a future time if the study has not yet ended, and add new associated objects to test groups.
To run Reach and Frequency in conjunction with Lift measurement, you must set up a Lift study first and make sure the duration of the Reach and Frequency is within the duration of the Lift study.
To begin, determine how many people recieve your ads and how many people do not. You must create a test group when you set up the study; pass a list of JSON objects in cells
under ad_studies
. See Ad Study Cell, Reference. A test group contains the following information.
Parameter | Description |
---|---|
| Name of test group. |
| Brief description of test group. |
| Defines the people who receive your ads. |
| Defines a holdout percentage of the people who will not see ads. Treatment plus control percentages must equal 100. |
| List of ad entities, such as |
Example - Read test groups in a study
curl -G \
-d 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<STUDY_ID>/cells
Example - Update or modify cell information and treatment and control percentages by providing the cell ID in cells
curl \
-F 'cells=[{id:<CELL_ID>,treatment_percentage:80,control_percentage:20}]' \
-F 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<STUDY_ID>
Example - Read all the studies that you created at ad_studies
for your business
curl -G \
-d 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
You can also see all studies associated with your ad account by making a GET
request at {ad-account-ID/include_all_studies=true}
with your access token.
Set up a study with multiple test groups of Facebook users. This helps measure incremental impact of different Facebook strategies on business goals, such as using different ads targeting options. To set up a study with multiple test groups, provide a list of test groups in cells
.
curl \
-F 'name="new study"' \
-F 'description="description of my study"' \
-F 'start_time=1435622400' \
-F 'end_time=1436918400' \
-F 'cooldown_start_time=1433116800' \
-F 'observation_end_time=1438300800' \
-F 'viewers=[<USER_ID1>, <USER_ID2>]' \
-F 'type=LIFT' \
-F 'cells=[{name:"group A",description:"description of group A",treatment_percentage:50,control_percentage:20,campaigns:[<CAMPAIGN_ID1>]},{name:"group B",description:"description of group B",treatment_percentage:20,control_percentage:10,campaigns:[<CAMPAIGN_ID2>]}]' \
-F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]' \
-F 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
control_percentage
determines the holdout for each test group respective to the total population. For example, you have a study with two test groups: group A is 50% treatment with 20% control and group B is 20% treament with 10% control. This results in ~28.6%, or 20%/70% of the population in group A, to be control users and ~33.3%, or 10%/30% of the population in group B, to be control users.
The sum of treatment and control percentages across test groups normally should equal 100. However, it can be less than 100 for some specific use cases. For example, when you have three test groups that are split evenly at 33%.
You can update, add, and remove test groups in a study.
cells
when you update the study:curl \
-F 'cells=[{id:<CELL_ID1>,treatment_percentage:60,control_percentage:10},{name:"group C",description:"replacing group B",treatment_percentage:25,control_percentage:5,campaigns:[<CAMPAIGN_ID3>]}]' \
-F 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<STUDY_ID>
Define advertising objectives you want to measure and how you pass conversion data to Facebook. A lift study requires at least one objective. You cannot modify objectives after the study starts running. See Ad Study Objective, Reference.
Example - Create and add the MAI
objective to a study
curl \
-F 'name="new study"' \
-F 'description="description of my study"' \
-F 'start_time=1435622400' \
-F 'end_time=1436918400' \
-F 'cooldown_start_time=1433116800' \
-F 'observation_end_time=1438300800' \
-F 'viewers=[<USER_ID1>, <USER_ID2>]' \
-F 'type=LIFT' \
-F 'cells=[{name:"test group",description:"description of my test group",treatment_percentage:90,control_percentage:10,adaccounts:[<ACCOUNT_ID1>,<ACCOUNT_ID2>]}]' \
-F 'objectives=[{name:"new objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID>}]}]' \
-F 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
Name | Description | Measurement Sources |
---|---|---|
| Measure the lift in purchases. The reporting will contain dollar amount. | Conversion Pixels, Facebook Pixels, Mobile App, Offline Event Sets, and Custom Conversions |
| Measures the lift in non-purchase conversions. | Conversion Pixels, Facebook Pixels, and Mobile App |
| Measure the lift in mobile application installs. | Mobile App |
If you use SALES
and NONSALES
and use Facebook pixel or Mobile App as event sources, you must provide a list of the event names that you want to capture for the objective. Facebook can then report results based on these specific conversion events.
Measurement Source | Event Names |
---|---|
Facebook Pixel |
|
Mobile App |
|
Create an objective by passing a list of JSON objects objectives
when you create a new study. Objectives contain the following information:
Parameter | Description |
---|---|
| Name of the objective. |
| A boolean specifying that this is your primary advertising objective. A study can only have one primary objective. |
| Objective value of |
| List of Facebook pixel IDs along with the relevant list of |
| List of your mobile apps including relevant |
| List of Conversion pixel IDs, if applicable. |
| List of Offline Event set IDs if applicable. Currently, we don't support event breakdowns for Offline Conversion. |
| List of Custom Conversion IDs, if applicable. |
You can also have multiple objectives per study. The result will be aggregated based on objectives. Below is an example of a study with multiple objectives—the MAI
objective as seen previously and another SALES
objective with Facebook pixel and application as measurement sources.
curl \
-F 'name="another study"' \
-F 'description="description of another study"' \
-F 'start_time=1435622400' \
-F 'end_time=1436918400' \
-F 'cooldown_start_time=1433116800' \
-F 'observation_end_time=1438300800' \
-F 'viewers=[<USER_ID1>, <USER_ID2>]' \
-F 'type=LIFT' \
-F 'cells=[{name:"test group",description:"description of my test group",treatment_percentage:90,control_percentage:10,adaccounts:[<ACCOUNT_ID1>,<ACCOUNT_ID2>]}]' \
-F 'objectives=[{name:"MAI objective",is_primary:true,type:"MAI",applications:[{id:<APP_ID1>},{id:<APP_ID2>}]},{name:"SALES objective",type:"SALES",applications:[{id:<APP_ID3>,event_names:["fb_mobile_purchase"]}],adspixels:[{id:<FB_PIXEL_ID>,event_names:["fb_pixel_purchase","fb_pixel_lead"]}]}]' \
-F 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<BUSINESS_ID>/ad_studies
You can update, add, and remove objectives in a study by doing so at the study level similar to modifying test groups. To update an existing objective, refer to its ID in the objectives
object. To add a new objective, provide a new objective object. To remove an objective, simply omit it from the objectives
parameter when you update it.
Example - Update an objective's applications
measurement sources and remove its adspixels
measurement sources
curl \
-F 'objectives=[{id:<OBJECTIVE_ID>,name:"new objective name",applications:[{id:<APP_ID>}],adspixels:[]}]' \
-F 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<STUDY_ID>
Example - Read objectives for a study
curl -G \
-d 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<STUDY_ID>/objectives
A study's objectives are defined during the study setup. See the setup guide on how to define your study's objectives
You can read the objectives that were created for a study by making a GET
call to the study's objectives
edge.
curl -G \
-d 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<STUDY_ID>/objectives
For more details on objectives, refer to the Ad Study Objective reference documentation
To retrieve results for an objective, you can make a GET
call to the objective node by specifying results
in the fields parameter. last_updated_results
field also tells you when the results data was last updated.
curl -G \
-d 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<STUDY_OBJECTIVE_ID>? fields=results,last_updated_results
The resulting data is a JSON object, containing strings with the following default fields:
Name | Description |
---|---|
|
|
| The lower bound of the 90% symmetrical credible interval for |
| The upper bound of the 90% symmetrical credible interval for advancedConversions.incremental after applying advanced adjustments. |
| Number of conversion events, such as purchases, from the experiment's control group after applying advanced adjustments. |
| Net increase in number of conversions as a result of running ads after applying advanced adjustments. Calculated as |
| Probability that the ads in the test group of the specified experiment cell had the lowest cost per incremental conversion across all cells in the experiment after applying advanced adjustments. |
| Probability that the ads in the test group of the experiment caused positive |
|
|
| Number of conversion events in the control group after applying advanced adjustments, factored by the ratio of |
| Number of conversion events, such as purchases, from the experiment's test croup, after applying advanced adjustments. |
| Incremental return on ad spend (ROAS). Calculated as |
|
|
| The lower bound of the 90% symmetrical credible interval for |
| The upper bound of the 90% symmetrical credible interval for |
| Amount of sales from the control group of the experiment after applying advanced adjustments. |
| Net increase in amount of sales as a result of running ads after applying advanced adjustments. Calculated as a |
| Probability that the ads in the test group of the specified experiment cell had the highest incremental ROAS across all cells in the experiment after applying advanced adjustments. |
| Probability that the ads in the test group of the experiment caused positive |
|
|
| Amount of sales from the control group after applying advanced adjustments, factored by the ratio of |
| Amount of sales from the test group of the experiment after applying advanced adjustments. |
|
|
| The lower bound of the 90% highest posterior density (HPD) credible interval for |
| Number of people from the experiment's control group who purchased. |
| The upper bound of the 90% highest posterior density (HPD) credible interval for |
| The 90% statistical confidence interval associated with the estimated |
| Net increase in number of people who made purchases as a result of the ad campaigns. Calculated as |
| Boolean indicating statistical significance. |
|
|
| Probability that the ads in the test group of the specified experiment cell had the lowest cost per incremental buyer across all cells in the experiment using Bayesian statistics with an uninformative prior distribution. |
| Based on regression analysis, the p-value associated with |
| Number of people reached by your ads who also converted. |
|
|
| Number of people in the control group, factored by |
| Probability that the ads in the test group of the experiment caused positive |
| Number people from the experiment's test group who purchased. |
|
|
| The lower bound of the 90% highest posterior density (HPD) credible interval for |
| The upper bound of the 90% highest posterior density (HPD) credible interval for |
| Number of conversion events, such as purchases, from the experiment's control group. |
| The 90% confidence interval associated with the estimate for |
| Net increase in number of conversions as a result of running ads. Calculated as: |
| Boolean indicating statistical significance. |
| Number of conversion events in the control group, factored by the ratio of |
|
|
| Probability that the ads in the test group of the specified experiment cell had the lowest cost-per-incremental conversion across all cells in the experiment using Bayesian statistics with an uninformative prior distribution. |
| The p-value for |
| Number of conversion events from people who purchased and were reached by your ads. |
| Probability that the ads in the test group of the experiment caused positive |
|
|
| Number of conversion events, such as purchases, from the experiment's test group. |
| The average number of times your ad displayed to each person in population.reached. |
| Number of billable ad impressions for ads associated with the lift study. |
| Number of people in the control group of the experiment. |
| Number of people in the test group reached by one or more ads associated with the study. |
| Number of people in the test group for the experiment. |
|
|
| The lower bound of the 90% highest posterior density (HPD) credible interval for |
| The upper bound of the 90% highest posterior density (HPD) credible interval for |
| Amount of sales from the control group of the experiment. |
| 90% confidence interval associated with the estimate for |
| Net increase in sales as a result of running campaigns. Calculated as: sales.test - |
| Boolean indicating statistical significance. |
|
|
| Probability that the ads in the test group of the specified experiment cell had the highest incremental ROAS across all cells in the experiment using Bayesian statistics with an uninformative prior distribution. |
| The p-value associated with |
| The amount of sales from purchasers reached by your ads. |
| Amount of sales in the control group, factored by the ratio of |
| Probability that the ads in the test group of the experiment caused positive |
| Amount of sales from the test group of the experiment. |
| Total amount spent up to the current time for your study in the currency USD. |
Sample response shown as parsed JSON for ease of reading:
{ "last_updated_results": "2019-04-30", "id": "<STUDY_OBJECTIVE_ID>", "results": [{ "population.test": 8008, "population.control": 8069, "population.reached": 7329, "impressions": 151928, "spend": 2367.74, "frequency": 20.72970391595, "buyers.test": 795, "buyers.control": 685, "buyers.scaled": 679.82153922419, "buyers.incremental": 115.17846077581, "buyers.reached": 720, "buyers.reachedPercent": 0.90566037735849, "buyers.baseline": 604.82153922419, "buyers.lift": 0.19043379460915, "buyers.delta": 60.074001807654, "buyers.pValue": 0.0016110673739798, "buyers.isStatSig": 1, "buyers.singleCellBayesianConfidence": 0.98727038188854, "buyers.multiCellBayesianConfidence": 0.78486645400638, "buyers.bayesianCILower": 30.473292849176, "buyers.bayesianCIUpper": 199.95922666997, "conversions.test": 871, "conversions.control": 736, "conversions.scaled": 730.43598958979, "conversions.incremental": 140.56401041021, "conversions.reached": 789, "conversions.reachedPercent": 0.90585533869116, "conversions.baseline": 648.43598958979, "conversions.lift": 0.21677391857774, "conversions.delta": 67.978575551168, "conversions.pValue": 0.00067032207485984, "conversions.isStatSig": 1, "conversions.singleCellBayesianConfidence": 0.99176024719258, "conversions.multiCellBayesianConfidence": 0.80160595182145, "conversions.bayesianCILower": 44.564010410212, "conversions.bayesianCIUpper": 236.54889081671, "advancedConversions.test": 870, "advancedConversions.control": 771.72138217184, "advancedConversions.scaled": 765.88732537268, "advancedConversions.incremental": 104.11267462732, "advancedConversions.baseline": 684.88732537268, "advancedConversions.lift": 0.1520143106323, "advancedConversions.informativeSingleCellBayesianConfidence": 1, "advancedConversions.informativeMultiCellBayesianConfidence": 0.819, "advancedConversions.bayesianCILower": 56.7105227, "advancedConversions.bayesianCIUpper": 156.976729, "incrementalROAS": 1.3104231791133, "sales.test": 54663.94, "sales.control": 51953.96, "sales.scaled": 51561.198621886, "sales.incremental": 3102.7413781138, "sales.reached": 49594.66, "sales.baseline": 46491.918621886, "sales.lift": 0.06673721950148, "sales.delta": 5708.9583320282, "sales.pValue": 0.37130299866185, "sales.isStatSig": 0, "sales.singleCellBayesianConfidence": 0.74013779586612, "sales.multiCellBayesianConfidence": 0.70927872163835, "sales.bayesianCILower": -4827.3888400561, "sales.bayesianCIUpper": 11330.643600358, "advancedIncrementalROAS": 1.4777176303831, "advancedSales.test": 54619.86, "advancedSales.control": 51510.417142093, "advancedSales.scaled": 51121.008857837, "advancedSales.incremental": 3498.8511421632, "advancedSales.baseline": 46095.808857837, "advancedSales.lift": 0.075903888636687, "advancedSales.informativeSingleCellBayesianConfidence": 0.97125, "advancedSales.informativeMultiCellBayesianConfidence": 0.7505, "advancedSales.bayesianCILower": 596.181983164, "advancedSales.bayesianCIUpper": 6470.00248918 }] }], }
In addition to retrieving the results per objective, you may choose to breakdown the results by providing the breakdowns
parameter.
curl -G \
-d 'access_token=<ACCESS_TOKEN>' \
https://graph.facebook.com/<API_VERSION>/<STUDY_OBJECTIVE_ID>?fields=results&breakdowns=['cell_id']
The following are the available breakdown dimensions:
Breakdown | Values |
---|---|
|
|
| IDs of the available cells in the study. |
|
|
| Two letter country codes ( Currently supported only when queried in combination with Example: |
The results return multiple JSON objects in the array based on the available breakdowns. For example, if cell_id
is provided, the results are broken down by the number of cells in the study. You may provide one or more breakdowns; however, the combination of breakdowns must at least 100 conversions from test and control groups combined for results to display.
{ "id": "<STUDY_OBJECTIVE_ID>", "results": [ { "cell_id": "<CELL_ID1>", ... Default fields where the values are specific to the <CELL_ID1> breakdown ... }, { "cell_id": "<CELL_ID2>", ... Default fields where the values are specific to the <CELL_ID2> breakdown ... }], }