The Optimizely app empowers editors and content managers to run experiments on structured content in a simple way.

Why Contentful and Optimizely?

Contentful and Optimizely work together to facilitate experimentation without the need for developer involvement. This means that once it is setup, editors can run experiments on content using the Contentful web app and do not need to ask for code changes to clients. This enables a powerful and flexible workflow for editors and content managers.

Getting Started

In order to successfully integrate Optimizely into your content model, we will cover the following topics:

1. What you will need
2. Terminology
3. Installation and configuration
4. Usage
5. Integration with your front end
6. FAQ and troubleshooting

1. What you will need

You will need a few main things in order to run experiments with Optimizely and Contentful.

1. A Contentful account with a space for your website
2. An Optimizely account with a Full Stack project
3. A developer to integrate the initial solution with your front end

2. Terminology

We will start by explaining term definitions.

What is a variation?

A variation is simply one possible option or variant in a test. If you are A/B testing, you will test two different options: option A and option B. Translated to real-world terms: if you have two variations of a button, you may want to see which one gets more clicks.

What is a reference field?

A reference field is Contentful specific. It is a field within a content type that doesn't hold its own value but instead points to one or more entries. In the image below, we have a content type called page. Page has a reference field called component. The component reference field is a list of items (content types) that can be found on a page.

What is a variation container?

A variation container is a content type created by the Optimizely app which is used in place of a value in a reference field. In the image below, we've replaced our Call To Action (CTA) entry with a variation container that holds two different CTAs.

This is how the Optimizely app transforms your content model and prepares it for experimentation. As noted in the "What you will need" section, you will need a developer to handle the new response from Contentful.

3. Installing and configuring the Optimizely app

Step 1 - Install the app

Optimizely will ask you to sign in and authorize access to your account in order to connect it with Contentful.

Select an Optimizely Full Stack project you wish to use for your experiments. Note: if you don't see a list of projects, this means that you do not have any projects starting in Optmizely. You will need to start a project and create an experiment first before continuing.

Hit install

Step 2 - Configure the app

Now that the app is installed, it is time to set up an experiment. We are going to walk through an example setup where we run an experiment on the CTA button of a page.

We will enable experimentation on the component reference field by enabling it in the Optimizely app. Clicking the "Add content type" button will present a modal where we will pick the page content type and enable component.

As you'll remember from our example above, component is a reference field:

In Contentful, it looks like this:

4. Usage

Now that we have the app installed and configured to experiment on component, we will edit our content to use the variation container. Again, the variation container is a custom content type which is installed by the Optimizely app.

In the flow shown above we do the following:

1. Add a variation container to our component
2. Pick the CTA experiment from the dropdown menu in the Optimizely app interface
3. Pick the existing "Buy Now!" CTA as our control group
4. Create a new CTA entry with the value "20% Off!" and use that as our varation
5. Publish the new CTA entry
6. Publish the variation container
7. Remove the old CTA block
8. Publish our page

Editors can easily swap out values and turn them into variation containers like in the example above. This makes it easy to take any reference field which is used to display content and change it into a variation container which displays content based on an experiment.

5. Integration with your front end

Variation containers are a new content type that are introduced into the Contentful response. They are containers that point to two or more actual values that we'd like to test. We now need to now integrate our front end with Optimizely and the variation containers. At this point, we need the help of a developer to change how our front end works to accept and programmatically handle variation containers.

From here, we need to take a more technical dive into the setup of the client and how it integrates with Optimizely. The guide below uses the same CTA example we have been following so far. It is beneficial for developers get acquainted with the content we outlined above before moving onto the technical guide below.

The goals of the setup are:

• Enable server side experimentation without pushing new code per experiment
• Separating concerns: controlling content in Contentful and experiments in Optimizely
• Speed: server side selection of variations for fast delivery and to avoid the "flash of content"

How the Optimizely App changes the Contentful API response

As mentionted above, the Optimizely app creates a new content type: the variation container. Let's look again at the CTA example that says "Buy now!":

Fig 1.1

Using the Optimizely app, the same content model is transformed. Where we originally received the CTA of "Buy Now!", we now are receiving a variation container holding all the different possible CTAs:

Fig 1.2

The variation container is simply a content type that nests the possible values for the CTA and holds meta data about the Optimizely experiment and the variation names.

Using Optimizely to pick the right variation

We are going to create a pseudo-backend which takes the Contentful API response and uses the Optimizely SDK to determine which variation to show to the user.

Let's start by looking at the variation container JSON response returned by Contentful. Notice that the content type exposes a meta, variations and experimentKey property.

{
    "sys": {
    "space": { ... },
    "id": "41nZggHEplcBOsrPXLOEU",
    "type": "Entry",
    "createdAt": "2019-07-17T14:32:24.306Z",
    "updatedAt": "2019-07-17T14:32:24.306Z",
    "environment": { ... },
    "revision": 1,
    "contentType": {
        "sys": {
        "type": "Link",
        "linkType": "ContentType",
        "id": "variationContainer"
        }
    },
    "locale": "en-US"
    },
    "fields": {
    "experimentTitle": "CTA Variation Experiment",
    "experimentId": "15249150297",
    "meta": {
        "control": "2hoYOxnZVV3Imvry1DhmtG",
        "variation": "5IlB5PCsca3zZnzlVd7WvI"
    },
    "variations": [
        {
        "sys": {
            "type": "Link",
            "linkType": "Entry",
            "id": "2hoYOxnZVV3Imvry1DhmtG"
        }
        },
        {
        "sys": {
            "type": "Link",
            "linkType": "Entry",
            "id": "5IlB5PCsca3zZnzlVd7WvI"
        }
        }
    ],
    "experimentKey": "CTA_test"
    }
}

Before we can choose which entry to pick out of our variation container, we need to see how to get the right variant from Optimizely. Below is an example with a hardcoded test to get a variation for a user.

import Optimizely from 'optimizely';
import datafile from './optimizelyDataFile';
import { getUser } from 'user';

const optimizelyClient = new Optimizely({ datafile });

const user = getUser();

const variation = optimizelyClient.activate(
  'CTA_test',
  user.userId
);

// variation => 'variation'

In the code above, Optimizely has determined that this user should see the variation option. This is a value we can get from the variation container meta field.

Now we will update our code to include pulling the variation container itself and using its values to populate the Optimizely experiment.

import Optimizely from 'optimizely';
import datafile from './optimizelyDataFile';
import { getUser } from 'user';
import sdk from 'contentful-sdk';

const optimizelyClient = new Optimizely({ datafile });
const user = getUser();

// we are using the id of the variation container from the JSON sample above
// In a real world implementation the ID would be set dynamically through e.g. a slug.
const variationEntry = await sdk.getEntry('41nZggHEplcBOsrPXLOEU');

const variation = optimizelyClient.activate(
  CTA_test
  variationEntry.experimentKey,
  user.userId
);

// variation => 'variation'

const ctaEntryId = variationEntry.meta[variation];

// ctaEntryId => '5IlB5PCsca3zZnzlVd7WvI'

We now have the entryId of the CTA content block we want to display! This means that the content we should show is the "20% off!" CTA from Fig 1.2 above. From here, you should be able to make minimal changes to your display logic in order to show the correct entry.

Important notes about this example

In this example we used userId. Optimizely requires that you identify the current user so they can determine which experiment group to place them into. This ID should stick with the user and properly identify them throughout different sessions. If you use a random number as an ID, your A/B test will not work correctly and the results will be meaningless.

A datafile was used in the example above to create the Optimizely client. This is supplied by Optimizely and will be unique to your project. You can read more about the data file here.

Only published content on Contentful will be exposed in the variation container. Using a Contentful environment, you can test how a variation container works before promoting your experiment to production.

6. FAQ and troubleshooting

How does the Optimizely app effect my environment?

The app needs to create a new content type called the variation container. The variation container is a custom type which makes use of a custom editor interface inside the web app and collates data from Optimizely and Contentful.

Where can I activate my experiments?

In Optimizely. We want to keep a clear separation of concerns. That means Contentful controls the content and Optimizely controls the experiment. The Optimizely app provides deeplinks on the sidebar into the Optimizely dashboards for easy access to your experiments where you can set up and launch experiments.

Why is my app configuration not being saved?

You must remember to click "Save" at the top right of the Optimizely app in order to save the full configuration.

Why does one or more of my variations not show up?

When you pick variations inside of a variation container, you will need to ensure that they are published and not a draft. Furthermore, you will need to ensure that your variation container itself is published for it to show up in the API response.

How can I define a fallback or default value for my experiment?

This is up to you. However, a default value or fallback is usually the control group. If you have a certain audience that you don't want to expose the experiment to, you will need to ensure your front end can handle the scenario of falling back to your control group.

How do I target a certain audience?

This is done in Optimizely. You will need to provide a unique value (usually a user ID) to let Optimizely know who is currently seeing an experiment. If you need a custom solution, you will have to provide more data points in order to correctly target your audience. In the case of excluding an audience, see above about defining a default or fallback value.

What do I do after an experiment is over?

There are a few options available. Currently, Optimizely offers the option to automatically start showing the winning variation after your experiment has gathered a meaningful data set. This means you would leave your variation container in place and allow Optimizely to always choose the winner.

Another option is to manually remove the variation container and replace it with the value that has won the experiment. This may be beneficial as you won't need to rely on Optimizely to serve the right content anymore.

Does uninstalling the Optimizely app break my experiments?

Your existing experiments will still run. We never tell Optimizely what to do with your experiments, we simply read experiment names, variation groups and experiment traffic data. However, the editor experience when dealing with experiments inside of the Contentful Web App will be highly degraded. Variation containers will still exist but the custom editor they use will be uninstalled.

Why does the variation container not show up as an option in my reference field?

Your reference field may have specific validations enabled. This means you will need to enable the variation container as a possible valid content type for that field. This can be done in the app or by finding the specific content type and changing the validations to include the variation container.