Facebook Analytics for Messenger Chatbots

In a previous post we saw how to use Facebook Analytics for Apps (FAfA) to log events from a simple front-end application. We noted also that using FAfA to log events in the back-end of an application is a bit more tricky.

In this second post about FAfA we look at how to use FAfA for tracking users for a Facebook Messenger chatbot. Fortunately, the people behind FAfA know how important are analytics for this new kind of applications, and FAfA provides a handy way to do this.

Default FAfA Events for Messenger Chatbots

We have some good news and we have some bad news.

The good news is that Facebook already provides the infrastructure for FAfA for every new Messenger chatbot by default. So, if you create a chatbot, essentially a Facebook application that uses the Messenger platform, your application automatically gets some basic analytics. In another words, you do not need to do anything to have some basic analytic data for your chatbot on Messenger!

The bad news is, however, that the default data that is logged by FAfA is very generic and in most practical applications is not useful on its own. In more detail, FAfA logs these events by default (found at section Activity -> Events):

These events are quite nice but they are generic and only scratch the surface of what is possible to track about the domain-specific chatbot you develop. For instance it is much more informative to log how many times a specific question was asked by the chatabot to the user and how many times an answer was given to the chatbot by the user. Or, we may want to count how many user requests were successfully handled by the chatbot, or how many times the chatbot showed a specific service or item to a user.

We can do this by using custom events.

Custom FAfA Events for Messenger Chatbots

Until now, we didn't have to touch a single line of code in order to get some basic insights on the Messenger chatbot usage. Now, it is time to code a bit in order to log some specific, custom, events for our chatbot. In fact it is possible to log with FAfA any custom event we can imagine that we think is interesting to track.

The behavior of a Messenger chatbot may be controled in various ways, e.g., using a tool for developing chatbots or using a custom back-end that handles the messages received by the chatbot and controls what response message is sent to user. A custom back-end typically provides more flexibility in setting up which events can be logged. Here we assume that we have a Node.js back-end for controlling the behavior of our chatbot. In this case, in order to log a custom event in FAfA, we need to make a call to the Facebook Graph API with the appropriate parameters.

The function below is a simple wrapper to the Facebook Graph API call that logs custom events in FAfA.

function logEvent(recipientId, event, parameters, valueToSum) {  
    let base_event = {
        _eventName: event,
        _valueToSum: valueToSum
    };

    let custom_events = [Object.assign(base_event, parameters)];

    request.post({
        url: "https://graph.facebook.com/" + app_id + "/activities",
        form: {
            event: 'CUSTOM_APP_EVENTS',
            custom_events: JSON.stringify(custom_events),
            advertiser_tracking_enabled: 0,
            application_tracking_enabled: 0,
            extinfo: JSON.stringify(['mb1']),
            page_id: page_id,
            page_scoped_user_id: recipientId
        }
    }, function (err, httpResponse, body) {
        logger.error(err);
        logger.error("Error in Custom Event Log; Status Code: %s", httpResponse.statusCode);
        logger.debug(body);
    });
}

First note that there are two configuration variables you need to set before calling the function:

Once the values to these variables are set, you can use the logEvent function to log custom events in the FAfA dashboard. This function takes 4 parameters:

For example, consider a chatbot that asks the user a set of questions before providing a personalized service. We can use a custom event QuestionAsked to track the questions made by the chatbot to the user. Essentially we need the back-end to log an event of this type, by calling the function above, whenever the back-end sends a question to the user.

logEvent(recipientId, "QuestionAsked", {}, 1);  

In order to differentiate between the various questions that are sent to the user we can use the parameters in the event. For instance, using the following two parameters we can log inside each QuestionAsked two more pieces of information: i) the specific question that was asked, and ii) the ID of the conversation session, assuming we keep something like this in the back-end.

{question: questionName, conv_session_id: sessionId}

So we need the back-end to calling the function above with the following parameters, whenever the back-end sends a question to the user.

logEvent(recipientId, {"questionAsked", {question: questionName, conv_session_id: sessionId}, 1);  

Using this information we can have more interesting insights on the way that users interact with the chatbot. For instance, if we have five questions in a row that the chatbot asks the user, we can use the total times each question was asked to find out at which one most users drop out the conversation, and perhaps adapt to remove or change the way the question is asked.

Open problems

Facebook Analytics for Applications (FAfA) is still at its early days and there are a lot of things that are not well documented or difficult to track.

One of the main problem is that "user properties" (that is the way to attach information such as gender or nationality to your users, we mentioned this also in the previous post on FAfA) are in beta and at the time of writing we could not find a nice way to use them for a Facebook Messenger chatbot. Probably this will be addressed in the near future but, for now, user properties do not seem to be usable.

Second, the dashboard is designed in a generic way so at to handle all sorts of applications, e.g., iOS and Android applications, and as a result for specific applications like chatbots some things may be difficult to fit it. In particular about custom events, the parameters of the events that are very critical (e.g., consider the questions example we used above) are in fact hard to visualize well. There is a "breakdown" option that can help on this (found at section Activity -> Breakdowns) which shows the distribution of events per parameter, but this can only fit on some basic insights we may want to investigate.

In any case, getting more experience with FAfA will certainly reveal ways to encode the custom events and insights we want in the existing functionalities, but the challenges pointed out above are important to consider if you are deciding now which platform for analytics to use for your chatbot.