How to Get Started with Built-in NLP

When designing your bot’s conversational flow, a key consideration is whether to use natural language processing (NLP). NLP allows for more natural interactions, and for people to feel understood in real time. When implemented well, it creates a positive and effective customer experience.

The built-in NLP feature enables developers to tap into the power of NLP right in Messenger. This feature understands and extracts meaningful information (called entities) out of free-form text messages, which can then be used to identify intent and automate a response such as sending a specific reply, routing the conversation to a human, collecting audience data, etc. Built-in NLP recognizes a set of predetermined entities and supports multiple languages. You can find full details on supported entities and languages here. Built-in NLP is powered by Wit.ai, which we will cover in more detail in a future tutorial, but if you’re interested, find out more here.

Scope

This tutorial covers how to use built-in NLP to understand your users’ intents. We will create a bot for a bank that simulates allowing people to transfer an amount of money to a person using an email address and a time to make the transfer. In this tutorial, we will set up a webhook using Node.js and express that does the following:

  • Handles incoming messages, and setup.
  • Parses incoming messages for relevant entities.
  • Handles outgoing messages using entities identified by built-in NLP.

What you need

MESSENGER FEATURES

  • Webhook: The Messenger Platform sends events to your webhook to notify your bot when interactions or events happen.
  • Built-in NLP: Allows you to understand and extract meaningful information from messages.

PREREQUISITES

  • Complete the general set up:
  • Built a bot for Messenger. You can do so quickly using the Quick Start guide.
  • Have a server with Node.js installed.

How doees it work? Step-by-step

Step 1: Enable built-in NLP for your app

To use built-in NLP, you need to enable it for a page under the Messenger > Settings of your app. Select the default language of your users, for example, Default English.

Enabling built-in NLP under Messenger settings

Step 2: Download the starter project

We’ve created a starter project that has the project structure set up for you. You can download it here. If you want to skip ahead and see the completed project, you can find it here .

Step 3: Install dependencies

To begin, you need to install some Node module dependencies. This tutorial uses Express to set up a web server for the webhook, the body-parser module to parse the JSON body of the incoming POST requests, and the Request module to send HTTP requests to the Messenger Platform. Issue this command in your terminal:

cd node
npm install
        

Also, add your authentication information to the ‘node/config/default.json’ file.

Step 4: Parse message text

Messenger’s built-in NLP parses incoming messages and look for occurrences of phrases and keywords it recognizes, consider the sentence:

Transfer $50 to yourname@email.com tomorrow at 2pm

This message consists of three pieces of information that Messenger can recognize and create entities from:

  • $50, which it splits into the unit ‘$’ and the value ’50’.
  • yourname@email.com, an email address.
  • tomorrow at 2pm, which it converts to an ISO timestamp.

Built-in NLP gives each entity a confidence value that shows how sure Messenger is that it has recognized the meaning of the text. The value is on a scale of 0 to 1, where 0 is no confidence, and 1 is fully confident. Represented as a JSON response, this is:

{
  "mid": "mid.$cAACFU2NCL81nc02tgVhQghkkuXzk",
  "seq": 69894,
  "text": "Transfer $50 to yourname@email.com tomorrow at 2pm",
  "nlp": {
    "entities": {
      "amount_of_money": [{
        "confidence": 0.9650075,
        "value": 50,
        "type": "value",
        "unit": "$"
      }],
      "email": [{
        "confidence": 0.96904833333333,
        "value": "your-name@email.com"
      }],
      "datetime": [{
        "confidence": 0.96793833333333,
        "values": [{
          "value": "2018-01-30T14:00:00.000+01:00",
          "grain": "hour",
          "type": "value"
        }],
        "value": "2018-01-30T14:00:00.000+01:00",
        "grain": "hour",
        "type": "value"
      }]
    }
  }
}
        

We use the searchNLP function searches the JSON response for the correct key. As we are only expecting one instance of each entity in the sentence, we return the first value of the array:

function searchNLP(nlp, name) {
    if (nlp.entities[name]) {
        return nlp.entities[name][0];
    } else {
        // TODO: Or remove
    }
}
        

We need to parse the response for the key pieces of information your bot needs to process the user’s request using the searchNLP function above. Inside if (messageText) {} add the following code:

if (messageText) {
  const money = searchNLP(message.nlp, 'amount_of_money');
  const email = searchNLP(message.nlp, 'email');
  const when = searchNLP(message.nlp, 'datetime');
}
        

To make the date that Messenger parsed more readable and conversational, convert the date to a different format:

if (messageText) {
  ...
  const formattedWhen = new Date(when.value).toDateString();
}
        

Step 5: React to the message

Now that we have the values extracted; we can respond to the user appropriately, depending on how confident built-in NLP is that it understood the input.

Starting with the money value, if the confidence value is less than 0.8, tell the user that you’re not sure what their message meant. Optionally at this stage, you could ask for further clarification.

if (!money || money.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand how much to transfer.");
}
        

Next, if the confidence for the email value is less than 0.8, tell the user that you’re not sure what their message meant, and again, optionally ask for clarification.

if (!money || money.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand how much to transfer.");
} else if (!email || email.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand who the recipient is.");
}
        

Next, if the confidence for the when value is less than 0.8, tell the user that you’re not sure what their message meant, and again, optionally ask for clarification.

if (!money || money.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand how much to transfer.");
} else if (!email || email.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand who the recipient is.");
} else if (!when && when.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand when to make the transfer.");
}
        

Finally, issue a confirmation message to the user. Coding the actual transfer is out of the scope of this tutorial, but as you have all the essential pieces of information, you could pass these to an external API, and update the user with the progress of the transfer.

if (!money || money.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand how much to transfer.");
} else if (!email || email.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand who the recipient is.");
} else if (!when && when.confidence < 0.8) {
    sendTextMessage(senderID, "Sorry, I don't understand when to make the transfer.");
} else {
    // TODO: Perform action
    sendTextMessage(senderID, `Great! I will transfer ${money.unit}${money.value} to ${email.value} ${formattedWhen} `);
}
        

Here are two examples of how the bot handles different requests:

Example of Messenger bot handling user request

Conclusion

Messenger’s built-in NLP gets you a long way down the road of creating a bot that people can interact with conversationally. As you saw in this tutorial, it can handle complex sentences that contain multiple instances of entities and extract the important parts for your bot. For use cases where your bot needs to recognize and react to more custom and complex language than what is handled by the default entities in built-in NLP, you can take the same principles you learned in this tutorial and apply them to custom entities you create yourself in Wit.ai.