Building Intelligent Bots Using The Microsoft Bot Framework

Scope

The following article introduces the concept of bots, demonstrates the basics of the Microsoft Bot framework, and showcases how Microsoft Cognitive Services can be used to power the intelligent bots.

Introduction

Conversational Applications or bots is a hot topic right now. Is it really useful or is it just  hype? To answer this question, let’s look at scenarios where bots are useful.

Messaging Applications

We all notice that Text Messaging is the most used smartphone feature and for the first time in history, people are using more messaging apps than social networks.


Source - http://www.businessinsider.com/the-messaging-app-report-2015-11?IR=T

What if you could interact with your application via this interface? This is possible via bots! Moreover, bots can adapt and work on most devices and platforms like desktop, mobile, or social media like Facebook, WhatsApp, or Skype. This will give your app a totally new selling point by interacting with your users using its favorite communication method and using his/her favorite platform!

Repetitive Tasks

In scenarios, there are repetitive tasks such as a call center, bots can be used, since there are lots of generic answers. The flow as described below would be perfect to handle the generic and most common requests and when there’s a more difficult query, the bot transfers the call to a human.

Getting Started

One of the tools to quickly build bots is the Microsoft Bot Framework which is Open Source. Bot Builder SDKs allow you to build simple to sophisticated dialogs; Cognitive Services enable your bot to see, hear, interpret and interact in more human ways. This will also be used in this article. To get started with the Microsoft Bot Framework, you must first download the Bot Builder SDK here.

Create the project

To set up the bot framework connector SDK .NET template in the Visual Studio 2015, this is a step-by-step guide.

2. Install Visual Studio 2015 (latest version).
3. Download bot application template from this download link here.
4. Save the zip file to your Visual Studio 2015 templates directory which is traditionally in "%USERPROFILE%\Documents\Visual Studio 2015\Templates\ProjectTemplates\Visual C#\". For example, “C:\Users\USERXX\Documents\Visual Studio 2015\Templates\ProjectTemplates\Visual C#”.
   
   
   
   In the Web.Config file, make sure that the keys are as follows for debugging and runing the project. These will be changed when deploying the bot.
   

   <appSettings>
       <!-- update these with your BotId, Microsoft App Id and your Microsoft App Password-->
       <add key="BotId" value="YourBotId" />
       <add key="MicrosoftAppId" value="" />
       <add key="MicrosoftAppPassword" value="" />
   </appSettings>

5. Download & Configure the Bot Emulator.
   
   https://download.botframework.com/botconnector/tools/emulator/publish.htm

Once the emulator is installed, copy the link of the running application in your browser and paste it in the emulator, leave the ID and password blank, and click on Connect.

The URL in the emulator must be in the format http://xxx:port/api/messages

Once you click on connect, when you check the Log, it should return status 200, meaning that the connection has been established.

Microsoft Bot Framework Basics

Let’s go back to the Bot Framework project to examine the code. The class here is the MessageController; this is where all the communication between the bot and the user will be handled.

All the input received from the user will go through the Post method where the type of the activity will be identified which will decide how to reply the user.

The main types of activities are,

1. Message - A simple communication between a user <-> bot
2. ConversationUpdate - Your bot was added to a conversation or other conversation metadata changed
3. ContactRelationUpdate - The bot was added to or removed from a user's contact list
4. Typing - The user or bot on the other end of the conversation is typing
5. Ping - An activity sent to test the security of a bot.
6. DeleteUserData - A user has requested for the bot to delete any profile / user data

Therefore, in the post method, you will ensure that ActivityType is checked. If it is of type message, the bot will send a reply to the user and output the length of the text that he sent. 

if (activity.Type == ActivityTypes.Message) {
    //default
    var connector = new ConnectorClient(new Uri(activity.ServiceUrl));
    var reply = activity.CreateReply();
    reply.Text = "You typed " + activity.Text;
    await connector.Conversations.ReplyToActivityAsync(reply);
}

Let’s amend the code to send a welcome message to the user when he joins the conversation and triggers a ConversationUpdate activity.

else if (message.Type == ActivityTypes.ConversationUpdate)
{
    var connector = new ConnectorClient(new Uri(message.ServiceUrl));
    var response = message.CreateReply();
    response.Text = "Hi there! I'm chervine's bot! How can I help you today? ";
    await connector.Conversations.ReplyToActivityAsync(response);
}

Now, let’s run the code and see the results in the emulator.

Adding Language understanding Capability to the Bot

Microsoft’s Language Understanding Intelligent Service (LUIS) offers a fast and effective way of adding language understanding to applications. With LUIS, you can use pre-existing, world-class, pre-built models from Bing and Cortana whenever they suit your purposes – and when you need specialized models, LUIS guides you through the process of quickly building them.

Before proceeding, it is important that you understand the basic concepts of LUIS.

1. Intents
   Consider intent as an intention or action. What action should a sentence trigger? Search for  news? Look for help? Proceed with sign up? Those can be examples of Intents.
   
   Intents match user requests with the actions that should be taken by your app.
   
   
2. Utterances
   Utterances are sentences representing examples of user queries or commands that your application is expected to receive and interpret. LUIS learns from these utterances and your app is able to generalize and understand similar contexts.
   
   For each intent, add example utterances that trigger this intent and include as many utterance variations as you expect users to say.
   
   
3. Entities
   An entity represents a class including a collection of similar objects (places, things, people, events or concepts). Entities describe information relevant to the intent, and sometimes they are essential for your app to perform its task.

Imagine a news bot that searches for news. How will the bot know which work in the sentence he should use to search for news? This item that he should search about is called the Entity.

Example, from the sentence search for news about Mauritius. Mauritius will be the entity.

Now, that you understand the basic concepts of LUIS, let’s go and create your LUIS application.

Creating a LUIS App

To create a LUIS application, go to www.luis.ai, register and click on create a new App.

Once you click "Create", you are redirected to the bot dashboard where you can see statistics about the bot.

In this example, we’ll create a bot that allows us to search for news, register for news, and get help.

Later in this article, we’ll build and enhance the bot to understand the sentiment of the news, therefore our bot will be able to detect whether a news has a positive or negative sentiment.

Intents

Therefore, the bot will have 4 intent, namely, SearchNews, AskHelp, Register and None.

All LUIS application has a None intent created which represents any text that it cannot interpret.

Next, click on intent on the dashboard and select Add Intent to add the 4 intents.

Define the Utterances and Entities

For each of the Intents, we now need to add the Utterance and Entities. Click on each of the Intents and start adding the Utterance.

In the example below, we add the utterance “What are the news in Mauritius”. Now, who will the bot identify what is the word he needs to search for? This is the entity.

Then, from the Utterance, we select the word news which represents the entity. We need to do the same for a few utterance which will be used to train the model.

Now, we also need to be able to identity the good and the bad news. How will the bot know which words to identify good and bad news? For this purpose, we also need to train our bot by adding utterances referencing good and bad news.

The next step is to add the utterance and entity for other Intents such as register for news or to ask for help.

Train the model

The next step is to train the model. To do so, from the left menu, click on Train & Test and click on Train Application.


Test the Application

Once the model is trained, you can now type in sample sentences and see if it find the intent and entities correctly.

Once done, click on publish and you'll be able to call the LUIS app from the Bot.

Setting up the LuisDialog

Now that the LUIS service is setup, we now need to add it in the bot project to be able to recognize each intents of the user.

What will happen now is that whenever the user input a text, this will be sent to the LUIS model to identify each Intent and their Entities.

To proceed, create a new class RootDialog which implements the LusDialog Object and is specialized to handle intents and entities from LUIS, 

[Serializable]
[LuisModel("LUIS APP ID", "SUBSCRIPTION KEY")]
public class RootDialog: LuisDialog < object > {}

You also need to add the keys of your LUIS App Id and Subscription key as above. Both can be retrieved in the Publish tab of the LUIS model.

Now, it’s time to map the LUIS intents to the code in out bot. To do so, we need to add specific functions in the class RootDialog and decorate each function having the name of the intent in the LUIS model.

Proceed with the same steps for all the other intents, also, always create the “none” intent for cases where the LUIS model could not map the utterance to any intent. 

[LuisIntent("")]
[LuisIntent("None")]
public async Task None(IDialogContext context, LuisResult result)[LuisIntent("AskHelp")]
public async Task Help(IDialogContext context, LuisResult result)[LuisIntent("Register")]
public async Task Subscribe(IDialogContext context, LuisResult result)

Let’s see how the bots behave. As you run the program and ask the bots questions related to the intents, you will see that the bot will now reach the code as per the intent and the entities.

Adding the Bing News API

At this stage, we have setup our project and we have successfully implemented Language Understanding in our bot. It’s now time to build the search functionality using the Bing News API.

What will happen is that once LUIS identify a sentence as a news, it will then return us the News Entity, that is, the item about which the user is trying to search for news. We’ll then take this entity and pass it to the Bing News API to get the results.

To connect to the Bing News API, follow the steps here to register and get the keys.

The API takes a text and input and returns a JSON of all the news item as shown below.

{
    "_type": "News",
    "readLink": "https://api.cognitive.microsoft.com/api/v5/news/search?q=football",
    "totalEstimatedMatches": 1980000,
    "value": [{
                "name": "Tennessee <b>Football</b>: Former Coach Phillip Fulmer Takes Shot at Lane Kiffin",
                "url": "http://www.bing.com/cr?IG=6E1D71ECB12642478D9B43CABFE6D6DA&CID=368054890FCB699507CA5ECA0EFA68F3&rd=1&h=hHf7Kln-pgk2Ygz0QhP5uZ6DaajSR6YII2JGClvmfYA&v=1&r=http%3a%2f%2fallfortennessee.com%2f2017%2f03%2f10%2ftennessee-football-fulmer-kiffin%2f&p=DevEx,5023.1",
                "image": {
                    "thumbnail": {
                        "contentUrl": "https://www.bing.com/th?id=ON.3093BAC7465DF3974947D6AFC8A00B69&pid=News",
                        "width": 700,
                        "height": 461
                    }
                },
                …

Once we have subscribed to the Bing News API, follow the steps below to add the code to the bot to retrieve the news.

1. Create function GetBingNews to fetch the news.
   

   private async Task < BingNews > getBingNews(string query) {
       BingNews bingNews;
       String bingUri = "https://api.cognitive.microsoft.com/bing/v5.0/news/search/?count=50&q=" + query;
       string rawResponse;
       HttpClient httpClient = new HttpClient() {
           DefaultRequestHeaders = {
               {
                   "Ocp-Apim-Subscription-Key",
                   "SUBSCRIPTION KEY"
               },
               {
                   "Accept",
                   "application/json"
               }
           }
       };
       try {
           rawResponse = await httpClient.GetStringAsync(bingUri);
           bingNews = JsonConvert.DeserializeObject < BingNews > (rawResponse);
       } catch (Exception e) {
           return null;
       }
       return bingNews;
   }

2. In the SearchNews intent, add the following code to retrieve the entity.
   
   It is this retrieved entity that represents the news that we need to search.
   

   EntityRecommendation newsEntity;
   if (result.TryFindEntity("NewsItem", out newsEntity)) {…}

3. Pass the intent to the BingNews API and call the function GetBingNews
   

   BingNews bingNews = await getBingNews(newsEntity.Entity);

4. Check if there are no news returned and inform the user accordingly
   

   var message = await activity;
   var reply = context.MakeMessage();
   if (bingNews == null || bingNews.totalEstimatedMatches == 0)
   {
       reply.Text = "Sorry, couldn't find any news about '" + newsEntity.Entity;
   }

5. If there is news received, we can build a carousel to display the news items to the user.
   

   reply.AttachmentLayout = AttachmentLayoutTypes.Carousel;
   reply.Attachments = new List < Microsoft.Bot.Connector.Attachment > ();
   for (int i = 0; i < 10 && i < (bingNews ? .totalEstimatedMatches ? ? 0); i++) {
       var article = bingNews.value[i];
       HeroCard attachment = new HeroCard() {
           Title = article.name.Length > 60 ? article.name.Substring(0, 57) + "..." : article.name,
               Text = article.provider[0].name + ", " + article.datePublished.ToString("d") + " - " + article.description,
               Images = new List < CardImage > () {
                   new CardImage(article.image ? .thumbnail ? .contentUrl + "&w=400&h=400")
               },
               Buttons = new List < CardAction > () {
                   new CardAction(ActionTypes.OpenUrl, title: "View on Web", value: article.url)
               }
       };
       reply.Attachments.Add(attachment.ToAttachment());
   }
   await context.PostAsync(reply);
   context.Wait(this.MessageReceived);

6. Test the bot in the emulator.
   
   
   

Adding Sentiment Analytics

At this stage, the bot is functional and retrieving news. Our objective now, is to make the user able to search for positive or negative news.

The first step is to subscribe to the Text Analytics API in Cognitive Services here. Note that you will get up to 5000 transactions free per month. Next, we’ll need to retrieve the Good/Bad news entity from the LUIS model to determine whether the user requested a good or bad news. We’ll then take the news results from the Bing API, calculate the sentiment score of each of them and filter it based on the Entity detected from the LUIS model.

The next step is to create the method getSentimentScore that will take a text as parameter and return us the sentiment score. It is this method that will be used to filter the results returned by the Bing API.

private async Task < double > getSentimentScore(string documentText) {
    string queryUri = "https://westus.api.cognitive.microsoft.com/text/analytics/v2.0/sentiment";
    HttpClient client = new HttpClient() {
        DefaultRequestHeaders = {
            {
                "Ocp-Apim-Subscription-Key",
                "XXX"
            },
            {
                "Accept",
                "application/json"
            }
        }
    };
    var textInput = new BatchInput {
        documents = new List < DocumentInput > {
            new DocumentInput {
                id = 1,
                    text = documentText,
            }
        }
    };
    var jsonInput = JsonConvert.SerializeObject(textInput);
    HttpResponseMessage postMessage;
    BatchResult response;
    try {
        postMessage = await client.PostAsync(queryUri, new StringContent(jsonInput, Encoding.UTF8, "application/json"));
        response = JsonConvert.DeserializeObject < BatchResult > (await postMessage.Content.ReadAsStringAsync());
    } catch (Exception e) {
        return 0.0;
    }
    return response ? .documents[0] ? .score ? ? 0.0;
}

Next, remember that in our LUIS model, we have 2 entities, GoodNews and BadNews? This is where we’ll use them!

In the search method, we’ll add the following line of codes to retrieve the Positive/Negative entities from the sentences if they are available. This will be used to detect whether we’ll filter by  good or  bad news.

var findPositive = result.TryFindEntity("GoodNews", out sentimentEntity);
var findNegative = result.TryFindEntity("NegativeNews", out sentimentEntity);

The next step is add the following code in the for loop of the search method. The only difference is that we’ll check the sentiment of each news item and decide whether it should be returned to the user of now.

var sentimentScore = await getSentimentScore(article.description);
if (findPositive && sentimentScore < 0.6) {
    continue;
} else if (findNegative && sentimentScore > 0.4) {
    continue;
}

Below is a screenshot of the bot when the code for Text Analytics has been added. The bot now understands the sentiment of the news!

Prompting the user for specific questions

In some scenarios, you may need to ask the user specific questions or fill a form. To do so, we make use of FormDialog.

You first need to create a class with all the “fields” for which  you need as attributes. To prompt something specific for an attribute, decorate the attribute with the question using the prompt keyword. Finally, add the method BuildForm to add the attributes to the form.

public class RegisterForm {
    [Prompt("Whats your name?")]
    public string Name {
        get;
        set;
    }
    [Prompt("Enter your Email)")]
    public string Email {
        get;
        set;
    }
    public static IForm < RegisterForm > BuildForm() {
        return new FormBuilder < RegisterForm > ().Field(nameof(Name)).Field(nameof(Email)).Build();
    }
}

The next step is to create the register Intent. The main item is to create a new FormDialog and provide a handler function that will be called when the user “registers” to process the data.

[LuisIntent("Register")]
public async Task Subscribe(IDialogContext context, LuisResult result) {
    try {
        await context.PostAsync("That's great. You will need to provide few details about yourself to proceed.");
        var registerform = new FormDialog < RegisterForm > (new RegisterForm(), RegisterForm.BuildForm, FormOptions.PromptInStart);
        context.Call(registerform, RegisterFormComplete);
    } catch (Exception) {
        await context.PostAsync("Something really bad happened. You can try again later meanwhile I'll check what went wrong.");
        context.Wait(MessageReceived);
    }
}
private async Task RegisterFormComplete(IDialogContext context, IAwaitable < RegisterForm > result) {
    try {
        var registration = await result;
        var success = true;
        if (!success) await context.PostAsync("I was not able to send your message. Something went wrong.");
        else {
            await context.PostAsync("Hi " + registration.Name + "! Your registration has been received, an email has been sent to " + registration.Email);
            await context.PostAsync("I remain here to help you. Do you want me to search for any news for you? ");
        }
    }…..
}

The screenshot below shows the result when this code is run – The bot will ask the user questions to basically fill in a form.

 

 

References

1. Bot sample using cognitive services (Bing News Search and LUIS) – Original Source Code
2. Bot Framework - Activities
3. Understanding Natural Language - LUIS
4. Bing News Search API
5. Messaging Apps are now bigger than Social Network – Business Insider