Leveraging Rasa to build on-premise intelligent chat bots

In this post we will explain and demonstrate how to leverage Rasa to develop intelligent chatbots and securely deploy them on premise. Chatbots are one of the most direct ways to integrate with existing chat platforms. Whether you require a simple bot that replies to specific queries by fetching some information, or a more complex NPL-powered interactive agent, chatbots provide a way to integrate external services and functions into the collaboration experience of the chat platform users.

Chatbot development since 2016

Modern chatbot development for the most part leverages the capabilities of cloud APIs such as Microsoft LUIS and Google DialogFlow that tend to abstract away from the developer most of the underlying complexities of natural language understanding and response selection tasks. This makes developing chatbot integration significantly easier than it would have been previously, where either more rudimentary approaches where necessary or intelligent chatbot development was significantly harder. This unfortunately means that this useful level of abstraction is unavailable to enterprise environments that have strict compliance or security requirements.

The use of cloud APIs that can read and store the information exchanged can lead to sensitive information being leaked to unintended parties, as well as not being as easlily traceable and archivable for compliance purposes. This means that many existing chatbot integrations that use these APIs are unsuitable for use on premise, and in-house development teams are prevented from using them as well. In order to achieve the same rich user experience talking to intelligent chatbots the in-house team has a lot more work to do, which may make these integrations unreasonably complex and time consuming to develop.

Enter Rasa

Rasa is the only natural language understanding and chatbot development API that is open source. The API itself can be deployed on premise in a controlled environment, with each message becoming easily traceable and subject to the same security constrains as any other message exchanged over the chat system. By using Rasa it becomes relatively easy to analyse natural language messages sent by chat users for intent categorisation and to build intelligent integrations with external systems.

We’re now going to take a look at a very simple bot built through Rasa and how to connect it to the MindLink API.

The interesting bit

For this example we’re going to use one of the tutorials for Rasa Core, and implement a simple group-listening bot connection to MindLink that will read all messages sent in a particular chatroom and reply with some information. It should be possible to resuse the connection code for other bots in the same manner as it is used here. We’ll implement the whole thing in Python as that is the simplest way to write bots against Rasa, but mutatis mutandis you should be able to re-implement the connection in your favorite language. Similarly to how the MindLink API works, you can setup the Rasa API as a REST service and use any language.

I’m going to assume you have Rasa Core installed locally, and you have your local Python environment setup (I’m assuming Python3, but the connection code is relatively straightforward and should not be too hard to port to Python2).

The anathomy of an intelligent chat bot

There are two independent pieces of a Rasa bot: an “interpreter” or intent-extractor and a dialogue model. The first is a mechanism to “parse” natural language input and extract the user intent, then feed this to the dialogue model together with raw natural language input. It is trained over a set of pre-classified data, and uses this training to classify new inputs. Similarly, the dialogue model is built from a set of response templates, actions and the same intents recognized by the interpreter. It is also trained over sample dialogues that represent possible conversations.

Both modular pieces are implemented in a “data-driven” fashion: to develop an intelligent chatbot a significant amount of training data is required for both the intent extractor and the dialogue model. Interactive training modes, where you basically have a dialogue with the interpreter/dialogue model and give it feedback allowing it to adjust its responses, are available out of the box. Both components are also independent from each other (you can use alternative dialogue models/intent extractor with them) and allow very fine tuning of the resulting trained models (if you know what you’re doing). Again, compelling chatbots are always the product of repeated training over a significant body of data. Depending on the application, there may be large data sets already publically available (e.g. healthcare etc).

Great, but how do I actually put these together?

Let’s focus on the intent extractor first. We’ll define a set of intents in a markdown file with some initial associated data: intents.md. This is the file that basically defines how the intent extractor is going to interpret the input¹.

## intent:greet
- hey
- hello
- hi

## intent:goodbye
- bye
- goodbye

## intent:mood_affirm
- yes
- correct

## intent:mood_deny
- no
- nope

## intent:mood_great
- very good
- great
- amazing
- happy

## intent:mood_unhappy
- I am sad
- super sad
- sad
- very sad
- unhappy
- so saad
- so sad

Secondly, we’ll need two separate files for the dialogue model, one to define the things it recognizes and the actions it can take, and another with example conversations that exemplify when to take an action. First we define a domain: domain.yml.

intents:
  - greet
  - goodbye
  - mood_affirm
  - mood_deny
  - mood_great
  - mood_unhappy

actions:
- utter_greet
- utter_cheer_up
- utter_did_that_help
- utter_happy
- utter_goodbye

templates:
  utter_greet:
  - text: "Hey! How are you?"
    buttons:
    - title: "great"
      payload: "great"
    - title: "super sad"
      payload: "super sad"

  utter_cheer_up:
  - text: "Here is something to cheer you up:"
    image: "https://i.imgur.com/nGF1K8f.jpg"

  utter_did_that_help:
  - text: "Did that help you?"
  - text: "Was that helpful?"

  utter_happy:
  - text: "Great carry on!"

  utter_goodbye:
  - text: "Bye"

As you can see the domain also contains the response templates the bot can respond with. Then we define sample conversations stories.md:

## happy path               <!-- name of the story - just for debugging -->
* greet
  - utter_greet
* mood_great               <!-- user utterance, in format _intent[entities] -->
  - utter_happy

## sad path 1               <!-- this is already the start of the next story -->
* greet
  - utter_greet             <!-- action of the bot to execute -->
* mood_unhappy
  - utter_cheer_up
  - utter_did_that_help
* mood_affirm
  - utter_happy

## sad path 2
* greet
  - utter_greet
* mood_unhappy
  - utter_cheer_up
  - utter_did_that_help
* mood_deny
  - utter_goodbye

## say goodbye
* goodbye
  - utter_goodbye

## no greeting happy
* mood_great
  - utter_happy

## no greeting sad
* mood_unhappy
  - utter_cheer_up
  - utter_did_that_help

Each story is an independent example of a complete conversation. As you can see, the dialogue model doesn’t understand natural language, it undertstands intents. So the raw input needs to be piped through the intent extractor and then into the dialogue model.

Now that we’ve defined our data in files we need to train (“compile” if you like) the actual models from the data. You can use config files or pass additional parameters to the python scripts:

nlu_config.json

{
  "pipeline": "spacy_sklearn",
  "path" : "./models/interpreter",
  "data" : "./data/intents.md"
}

Training the intent extractor:

python -m rasa_nlu.train -c nlu_config.json --fixed_model_name current

Training the dialogue model:

python -m rasa_core.train -s data/stories.md -d domain.yml -o models/dialogue --epochs 300

Hopefully, you’ve got a working bot at this point. You can talk to it over the command line²:

python -m rasa_core.run -d models/dialogue -u models/interpeter/default/current

Now we are ready to wire this up to the MindLink API.

A simple Python MindLink API connection

We’re going to define a simple API connection class. This class will be responsible for interfacing with MindLink, relaying messages received to a consumer (which will be the bot in our example) and allowing for sending messages.

class ApiConnection:
    def authenticate(self):
        requestUrl = '{}/Authentication/V1/Tokens'.format(self.host)

        response = requests.post(
            requestUrl,
            json = {
                'Username': self.username,
                'Password': self.password,
                'AgentId': self.agent
            },
            headers = self.RequestHeaders)

        self.token = response.json()

        return self.token

    def getMessages(self, channelId, count):
        requestUrl = '{}/Collaboration/V1/Channels/{}/Messages'.format(self.host, channelId)

        parameters = {
            'take': count,
        }

        response = requests.get(
            requestUrl,
            params = parameters,
            headers = {
                'Accept' : 'application/json',
                'Authorization': 'FCF {}'.format(self.token)}
            )

        return response.json()

    def sendMessage(self, channelId, message):
        requestUrl = '{}/Collaboration/V1/Channels/{}/Messages'.format(self.host, channelId)

        response = requests.post(
            requestUrl,
            json = {
                'Text': json.dumps(message),
                'IsAlert': False
            },
            headers = {
                'Accept' : 'application/json',
                'Authorization': 'FCF {}'.format(self.token)}
            )

        if response.status_code != 200:
            print('Failed to send message to channel', channelId)

    def _getEvents(self, channelId, callback):
        self.running = True

        while self.running:
            requestUrl = '{}/Collaboration/V1/Events'.format(self.host)

            parameters = {
                'last-event': self.last_event,
                'types': ['message'],
                'channels': [channelId],
                'regex': '',
                'origins': ['remote']
            }

            response = requests.get(
                requestUrl,
                params = parameters,
                headers = {
                    'Accept' : 'application/json',
                    'Authorization': 'FCF {}'.format(self.token)
                    })

            if response.status_code != 200:
                print('Failed to get new messages from channel', channelId)
                continue

            for event in response.json():
                eventId = event['EventId']
                if (eventId > self.last_event):
                    self.last_event = eventId

                if (event['Sender'] != self.localUserId):
                    message = Message(event['Sender'], event['Content'], event['Time'], event['ChannelId'])
                    callback(message)

    def startStreaming(self, channelId, callback):
        self.requestThread = threading.Thread(target=self._getEvents, args = (channelId, callback))
        self.requestThread.start()

    def stopStreaming(self):
        self.running = False

What’s all this?

Let’s breakdown the methods on by one. First off, the authenticate should be relatively self-explanatory, it makes a request to the authentication endpoint of the MindLink API in order to log on an agent. It’s going to keep a hold of the received token and send it back with each subsequent request (this needs to be renewed if it expires).

The getMessages method is going to read a certain number of messages from the messages endpoint for a specific channel. Similarly with the sendMessage method.

Then we get to the more complicated bits: _getEvents is a perpetually running method that repeatedly makes get requests to the events endpoint of the API, fetching new events for a given channel ID. This method is invoked from the public startStreaming method on a separate thread, and will keep that thread running forever until the stopStreaming method is invoked. As you can see, what happens with each event received is not defined by this class, but needs to be forwarded by the caller in the form of a callback.

Now that we’ve covered the core connection code, we’ll build a basic example of how to use it.

MindLink + Rasa

Let’s make a python script entry point file, this is the actual sauce that feeds the message content to the bot and posts back a respone:

def handleMessage(message):
    response = agent.handle_message(message.content)
    connection.sendMessage(channelId, response)

if __name__ == "__main__":
    print('Loading Rasa agent (this may take a while).')
    agent = Agent.load(dialogue_model, interpreter_model)

    print('Creating connection to MindLink.')
    connection = ApiConnection(
        u'http://localhost:8081',
        u'userdomain\\moodbot',
        u'some password',
        u'moodbot_agent',
        u'contact:moodbot@userdomain.local')

    connection.authenticate()

    connection.startStreaming(
        channelId,
        lambda _, message: handleMessage(message))

    input('Type anything to stop.\r\n')
    connection.stopStreaming()

This will “start” the API connection on a given channel, with some preconfigured agent settings and a given channel ID (in a real application these may be passed as command line arguments, read from file etc). For each message received in the channel, this will be forwarded to the Rasa bot and the response will be sent back to the channel.

Conclusion

We covered a fair amount of ground in this post, explaining why Rasa is interesting for enterprise development, touching on core Rasa bot development concepts and demonstrating a basic way to wire everything up the MindLink API.