Agenda
Overview
Natural language processing - NLP
Chatbot
About
Rasa
Build Simple College Admission Chatbot
Best Practices
Conclusion
“[AI] is going to change the world more than anything in the history of mankind. More than electricity.”— AI oracle and venture capitalist Dr. Kai-Fu Lee, 2018
In today's smart phone world everyone of us have been users of the technology AIML. From the video predictions, shopping, service centres, social media, surveillance, food delivery, transportation, self driving and so on. Interestingly we are as well the producers of these data.
NLP is a discipline of AI, used to helps in understanding, interpret and manipulate human language.
We would briefly see today what is NLP and its applications. Build a chatbot from scratch using Rasa framework.
Natural language processing helps computers communicate with humans in their own language and scales other language-related tasks. For example, NLP makes it possible for computers to read text, hear speech, interpret it, measure sentiment and determine which parts are important.
Today’s machines can analyse more language-based data than humans, without fatigue and in a consistent, unbiased way. Considering the staggering amount of unstructured data that’s generated every day, from medical records to social media, automation will be critical to fully analyse text and speech data efficiently.
Sample Applications of NLP
categorising animal specials
categorising fake news
categorising bank transactions
A chatbot is a computer program that simulates human conversation through voice commands or text chats or both. Chatbot, short for chatterbot, is an artificial intelligence (AI) feature that can be embedded and used through any major messaging applications.
Chatbots, also called chatterbots, is a form of artificial intelligence (AI) used in messaging apps.
This tool helps add convenience for customers—they are automated programs that interact with customers like a human would and cost little to nothing to engage with.
Key examples are chatbots used by businesses in Facebook messenger, or as virtual assistants, such as Amazon's Alexa.
Chatbots tend to operate in one of two ways—either via machine learning or with set guidelines.
Rasa helps in creating virtual assistants. Used to automate human-to-computer interactions anywhere from websites to social media platforms.
Rasa supplies conversational AI infrastructure for a global community of developers, providing the tools to build chat-based and voice-based contextual assistants.
As Rasa is powered by open source software and runs in production everywhere from startups to Fortune 500s, across industries like healthcare, financial services, retail, and insurance.
Rasa Open Source provides three main functions. Together, they provide everything you need to build a virtual assistant:
Natural Language Understanding
Convert raw text from user messages into structured data. Parse the user’s intent and extract important key details.
Dialogue Management
Machine learning-powered dialogue management decides what the assistant should do next, based on the user’s message and context from the conversation.
Integrations
Built-in integration points for over 10 messaging channels, plus endpoints to connect with databases, APIs, and other data sources.
The first step before we proceed, lets install rasa.
% python -m venv env
% source env/bin/activate
% pip install rasa
For UI: pip install rasa-x -i https://pypi.rasa.com/simple [optional]
% rasa init # setup the basic file structure
% update config.yml and endpoint.yml [optional]
% rasa train # train your model
% rasa train nlu
% rasa train core
% rasa shell # run the bot from shell
% rasa run actions # action server
% rasa data validate # validate if the data given in the configuration is good
% rasa run -m models --enable-api --cors "*" --debug # run the rasa as rest http server
Sample UI Installation (not associated with RASA)
To have conversations with Rasa in UI, there is a module called chatroom. It is reactjs based.
% mkdir ui; cd ui
% git clone https://github.com/scalableminds/chatroom.git
% cd chatroom
% yarn install
% yarn build (after any customised changes)
% yarn serve # to run the ui server
% update the index.html with the address of the rasa server running
Fig1: Initial page loaded using the chatroom module
Sample Conversation:
Here is the sample conversation which was written from scratch using Rasa.
Fig2: Sample rasa conversation using rasa shell
Source Code:
Now that we have seen the conversation, lets see how each scenario is written step by step using Rasa framework. We will also see gradually learn the concepts of Rasa from these scenarios.
For each scenarios we would see the output in UI form.
Scenario 1:
College timings
nlu.yml
- intent: timings
examples: |
- I would like to college timings
- college time please
- what would the class start and end time
- when college reopen
stories.yml
- story: college time path
steps:
- intent: timings
- action: utter_timings
domain.yml
intents:
- timings
responses:
utter_timings:
- text: "The college is not open now we are still working through online"
Description:
rasa run :
rasa train
Need to run this after every changes we do the configuration files
rasa nlu train
If only the nlu is updated, such as nlu.yml, stories.yml and rules.yml
Once the train is completed the model will be saved under model/ directory
eg: below message will get
Your Rasa model is trained and saved at '/Users/dev/Desktop/Technical/Blogs/Chatbot/rasa/models/20210925-140501.tar.gz'.
nlu.yml
NLU(Natural Language Understanding) used to store the training data and extract structured information from user messages.
This usually includes the user's intent and any entities their message contains.
From the above example the student can know the timings of the college, so given some sample input for the bot to learn the student's intention.
stories.yml
Stories helps the bot to learn the dialogue management.
Stories can be used to train models that are able to generalise to unseen conversation paths.
From the above example, when the user's intention is to know the timing of the college the bot will respond the message back to the student, saying the “colleges are still operating online”.
domain.yml
Domain is the key file for the Rasa framework.
It specifies the intents, entities, slots, responses, forms, and actions your bot should know about. It also defines a configuration for conversation sessions.
From the above example, we have specified the intents and our responses here.
Output in UI:
Fig 3: Scenario 1 chat with bot, simple chat
Scenario 2:
Course duration for department
nlu.yml
- lookup: department
examples: |
- civil
- mechanical
- computer
- textile
- printing
- intent: course_duration
examples: |
- what is course duration for [civil]{"entity": "department"}
- would like to know the [computer]{"entity": "department"} course tenure
stories.yml
- story: college course duration
steps:
- intent: course_duration
- action: action_course_duration
domain.yml
intents:
- course_duration
actions:
- action_course_duration
entities:
- department
responses:
utter_course_duration:
- text: "For {department} course is of {duration} months"
actions.py
class ActionCourseDuration(Action):
def name(self) -> Text:
return "action_course_duration"
def run(self, dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
# It will return array of entities
entities = tracker.latest_message['entities']
print(entities)
course_duration = {
'civil': 10,
'computer': 12,
'mechanical': 14,
'printing': 16,
'textile': 18
}
entity_department = None
# Iterating through the array to retrieve the desired entity
for e in entities:
if e['entity'] == "department":
entity_department = str(e['value']).lower().strip()
duration = course_duration.get(entity_department, 0)
dispatcher.utter_message(
response="utter_course_duration",
department=entity_department,
duration=duration
)
return []
Description:
nlu.yml
Here we could see something new, it is called as entity and lookup tables.
Entity can be said as the information extracted from the intent.
Lookup tables are lists of words helps to extract entities.
Here we are defining list of departments/courses offered in the college, as it will help the bot to reply better based on the student's department.
stories.yml
When the user ask about the course duration
Reply is not plain uttering the message, rather its an action taken.
actions.py
Actions file holds the custom action that can run any code you want. This can be used to make an API call, or to query a database for example.
Rasa framework as beautifully decoupled actions server, to run it we need to call “rasa run actions”
From the example we are returning the course duration saved from the dict, but it could from anywhere.
Fig 4: Scenario 2 chat with bot, using entities
Scenario 3:
Exam Results
nlu.yml
- regex: rollnumber
examples: |
- \d{10,30}
- intent: get_roll_number
examples: |
- my roll number is [1234567891](rollnumber)
- This is my roll number [1234567891](rollnumber)
- [1234567891](rollnumber)
- intent: request_result
examples: |
- may I know the exam results
- can you please help me to know if I have passed
- am I all clear
rules.yml
- rule: activate result form
steps:
- intent: request_result # intent that triggers form activation
- action: result_form # run the form
- active_loop: result_form # this form is active
- rule: submit form
condition:
- active_loop: result_form # this form must be active
steps:
- action: result_form # run the form
- active_loop: null # the form is no longer active because it has been filled
- action: utter_submit # action to take after the form is complete
- action: utter_slots_values # action to take after the form is complete
- action: action_show_result
domain.yml
intents:
- get_roll_number
- request_result
actions:
- action_show_result
forms:
result_form:
required_slots:
rollnumber:
- type: from_entity
entity: rollnumber
slots:
rollnumber:
type: any
entities:
- rollnumber
responses:
utter_result:
- text: "For {roll}, result is {result} with {score} score"
utter_ask_rollnumber:
- text: "Please provide your roll number"
utter_submit:
- text: "All done!"
utter_slots_values:
- text: "I am going to run a result search using the following parameters:\n
rollnumber: {rollnumber}"
actions.py
class ActionShowResult(Action):
def name(self) -> Text:
return "action_show_result"
def run(self,
dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
roll = tracker.get_slot("rollnumber")
print("Rollno: ", roll)
if( roll ):
score = 98
roll = 100
else:
score = -1
roll = 0
result = "Fail"
if score >= 50:
result = "Pass"
dispatcher.utter_message(
response="utter_result",
score=score,
roll=roll,
result=result
)
return []
Description:
nlu.yml
It would be hard to configure all the possible values at times, to rescue regex is the solution.
We could see regex is used to identify the rollnumber
rules.yml
Rules are a type of training data used to train your assistant's dialogue management model.
Rules describe short pieces of conversations that should always follow the same path.
Forms are used, inorder to save the student's roll number.
Forms one of the most common conversation patterns is to collect a few pieces of information from a user in order to do something (book a restaurant, call an API, search a database, etc.).
Note: Don't overuse rules. Rules are great to handle small specific conversation patterns, but unlike stories, rules don't have the power to generalize to unseen conversation paths.
domain.yml
Slots are your bot's memory.
They act as a key-value store which can be used to store information the user provided (e.g their home city) as well as information gathered about the outside world (e.g. the result of a database query).
Here we have save the student's roll number in slot and it is extracted from the entity.
Output in UI:
Fig 5: Scenario 3 chat with bot, using slots
Scenario 4:
Fees Enquiry
Provide if only roll number was provided
Else ask for the roll number and provide the fees structure
- regex: rollnumber
examples: |
- \d{10,30}
- intent: get_roll_number
examples: |
- my roll number is [1234567891](rollnumber)
- This is my roll number [1234567891](rollnumber)
- [1234567891](rollnumber)
- intent: fees_enquiry
examples: |
- may I know the fees structure
- how much fees do I need to pay
- do I have any pending fees to be paid
stories.yml
- story: Ask for rollnumber and say fees
steps:
- intent: fees_enquiry
- slot_was_set:
- rollnumber_provided: null
- action: utter_ask_rollnumber
- intent: get_roll_number
- slot_was_set:
- rollnumber_provided: true
- action: action_save_roll_number
- action: action_fees_details
rules.yml
- rule: Only say `fees` if the user provided a rollnumber
condition:
- slot_was_set:
- rollnumber: true
steps:
- intent: fees_enquiry
- action: action_fees_details
domain.yml
intents:
- fees_enquiry
actions:
- action_fees_details
entities:
- department
- rollnumber
slots:
rollnumber:
type: any
responses:
utter_fees:
- text: "For {roll}, fees is {fees} INR."
actions.py
class ActionShowFeesStructure(Action):
def name(self) -> Text:
return "action_fees_details"
def run(self,
dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
roll = tracker.get_slot("rollnumber")
print("Rollno: ", roll)
fees = 0
if( roll ):
fees = 10000
dispatcher.utter_message(
response="utter_fees",
fees=fees,
roll=roll
)
return []
class ActionReceiveRollNumber(Action):
def name(self) -> Text:
return "action_save_roll_number"
def run(self, dispatcher: CollectingDispatcher,
tracker: Tracker,
domain: Dict[Text, Any]) -> List[Dict[Text, Any]]:
#text = tracker.latest_message['text']
entities = tracker.latest_message['entities']
roll = None
for e in entities:
if e['entity'] == "rollnumber":
roll = str(e['value']).lower().strip()
dispatcher.utter_message(text=f"I'll remember your rollnumber {roll}!")
return [SlotSet("rollnumber", roll), SlotSet("rollnumber_provided", True)]
Description:All the configuration we have learnt so far would remain the same.
But this scenario is written to test the bot's memory (slot)
The student here does not require to enter the roll number again.
And in story we have written, if the student has not provided the rollnumber it will asked and then the feel structure information will be provided.
Output in UI:
Fig 6: Scenario 4 chat with bot, using slots and conditions
Scenario 5:
Change of department request
nlu.yml
- intent: department_have_been_changed
examples: |
- I have changed from [civil]{"entity": "department", "role": "from"}
- Have moved from [civil]{"entity": "department", "role": "from"}
- intent: department_going_to_change
examples: |
- I am going to [civil]{"entity": "department", "role": "to"} department
- I am changing to [civil]{"entity": "department", "role": "to"} department
- Will be moving to [civil]{"entity": "department", "role": "to"} course
stories.yml
- story: The student moving from another department
steps:
- intent: department_have_been_changed
entities:
- department: Civil
role: from
- action: utter_ask_about_experience
- story: The student is going to another department
steps:
- intent: department_going_to_change
entities:
- department: Computer
role: to
- action: utter_wish_luck
domain.yml
intents:
- department_have_been_changed
- department_going_to_change
responses:
utter_ask_about_experience:
- text: "How was your experience with the department."
utter_wish_luck:
- text: "Wish you best luck in the new department."
Description:nlu.yml
Here we are using the feature Entity Roles and Groupswhere we need to specify the list the roles and groups of an entity can belong to.
Output in UI:
Fig 7: Scenario 5 chat with bot, using roles
Real world test data
Test conversation
Managing conversation data files modularly as it can be ease for writing and maintaining
Rasa can help us with quickly build a chatbot for use case. Adding its open source and holds state-of-the-art models in building the chatbot. The scenario given here are just the basics, there is lot more Rasa provides.
References:
https://github.com/scalableminds/chatroom
https://www.techfunnel.com/wp-content/uploads/2019/10/Best-AI-Chatbot-Platforms-that-Increase-Conversions-_-Techfunnel.jpg [credits for front page image]
Original Blog Posted in OSFY
https://www.opensourceforu.com/2022/01/using-the-rasa-framework-for-creating-chatbots/
For further research and updates maintaining the blog here.
No comments:
Post a Comment