Earlier this week we published a post called "How to make your chatbot more human-like" where we expanded on some of the most common issues users experience while talking to a bot and how developers can solve them by using Natural Language Processing.
We conducted a research trying all the major bot development platforms and we realized they need a long and intensive training to provide accurate answers to users' requests.
Chatbot training is a resource intensive task. Linguistic analysis provides different solutions that speed up training and, most important, solve some structural issues with bot development.
- The case of double intent as an example problem in bot training:
Most chatbot frameworks are based around the concept of intent and entity detection, which involves identifying both the intent of an utterance and the entities relevant to that intent. For example, for the sentence “I want a pepperoni pizza”, most chatbot frameworks – after being properly configured and trained – would detect “order food” as the intent, and “pepperoni pizza” as the “food type” entity.
For simple utterances such as the previous example, most chatbot frameworks work correctly. But when users ask for assistance with more complex requests, existing solutions are often not able to cope.
For example, consider the utterance “I want a pepperoni pizza and a soda”, which has two entities “pepperoni pizza” and “soda” which could be the object of the intent. Most frameworks only support a single entity for each intent, so they cannot easily handle natural requests with two intents such as this one. The problem is known as “double intent”.
This is usually a design limitation, because intent detection is typically handled as a text classification problem, and text classification models are designed to output a single class for a given text. For example, for the sentence “I want to order a pizza and rent a movie”, there are two separate intents: “order food” and “rent movie”.
At Bitext we have developed a solution to this problem. Using our Deep Linguistic Analysis Platform we rewrite the input request into distinct sentences with one intent each. This allows existing chatbot frameworks to deal with this type of user requests without having to redesign their architectures. Instead, our system takes “I want to order a pizza and rent a movie” and outputs “I want to order a pizza” and “I want to rent a movie”, which can be processed sequentially by existing frameworks.
Our Platform also allows existing frameworks to deal with other complex linguistic phenomena like negation (“I want a deluxe pizza without mushrooms”) or conditional structures (“If it is included in the offer, add a beer”) without having to add extensive intent detection rules and hundreds of examples for training. Our Platform provides these services mainly through its syntactic component or parser, available in more than 20 languages.