Lately, chatbots seem to be surrounded by controversy due to their limitations to interact with humans using their same language. In the process of finding a solution for this issue, Natural Language Processing techniques have a relevant role to make bots not only understand language, but also make them more human-like.
However, if we want to develop a chatbot able to truly understand users and give them appropriate replies avoiding predefined options, we should ask ourselves, what do we need to communicate? If we look back to basics, the answer comes out naturally: language.
Therefore, a successful bot must be able to understand language, and here is where using a linguistic approach, based on knowledge of language and its structure becomes key.
There are three different concepts we should take into consideration while developing our bot:
- Syntax: the subfield that studies language structure: the way in which words are put together to create grammatically correct structures.
- Semantics: the branch of linguistics concerned with meaning. When we talk/write we normally do so, trying to convey a specific meaning.
- Pragmatics: the part of linguistics that studies the influence of context and shared knowledge in meaning.
Syntax and semantics allow a bot to process grammatically correct sentences, and it will understand the literal meaning of what we are saying. However, without pragmatics, our bot will never sound like a human, and what is more important, it will not understand the user when she talks like one. If the user uses an idiom, cracks a joke, or uses the word ‘it’ referring to the skirt she was trying to buy, the bot will not understand her. Pragmatics is key for the bot to have successful interactions with humans.
You can input syntax and semantic knowledge into a chatbot but if you tell it to ‘give you a hand’ or ‘break a leg’, it will understand the literal meaning, instead of ‘I need your help’ or ‘good luck’.
With the right syntactic training, it can understand that ‘give’ is a verb and ‘a hand’ is the direct object. With the right semantic training, it might understand the meaning of the individual words, but it will still fail to infer the meaning of these expressions because it does not share our knowledge of the world.
As a conclusion, Machine Learning processes need improved features or features vectors. Linguistics can add significant value to the feature selection process. By doing this you will make sure that you are feeding all the significant information to your bot, helping it to understand natural language.
If you want to know more about how linguistics can help in the process of creating and training chatbots download our benchmark.