For a monolingual person, it comes as an uncomfortable reality that most people in the world (60%) speak more than one language in their day-to-day lives. It is very common that a person who speaks one language at home and then another one outside, be it at school or at work. But many more scenarios are possible.
For example, people living in countries with open borders with other nations that speak different languages are very likely to end up learning one or several of those languages. In addition to that, e-learning platforms are pleasing the curiosity of the most eager people, making it easier and easier to study foreign languages.
The internet has allowed businesses to broaden their operations worldwide and that’s why the expression “The market is the world” has become popular. With these facts in mind, if the value of AI systems such as virtual assistants or chatbots is based in being able to have conversations with anyone, anywhere, and at any point in time automating processes, mustn't the AI industry be prepared to receive multilingual inputs from customers?
Most AIs, as most NLP systems, have been trained with monolingual data, and so when they are put to work, they fail to understand most of the user queries. A high share of these failures can be traced back to multilingual inputs the system wasn’t prepared to handle.
So what can a language identifier do for bot builders?
Here are a couple of ideas to make use of a language identification tool to improve your AI solution:
- You can set it to respond in the language(s) your customers are speaking. As simple as: if they talk in French, the answer of your bot should be in French, right?
- You may be collecting data from the web to form the training corpus for your chatbot or assistant. In this case, language detection can be used as a pre-filtering step to improve its quality if you can only find bilingual sites. This is the typical case for low-density languages, which are almost always found in multilingual websites.
Of course, you can only implement any of these ideas if the tool performs acceptably for small texts, not just for the large ones, and if it allows to know exactly which parts of the conversation were said in which languages, not only the percentage of presence of each language in the text, as other approaches offer.
That’s why we are so thrilled about our new Language Identification service, which is now available in our recently renewed API platform. The advantages? It provides sentence-level language detection and high accuracy even for very small texts. Yeah, that's right! Just what you need.
Try it now for free and start talking to your users in their own language.