Any improvement you want to make to your chatbot must be based on real facts to achieve more favorable results; there is no time to shoot in the dark. This is where metrics have an important role to play: telling you what needs to be modified to assure a better customer experience and increase your revenue rates. What gets measured, gets managed.
General metrics of bot performance
These metrics allow you to have a bird’s-eye view of the bot conversion rates from top of the funnel to bottom.
- Users reached. Your bot needs to interact with a big network of users so that you have enough data at disposal to properly optimize its conversation skills improving, in the meanwhile, the user experience.
- Click-through rate. Measure which is the conversion rate from the users reached to those that make any interaction with your bot.
- Active and engaged users. Active users are those who just read the messages sent by your bot, while engaged users will also reply to its calls. Those users who are constantly engaged will help you enhance your bot by providing you with raw data to be analyzed.
- Retention rates. Retention rate is considered the number of users who turn back to the chatbot over a particular time period. This interval would vary depending on the purpose of your bot. If your bot helps users place orders, for instance, they must be retained until their order is closed.
These metrics focus on measure how engaged users are with the bot.
- Conversations per user: Supervising the number of sessions per user is of vital importance, since it could be a result of a job well done.
- Interactions per user: The right number depends on the aim of your bot. If its purpose is to sell, many interactions can be translated to potential clients interested in your product. On the contrary, if the main purpose of your chatbot is to provide general information, many interactions can mean that users aren't getting what they need.
- Average session per user: Sizing up average conversations per user over time is useful for bot developers to detect potential weak points and try to fix them. If average conversations across users dropped, it means that they are not engaging with what is offered. A good idea here would be asking users for more particular information to deliver customized answers.
Bot conversation metrics
These are the key metrics to assure a good user experience. If the conversation between the bot and users is good, all your other metrics will improve.
- Confusion triggers: Natural language of humans frequently makes chatbots not understand or even misinterpret what users really mean. Every time your bot replies with any version of ‘I don’t understand’ there is a call for action. Focusing on these metrics provides a great opportunity to learn about customer needs. A piece of advice: check how many of those triggers your bot shows and analyze them all to find out how to avoid these unpleasant situations. Once those are detected, it will be much easier to generate data for those intents.
- Conversation steps: Any conversation that either is too long or fall short of steps often leads to a bad customer experience. If it takes your bot a while to assimilate users’ inputs, that will increase users’ exit rates as well. Again, this metric depends on the main purpose of your bot. If it is all about straight-to-the-point conversations, your bot must not beat around the bush. While on the subject of meaningful connections with your audience, your bot must be as talkative as possible.
- Response time: Time is gold and users want quick responses. For this purpose, you can simply test your bot to check how long it takes him to respond.
Working on customer engagement is a metric that thriving companies adopt to be fully customer-focused. Remember, keeping a current customer happy is where your money really is. If after reviewing all these metrics the results are still not good enough, you’d better consider improving the conversational skills of your bot for the better.
Bitext offers you two solutions to enhance the linguistic skills of your bot: on the one hand, artificial training data to train it easier and faster and, on the other hand, a query simplification tool to make it understand exactly what the user really wants. Want to see how it looks like? Try it out in our API now.