Here’s how chatbot metrics differ from traditional apps

This story was delivered to BI Intelligence Apps and Platforms Briefing subscribers. To learn more and subscribe, please click here. Chatbots are being touted as the next platform in which businesses will engage with consumers. Because of this, developers must find methods to measure the success of the nascent technology, according to media and marketing firm Topbot’s head of design and engagement Mariya Yao. That’s because existing metrics used to measure app engagement, such as daily active users and session length, aren’t quite efficient at capturing the unique conversational nature of chatbots. For example, session length — used to measure engagement on apps — could be misleading when used for chatbots, since interactions may reach a successful conclusion in a short amount of time.  Instead, developers should follow metrics that more closely align with what a successful chatbot interaction should look like, Yao suggests. For example, since 40% of users only converse with a chatbot once, it makes sense to measure and identify active and engaged users.  Similarly, because the industry is still nascent, many developers are still working out the kinks in an attempt to optimize the user experience. Measuring “confusion triggers,” or incidences in which a bot replies with “I don’t understand,” can help developers identify friction areas that need to be addressed.  Looking ahead, setting standardized metrics for measuring chatbots will be necessary as brands and platforms, like Messenger, begin looking at leveraging the tech to drive monetization through things like ads. The good news is that thanks to the app market, developers are now well-versed in different strategies that go into measuring engagement, which likely means that a standard for chatbots will be set much faster than it was for apps.  Advancements in artificial intelligence, coupled with the proliferation of messaging apps, are fueling the development of chatbots — software programs that use messaging as the interface through which to carry out any number of tasks, from scheduling a meeting, to reporting weather, to helping users buy a pair of shoes.  Foreseeing immense potential, businesses are starting to invest heavily in the burgeoning bot economy. A number of brands and publishers have already deployed bots on messaging and collaboration channels, including HP, 1-800-Flowers, and CNN. While the bot revolution is still in the early phase, many believe 2016 will be the year these conversational interactions take off. Laurie Beaver, research associate for BI Intelligence, Business Insider’s premium research service, has compiled a detailed report on chatbots that explores the growing and disruptive bot landscape by investigating what bots are, how businesses are leveraging them, and where they will have the biggest impact. The report outlines the burgeoning bot ecosystem by segment, looks at companies that offer bot-enabling technology, distribution channels, and some of the key third-party bots already on offer. The report also forecasts the potential annual savings that businesses could realize if chatbots replace some of their customer service and sales reps. Finally, it compares the potential of chatbot monetization on a platform like Facebook Messenger against the iOS App Store and Google Play store. Here are some of the key takeaways: AI has reached a stage in which chatbots can have increasingly engaging and human conversations, allowing businesses to leverage the inexpensive and wide-reaching technology to engage with more consumers. Chatbots are particularly well suited for mobile — perhaps more so than apps. Messaging is at the heart of the mobile experience, as the rapid adoption of chat apps demonstrates. The chatbot ecosystem is already robust, encompassing many different third-party chat bots, native bots, distribution channels, and enabling technology companies.  Chatbots could be lucrative for messaging apps and the developers who build bots for these platforms, similar to how app stores have developed into moneymaking ecosystems.   In full, the report: Breaks down the pros and cons of chatbots. Explains the different ways businesses can access, utilize, and distribute content via chatbots. Forecasts the potential impact chatbots could have for businesses. Looks at the potential barriers that could limit the growth, adoption, and use of chatbots. To get your copy of this invaluable guide, choose one of these options: Subscribe to an ALL-ACCESS Membership with BI Intelligence and gain immediate access to this report AND over 100 other expertly researched deep-dive reports, subscriptions to all of our daily newsletters, and much more. >> START A MEMBERSHIP Purchase the report and download it immediately from our research store. >> BUY THE REPORT The choice is yours. But however you decide to acquire this report, you’ve given yourself a powerful advantage in your understanding of chatbots.