Since Facebook and Kik debuted their chatbot channels last year, thousands of bots have been built by brands, media companies and developers on the platforms, with more expected in 2017. (Just last week, Facebook unveiled its Messenger 2.0, with brands like Subway, Aeromexico and Food Network coming onboard.)
However, while bots might seem like just another extension on social media, there is still a lot to figure out in terms of performance and measurement. And just like the early days of social media, understanding what clicks and what doesn’t is still a work in progress.
“When we were creating chatbots and thinking about it as a new medium, we wanted to go back to the drawing board to assess how these things should be measured,” said Kik product manager Laura Newton. “Although bots are still a niche product and aren’t super widespread, they cover a vast variety of use cases and we think they need to be treated different.”
Despite all the early hype, brands need to be careful to set realistic expectations for what bots can do now and what they can do a year from now.
“It’s a long-term kind of game,” said Octane AI co-founder Ben Parr, whose company has built bots for musicians and actors such as Maroon 5, Lindsay Lohan and 50 Cent. “The Messenger bot platform is only a year old, and after a year of the iOS store, there was no Instagram or Snapchat. We hadn’t hit that technological level, and we’re still at that early stage [with bots].”
Since Kik’s Bot Shop debuted a year ago, more than 2 billion messages have been exchanged with chatbots on the messaging app. Here are some examples of the types of bots and how they’ve fared.
Bots created to thrive within group chats are best measured by their retention rate and viral ratio. Roll, a group bot created by Kik’s developers, now has 1 million users, with 95 percent coming from friends rather than from the bot shop.
The value of bots that aim to entertain should be gauged by total messages sent and average messages per user. A collection of four bots created to promote the movie Teenage Mutant Ninja Turtles saw average engagement of 7 minutes, with 40 million messages exchanged. On average, users exchanged 60 messages with the bots.
Content distribution bots
Brands and publishers pushing out content should focus on the read rate of their bots. However, they should be sure to avoid merely pushing content. CNN’s bot for the 2016 campaign grew 56 percent within three weeks, with 4.5 million messages exchanged on Election Day. (Users asked “who’s winning” 516,000 times that same day.)
If a chatbot is a part of a group conversation, it should facilitate conversation between users rather than become the main attraction. It should feel “lightweight,” an added form of expression.
Forget the yardstick
Don’t worry about a single metric for all bots. Because each can serve a different purpose, their success is defined in different ways. For example, read rate might be good for content distribution, but engagement rate is better for entertainment.
Focus on one thing and do it well so that the user has a clear understanding of what the bot is intended to accomplish. Is it meant to entertain? Inform? Assist? Later on, consider adding complementary features.