What do curated content, personalization and machine learning have in common?
This week Earley Information Science (EIS) is hosting two excellent executive roundtables – the first is with Anjali Yakundi, an analyst from Forrester Research and the topic is personalization. Our second webinar this week is with Rowen Curran also with Forrester and the topic is about how we need to consider approaches for training AI based systems. Training the Robots will touch on several of these topics and help attendees understand that there is no magic to AI.
Personalization is all about giving people appropriate information when they need it and anticipating what they need next so they can solve their problems and satisfy their needs more quickly and easily. No one wants to waste time on useless tasks, and certainly slogging through endless links on a poorly organized set of web search results is a useless task if it does not lead me to what I want. Consider also the transformations in processes and technologies that this requires. We need to know what the customer wants and needs in many different contexts and have to have the data and content models to support pulling the correct products and related information in those contexts. We need a mechanism to segment audiences and we have to recombine content through a rules engine or machine learning algorithm that optimizes offers, content and product information.
Doing this requires rethinking multiple processes, including understanding how customers behave and when and why they need products and services under a variety of circumstances. This problem is difficult to solve but there are several ways to approach it.
In our “Anatomy of a Personalized Experience” infographic (available as a 12 page ebook here) represented in schematic form in the image above, the personalized and contextualized user experience is the result of many upstream processes, and reflects organizational maturity across a number of different areas.
Starting with product
Without good product data, users cannot find what they need, and the content management machinery behind ecommerce and user experience cannot assemble the necessary components (images and related content) with products. Search engines will not have the correct facets to index, and product details will be invisible. Therefore good product data is a must-have.
Understanding the user
Knowing the characteristics and needs of your target customers is the next most important piece of the puzzle. What product combinations will make the most sense for users? Under what circumstances? How can customers’ past behaviors provide clues about how to sell to them successfully? What products can be correlated with their current browse paths in order to increase the likelihood that they will convert? Information about user needs and interests can be mined from their behaviors and the content they consume. We can also ask questions about problems, needs, interests and in the case of B2B applications, the industry and type of job.
Mining data sources
Product combinations can be derived through shopping basket analysis, but that is not enough to determine the most important product combinations. We also need to understand the inherent product relationships at the level of specific solutions. These can be captured in a product and solution ontology. Other data sources include the knowledge of merchandizers and subject matter experts, mining of solution content from reference manuals and troubleshooting guides, speech to text translation of YouTube videos, and other knowledge base content.
What’s this got to do with robots?
Funny you should ask that. Here is the big secret of all of the AI, intelligent assistant and user support bots – they all need content for training. And the big aha: The same content that is needed to train humans is needed to train robots.
Training robots with curated, structured content
The content for personalization tuning is the same content that robots need in order to answer questions and support users. The two are really the same thing, though with varying use cases and varying specificity of results. A more defined process might have only one answer for a next step, whereas a more ambiguous use case might require more detail or present a larger result set that users can then filter on.
Chat bots are trying to help the user by providing an answer or, more frequently, walking users down a path so they can get to their answer or make a selection. The task may be complex, there may be multiple steps or the users may not know what they want until the do some exploration. In order to retrieve content for a user, the bot needs to access it in the correct context. Context is facilitated by metadata and tagging. Even if the content is not explicitly tagged, the AI system ingests and indexes the content and enriches it with derived meaning and characteristics. That can done simply through knowledge about relationships with other content, or it can be achieved through automated extraction of characteristics, clustering, or some other type of text analytics.
Search operates on the same paradigm. A user provides signals by entering a search term. The signal can be enhanced by looking at previous searches the user has conducted before (if the user is authenticated) or by looking at what was previously purchased, or by looking at what others with the same behavioral or interest characteristics purchased.
Rather than using the search term as the sole signal, the term is one of many that the search engine uses to make recommendations in addition to the result set that is returned through full text search or searching on metadata fields (facets) that use product data and attributes.
Taking a step toward AI
Curating and structuring knowledge bases and product information is important not only because it helps to short-term findability problems, but also because it is a stepping stone and foundational component of more automated approaches. In fact, many search algorithms, including those developed by EIS, use AI and machine learning without calling it AI. A chat bot is simply a front end to content. It is how you create and structure that content where the greatest value is created. From that perspective it is a good first step toward AI.