Earley AI Podcast – Episode 35: Applying AI to Your Industry with Sanjay Mehta

Beyond the Hype: Practical Strategies for Deploying Generative AI in B2B Enterprises 

Guest: Sanjay Mehta, Head of Industry Commerce at LucidWorks 

Hosts: Seth Earley, CEO at Earley Information Science 

             Chris Featherstone, Sr. Director of AI/Data Product/Program Management at Salesforce

Published on: October 16, 2023

 

 

In this episode, Seth Earley and Chris Featherstone speak with Sanjay Mehta, Head of Industry Commerce at LucidWorks, about the rapidly evolving landscape of generative AI and its real-world applications across industries. They explore why AI is far from turnkey for B2B organizations, how combining LLMs with search and knowledge graphs unlocks enterprise value, and what executives must consider around ROI, data quality, bias, and ethical guardrails before deploying AI solutions at scale. 

 

 

Key Takeaways:

  • Generative AI is not a plug-and-play solution for B2B, as significant complexity, nuance, and integration work are required before organizations see real value.
  • Executives must rigorously evaluate ROI before deployment, as the costs of running large language models at enterprise scale can become unsustainable.
  • LLMs supercharge query understanding and conversational search, enabling users to express intent naturally rather than relying on keyword-based retrieval alone.
  • Combining generative AI with search platforms, recommenders, and data enrichment tools creates far more compelling and reliable use cases than standalone chatbots.
  • High-quality, well-structured product data is foundational; enriching and normalizing your catalog before ingesting it into a vector space dramatically improves AI performance.
  • Intelligent content chunking, prompt engineering, and temperature controls are critical levers executives should use to govern model accuracy and reduce hallucination risk.
  • Ethical guardrails including tone moderation, privacy protection, and human escalation paths are essential for responsible AI deployment in customer-facing environments.

 

Insightful Quotes:

"It's not turnkey. From a consumer side, maybe, but from a B2B perspective, there's quite a bit of complexity, especially when you think about how it's applied across different businesses and industries." - Sanjay Mehta

"You can't just let the large language model run off and figure out your support questions. You need to specify where you're getting your data, not relying on the model generally to answer questions." - Seth Earley

"Bias is a concern, especially if you're using external data. The machine has a hard time discerning accurate from inaccurate sources, and the validity of the source is a critical piece." - Sanjay Mehta

Tune in to discover how B2B organizations can cut through generative AI hype and build practical, ROI-driven implementations that combine search, knowledge graphs, and responsible guardrails for lasting enterprise impact.



Links:

Ways to Tune In:

Thanks to our sponsors:

 

Podcast Transcript: Applying Generative AI to Industry, Separating Hype from Reality

Transcript introduction

This transcript captures a conversation between Seth Earley, Chris Featherstone, and Sanjay Mehta about the practical realities of deploying generative AI in B2B enterprises. They explore high-value use cases across retail, financial services, travel, and customer support, while examining the technical architecture of combining LLMs with search, vector stores, and knowledge graphs, and the executive considerations around ROI, data quality, and ethical guardrails.

Transcript

Seth Earley: Welcome to today's podcast. My name is Seth Earley 

Chris Featherstone: And I'm Chris Featherstone.

Seth Earley: And today our guest is a seasoned technologist and solutions evangelist with rich experience in Internet software engineering. He's the head of industry commerce at LucidWorks. Sanjay Mehta. Welcome to the show.

Sanjay Mehta: Thank you for having me. Yeah, it's great to see you. And you know, as we get started here, one of the things I like to ask is, you know, what is a common misconception about our world right now? That is being kind of overrun with hype around things like generative AI, and I know you have thoughts there. Let's jump in.

Sanjay Mehta: Yeah, a couple of things that we've been observing in the industry. The myth that it's a panacea, right? And it's gonna solve everything and that it's super easy to drive results and value, that's most likely not true, as far as we're seeing. There's a lot of complexity, a lot of nuances as it relates to even different businesses and industries and how it's applied. And really, it's not turnkey. From a consumer side, definitely, we're all seeing aspects of it being easy, from a ChatGPT or Bard perspective. But from more of a B2B perspective, there's quite a bit of complexity.

Seth Earley: And you know, we met at that conference, San Diego, I think, and I know you had a presentation, and I stole a couple of your slides because it reinforced the message so well. You were going through a number of use cases. What do you think are the most compelling and important use cases? And what do you think executives really need to know to support those types of use cases?

Sanjay Mehta: Yeah, absolutely. I think the main area right now is that folks are, there are probably technically millions of use cases. A lot of brains are working on this problem area. I think, mostly for businesses, identifying where the most value and lowest risk can be generated is key. And they really have to think about the cost, the ROI. These are very expensive, as you can imagine. You can't just let it answer every type of request or send all your data there. You have to be very careful about use and adoption and making sure you're measuring that you're getting the ROI you want. Otherwise, you're gonna end up being a sinking ship if you think about the costs that might evolve.

Seth Earley: Yeah, no, that makes a lot of sense. What are the things you're seeing that organizations want to do with this? I know there's a lot of interest in customer support, whether supporting the customer directly or supporting the agents who support the customer. You want to talk about some of those?

Sanjay Mehta: Yeah, that's a great use case. MIT had cited some recent research showing that using large language models in chat is improving resolution of inquiries and problems by over 17%. So that's obviously a lot of value. And that's a great one where you think about the cost, the cost involved in getting a human or call center agent to respond versus the cost for a chatbot using ChatGPT to respond. There's a big cost savings from that perspective.

Seth Earley: And best practices there, the chatbot and the LLM needs to know how to solve those problems. You don't want to just let the large language model run off and figure out your support questions. So how do organizations deal with that?

Sanjay Mehta: Yeah, a lot of these are foundational models, and there's a couple of things you can do. We think about prompting, you have a lot of control over the way you prompt, your prompt templates, the parameters you send in your prompt. And then what we call controlled generation, for example, temperature or penalties, things you can send where you don't allow the model to improvise too much, kind of stay close to the source. Citations are also important from a quality perspective. And most models allow fine-tuning and training. Using historical QA data or synthetic QA pairs, synthetic training data generated using these models, to help refine capabilities and eliminate potential risk in interactions.

Chris Featherstone: You mentioned this notion that nobody was really talking about this until about December, and then it started to come on strong. And I see that especially, like you were saying, there's nothing that is a one size fits all, especially in the consumer world. But how has that consumer bias affected business?

Sanjay Mehta: Yeah, that does vary by industry. In retail, there's definitely a lot of bias in certain use cases, generating content for campaigns, sites, SEO. Versus if you look at financial services, the focus might be more on driving productivity of analysts and researchers who are mining data. In retail, we might see more consumer-facing personalization, versus an internal use case that's more focused on roles. And of course, privacy is a big focus in compliance-heavy industries versus enterprise workplace applications.

Chris Featherstone: Have you seen folks missing the ball, as well as way ahead of it? And how are you helping pull that back?

Sanjay Mehta: Yeah, I think we're building really concrete solutions. It's a fallacy that GPTs can do everything alone, it actually has to be combined with other capabilities to create a compelling use case. In our case, it might be combining it with search, with recommenders, with data enrichment. Just plugging a chatbot into your site, there's a lot more required, and you might not get the results you expect.

Chris Featherstone: And I think people get a misconception that it's a one size fits all for all of AI. And the flip side is knowing how to use discriminative approaches to figure out patterns and pull that into a scenario where you can utilize a generative model to create something really interesting, especially as a filter for human-related content. Plus the hundreds of models out there today that keep exponentially growing.

Seth Earley: Talk a little bit about your space, search, and how it's impacting search. Because my belief is search is an integral tool, especially if we have a target knowledge or content source and we want to turn the temperature down and tell it to only use the sources, and then process the query plus process the results. How do you see LLMs in search?

Sanjay Mehta: Yeah, in its simplest form it supercharges query understanding. The primary use case is conversational search, not just lexical or semantic, but where you can use your natural language and let the machine understand what you mean. That's particularly important for voice applications. We're getting used to machines doing certain things for us, but now having more of a free-form, natural conversation and extracting intent, not just entities and results, but really extracting the intent, is key.

Seth Earley: I say it's autocomplete on steroids. What it's doing is understanding how phrasing is ingested into a vector space and looking for common terminology and common intent for different ways of representing ideas. And then that becomes the intent that can be compared to other vectors which have the answers. You'd agree with that?

Sanjay Mehta: Exactly. And the product catalog use case is a big one. Most folks in my 20 years in industry have always struggled with product data, taxonomy classification, labeling. This is really where you get the intelligence available around the entire web to actually inform your products, enrich them, and inform your categorization, based not just on your own opinion of how the catalog should be, but on what the universe of content in that industry is telling you.

Seth Earley: And I think that's a double-edged sword. You don't want to be too standard and look like everybody else, you still want some differentiation. But some things don't have to be that creative. I asked you at the conference: did you ingest the product catalog into the vector space?

Sanjay Mehta: Yes. And in that case you need to start with good product data. There's the enriching of the product data that generative AI can help with. Once you enrich it and normalize it, then you ingest it into the vector space. Most folks already have some level of description or title. The key is using AI to enrich that data, and then serving it intelligently through search.

Sanjay Mehta: We support graph databases and knowledge graphs too. Vector databases are a big piece to support neural hashing, combining approximate nearest neighbors with embedding proximity. What you're trying to get is nearest neighbors for certain topics, using cosine similarity or neural hashing to find the proximity between certain embeddings. And then there's the query engine: analyzing queries, wrapping queries to different models, breaking down and creating snippets, and then filtering and faceting the navigation depending on your use case.

Seth Earley: Talk about the laundry list of things executives need to think about when looking at LLMs, corporate data, and search and retrieval. What does that full checklist look like?

Sanjay Mehta: So the models can cite their sources, that's used for fact-checking. Retrieval augmented generation means you already know what you're giving to the model. For folks with very sensitive data, it gets into what snippets of a document you actually can send, breaking a document down into chunks and determining which chunks you may allow. Identifying the right models or technologies like LangChain to use multiple models together. Training using historical data or, in retail, things like Google Trends to seed your models. And executives need to consider measuring attribution models, for each use case, measuring conversion events, revenue events, deflection events, resolution time, those sorts of things.

Seth Earley: When is it more beneficial to take knowledge from a knowledge base and ingest it into a vector store, versus searching and retrieving and then processing that output with an LLM?

Sanjay Mehta: From a search perspective, there are a lot of pre-existing knowledge graphs for different industries. When you're building your index, that's a great vehicle both to enrich data and to understand users' context. It links all the topics in a very similar fashion to a vector database, but it's already built and pre-trained. You can use a knowledge base to build a vector or combine knowledge to build your own vector space. But vector databases have more dimensions, context layers that keep going deeper, you don't see that kind of traversal in a graph database alone.

Seth Earley: And what are your thoughts about data quality, metadata, and chunking?

Sanjay Mehta: Yes, that's correct, data quality is critical. From an executive perspective, cost is a big one, you'd want to send only the right data. You're paying by tokens, and input sizes are limited. So intelligent chunking is absolutely necessary. When thinking about model training, bias is a concern, especially with external data. There's a lot of fake news out there, and the machine does have a hard time discerning accurate from inaccurate sources. The validity of the source is a critical piece.

Seth Earley: What keeps you up at night?

Sanjay Mehta: Bias and adversarial attacks. There are things that put certain models at risk. Are the responses the machine is generating causing more harm than good? That keeps everyone awake. Fortunately, there's a lot of research being done to detect and mitigate those things. Guardrails are what we all try to put in place to eliminate as much of that risk as possible.

Seth Earley: Can you talk about a recent problem you've solved? A class of example without naming the client?

Sanjay Mehta: Travel and hospitality is an interesting space. People are basically re-thinking travel. You want to go on vacation and you define parameters, kid-friendly or pet-friendly, specific activities you like, Wi-Fi, open bar, whatever it may be. It's rich data that's more of a dialogue, as if you were speaking to a travel agent, a virtual travel agent. That's one example. Food and beverage is another, virtual sommelier use cases, drawing on nutrition and dietitian data for personalized recommendations. And then on the back end, workplace employee productivity in banking, finance, and healthcare, a co-pilot summarizing millions of documents. Information discovery where instead of reading massive clinical trials or corporate filings, you can interrogate the data and get just the answers you need.

Seth Earley: Have you experienced any ethical concerns or things you want the audience to think about?

Chris Featherstone: Yeah, you alluded to some earlier, any specifics?

Sanjay Mehta: Ethics does come up quite a bit. The tone in conversational AI, is it recognizing the person's emotional state? Making sure it's not responding in an offensive way. Making sure it's not using customer information without their knowledge, especially if customer data from history is being used to inform the model. Sentiment and tone of the end user, making sure you route to a human when it gets complex or emotionally charged dialogue. Those are the big ones when we think about guardrails, workflows, and orchestration.

Seth Earley: I wanted to switch gears and ask you about Sanjay the person. How did you get to where you are?

Sanjay Mehta: I was always interested in technology. Back when PCs came out, Atari, Commodores, Apples, having exposure to that equipment and gaming at a young age in the Chicago area helped me. Programming became something I was very interested in early on. Then through academia with data science, computer science, psychology, and economics. I saw the Internet opportunity in the late nineties, survived the dot-com boom, and worked in Internet applications, media, personalization, ecommerce, cloud computing, AI search over the years. I've been acquired by Oracle three times. Most of my focus has been in the ecommerce arena, which I've built a real passion for.

Seth Earley: If you could go back after graduating college, what advice would you give yourself?

Sanjay Mehta: Taking bigger risks. Especially at a younger age, you get a little comfortable. Don't be afraid.

Chris Featherstone: Where can people find you?

Sanjay Mehta: LinkedIn is the best way, I'm very active there. A lot of papers and webinars. And through LucidWorks, as Head of Industry there.

Chris Featherstone: I know you live in Southern California. Tell everybody about what you do outside of work.

Sanjay Mehta: Yeah, my passion for music, I had a music scholarship in school. I play piano, I still play, I played professionally and did some tours. A lot of jazz and blues stayed with me in Los Angeles. And I have a hobby in farming, I keep an avocado grove in Southern California and a citrus grove in Riverside. That keeps me busy.

Seth Earley: Well, Sanjay, it's been wonderful to have you. Thank you so much for your time and for sharing your knowledge and your expertise and your experience. It's really been a pleasure.

Sanjay Mehta: Cool. Thank you again for having me, I enjoyed it.

Chris Featherstone: Yeah, that's great. Thanks, Sanjay, appreciate it.

Seth Earley: Thanks to our audience for listening, and I hope you learned something today. Again, Sanjay, Chris, thanks for being on the show.

Chris Featherstone: Thanks, Sanjay. Appreciate it. Have a great day everyone.

 
Meet the Author
Earley Information Science Team

We're passionate about managing data, content, and organizational knowledge. For 25 years, we've supported business outcomes by making information findable, usable, and valuable.