Earley AI Podcast - Episode 15: The State of AI in the Enterprise with Tom Davenport

In this episode, Seth Earley and Chris Featherstone speak with Tom Davenport, one of the world's foremost authorities on AI, analytics, and knowledge management, and the author of more than 20 books including "The AI Advantage," "Only Humans Need Apply," and "Competing on Analytics." Tom traces his career from academic sociologist to business researcher, explains why only 13% of AI models actually make it into production deployment, and challenges the assumption that AI always reduces cognitive load - arguing that it sometimes does the opposite by eliminating easy tasks and leaving only the hard ones. He discusses the Allen Curve and what it means for innovation in a remote work world, the elusive promise of hyper-personalization, Morgan Stanley's next best action system, the democratization of AI through automated machine learning, and why knowledge management - despite going quiet for years - is now essential infrastructure for any serious AI strategy.

Key Takeaways:

• Only around 13% of AI and machine learning models actually get deployed into production - organizations must think about deployment from the beginning, not as an afterthought after experimentation, because no value is created until the system goes live.
• AI does not always reduce cognitive load - it sometimes increases it by automating the easy, routine parts of work and leaving humans responsible only for the difficult, ambiguous decisions that remain.
• The Allen Curve suggests that people more than 25 meters apart have effectively zero chance of exchanging ideas - the remote work era has made everyone thousands of meters apart, and organizations are still figuring out what replaces the physical proximity that drives innovation.
• Knowledge management for AI is no longer optional - content-rich AI systems like chatbots, intelligent agents, and recommendation engines require well-structured knowledge about concepts, relationships, and products before they can function effectively.
• Hyper-personalization is still more promise than reality - after decades of "one-to-one marketing" aspirations, most organizations still cannot deliver truly personalized experiences, and Morgan Stanley's next best action system is one of the few genuine success stories.
• Democratization of AI through automated machine learning tools will allow domain experts with business knowledge to build useful models without needing data scientists, which is critical because there will never be enough data scientists to explore all the data being generated.
• "Think big, start small" - organizations should have an ambitious long-term vision for how AI will transform their business, but execute in small, deployable pieces rather than treating digital transformation as a one-time project rather than a permanent way of operating.

Insightful Quotes:

"I agree that AI reduces cognitive load - except that some of my recent research suggests that sometimes AI increases the cognitive load on humans, because it does the easy part that humans used to do and leaves the hard part for people to deal with." - Tom Davenport

"I think knowledge is the most important resource that we have. To do a good job of AI you have to have your knowledge in order. If you want to have a unique AI system, you need well-structured knowledge - whether it is chatbots, intelligent agents, or recommendation engines, you have to know the relationships between your concepts and your products." - Tom Davenport

"Think big, start small. Have a big plan for how you are going to really change your business dramatically - but it is probably smarter to do it in small pieces. And digital transformation should be a way of life for those companies, not a project." - Tom Davenport

Tune in to hear Tom Davenport discuss why Gary Loveman left a major health insurance company because it would have cost $30 million just to add email addresses and cell phone numbers to the member database, how Morgan Stanley's AI cut the time to generate a personalized investment recommendation from 45 minutes to one second, and why he is cautiously optimistic that knowledge management - after years of being overlooked - is finally returning as a first-class discipline in the age of AI.

Links:

• The AI Advantage
•  Only Humans Need Apply
• TomDavenport.com

Thanks to our sponsors:

• Marketing AI Institute
• CMSWire
• Earley Information Science
• AI Powered Enterprise Book

Podcast Transcript: Operationalizing AI, Knowledge Management's Return, and Why Hyper-Personalization Is Still Hard

Transcript introduction

This transcript captures a conversation between Seth Earley, Chris Featherstone, and Tom Davenport about the real state of enterprise AI in 2022. Tom draws on decades of research across knowledge management, business process reengineering, analytics, and AI to explain why most models never reach production, why physical proximity still matters for innovation even in a remote-first world, why knowledge management is more important than ever, and why the promise of personalization has been deferred for decades - with a few notable exceptions like Morgan Stanley's next best action system.

Transcript

Seth Earley: Welcome to today's podcast. I'm Seth Earley.

Chris Featherstone: And I'm Chris Featherstone. Good to be with you again. Last time we spoke I was in Salt Lake - now I've relocated to Montana.

Seth Earley: Montana - you're going to be a dental floss tycoon. A Frank Zappa reference from "Weasels Ripped My Flesh." Today's guest is the author of many, many books. Recent ones include "The AI Advantage: How to Put the Artificial Intelligence Revolution to Work" and "Only Humans Need Apply: Winners and Losers in the Age of Smart Machines." He also wrote "Competing on Analytics" and "Working Knowledge" - that goes way back to the knowledge management days. His articles have appeared in Harvard Business Review, Forbes, Sloan Management Review, and the Wall Street Journal. He is a world-renowned thought leader on AI, analytics, information and knowledge management, and process management. He is President's Distinguished Professor of Information Technology Management at Babson College and a Fellow at the MIT Center for Digital Business. Welcome to the show, Tom Davenport.

Thomas Davenport: Glad to be here, thanks for having me.

Chris Featherstone: Tom, I have to ask - why do you have a Texas shirt on?

Thomas Davenport: Despite my virtual background from Oxford, where I'm currently a visiting professor, I'm actually in Austin. I was a professor here in the mid to late 1990s and I realized I didn't have any Austin shirts, so I bought this Longhorn shirt to reflect my background. Burnt orange, I believe.

Seth Earley: Tom, we've known each other for a number of years, but I never really heard your origin story. How did you get into this space?

Thomas Davenport: I was trained as a sociologist, worked for a couple of years as an academic sociologist, and said - this is a little boring, writing papers for a few other sociologists in the world. I decided I wanted to go into consulting, and eventually I got a job at an IT strategy boutique called Index. I immediately gravitated toward the research side. We had a multi-client program with different sponsors and I did work on a lot of topics, including work with the late Michael Hammer. Since then I've gone back and forth between business schools and consulting firms, and I've always focused on - if there were such a thing as the sociology of business information and technology, that would be my domain. I'm always interested in how people in organizations use these tools effectively.

Seth Earley: That's really the crux of things - it's all about humans and how people are interacting with technology and reducing the cognitive load on people.

Thomas Davenport: I agree, except that some recent research I've done suggests that sometimes AI increases the cognitive load on humans, because it does the easy part that humans used to do and leaves the hard part. You're getting rid of the routine and unambiguous cases - and now here is the real hard problem.

Seth Earley: What is most important for organizations today when they're trying to operationalize these things?

Thomas Davenport: That's a big problem - getting a little better, but for the last couple of years I've been quite focused on how do we get return on AI. How do we get beyond the experimentation stage and really get some production deployments in place? Because we don't get any value unless we actually put something into production. The number of AI and machine learning models that actually get deployed is quite low. One VentureBeat survey said it was 13%, which seems crazy low. So we need to think about deployment from the beginning - not just play around with something and then wonder how to deploy it later. You have to invest a lot more to build a deployed AI system than a proof of concept, because you've got to integrate it with existing systems, get a lot of data, build an API so you can get a good answer quickly.

Chris Featherstone: A lot of organizations think this is a one-hit wonder - set a model and go. But then it's great at finding the easy stuff and they don't understand how much work the next increment requires.

Thomas Davenport: In terms of the people analytics space - this is a really exciting time. We are genuinely rediscovering how work should be done. You just moved to Montana, where I'm assuming there's not a big Salesforce presence. What does that all mean for how we work effectively and how we innovate and collaborate? Fortunately we have all these tools, analytics and AI, and lots more data in various transactional systems to help us answer those questions.

Seth Earley: What's holding organizations back from really capitalizing on knowledge collaboration tools and remote work?

Thomas Davenport: A lot of it still has to be figured out and invented. Someone mentioned the importance of tacit knowledge to innovation at a conference I did this week, and I brought up this work at MIT by Tom Allen - he's known for the Allen Curve, or what some call the 25-meter rule - suggesting that if you're more than 25 meters away from someone, your chances of exchanging ideas with them are effectively zero. So how do we translate that belief into an environment today where we're more than 25 meters away from everybody? Chris is more than 25,000 meters from most of his colleagues. Technology can clearly help, but we don't yet know under what circumstances. Seth is a big fan of getting a bunch of people around a fire pit to exchange ideas. What's the logical equivalent of that?

Chris Featherstone: Technology moves fast, organizations move slow. Is that executives moving slow while their people try to move fast, or is it the Peter principle - people rising above their level of competence?

Thomas Davenport: There is certainly some of that, particularly in the return-to-work environment, where you have some executives - I would almost want to call them dinosaurs - who say by God, the only place to really get work done is in the office, five days a week. No data to support that assertion, no analytics, no evidence at all. And I think people are going to vote with their feet. They will not want to do that.

But interestingly, at the conference I attended, there was some data suggesting the Great Resignation isn't really a new trend - it was happening even before COVID and is just continuing. People were slowly figuring out that life is too short to spend on commuting and airplanes. I have 7 million miles on various airlines and I sometimes wonder why I did it.

Seth Earley: Is it IT infrastructure slowing organizations down, or is it culture and executive understanding?

Thomas Davenport: Things are changing. Historically, technology was sort of ahead of what people could do with it, and people were content to let centralized IT groups tell them what technology they needed. But I was interviewing someone this morning at a relatively small company, talking about great new manufacturing applications using sensor data and pulling it all together. I asked whether he needed to educate people. He said, no, the head of manufacturing is the primary advocate in the organization and we are just trying to keep up.

As we get generational change - people who have had technology available in real time and on demand their entire lives - the centralized IT people and old technologies will become the biggest problem we have. Even so, all of us need to push ourselves to explore and adopt. I could do better. I was laughing with a friend recently about the fact that we still create things and send around Microsoft Word documents. I send them to a lot of companies and they immediately convert them into Google Docs and I wonder where my file went. We have to get over that.

I did a lot of work with Gary Loveman - a Harvard professor, MIT-trained economist who left Harvard and eventually became CEO of Caesars, the gaming company. He then went to a large health insurance company. He wanted to do interesting work in consumer health and proposed adding members' email addresses and cell phone numbers to the database so they could send reminders - get your exercise, eat better. The IT team estimated the cost of doing that at $30 million. Eventually he said: I cannot take this anymore, I am going to create a startup to do it, because startups do not have all that technical debt and can build it from scratch. That is one of the reasons why I find it so interesting how to change a large organization in this regard.

Seth Earley: What is your take on Gartner's prediction that knowledge management for AI will be the fastest growing segment and the largest spend?

Thomas Davenport: I don't know exactly what that phrase means, but I am a big supporter of it. I'm not quite as loyal to knowledge management as you are, Seth, because I want to move to where companies have high levels of interest. For years I did knowledge management and then people stopped wanting to talk about it. But I realized some aspects of knowledge management could be applied to the whole area of analytics, big data, and then AI - the human side of it was quite well known in knowledge management but not well known in those other areas.

I would love it to come back. I think knowledge is the most important resource we have. I am occasionally quite pleased to see companies combining AI and knowledge management. To do a good job of AI you have to have your knowledge in order. If you want a unique AI system - anything related to chatbots, intelligent agents, even recommendation engines - you have to know the relationships between your concepts and your products. For content-rich AI systems you are going to need that kind of structured knowledge. I think we will see AI helping in many areas of data and knowledge content management, but we are in the very early stages of it.

Before knowledge management, I did most of my work in business process reengineering. I keep hoping that will come back, and I have friends who have been saying it is coming around the bend any day now - for decades.

Seth Earley: Are organizations using AI for customer experience and personalization effectively?

Thomas Davenport: That is one of the areas we have been waiting for a very long time to materialize. Do you remember the book "The One to One Future" by Peppers and Rogers? I have yet to receive any truly good one-to-one marketing. Once I got a Groupon discount offer to a restaurant I actually wanted to go to, and it almost brought a tear to my eye.

Now people are calling it hyper-personalization - because we never really got personalization right in the first place and we need a whole new name. I do think there are fantastic opportunities to use machine learning to hyper-personalize interactions with customers, but I have yet to see great examples of it at scale.

One interesting case is Morgan Stanley. Their vice chairman - this was about a decade ago - had the vision that just as Netflix makes movie recommendations, Morgan Stanley should make personalized investment recommendations. They developed a next best action system. Over time they integrated it with their primary CRM system, which happens to be Salesforce, and they realized that the recommendations matter, but what is really important is making the client feel that you are engaged with them, paying attention to them, and that you care. Now it is roughly a 50/50 system of recommendations and proactive communications, and it is working very well. Their financial advisors have about 200 clients each on average. Coming up with a good personalized idea used to take 45 minutes per client. Now it takes a second.

Seth Earley: What about democratization of AI?

Thomas Davenport: We are moving toward a situation where, through automated machine learning, automated data discovery, and similar tools, we no longer have to rely entirely on data scientists or professional analysts to find out what is going on in our data. We are still in the early stages of that - not too many organizations have made a substantial commitment to that approach. But there just are not enough data scientists in the world, and there never will be, to explore all the data we are generating. If you have domain knowledge and you can combine it with the ability to create a good model and choose the right features or variables, that is going to be a superior approach. In the future we will see a lot more of that.

Seth Earley: What is in store for 2022 for you personally?

Thomas Davenport: I hope to get back to Oxford - I was there last fall, but there were really no other faculty there and my teaching ended up being virtual instruction to Dubai, so I was wondering what exactly I was doing in Oxford other than some favorable time zone benefits. My first grandchild is supposedly arriving this year, so that is a big thing. We bought a house in Santa Barbara for family proximity reasons. And I have three more books on AI coming out this year - one on companies using AI aggressively in their business, one on people who work with AI on a day-to-day basis called "Working with AI," and one on healthcare AI.

Seth Earley: Any final words for organizations that are still figuring things out?

Thomas Davenport: Think big, start small. Have a big plan for how you are going to really change your business dramatically, but it is probably smarter to do it in small pieces. And digital transformation should be a way of life, not a one-time project. Going from pilot to production is a big leap - but it is always just version one. Increment from there.

Seth Earley: Tom, this has been a tremendous pleasure. Thank you for catching up with us. And I look forward to taking you out on the Thundercat again this summer.

Thomas Davenport: I look forward to that. I have never been moved so fast so close to the water as in that experience. See you soon.