Earley AI Podcast - Episode 25: Human-Centered AI, Personality Intelligence, and the Future of Empathetic Chatbots with Michelle Zhou

From Psycholinguistics to Conversational AI: How Juji Is Democratizing Deep Human Understanding

Guest: Michelle Zhou, Co-Founder and CEO at Juji, Inc. 

Hosts: Seth Earley, CEO at Earley Information Science 

             Chris Featherstone, Sr. Director of AI/Data Product/Program Management at Salesforce 

Published on: February 26, 2023 

 

 

 

In this episode, Seth Earley and Chris Featherstone speak with Michelle Zhou, Co-Founder and CEO of Juji, Inc., whose career spans AI research at IBM Watson, computational psychology, and human-centered conversational AI. Michelle explains how psycholinguistics - the science that language patterns reveal personality - underpins Juji's approach to building chatbots that go beyond transactional Q&A to genuinely understand users as individuals. She discusses why general large language models like GPT-3 need proprietary data and contextual layers to deliver real enterprise value, how universities are using Juji to match students with the right programs, and why the future of AI lies in empathetic automation of high-touch human services. 

 

Key Takeaways:

  • Psycholinguistics shows that how people express themselves in free text reliably reveals their personality traits, interests, and how they handle challenges.
  • Reliability and validity - not just accuracy - are the key measures for psychometric AI systems, requiring careful calibration of minimum evidence thresholds.
  • General large language models like GPT-3 are powerful starting points but cannot replace proprietary data and context-specific training for enterprise use cases.
  • Organizations must curate and format their knowledge content for conversational AI - simply migrating FAQ data into a chatbot without redesign produces poor user experiences.
  • Universities using Juji have found that students often choose majors based on family influence rather than personal fit, and AI-guided advising can dramatically improve outcomes.
  • Transparency drives responsibility - powerful AI systems that infer personality and guide human decisions must be built and deployed with ethical accountability.
  • The future of work will see domain experts supercharged by AI, executing strategy 100 times faster while AI handles the conversational and empathetic execution layer.

 

Insightful Quotes:

"In this data there are some golden nuggets, and there are some things you don't need to know - basically it's just noise. How can you pick out the golden nuggets and thread them together to show a very coherent and meaningful summary to the people on the receiving side?" - Michelle Zhou

"I want to really democratize the use of this cutting-edge technology. Not talking about big companies who can pay IBM to use this product - I'm talking about anybody who just says, hey, I want a career advisor, I couldn't afford one right now." - Michelle Zhou

"Chat GPT really gives you a glimpse of what the future could hold. But there's still a lot of work, a lot of training that still needs to be done by the specific entity, the specific organization. And this will make knowledge management and knowledge engineering even more important." - Michelle Zhou

Tune in to hear how computational psychology, psycholinguistics, and human-centered AI design come together in Juji's platform to deliver empathetic, personality-aware AI assistants that go far beyond what general-purpose language models can achieve on their own.

 

Links:

 



Ways to Tune In:

 



Thanks to our sponsors:

 

 

Podcast Transcript: Human-Centered AI, Computational Psychology, and Empathetic Automation

Transcript introduction

This transcript captures a conversation between Seth Earley, Chris Featherstone, and Michelle Zhou about the science and business of human-centered AI - from Michelle's accidental entry into computer science and her PhD work on AI data storytelling, to her 15 years at IBM Watson developing personality inference from text, to founding Juji to democratize access to AI that genuinely understands people as individuals. The discussion covers psycholinguistics, psychometric reliability and validity, the limitations of general LLMs, and the emerging vision of AI companions that provide empathetic high-touch service at scale.

Transcript

Seth Earley: Welcome to today's podcast. I'm Seth Earley.

Chris Featherstone: And I'm Chris Featherstone.

Seth Earley: Our next guest is passionate about creating human-machine symbiosis. She envisions a world where everyone will have their own AI companion - a companion who can truly understand us as unique individuals and help us in our personal and professional lives. To that end, she built a company that powers chatbots using human-centered AI. Please welcome co-founder and CEO of Juji, Michelle Zhou.

Michelle Zhou: Thank you for having me, and thank you, Chris.

Seth Earley: Michelle, give us a sense of your background. How did you get into this space?

Michelle Zhou: I'm glad you asked. I would say I stumbled into computer science by accident, because both of my parents are physicians. I didn't want to become a physician, but I really wanted to become a biologist. However, I wasn't accepted by the biology department. They said I had to pick a second major, otherwise I might not be accepted by the university. So I chose computer science almost by pointing at something randomly, because I had never touched a computer before I decided to go to computer science.

I wasn't really fond of computer science at the very beginning. In computer science you have to learn binary code, only computers understand it. I remember at the beginning many people struggled to understand why X equals X plus one. And I really wasn't a fan of computer science until I came to the US. I attended Michigan State University to study toward my master's degree. There were two professors who really changed my perspective about computer science. One professor gave me the opportunity to work on building graphical user interfaces for network and power management. That really gave me a sense of gratification - you build a graphical user interface that people can use to manage something with real-world impact.

The second professor was the late Professor Carl Page, who taught our AI class. He was very passionate. You probably know - he was the father of Larry Page, who is the founder of Google. Of course, Google didn't exist then. But he was very passionate about AI and told us lots of stories about AI in the classroom. So I decided I really wanted to do something at the intersection of AI and user interface design - what we now call human-centered AI. It's really at the intersection of human-computer interaction and artificial intelligence.

Seth Earley: So tell us about the work you did on AI data storytelling.

Michelle Zhou: So I pursued a PhD in this area. My thesis was working on creating what I called the AI data storyteller. Given a set of data, given a task of the user, and given the user's visual preferences - for example, some users want to see the overview first and then the detail, while other users prefer to see the detailed examples first and then the overview - the system would automatically generate that story. It consists of a series of animated data visualizations. Part of the system was actually used by Columbia Presbyterian Hospital to brief nurses about the patient's status - telling the nurse what happened during surgery, what kind of treatment this particular patient is under, and what treatments should be given after surgery.

After I graduated from Columbia I joined IBM Watson Research Center. I realized the storyteller for Presbyterian Hospital caregivers was not very interactive - meaning the story was a video, you could click on it and look at detail, but nurses couldn't say, "Hold on, I don't want to see this data, I want to see a different set of data" or "Can you compare this with yesterday's data?" So my first really significant project was to make an interactive conversational data storyteller, where users can interject and change the trajectory of the story based on context.

Seth Earley: That's really interesting. And how far along did that go? And you know, there's the area you touched on called psycholinguistics. What is that really?

Michelle Zhou: So in that conversational data exploration storyteller, we tried to understand the user's preferences, because in human-centered AI, one very important area of study is user modeling - how can you understand users so that computer systems can adapt to users, rather than the other way around?

My first 15 years of research was working on understanding users in a task context - what their tasks are, what they're looking for, what their visual preferences are, and their verbal preferences. Not just their psychological characteristics - for example, is this person very open-minded, or very careful and cautious? Is this person very social, or very independent? If you know these characteristics, your information presentation and your interaction with this person might be completely different.

I then worked on a project at IBM that became Watson Personality Insights commercially - trying to use users' communication text to infer people's personality. This is the line of research behind what's called psycholinguistics. Psychologists and linguists have found, going back as far as Darwin, that how people express themselves is highly related to their personality. For example, people who are very extroverted and social often use plural pronouns like "we," while people who are more self-contained are more likely to use "I."

That's the foundational theory. In the past, experts would look at your writing and try to figure out who you are - like a second persona analysis. My co-founder, who is a psychologist and a computer scientist, said that using modern psychometric theory, maybe we can extract evidence from people's free text writing, and using what's called item response theory, automatically infer people's psychographic characteristics - their passions, their interests, how they handle life's challenges.

Chris Featherstone: How did you get to testing accuracy, and how did you work against bias in those situations?

Michelle Zhou: In the psychology world they don't call it accuracy. They use two metrics: reliability and validity. Reliability means - like any measurement instrument - it has to be stable. If today I measure you and get a certain personality profile, but tomorrow using a different 500 words you come out with a completely different personality, that's problematic. We found that a minimum number of words is needed to reach acceptable reliability. IBM required around 3,000 words. Our first version needed around 700 words. Now a coming third-party evaluation paper from psychology professors suggests that around 500 words may actually reach acceptable reliability.

Beyond reliability, more importantly there is validity. In psychology, validity means: can you use the inferred characteristics to predict people's real-world behavior? Because the crux of this work is - if you learn about people's characteristics, you want to use that to help people. For example, help them be healthier, help them find a career. If you can't demonstrate that your derived personality traits actually predict real-world behavior or performance, it's useless. The concept of incremental validity is very important - meaning that the traits we infer can be used to predict certain real-world user behaviors that cannot be predicted, or are less predicted, by traditional self-report tests.

Seth Earley: All right. Let me pause a moment because we're about halfway through our time. I wanted to remind our listeners we're talking with Michelle Zhou, who is an expert in digital assistants, and we're discussing computational psychology. What motivated the move from IBM to starting your own company?

Michelle Zhou: To be honest, I never thought about working at a startup. I always thought I would be in research for life, or maybe go to a university and teach. My co-founder really persuaded me into it, for two reasons. First, because of the success of Watson Personality Insights at IBM, many people said this is a really exciting research area. But if this whole area of being able to deeply understand people just remains in research and doesn't benefit the general public - not talking about big companies who can pay IBM to use this product, but talking about anybody who says "I want a career advisor, I couldn't afford one right now" - that would be a pity. I remember in our area, people pay $25,000 for their kids to figure out what major to apply to in college. That's why I want to really democratize the use of this cutting-edge technology.

Number two, it's very hard to do this at a big company because you don't have the freedom to move as fast. So my co-founder convinced me, and we said we have to build it from scratch. Very naively, we came out not knowing what the product would be. It took us a long time to figure out exactly what to do.

Chris Featherstone: You're now working with universities and HR departments. I'd love to get a take on how AI centered around intent-based frameworks can be put into a university setting - where a student doesn't know what they want to be when they grow up, and yet they might receive a recommendation from the university that's more about filling seats in certain programs than about their actual fit.

Michelle Zhou: Actually, you might be surprised - it's the opposite. Universities really want students to get into programs that are the best fit. I remember speaking with the Dean of a local university business school. He was saying that tools like Juji would be so useful not just for students, but for the university to better manage their resources. Students at a young age don't know what will be great for them. Not every student can afford a personal advisor.

I gave a talk at this university and I asked the audience which major they were in - 80% raised their hands saying accounting. I asked, do you love accounting, why did you get into it? And they said: "Because my uncle said so," "My cousin graduated with it," "I heard it's easy to find a job." The problem is that accounting classes are now completely full. Students who can't get the classes they need take six years instead of four. But some of these students would be brilliant as marketing students or in an entrepreneurship program - they just didn't know. The university was scrambling to hire more accounting faculty. Everyone is born with a gift. Not everybody needs to be a computer scientist or an accountant. Identifying those unique abilities and making the best match - that's what the university actually wants.

Chris Featherstone: I think this is where we get into the ethics behind recommendation engines - where transparency drives accountability.

Michelle Zhou: And responsibility as well. As I often say - with great power comes great responsibility. Our AI now has this power, and we want to make sure it is used in a very responsible way.

Seth Earley: What are the lessons learned from your startup journey?

Michelle Zhou: One lesson: you really, really have to build a product that can help people. You have to achieve your customers' outcomes, not your outcomes. When we created Juji at the very first time, we wanted it to be an interviewer - to help assess candidates' fitness and characteristics. But first of all, we had to customize it for every person who wanted to use it. Second, the results couldn't be easily visualized - they had to download a huge spreadsheet. One of my dear friends gave me very honest feedback: "Michelle, your tool sounds very interesting and very useful. But in the current state, it doesn't help me at all. It just adds burden to my work - I have to ask you to customize the questions, customize the AI responses, and then analyze all the free text. Why should I use your product?"

That really woke us up. You have to build something that goes 100% of the way, not 80%. From my reading before becoming an entrepreneur, I knew about the Lean Startup and MVP. But from our experience, something that's not quite working - nobody wants it even if it's free, because it wastes their time.

Chris Featherstone: What does it look like when you go in and work with an organization around data governance and discovery? Most organizations either don't know the data they have, or they have a subset of data they want to utilize that isn't quite complete.

Michelle Zhou: Very good question. When we're working with universities or HR departments, they often say "we have this data." We always try to figure out what kind of data is available. Even with FAQ data - sometimes the answers are very, very long, because they were made for website consumption. But if you just move it directly into a chat window, people get very annoyed because it's so long. Nobody wants to scroll in a tiny chat window. So we always ask: first, what content is available? Second, what is the format of that content?

Recently our team has been discussing how, even with content, you have to decide how to best present it in context. A tool like ChatGPT already gives you a certain kind of content generation. But you still have to decide how to best present it in a conversational context. There's a multi-step process that people often aren't aware of - a lot of work needs to be put in place, and that work is not being done.

Seth Earley: The knowledge sources need some degree of curation. You still come across vendors who say their AI does it all - and that's just not realistic. Talk a little bit about general language models such as GPT-3 and ChatGPT, and how they're impacting this space.

Michelle Zhou: GPT-3 is a large language model. It gets a lot of training data and from that training data tries to synthesize and generate different types of answers. ChatGPT is built on top of GPT-3 - it's an interface that allows you to access information that has been synthesized from the underlying model.

Recently I was actually interacting with ChatGPT and I was very impressed by some capabilities. For example, I asked it to help persuade a person who is very family-oriented to quit smoking, and then to persuade a person who is very independent and self-sufficient to quit smoking. The two persuasive messages were very different. Then people said - if ChatGPT already does this, why would a company like Juji still exist?

Here's what I said: I had to put into the query "family-oriented person" and "self-sufficient, independent person." Where does such knowledge come from? How did you know which type of person you're dealing with? That's exactly the first point - in order to use ChatGPT in a reasonable way, you need context. You need parameters. A system like Juji gathers that context through conversation first, and then uses it.

Also, ChatGPT doesn't know about proprietary enterprise data. I asked it recently to compare two specific rose varieties - David Austin's Golden Celebration versus a Souvenir de la Malmaison. It had no idea what they were. None of the enterprises you work with on knowledge engineering are in the public model. It has to be proprietary data. You have to train your own proprietary models. ChatGPT really gives you a glimpse of what the future could hold. But there's still a lot of work and a lot of training that still needs to be done by the specific entity and the specific organization. This will make knowledge management and knowledge engineering even more important - it will guide tools like GPT-3 to be much more targeted and much more accurate.

Seth Earley: We know these models are getting better and we know more things are being automated. There will be disruption in the marketplace by AI. What new jobs will be created to support cognitive AI, and how will AI assistants help with job disruption?

Michelle Zhou: The new types of jobs will involve domain experts - people who know about marketing, recruitment, sales. Those people will be supercharged by computational psychology and tools like Juji and ChatGPT, so that they can actually do their job maybe 100 times faster and better. But they will remain the brain. They devise the strategy, they design what I call the business rules. The execution would be the AI assistants.

It's almost like everyone could have an army of AI systems. We want to become the platform where anyone can create their own custom AI beings - what we call AI beings rather than AI assistants. These could be your companion, could be a learning buddy, could be an advisor - really not just helping you with mundane operational transactions, but actually being what I would call your emotional partner, to encourage you, to keep you on course, because it really knows you.

Chris Featherstone: The notion of a personal assistant that understands you at scale is incredibly rich, especially for small to medium businesses that need an EA-type perspective but can't afford one. You could have a text-based and voice-based assistant in the same conversational structure, able to take action and automate specific tasks. What sets Juji apart from others in this space?

Michelle Zhou: You're absolutely right that AI assistants can help small and medium businesses. When those businesses hire a human assistant, they want that person to understand English - but that's not sufficient. They also want a certain level of social and emotional intelligence - knowing how to work with people, how to interact with people, especially when they don't have all the knowledge.

What sets Juji apart from the rest is that our goal is to teach the AI assistant what we call advanced human skills - active listening, and even when they don't know how to answer a question, doing so in a socially and emotionally skillful way. Another skill is reading between the lines - really understanding what a person wants. Thinking about it this way: if you have your own digital AI assistant and I reach out to you, it doesn't know who I am yet. But after I converse with it, it can tell you: "Today I spoke with Michelle. She seems very open-minded about technology and wants to talk to you." Then you can decide - is this the right fit for your consulting or not? That's how an AI assistant can take a step further from mundane transactional operations like taking messages and making appointments. It can help qualify, understand context, and provide empathetic automation of high-touch services that weren't possible before.

Seth Earley: Well, we've come to the end of our time today. It's been a real pleasure talking to you, Michelle. Fascinating topics, and I'm really looking forward to following more of what Juji is doing. Thank you again, and we'll look forward to perhaps having you back on to continue this conversation.

Michelle Zhou: Thank you, Seth. Thank you, Chris, for the great questions, and thank you for having me.

Chris Featherstone: Thanks, Michelle. It's our pleasure, and we look forward to working with you in the future. Good luck.
Meet the Author
Earley Information Science Team

We're passionate about managing data, content, and organizational knowledge. For 25 years, we've supported business outcomes by making information findable, usable, and valuable.