Earley AI Podcast – Episode 36: AI Developer Tools and the Path to Production with Amar Goel

From Demos to Real Products: Navigating AI Costs, Governance, and the Developer Productivity Revolution

Guest: Amar Goel, Founder and CEO at Bito 

Hosts: Seth Earley, CEO at Earley Information Science 

             Chris Featherstone, Sr. Director of AI/Data Product/Program Management at Salesforce 

Published on: October 30, 2023

 

 

 

In this episode, Seth Earley and Chris Featherstone speak with Amar Goel, founder of Bito and former founder and chairman of PubMatic, a publicly traded digital advertising company. They discuss the gap between impressive AI demos and reliable production-ready products, the hidden costs of deploying LLMs at scale, and how AI developer tools like Bito are transforming software development. Amar shares candid insights on enterprise AI governance, data privacy concerns, and why behavior change—not just technology—determines whether developers truly adopt AI tools. 

 

Key Takeaways:

  • Building a compelling AI demo is easy, but turning that prototype into a reliable, production-ready enterprise product remains surprisingly difficult.
  • AI inference costs are not trivial—GPT-4 runs roughly 30 times more expensive per token than GPT-3.5, forcing businesses to rethink pricing strategies.
  • Ground truth and structured context are essential; proper code ontologies and vector embeddings dramatically improve the quality of AI-assisted developer tools.
  • Enterprises are managing AI adoption through AI councils, security teams, and procurement processes, with governance models still rapidly evolving across industries.
  • Developer concerns center less on documents and data and far more intensely on protecting proprietary source code from leaving internal infrastructure.
  • Shadow IT is accelerating AI adoption despite official bans—employees use ChatGPT in browser windows even when enterprise policies prohibit AI tool usage.
  • Over the next five to seven years, AI is projected to deliver a 5–10x productivity improvement for software developers across code quality, security, and test coverage.

 

Insightful Quotes:

"Making a demo is really easy. But actually then making a real product is really still quite hard—one that people want to use and that actually does what you care about." - Amar Goel

"We don't know what we don't know yet about AI ethics and privacy—everyone is learning on the job." - Amar Goel

"Using a knowledge architecture increased the validity of answers from 53% to 83%. It still comes down to knowledge and content and data quality." - Seth Earley

Tune in to discover how AI is reshaping developer workflows—and why the real challenge isn't building the demo, but navigating costs, governance, and human behavior change to get AI reliably into production.


Links:

Ways to Tune In:

Thanks to our sponsors:

 

Podcast Transcript: AI Developer Tools, Ground Truth in LLMs, and the Journey Behind Bito

Transcript introduction

This transcript captures a conversation between Seth Earley, Chris Featherstone, and Amar Goel about the common misconceptions surrounding generative AI, the complexity of building production-ready AI products, and how Bito is helping software developers leverage AI to write higher-quality, more secure code. The discussion also covers enterprise governance models, the real cost of running LLMs at scale, and why human behavior change remains the biggest barrier to widespread AI adoption.

Transcript

Seth Earley: Welcome to the Early AI Podcast. I'm Seth Early.

Chris Featherstone: And I'm Chris Featherstone. And we're very excited to introduce our guest for today.

Seth Earley: And we're going to discuss the new wave of AI tools, ways to incorporate ground truth in things like large language models, which is very important, how AI developer tools are really going to change the industry, and we're going to hit on a number of other topics. Our guest has been a trailblazer in the world of technology and entrepreneurship. He has an impressive track record spanning two decades. He's currently the founder and CEO of Bito, where he continues to shape the future of the emerging AI industry. For the past 17 years, he served as the founder and chairman of PubMatic, which was a public company, establishing it as a powerhouse in the digital advertising landscape. He's also made contributions as board member for Finexcel, and he's brought a wealth of experience to the table. We're really honored to have Amar Goel, a true leader in the tech world, on the show today. Amar, welcome to the show.

Amar Goel: It's nice to be on the show. And I'm kind of laughing as you're reading the bio. I think my wife would have a very different bio.

Seth Earley: I know it's always hard when someone's reading your bio, you're like, maybe I should just listen out loud to myself, right?

Amar Goel: But excited to be here and a student of AI like so many of us now.

Seth Earley: Okay, well, I want to start off with a question that I usually ask our guests, and that is, what are the myths and misconceptions about AI today? It's moving so fast, it's moving so quickly, and it's really hard to kind of keep up with this. And what's really surprising and interesting to me is really how quickly some of the organizations we're working with are adopting some form of generative AI, even with some of the problems that we have with large language models and ChatGPT. But tell me, what are the things that you see as the biggest misconceptions or the biggest fallacies in the space?

Amar Goel: Yeah, I mean, there's definitely a couple. So I think the first I would say is if you think back to, let's say, when ChatGPT came out in, I think, November of last year, you know, it was kind of mind-blowing for a lot of people. And it was obviously new for the vast majority of people. And it was really amazing. And every day you would see something in the headlines on Twitter, some demo where somebody would say, oh, I just rigged this up over the weekend and it would look incredible. Like, oh my God, I could make a picture, or I just made a video based on this thing, or where we help developers—I can make this thing where you can now write an entire program in 5 minutes. Or writers were showing all kinds of things like, I just wrote a book using ChatGPT.

Seth Earley: Yeah, exactly. 2 years, and today it took me a day.

Amar Goel: Exactly, you know. And so I think it seemed at the time like, oh my God, everything's gonna change overnight. And there was this whole idea like, oh my God, everyone's gonna lose their job and all this kind of stuff. And I think if you fast forward—

Seth Earley: That's right, that's what I thought when I first saw it. I was like, oh, is that the end of knowledge architecture and information architecture and all the stuff I've been doing for 25 years?

Amar Goel: Yeah. I mean, I'm on text threads with other tech CEOs and people are like, this is gonna change everything. You don't even need software anymore and everyone's gonna lose their job by the end of next year. And other people are like, what are you talking about? And it's like totally huge differences of opinion. And so if you fast forward here, we're not quite a year past ChatGPT, but almost—10 months, 11 months. In some ways I would say like everything's changed and nothing's changed. I mean, if you look at how you're still doing podcasts, Seth, you're not out of a job. I'm still working my butt off trying to make software. It turns out that making a demo is really easy. But actually then making a real product is really still quite hard—one that people want to use and that actually does what you care about. And so I think turning these quick demos, these quick prototypes, into a real product that actually repeatably, reliably works has proved to be non-trivial.

Chris Featherstone: I was just letting Seth know that I had a thought for you too. I'd love to get a sense, because we see this get announced around the beginning of the year and then the flurry of activity and then everybody pivots to this idea that that's AI, right?

Seth Earley: You mean this idea that I'm an AI, everything's AI now. Yeah.

Chris Featherstone: Yeah. And broadly because it was released to consumers and everybody was like, no, we've been doing this a long time. How much time do you find where you're having to reeducate, in addition to educate, in terms of pulling them back? Okay, hold on guys, don't get ahead of yourself. Let me pull you back. Let me reorient you on where we're at. And then let's get you pointed in the right direction. Do you find yourself doing that quite a bit?

Amar Goel: I find myself doing so. Yeah, no, I think definitely, because I think one of the things we've seen is that a lot of people just haven't had the time and energy to get deep into this. And so they're kind of like, I mean, they read the headlines, but in terms of really understanding what's happening or what's possible, that isn't something that they've really done. So I mean, I have a friend who runs a software company and he was like, hey, I haven't really spent time on this, and part of it's because it's not clearly obvious in 2 minutes that it's gonna turn his business upside down or that there's some crazy big opportunity. So it's just sort of like, yeah, I haven't really spent any time on this yet. And hey, what is this all about? So I think there's a certain element where people are like, is this like another crypto moment? I mean, I hear that come up a lot where they're like, yeah, crypto—you can argue about whether it's a fad or not. But there's definitely a sentiment that it kind of rose and then crashed. So is this just another kind of Silicon Valley's gone run amok? And then now we'll return back to reality. So I think there is a certain amount of education and reeducation where some people just haven't gone deep on it yet. And it is a new tech stack and it requires—it's not like rocket science to understand it from a usability standpoint, but it takes a minute and there's new frameworks, new approaches, new technologies. RAG, retrieval augmented generation, is coming up in a big way. So it's like, well, what does that mean? So there's just all kinds of different components that people are thinking about. The first thing I've noticed is that making that demo is really easy, but then actually turning it into a real product is hard.

Seth Earley: Right. Well, I found that executives and many stakeholders had a lot of misconceptions about AI before ChatGPT. So there's already a lot of level setting that had to be done. There was already a lot of misconceptions in the C-suite. But then you also have parts of organizations that have gone through a very rapid maturity evolution. And we're talking with one large bank recently, and they had some very sophisticated questions. And what we were introducing to them was retrieval augmented generation, because that will eliminate hallucinations if you do it correctly, turn down the temperatures so that it's not creative. And then the idea is to say, what's a creative result versus what is a factually inaccurate result? Creativity is fine. We have to be factually correct and factually accurate. And then designating a particular data source and saying, if you don't have the information from this data source, say I don't know—that stops the LLM from using its own language understanding from extrapolating and creating something. So I think that there's a lot of education that needs to happen. And organizations are still kind of stuck on this, "Hey, is this the next magic bullet?" And do we have to curate our information? Do we have to have a knowledge base? When I start talking about knowledge architecture, I actually have a result of a study where we found that using a knowledge architecture increased the validity or the accuracy of answers from 53% to 83%. Without the knowledge architecture, it was 53% accurate. With it, it was 83%. And what that did was it helped contextualize those answers and provide the confidence that those answers were correct. So it still comes down to knowledge and content and data and data quality. And no matter how many times people kind of hit on that theme, there's still that belief out there in the marketplace that this stuff is going to obviate the need for data curation or data hygiene or content curation or knowledge hygiene. So are you seeing that and how are you kind of mitigating that or addressing that when you come across it?

Amar Goel: You mean just where people are kind of unclear about the need for this whole pipeline of information, if you will?

Seth Earley: Right. So the whole structure of the reference architecture, the ontologies, the metadata—what is your experience there? Because even a lot of vendors are not quite understanding that. I was interviewing one vendor for an article and they said, oh, we don't need any of that. We don't need taxonomies, we don't need ontology, we don't need metadata. And I said, well, what do you do with the data? And they go, well, we have to do some data labeling. And it's like, duh, data labeling is metadata.

Amar Goel: Yeah. No, I mean, I think we launched a product recently where we deploy a vector database on your machine and we index all of your code and build embeddings. And I think we're finding there that there's a lot of complexity to do that well. One of my co-founders who runs product was like, we could probably just build a company around this. There's so much complexity around just understanding—we kind of mostly live in the world of code and code has its own nuances. Like you said, it has an ontology and a taxonomy. Every language is a little bit different. What's the structure? Understanding what's a function name, a class, variables—all those kinds of things matter. When a developer's asking a question or trying to do something, if you can bring that appropriate context and an understanding of the code to bear, then you can provide a much better result. And you want to provide that context to the LLM to help inform it in the appropriate way. So we're definitely spending a lot of energy on those kinds of things.

Or I'll give you another example where building an agent that allows you to explain a code base, document a code base—originally we started with just basically going through every file in a folder and just passing it to the LLM saying, explain this. And that's a good start. But then people were like, but that's not really how you navigate a code base. I want to really follow the code flow—this is the main module, this is the entry point, then where do you go? So again, it's kind of like, well, now you gotta understand how that code flows. These are the functions first, and then if you follow down this path, you would then visit these functions. So it's understanding the organization of that dataset, if you will, and then bringing that to bear. So all of those points you're mentioning are super relevant. And if you go back to what I was saying earlier, in the quick 10-minute demo that people want to make, you just ignore all of that complexity and you pass the function to the LLM and you're like, oh, it explained it, we're good to go. And it's like, well, that's not the most useful thing.

Seth Earley: It really is. And I think you really hit the nail on the head when you talked about the fact that it's so easy to demonstrate a concept, but going from that demonstration to something that's truly production-ready—that could be a half-million-dollar project.

Amar Goel: Yeah, totally, 100%. And actually, there's one other thing I really want to mention since you just mentioned money. AI costs are not insignificant. You know, you look at different models and their costs can be really significant. On one hand, you have Moore's Law working for you—the cost of these chips are gonna keep getting cheaper. But you're also seeing the LLM providers themselves do a lot of optimizations to make their inference engines more powerful or more cost effective. So you're getting those benefits, but at the same point, new capabilities keep getting released. And so it's a little bit like, well, if you stay at the cutting edge, your costs don't really come down. But the other point I really want to make is that the cost to run these models to perform outputs is not insignificant. What I mean by that is if you look at GPT-3.5, that costs X per 1,000 tokens. Well, running GPT-4 is basically about 30 times more expensive. 1,000 tokens of GPT-4 is 30 times more expensive than that. And just to put that into dollars and cents, 1,000 tokens—which would be about 3,000 characters—costs about 4 cents to process input and output. So on one hand you say, ah, 4 cents, 3,000 characters. But if you gave it like a 10-page section of a book and you said, hey, could you summarize this, that might cost you like a dollar. And so as a consumer you're like, well, I did that a couple times a month, no big deal. But for the company doing this thousands of times a month, those costs are significant. A lot of people are like, hey, I want to include this in my product and give this away for free. It's not so easy. And that's why you're seeing a lot of companies saying, hey, if you want the AI capabilities for our tool, that's an extra $5, $10, $20, $30. My point is just that it's not like compute and storage costs where for the average app it's a couple cents a user per month. If you got another million users on your AI app, it might cost you $100 grand a month—or maybe half a million dollars a month depending on what they're doing.

And then even just getting capacity to some of these high-end models is hard. If you look at OpenAI GPT-4, they have an 8K model and a 32K model. The 8K is 8,000 tokens that you can handle input and output. Most people don't have access to the 32K model 6 months after it was announced. So in theory it's available, but in reality it's not really available. And so all of these things also limit what you can do. Now Anthropic has some amazing models. Their Claude 2 model has 100,000 context tokens and it's one quarter of the cost of GPT-4. So anyway, my point is that there's a lot of complexity in managing these models—managing your costs, thinking about capacity, throughput, context length—which is kind of a new paradigm. It's a little bit like when cell phone plans first came out—how many minutes do you get? And I'm sure 10 years from now, it'll be like water, no big deal.

Chris Featherstone: Well, yeah, maybe we'll get rollover tokens at some point, right? Then we can go to the next month.

Amar Goel: We actually put that in our paid plan launch—we said you get 100 GPT-4 requests, which users don't love so far. But we said they don't roll over. Maybe I need to roll them over.

Chris Featherstone: I'd love to get a sense too, since you guys are deep in the developer community. Right now all these code companions are just, hey, let me see what you wrote and let me look forward a bit, a few lines. We do it from the commenting side, but we're missing good core refactoring. We're missing good coding concepts because we see a lot of organizations that are stepping into it and just doing the baseline. However, they don't know what good, well-formed code is yet. And coding practices that are good because there's also a lot of semantics, and then you get code migration. I'd love to get your sense around—with large language models, code migrations, code modernization techniques—how are you seeing all those things come together? How do we get legacy to new? And then what do we do from the perspective of what I'm doing real time? Do we see those things converging in your mind?

Amar Goel: I think there's room for AI to help in all of these things. Let me start at a high level and then come down into some of the specifics. One of the big trends over the last 5, 10 years is about shift left—do it earlier in the process. Security by design, privacy by design, test by design. And I think for the developer, they're kind of like, this is a lot. I don't really get the time to just think and write code anymore. There's so many other things I'm supposed to be doing, and I'm not an expert in all of these things—I'm not a security expert. And so people bolt on tools to address some of those issues, but I think it's made the modern developer life a little bit miserable. And we actually think this is a big opportunity for AI to come and help, which is like, hey, let us take some of this drudgery off your plate to free you up to be who you want to be. Let us help take some of that test load off your plate. Let us help analyze all your code for security issues. You mentioned tech debt and refactoring—huge issue. One of our angel investors is Sri Sivananda, the CTO of PayPal, 10,000 engineers. And he's like, why can't we just use this—we got a new version of React, why do I have to have teams working on this for weeks and months to upgrade? And the business is like, what did I get from all that time spent upgrading the version? He's like, why can't I just put AI on this and have it do it?

I think the issue becomes like, well, when PayPal is going to use it, when a Fortune 1000 company is going to use it, that stuff better work. And that stuff better be right and work well. And that's where the complexity really creeps in. We're also seeing developers all across the spectrum. 40% of the developers using Bito today have over 5 years of experience. So a lot of people think it's only newbie developers. But it definitely helps everybody. I have a 9-year-old son who basically used Bito to make a Chrome extension in 2 minutes—he gave it a product name and wanted a little popup that searches eBay, brings back a list of all the products and ranks them by price with clickable links. It produced all the code, he ran it, and then he's like, I don't really like the UI, can you center this and move it over? And it rewrote it. He's a bright kid, but he shouldn't be able to do that at 9. It made him like a 15-year-old developer who knew what he was doing. And even if you're a pro, it would still take you an hour to do that. So it democratized that access. Going back to that idea of helping developers write high-quality code, secure code, and properly refactored code—I really think we should live in a world eventually where all of that's just kind of getting taken care of for you. And as a developer, you should be able to move at a much more rapid pace, produce much higher quality code, more secure code, more performant code, and have much tighter processes with better test coverage.

Chris Featherstone: I like the shift left concept in theory because it does bring the microscope into the right areas. As a product manager for a long time, I utilize these types of techniques to say, okay, let me create a requirements document that can break it down into work items—from epics into bite-sized chunks—and then feed that to the work item tracker, Jira, whatever, and potentially stub things out. However, it takes a very specific type of product manager to do that level of work. But now we're getting into some of these areas around—can I stub that out? Should I stub that out? How much should I give the developer to not do? Because I still am a fan of wanting them to be creative and get into the nuances of why they're building things and not take away all their creativity.

Amar Goel: Right. And I think it's a little bit about what is the core work that the developer is doing, and what is sort of the ancillary or add-on things? It's not to say those aren't important. Obviously security is important, privacy is important, test coverage is super important. But for most developers, those aren't the things they're really strong at. All these tools that have been added on or gates put into the process are a little bit like bolt-ons.

Chris Featherstone: Are you having to get into indemnification discussions and warrants centered around—if you generate code for me and it doesn't work or causes some type of exploit?

Amar Goel: Yeah, it's come up a little bit. It hasn't been a huge topic of concern, but it comes up. And I do know there are some organizations that have completely outlawed these technologies right now. I was talking—I won't name any names—to a huge cell phone and cellular infrastructure company, and they basically don't allow any of their developers to use any of these tools. That was really, really rare to me. I was also talking to a consumer internet company—pretty big, 3,000 software developers—and a senior executive there said, you know, we just decided that the reward is worth the risk. This is gonna help make us so much better that we're okay with the potential risks around this.

I would say we get a lot more questions about protection of my data. Is my code being used to train a model? What's happening with my code? It's interesting—people generally don't seem to care as much about their documents and their data, like numbers. They're happy to plug in all kinds of SaaS tools and OAuth their Salesforce data in. But the moment it comes to their code, every hair on their neck goes up. They're like, hey, what happens with my code? Where is it stored? How is it treated?

I mean, we had a call from a top 5 bank. I was surprised they called us—we're a small startup. But some of their developers were using our tools for open source work, and so they wanted to see about using it in their organization. And they were like, no code—frankly nothing, but no code for sure—can leave our four walls, can't leave our private cloud. I was like, well, your code doesn't need to go to the LLM for it to operate on it. And now there are a number of pretty cool services like Amazon Bedrock that allow you to keep the inference part in your own private cloud. So I think I see this as becoming a solved problem. Llama 2 from Facebook is pretty decent, and maybe Llama 3 will be there. So between a number of options developing—tools that allow inference in your private cloud, standing up your own model—I think a lot of these privacy issues are gonna go away.

Chris Featherstone: It's interesting. There's that whole notion—it feels like defined irony where we're getting into these discussions. Their legal teams have representatives in meetings. They don't understand the technology. They're asking these questions, and yet we're talking about all their infrastructure that for the last 5 years has been put in the cloud—and we had no legal discussions then. But now we're talking to you because this scares the hell out of people. What is the paranoia and the FUD that you're having to work through?

Amar Goel: I think I would agree with what you were just saying. I think they're a little bit like, we don't know what we don't know yet. We're still trying to figure out what are the right questions to ask. I mean, we were on a call with a top 5 travel company in the US—some developers want to use the tools, security procurement are involved. And there was a little bit of like, hey, what are the things that we do care about here? And it wasn't like they haven't done this enough yet where they have a game plan in place. I mean, I kind of get it—it's new and everyone's learning on the job, if you will. But it is interesting where there's concern—and then you read those articles, I think it was Samsung, where some developers had put something into ChatGPT. And of course a thousand articles come out about it and it makes everybody freak out.

Chris Featherstone: Fortune favors the brave. I've always been a fan of that kind of stuff. And so I've actually talked to that team and I'm like, listen, I'm not going to consider you guys did anything bad because I think you pioneered some stuff. There were two historical moments—they released the model and then we have these pioneering experts that were like, let's try it and see what happens. Fail fast. And the problem is there was an exploit, and the mob's fickle, especially in the public world. So we got all the adverse reactions. But it did cause a lot of people to take a step back. Did it cause the pendulum and some knee-jerk reactions? Absolutely. But did it ask some really good questions? Absolutely.

Amar Goel: I think all these things just slow adoption—and I don't mean that in a negative way. They're just reasonable concerns that people have. And some companies will say we're not gonna use this or we need it done better. I mean, I actually think it kind of happened in the cloud too. I remember around 2005 or 2006, I have a friend who's a pretty smart analyst on Wall Street and he's like, Salesforce—what a joke, man. What company is going to put their data in the cloud? And I was like, oh yeah, is that what you hear from your customers? And clearly he was wrong. But it didn't happen overnight either.

Seth Earley: No one's gonna use a credit card to buy something on the internet. It's crazy.

Amar Goel: Yeah, exactly. All these things are just a cycle. And enterprises are of course way—so I'll tell you another funny thing. We were doing a call with a media company and we had a team of maybe 10 engineers on the phone. And the manager was on the Zoom too. And he was like, well, I don't know if we can use these tools—security is gonna have a lot of questions. And then one of the engineers goes, isn't it funny that security is gonna stop us from using these tools, but everybody has ChatGPT up in a browser window and they're using it all day long? And it was sort of like, it's the whole bring-your-own-tech-to-work kind of thing. All of the employees are like, I need to get this task done. This is a useful tool. I'm going to use it until you block it.

Chris Featherstone: Shadow IT. Exactly. Shadow IT. Well, that shadow IT is funny because then people start bringing in things like, well, okay, I'm going to do it on my personal iPad and I'm going to copy and paste it across. So talk about the shadow of shadow IT.

Seth Earley: So it kind of begs the question of governance and decision-making, right? And governance is usually a bad word in a lot of organizations. When can you meet to talk about governance? And the response is, how about never? I like to call it metrics-driven decision-making. How do you make decisions and how do you measure them? So where do you see that kind of evolving? Who owns it? It seems like a lot of people have to be brought to the table. Governance has always been a challenge with organizations—you stand things up, you get them deployed, and then there's no long-term mechanism for ownership and evolution and continuous improvement in many cases. Have you seen those kinds of structures put into place and what are organizations gravitating toward when it comes to longer-term decision-making?

Amar Goel: When I said governance, a small part of me inside died a little.

Seth Earley: Exactly. So I like to call it metrics-driven decision-making.

Amar Goel: Yeah, well, I think there's a lot of models out there right now just because it's so new. And there's a little bit of this gold rush mentality—this could be super big for our business, so let's try this out or we should be experimenting. So I think I've seen a bunch of different models. One, we've kind of seen people talk about having an AI council—a group of people from around the business, IT development, whatever, helping evaluate these things and thinking about what are the big use cases, what are the big opportunities for our business, what is our evaluation process from a deployment cost, security, data management perspective. That's one thing we've seen.

Another model is where you have a security team that runs a questionnaire to understand how you handle data—do you have SOC 2 compliance? What happens with our code? All the things they normally use to evaluate a new tool. So it's kind of been thrown onto them like, hey, you should also now add in whatever you need to understand from an AI perspective. Those are probably the 3 biggest approaches we've seen so far. I think when there's an AI council, they tend to be a little bit more organized about it because they've been tasked with figuring this out. When it's thrown over the wall to legal or security, they're scrambling a little bit more. But teams are skilling up. My guess is by the end of next year, it'll be a more well-understood new technology.

Seth Earley: So when you think about the future and your organization, where do you see the biggest opportunities and what are the biggest risks?

Amar Goel: Yeah. So I mean, I think we really envision a world where there's going to be a huge unlock in productivity capabilities for software developers. We think over the next 5 to 7 years, they're gonna see a 5 to 10x improvement in capabilities—their ability to create code, have it be more secure, all the things we've been talking about. And so I think that's a huge set of capabilities that are gonna come to help software development improve.

A lot of the things we're doing today are kind of more like toys, frankly. Like line completions and things like that—I feel like it's a little bit like Excite Search before Google came along. It's cool, it's not gonna 100x you. And so we think that when we look at the North Star, there's a lot of possibilities here and software developers are gonna go at a really rapid pace.

I think one of the risks is that it slows down adoption—the behavior change required from people. We see that younger people are more adaptive to these things, and people who are more set in their ways or have been doing something a certain way for a long time are less interested in trying things out. They're just like, I have a process that works. Just to give you an example—my 9-year-old—I watch him work with these tools and I'm like, oh, this is gonna be how people probably do it. He doesn't really know how to write code but he just grabs some code from somewhere, puts it in his IDE, tries to run it, gets a build error, throws it into the LLM and says, here's my error, fix it, rewrite the code. And it rewrites the code and then he tries to build it—no, make this change, rerun it, fix it. Just this iterative loop. I was like, what? You can't write software like that. You've gotta look at each line and figure out what's the problem. And I'm like, that's because I'm old. So it's a behavior change for me. I mean, think about when you have a million context tokens and we're on GPT-7—it's just gonna be a totally different set of capabilities.

We're trying to make these tools work how people work today while looking at the future. Some people love it—we have a chat product in the IDE where you can ask questions and have a conversation, and a lot of people love it. But other people are like, I'm not into chat, I just like to write my code in my IDE in my text editor. I'm not saying anything's wrong with those people. I'm just saying it's a behavior change and some people want to make it and some people don't.

Chris Featherstone: That was like many moons ago I was at Microsoft and I remember working with the developer community and we looked at this guy's code and it was all tightly bound to a very thin column. And I was like, what is wrong with his code? Why isn't it expanded out? It's because he was copying code from the MSDN magazine column. And so he thought his code could only be this long. I was like, oh my gosh, think about the behavioral trends of following somebody else. And those same ideas still apply. Because just because code is there doesn't mean it's good. But that's where great tools will come in—hey, that's one way to do it, but just so you know, it's not optimized. Let me help you re-optimize that. But yeah, really, really fun problems to solve. So my hat's off to you.

Seth Earley: So, we're almost at the top of the hour and I did want to ask you: where are you located and what do you do for fun? And then I have one other quick question for you.

Amar Goel: Yeah, I'm located in the Bay Area. I'm in Menlo Park, California, right next to Stanford. I was actually born here in Mountain View, and then I lived in New York for a long time, lived in India for about 10 years, but then me and my family moved back here about 6, 7 years ago. What do I do for fun? To be honest, right now I'm working pretty hard. But I like to play golf. I don't play a lot, but I enjoy it. And I feel like I'm getting better the less I play, which is a little bit weird. My kids are at a fun age right now—went camping with them last weekend—just doing all that stuff before they don't want to spend time with mom and dad anymore. They're 9 and 11, two boys. They're still in their puppy dog phase.

Chris Featherstone: They still love mom and dad, and then they turn into cats. They won't come back until their mid-20s.

Seth Earley: So one last introspective question: if you can go back in time to when you graduated from college and give yourself a piece of advice, what might that be?

Amar Goel: That's a good question. You know, I think as I've gotten older, I've definitely thought about that. And I guess I would say it's a little bit like a Buddhist or Hindu philosophy—just lower your expectations, in the sense of like, I think when you approach a situation with "it has to be this way" and then it's not, that's what creates anger and frustration and tension. And so just kind of accept the world and people for who they are and deal with the situations as they come.

Seth Earley: No expectations, right?

Amar Goel: Yeah. It's not to say you have a low bar or that you're fine with whatever happens. But it's about trying to accept the situation for what it is versus what you wanted it to be, and then deal with it from there.

Seth Earley: That's a great philosophy and one I try to abide by.

Chris Featherstone: If you're flexible, you won't get bent out of shape. And that's it.

Seth Earley: It's been great to have you. Really, thank you so much for your time today.

Amar Goel: Yeah, thank you. Really, really great to chat with you guys and thank you so much. It was fun. And yeah, look forward to the feedback from your listeners.

Seth Earley: Absolutely. And thank you to our audience and we will see you next time. This has been another episode of the Early AI Podcast and we'll see you soon. Thanks, Amar.

Amar Goel: Thank you. Thanks, guys. Appreciate it.
Meet the Author
Earley Information Science Team

We're passionate about managing data, content, and organizational knowledge. For 25 years, we've supported business outcomes by making information findable, usable, and valuable.