Earley AI Podcast – Episode 46: The Future of Neurosymbolic AI with Erdem Özcan

Guest: Erdem Özcan, VP of Intelligent Applications at Elemental Cognition

Hosts: Seth Earley, CEO at Earley Information Science

             Chris Featherstone, Sr. Director of AI/Data Product/Program Management at Salesforce

Published on: March 27, 2024

Erdem Özcan is an esteemed expert with a rich background in computer science, focusing on innovations in AI. With a PhD in computer science and significant industry experience, including work on IBM's Watson and at Elemental Cognition, Dr. Özcan has been at the forefront of blending symbolic AI and deep learning systems. Today, he is actively engaged in developing solutions that enhance the reliability and explainability of AI applications.

Tune in to this enlightening conversation and gain deeper insights into the future trajectories and current challenges within the world of artificial intelligence as explained by one of the leading thinkers in the field.


Key Takeaways:

• LLMs create demand they cannot meet alone, making prototyping easy but failing at deployment due to reliability and explainability limitations for high-stakes decisions.
• Prompt engineering is not true engineering because natural language prompts lack semantic precision, making system behavior unpredictable and difficult to control reliably.
• Retrieval augmented generation provides knowledge access but cannot perform reasoning over rules to make synthetic decisions from combined data and constraints.
• Neurosymbolic AI architectures use LLMs as an interface sandwich around formal reasoning engines, ensuring 100% reliability while maintaining conversational fluidity.
• The Cogent platform compiles a controlled subset of English into executable reasoning code, democratizing PhD-level optimization expertise for business analysts without programming.
• Formal reasoning systems outperform LLMs at complex optimization tasks and can validate their own outputs, while LLMs show overconfidence when checking themselves.
• The AI field is converging toward hybrid architectures after decades of pendulum swings between symbolic and statistical approaches, driven by deployment realities.

Insightful Quotes:

"LLMs have created a demand which has not been really met by LLMs. It's going to be very challenging for LLM to meet this alone—that challenge is to be a reliable assistant to people so that humans can automate some workflows or make high stake decisions." - Erdem Özcan

"Prompts are tokens that are statistically interpreted by an engine which has a very big knowledge of things that we don't even know how they are represented in its own knowledge. This is why I don't like calling it prompt engineering because engineering is a well known methodology discipline and with prompts it is impossible to follow such a discipline." - Erdem Özcan

"You're basically taking this PhD level skill and then bringing it to business analysts." - Elemental Cognition Partner on the Cogent Platform

Tune in to discover how neurosymbolic AI bridges the gap between generative AI promises and enterprise reliability requirements—combining formal reasoning with conversational interfaces for trustworthy, explainable decision-making.

Links:

LinkedIn: https://www.linkedin.com/in/aerdemozcan/
Website: https://ec.ai/


Ways to Tune In:

Earley AI Podcast: https://www.earley.com/earley-ai-podcast-home
Apple Podcast: https://podcasts.apple.com/podcast/id1586654770
Spotify: https://open.spotify.com/show/5nkcZvVYjHHj6wtBABqLbE?si=73cd5d5fc89f4781
iHeart Radio: https://www.iheart.com/podcast/269-earley-ai-podcast-87108370/
Stitcher: https://www.stitcher.com/show/earley-ai-podcast
Amazon Music: https://music.amazon.com/podcasts/18524b67-09cf-433f-82db-07b6213ad3ba/earley-ai-podcast
Buzzsprout: https://earleyai.buzzsprout.com/ 

Podcast Transcript: Neurosymbolic AI, Formal Reasoning, and the Future of Reliable Enterprise Systems

Transcript introduction

This transcript captures a comprehensive conversation between Seth Earley, Chris Featherstone, and Erdem Özcan about the fundamental limitations of LLMs for enterprise decision-making, exploring how neurosymbolic AI architectures combine formal reasoning engines with natural language interfaces to deliver reliability, explainability, and accountability while maintaining conversational fluidity for complex optimization problems.

Transcript

Seth Earley: Welcome to the Early AI Podcast. My name is Seth early

Chris Featherstone: and I'm Chris Featherstone. It's good to be with you, Seth. It's been a while,

Seth Earley: it's been a little while since we've been able to do a show together. Today I'm really excited to introduce our guest. We're going to talk about knowledge management applications and the importance of knowledge and reasoning, especially with regard to generative AI. Generative AI applications and how they depend on knowledge, the benefits of centralization and of AI capabilities within an organization. Looking at how to solve problems with generative AI. Again, especially around reasoning, a lot of generative AI initiatives don't handle reasoning very well. So our guest today is the VP of Intelligent Applications at Elemental Cognition. He's leading the development of a platform that combines generative AI with the formal reasoning methods to deliver reliable and explainable outputs. So, Erdem Ozkan, welcome to the show. Hi, nice meeting you all.

Erdem Özcan: Thanks for this nice introduction. Absolutely. So, you know,

Seth Earley: we'd like to start, we'd like to start the show with, you know, talking about misconceptions and things that you're running into in the marketplace. And what are you seeing in terms of major misconceptions around AI, LLMs, knowledge and reasoning. But, you know, take those however you'd like to. But what are you hearing when you go into organizations and start talking about solutions? Sure.

Erdem Özcan: I think one of the things that are actually really exciting about AI is that over the last five years to a decades, AI has been a very compelling technology for a lot of enterprises. A lot of enterprises are trying to figure out what their AI strategy should be. And over the past decade, a lot of companies has transformed their data to make that happen. And then so there's a lot of investment and attention going in that direction. And then that has only get. That only got amplified with, you know, the lens, the large language models which have become more and more available to all, like, you know, to all of us. And, and so this has increased the amount of attention and investment that the companies want to. Want to make. So this is all good news for, for the field and I think there is a lot of great Things that are going to come out of that. On the other hand, like speaking of misconceptions, I think there is sort of a, you know, the LLMs have created a demand which has not been really met by LLMs. And I think that it's going to be very challenging for LLM to meet this alone. And then that challenge is to be a reliable assistant to people so that humans can automate some workflows or make some decisions and you know, make high stake decisions. What is happening with the LLMs is that they make it incredibly easy to start a project, create a prototype and see some of the lights that could come out of the AI. But then they actually fail at delivering on the reliability, accountability and expendability levels. And so I think that misconception is leading a lot of companies to start prototypes and failing at deployment. And so this is basically the main challenge that we are focusing on at Elemental Cognition. How to, you know, combine deep reasoning and formal reasoning with the LLMs to deliver on that kind of promise.

Seth Earley: So, so, you know, we had talked a little bit about prompt engineering being a bit of a misnomer that it's not an engineering discipline though people use it as such. And I'm actually that's a topic of one of our upcoming webinars. But why don't you talk a little bit about your, your thoughts around when people talk about prompt engineering, what they're considering and what the value to organizing and managing prompts, or at least coming up with some mechanism to develop prompts. What are your thoughts there?

Erdem Özcan: Okay, great question. I think like maybe let's first focus on what is good about the prompts and then talk about like, you know, what is challenging with them. I think what is good about the prompts is that over the last 10 to 15 years deep learning has been like the main technology leading AI for different enterprises. And the difficulty with deep learning in general is that it is very data hungry. You need to have a lot of data to tune systems to behave as as you wish. So this has been a very big challenge like how do I collect this data to start with and then how do I evaluate my systems and once I evaluate my systems, how do I actually fine tune them to get them to exactly to the behavior that I want. So what prompts have opened as a new possibility is that when you don't have as much data, you can actually instruct the system in natural language. You can say what the instructions and the rules that you want the system to consider when making its decisions and subsequent follow up conversation. And so this is great in that regard. It drastically reduce the amount of data that applications need to be deployed. Now the challenge that comes with that is that the prompts are described in natural language. They are interpreted by the underlying deep learning systems as tokens of words.

And some tokens of words are produced on the other side. What happens in between is the magic that the new LLM tools are enabling us with. Then the challenge there is that when you express something in natural language, first of all you do not know if the other part has understood it exactly the way you meant. Like, you know, within a conversation like this, I am constructing words, sentences, paragraphs and my speech around how I see the world. But now you may perceive it, or the audience may perceive it in a different way based on their context, their knowledge, etc. So like there is this, this gap of like this missing knowledge which is not said in the prompt. And obviously people can think that they can put that more and more into the prompt. Then the next challenge becomes that the prompt is statistically interpreted by this like, you know, token engine. It is not really understood in its meanings. There is no semantic backplane to map to, to really understand how the system is going to behave. So then what happens is that people create prompts, they evaluate the systems, the system may, may or may not really work the way they, they wished. They go back and then change the prompt and, and in an effort to make it better. But then what happens is that again, because we don't know exactly how this is understood by the underlying system, the updates that we're making may not have the consequences that we were wishing for. And as a result, this field that like, you know, has been called the prompt engineering is more of a like, you know, a process, an undeterministic process in which, you know, you, you try to introduce new things or adjust the behavior of the system, but you don't know how the underlying LLM is going to behave with that. And this is why I don't like calling it a prompt engineering because engineering is a well known methodology discipline and with prompts it is impossible to follow such a discipline and to get to a predictable outcome. So do you think that erdem, the prompt engineering

Chris Featherstone: is basically just like we used to do with short form based utterance design, that it's just a version 2.0 of that? Does that make sense? You know, things like that. Anyway, go ahead. Sorry, I didn't mean to. Yeah, I think this is

Erdem Özcan: a good analogy. I think what we should understand about prompts is that again, these are tokens that are statistically interpreted by an engine which has a very big knowledge of things that we don't even know how they are represented in its own knowledge. And then as a result of that, the systems are behaving in the right way in the sense that there are people building applications using prompts. The difficulty is how to align the behavior of the application that you're building exactly to what you need. This is really critical for businesses because a business cannot deploy a system which is not complying with the rules that need to govern the decision making. LLMs have other challenges too. But I think the nice thing about the prompts is that it makes it easy, easy to enter the world of AI and it makes it extremely easy to prototype things. With that on the other end, it makes it as hard to actually create a system. And this is mainly because of this lack of reliable understanding of what is in the prompt and the absence of a semantic backplane or representation underlying the decision making in the system, which makes it impossible to actually predict how the system is going to behave.

Chris Featherstone: I go at this. Sorry, Seth, I usually go at this in terms of. Because, you know, the questioning comes at us around, well, how are you going to define accuracy and you know, so that it doesn't hallucinate again to derive the outcomes that you're trying to drive. So I usually hit it from the perspective, well, you have an opportunity to build your own model. You can take an existing foundation model and fine tune it. You can put together prompt engineering type of perspective, or you can use rag. And each one of them will use a prompting technique, of course, but that's generally the crescendo. Would you agree with that? Would you disagree with that? You know, or what would you say as a, as a plus plus to that? I think, you know, there's a lot of methods to

Erdem Özcan: control the behavior of an LLM, like you mentioned, the most prominent ones, like fine tuning, better prompting, RAG and all that stuff is mainly aiming at, you know, fine tuning the parameters in some way of a model so that it behaves the way we want it to behave at the end. It is an empirical process. It is a statistical process that you're going through. The difficulty with that is you are not really controlling the behavior. You're just trying to put statistical boundaries around them.

This is something that we are doing very differently in my company at Elemental Cognition, where we are actually using formal reasoning. And in some ways you can actually think that this is similar to rag. The difference that I would actually draw is that RAG is basically retrieving knowledge from text or information or maybe databases, right? And then, and then prompting the system with that so that you get a more reliable outcome. Now what RAG misses in terms of its own architecture is that when answering the question or making the decision that you expect out of your model is not based on information which is readily available in searchable data or text, then your system cannot help you. To give you an example, we work with companies to help them build workforce planning tools. Now, every company has some documentation around how people should take vacation, how they should be scheduled into projects, etc. So with REG you can take this information and present those rules accurately to someone who is asking, you know, should I let Andy go on vacation next week? The system with a REG can tell you what the policy is. It can tell you that if Andy didn't take like three weeks this year, which is the allowed thing, and then they, they agreed with their manager, then they can actually like take vacation. Otherwise they cannot take vacation. So this is what you can expect from a Rack system, but then you can bring it to the next level where the system can tell you, yes, Andy can take vacation because he already took like only 10 days this year. He has that much of available and then there is no upcoming project for them. In order for the system to behave like this, it needs to put on top of this Bayes rule set that it understands. It needs to apply it with the data and reason through to give you this synthetic answer at the end.

This is the kind of applications that we are developing for. At the basis, you can query the system about the rules, but beyond that you can expect the system to apply those rules to a particular situation and make the decision and explain why that decision is the right one to you.

Chris Featherstone: Nice. It's a very powerful

Erdem Özcan: paradigm in which you can actually rely on these systems to make decisions. You can rely on these systems to understand the trade offs that you might need to make as human being. And then you can do that in a conversational setup which is actually enabled like this fluidity being enabled by the LLMs, but not the decision making itself.

Chris Featherstone: Nice. So when you, when you were saying that you

Seth Earley: don't believe that prompt engineering is a valid discipline or it's hard to engineer, it's more of an art than it is a science. I guess you could call it prompt design. Right. And put a methodology behind it. Because I do think there are certain characteristics and prompts that you can use to both guide your user and to get better results. And especially, I don't think there's any doubt in many people's minds that, you know, retrieval, augmented generation, and there's a lot of different ways to do that is really the way you're going to get the best results based on your data. Right. That's the source of ground truth in the organization. So that does require a reference architecture, it does require some content conversion or transformation. Right. Because many people don't, many organizations don't have the knowledge in the first place. Right. To support the questions that are being asked. So developing that library, if you use cases, developing the metadata that those use cases represent, when people ask questions, they're asking questions about something, you know, a contract or, you know, an article or a procedure or whatever it is. And then again, having some of those signals as part of the enriched ingestion and the enriched embeddings in the LLM and the vector space can help retrieve more precisely that content. So I, I agree that is, it is part art, but then there's lots of different ways of retrieving that information. And where does the logic and the reasoning layer in? I know we talked a little bit about ontologies and how in ontologies you can do some inference and some reasoning, but where, what is your reasoning engine or how does that developed, you know, does it sit on top of the LLM, is it, where, where is it and, and how is it configured?

Erdem Özcan: So at the topmost level of the architecture, we actually call our architecture an LLM sandwich. So LLM on one side to acquire knowledge, which is one of the breads, it turns your documentation or the knowledge of your subject matter experts into a very well structured document that would govern the decision making. And then in the middle, what we call the MEET is a formal reasoning engine that is developed by our company, Elemental Cognition.

And then this reasoning engine is able to do formal reasoning in terms of logical reasoning, logical inference, abductive reasoning, abductive inference. And also it is able to do optimization, like finite domain optimization, integer linear optimization, etc. So it has these base capabilities which then is driven by a document in English which tells it what are the objectives of optimization, what are the rules and constraints that the decisions need to be made. And so that English is not the full extent of the English language, but instead it is a subset of the English language which is allowing us to capture these rules and objectives in a very precise and unambiguous way. And then that English gets compiled into reasoning code and then gets executed formally. That's interesting.

Yeah. And so this is A very big invention like how to compile, how to define a subset of English which allows you to capture this kind of business rules and objectives and then compile that into code that can execute, you know, on a formal basis. And then on the other side, like this is the meat. And then on the other side of the LLM sandwich there's another brand which is again an LLM which brings this reasoning, formal reasoning into life through a fluid conversation with the end users where the reasoning engine can make decisions and then it can communicate it with end users using the natural language. Again so you know LLMs on one side to acquire knowledge from your documentation and subject matter expert to capture it in a precise form of English, which then gets executed by a reasoning engine informal basis which allows you to be 100, reliable, explainable and all these like, you know, features that, that matter to businesses. And then it gets used with by an LLM on the other side of the sandwich to deliver that to humans through conversation. Right,

Seth Earley: right. Making the response or the answer conversational because it'll just pull the data from the source after it reasons, but then it needs to communicate that to a human. Yeah, that makes a lot of sense. And then you mentioned this is you're able to compile in subset of English. So this is one of your tools, right? Or one of your languages. What is it called again? Is it the coaching?

Erdem Özcan: Yeah, we call this the cogent language. We also call the platform that is using it as cogen. So cogent is a, you know, as a language, it is a form of English which allows you to define, you know, the concepts that needs to be taken into account, their relationships, the rules that govern the interactions between them. And then in case it is like the decision making is not just about logical inference, but it needs to take some form of optimization into account. It also allows you to define optimization objectives. So for your, your example with the scheduling,

Seth Earley: scheduling nurses or coverage, you know, you can't, you know, some of the rules are you can't work more than two shifts in a row and you can't work more, you know, you can't have too many back to backs or whatever it might be along with vacation policies and the amount of vacation people have and you know, what the predicted load would be. So that that's taking into consideration all of those different variables and then using the reasoning language cogent, which is again a subset. So can you give us some examples of what would be a cogent term or variable? So

Erdem Özcan: for instance, Yeah, I mean if we take the example of Nurse scheduling. For instance, in Cogent you would say nurse is a type a concept, a shift is a concept. And then you can, you would start defining that a nurse may be working on a shift. And then you can say the system should decide which shift a nurse should be working on. So that's the optimization piece. And then you can start constraining the system in the decision making that it needs to operate with. So for instance, you can say things like a nurse cannot work more than two shifts on a day. Then you can introduce the concept of departments. You can say like department is a new concept, a nurse may be working for a department. And then you can say each department needs to be filled by at least a nurse or like, you know, at least three nurses depending on the situation, for every shift on every day. And then you can layer in the vacations like, you know, you can say each nurse must take at least one day of a vacation per week. And like, you know, you can define whatever rules that you need in that language. And then it leads the, at least the, you know, reasoning. And maybe I should also say that like, you know, we, like when we talk about optimization, we very quickly focus on scheduling and planning applications because this is a place where optimization has been very valuable for decades. What we see is that as we are democratizing access to optimization applications using cogent, there's a lot of different fields where people want to use it. So a couple of interesting examples is that we're applying cogent in space planning, like generating a, a kitchen layout or a wardrobe layout. You know, like again, you can express your governing rules like, you know, and then you can actually express the compatibility between the different products. And then you can express the user user wishes. And then you can find the most optimal way of generating these, you know, room layouts and kitchen layouts, et cetera. We're using it with, you know, some universities to help students make their degree plannings. You know, people can, more and more students want to take their own like, you know, pathway into, into university studies. They want to combine a major with a minor. They want to take some classes that are to interest them and it is fairly easy for them to miss some prerequisites. And like when they think they can graduate, they discover that they didn't take some of the classes to graduate. And then this can induce additional expenses they need to drop, et cetera. So like, you know, we, we built a tool using Cogent to, to solve that problem. So there's a lot of interesting problems that can be, you know, like Expressed in cogent, that general purpose tool to build solutions. AI solutions for. You had

Seth Earley: mentioned a quote where that AI, large AI vendors have written checks that they can't cash. And that is in terms of the limitations, how do you think the big AI vendors are looking at that or thinking about this and is the inherent limitation of what an LLM does is just it's not going to be able to do reasoning or do you see that evolving as these models evolve? Well,

Chris Featherstone: even, even define what, you know, what, what do you mean by they, you know, checks they can't cash? Right. I think that would be. Yes, yes, yes, that. Okay,

Seth Earley: why don't you explain that again, it's part of this idea of reasoning. But how are they messaging and how is that messaging falling short of reality?

Erdem Özcan: I think what is happening is that there is a lot of like, you know, in, in a lot of enterprises there is some departments that are responsible of creating their AI strategy. And then so these people look at the latest technology that is available and then they build proof of concepts and the proof of concept level. You have happy paths and then you tune the systems to deliver that happy path and then you create a proof of concept at demo. Then it comes the question of how do we deploy this and how do we make sure that it is accurate all the time? Because it's going to fail our customers or people at the company to make decisions, high stake decisions. Right. And this is where like the reality confronts the proof of the concept. And what we're seeing happening is that when it comes to that level, some of the projects have to drop because they actually discover that they cannot deploy it in these circumstances. Some other projects very quickly become a hybrid solution. You know, like, you know, some code gets written to handle some of the situations, some if and else gating functions get written to, to control the behavior of the LLM. Some precise procedural code gets inserted into some head situations which we know this is going to happen and it's not actually using the AI. And so this, the actual deploy solution very quickly becomes a hybrid solution which is using AI plus some procedural code in various different places. Right. And I think, you know, obviously the tech companies are seeing that as a problem and then there are some methodologies which are being developed and, and I would say that like one of the most promising approaches that I'm seeing these days is that LLMs get combined with tools and then LLMs can invoke a tool to perform a certain operation and then that tool is actually like formal code doesn't need to Be reasoning all the time. But like one of the examples is search for rag so that these tools deliver more reliable outcome that the LLM can leverage to output the end result. And so but this is done in a very, very ad hoc way. You can see what we're doing at Elemental cognition as one way of architecting this whole thing. You can actually think about the reasoning engine as a very capable tool, you know, sitting at the core of the decision making and the LLMs using that tool to help them like deliver high stake decisions to the end users.

So I think you know the reality right now is a lot of hybrid codes deliver whatever it is needed. I see a lot of academic research now being called like neuro symbolic approaches to AI which also we used to call it hybrid systems in the past like combining symbolic and deep learning systems. And a lot of companies are working on that. A lot of big Companies like Microsoft DeepMind have recently published papers illustrating that they're working on these architectures.

What we do is falls exactly into that category. We are delivering an industrial solution to build neurosymbolic AI that can deliver these reliability and explainability as features of its output. I want to dive into that neuro symbolic

Chris Featherstone: AI. But first which maybe as a follow on to this, I know you guys did a benchmark study in terms of what types of reasoning and inference are still challenging for LLMs. That's a, it's a, it's a super important topic that to your point you were alluding to it with some of your, you know, other comments that I just don't think people grasp the, the gravity of, of, you know, because I feel like they still think that it can bend light, it can solve world hunger and it can predict, you know, who the President of the United States is going to be. Right? That kind of stuff which is all fictional. So you know, give some, shed some light on the benchmark that you guys did and what that looks like in terms of where they're still missing. Yeah, thanks for bringing this

Erdem Özcan: up because I think this is a very good segue to like this deployment issue. How do we trust the systems that are working? So what we've done is because we see a lot of companies trying to use LLMs to solve these challenges which require reasoning. We took a couple of illustrative examples and then ran our system against GPT4 to see like where both systems stand. So we took three different use cases. One of them is workforce workforce management, the one that I just mentioned earlier. You know how to schedule people to projects taking into account their availabilities and skills, etc. Another one was, you know, an application, as I mentioned, that we built how do, how do we help students make, you know, their plans for their university pathways? And another one was around complex travel planning, like how do I, how do I create a travel plan to travel the world, for instance? And this was also motivated by one of the applications that we deployed with a partner of ours. So we took these three systems and then we created these urgent descriptions leading to actually prescribing their behaviors. And then we ran this with our system and we ran this with GPT4, giving them scenarios at different levels of complexity. So for instance, if I picked the workforce management to illustrate the complexity, the number of people that are available and the number of projects that needs to be ran is a level of complexity. And then we started with very simple things which would be a trivial decision to make for a human. And then we brought it to a much more complicated amounts where decision making suddenly becomes very difficult. And bear in mind that all these optimization issues are NP complex optim so like they're like non trivial things to be sold by humans. And so what we saw there is that GPT can, and like, I think this would be representative for all the state of the art LLMs. So the LLMs are able to actually answer some of these questions when especially like in the travel space when there is some world knowledge which gets into the place play like, you know, where the cities are located through the embeddings, et cetera, it is doing a slightly better job than, you know, a degree planning which may not be as common problem that like, you know, everybody knows. And so what we saw is that like when the complexity is really like, you know, small then LLMs are like usually able to deliver the good outcome. But then as the complexity grows and gets to levels which would actually motivate for using a technology and not relying on human decision making, LLMs start failing. And this happens in a gradual way way.

Whereas with our System it was 100% correct across the board of complexity. And the reason for that again is that we're doing formal code. We are taking this English description, we're compiling into a very efficient reasoning code which gets executed in the reasoning engine. So it is expected that we can actually get this right across the board, which is why we're developing this technology. This was one aspect of that.

I think the next aspect, which is maybe even more critical than being right all the time, is that when you do not have a system which is Reliably correct all the time. You need to know when it is right versus when it is wrong. It is really critical to be able to make that distinction because otherwise you can actually serve outputs which are going to mislead users.

The second test that we have done in our study is to ask our system, like give it an output and ask our system versus the LLM. Do you think this is valid given this rule set that are prompted to you? And again, our system was getting it 100% right for all the reasons that I mentioned.

What was really interesting is that when we did that with the LLM, we tried different ways of asking this question. When we asked, as another person has created this plan, can you validate that it is correct? It was actually, was actually finding the invalidities more consistently than if he said, you have returned this plan, can you please double check that you were correct? Then it was actually overconfident that it would have done it in the correct way. And as a result it was doing the operations in such a way that it is actually misleading the humans. So this is again like a place where we see that this reliability comes to play into play a lot and then the actual ability to classify whether an outcome is correct versus not becomes very important. And so again, formal reasoning is able to cope with that, LLMs or not.

Seth Earley: So just a quick question, when you say your system versus an LLM, I mean the optimization, the traveling salesperson problem is a very difficult problem to solve, right? That's the optimization with multi, multivariant analysis. And so your, your engine, I imagine, would use some machine learning in, in doing that optimization. Correct. It actually doesn't

Erdem Özcan: use machine learning. Instead it uses, you know, some operations research tools to do that. So again, the execution is formal, not statistical. Okay. And then you know, when

Seth Earley: you say your system versus an LLM, you're still using an LLM in some way, but you're saying using your reasoning engine, you're still using the sandwich. Right?

But you're saying that your reasoning engine versus the LLM by itself is what's correct. Yeah, correct. Because as it increases in complexity,

the LLM does not keep up. Correct. But

Erdem Özcan: the interesting thing to see here is that the input is in English. It is not. You're not comparing an English prompt versus code that is written in Python or whatever. The, the input is in English and then that English input is compiled into reasoning code on one end or statistically managed on the other end. So that's the main difference. Otherwise, if we were comparing like code written in Python versus like a, a specification written in English. That wouldn't be a fair comparison. Isn't. Now when

Seth Earley: you mentioned symbolic. Symbolic is. Is. Well, why don't you give your definition of symbolic AI versus versus the statistical approaches. But it's really representation of knowledge, right? It's really representation of the, the, you know, the isness and aboutness. I like to call it the metadata structures, the metadata models, the, the content architecture, the knowledge architecture. Do you want to talk a little bit more about. Because it always seems to me it's always had to be a combination of both. But do you want to talk a little bit more about. You started to mention this before the, the neuro symbolic approach.

Erdem Özcan: Sure. I mean, my definition of the symbolic AI would be a process in which we encode the knowledge in some formalism using symbols and then we can translate these symbols into something that can be executed on a computer. Right. And then we would have a defined process of getting the symbolic knowledge into something that can be executed by the compiler. Now you can. And again, symbol. Symbol is metadata,

Seth Earley: essentially. Right? Yeah, yeah. What makes symbol, you know, that kind of

Chris Featherstone: thing. Like one of the things that, I think a popular

Erdem Özcan: view of symbolic AI, as you said, like ontology, is knowledge representation. We typically start indexing on the hierarchies between things, like as defined in an ontology. We typically start indexing on what subsumes the other thing, what is the subtype of another thing, how one thing relates to another thing in terms of singular, multiple relationship, et cetera. So I would say that it doesn't need to be limited to that, that it can start getting into rules like, you know, again, like a nurse cannot work more than two shifts a day. There is, yes, an ontological relationship between, you know, the nurse and the shift nurse working a shift from an ontological perspective. But on top of that there is a constraint that comes on that like they cannot work more than two shifts in the same way. Optimization objectives can be represented in that way. You know, I want, you know, at least, least that much people to be scheduled and maybe the cost of missing a. Like, you know, the cost of not fulfilling a department on a day is much higher than not allowing a person to go on vacation. So you can start also introducing this kind of concepts in symbolic ways. So I think the, the main difference there is that there is a challenge of representing things that can be said in natural language in these symbolic forms. And this is all like the, all what the research around symbolic AI is like, how can we represent different things that are happening in the world in symbolic ways that then can be executed in a reliable fashion by a computer. And the difference there is that, because this is a very big challenge, statistical systems comes in and then they actually introduce a way of making, making that mapping not maybe in such a reliable way, but in more flexible ways that can cover a larger basis. And I think that. And then this is where the fluidity comes from, you know, like, how do I actually translate language into this symbolic knowledge? And I think there are schools of thought, I would say in AI, like some people are like only believing into statistical systems because they are thinking that there needs to be the learning part needs to be the only part of the equation and everything can be learned, which is like true up to a certain degree, which we all humans learn in that way. Like we start almost from scratch and then we learn things along the way. But yet at the same time, you know, some of the things that can be learned do not necessarily result in very efficient representations, which is where all these mistakes are coming in, et cetera.

And then humans have developed symbolisms to deal with this kind of stuff. We invented mathematics to, you know, like, rather than then talking about equations or like, you know, mathematical things between each other to make sure that we can solve this in a reliable way, we invented maths, right? We invented mathematical representations and methodologies. So I guess the symbolic AI camp in this is, wants to leverage all these methodologies which are like known to be very efficient and neurosymbolic. AI is all about combining these two. It is not belonging to one of these extremities, but saying that probably the better outcome is going to come from the combination of these two tools. Right? And if you're

Seth Earley: using RAG at any point of that equation, then you need to have some curated and quality knowledge, as you had mentioned, you know, you need to structure that in the right way. What is it, what does it take to become, become proficient or fluent in cogent. So because it sounds like that would be the key to an organization being able to kind of leverage this fully, is, is that that a business analyst role?

And what is it? What's, what's learning curve on that? I would add on to

Chris Featherstone: that too, Seth, like, and just in terms of, you know, the, the fortunate or unfortunate perspective is that everybody, if they've got one, you know, they define AI by, you know, by GPT 3 and 4, right? Which, you know, is, is not the case. But at the same time, what does that look like in terms of, you know, it's, it set the Precedents, Right. And the unconscious bias. So what does that look like comparatively with those maybe as a, as a predecessor. So to Seth's question, you know, what does it look like within your platform? And two, what does that look like as to why your platform over something else? Right. Maybe like a GPT that people have experience with, you know what I mean? Well, GPT4, I shouldn't say just GPT because they're all large language, but you. Know what I'm saying, I'll

Erdem Özcan: maybe answer the second question first and then go to the first one. So if you take the LLM world, the main reason would be the requirement for reliability and explainability, basically accountability. You know, like you don't want to be, if you don't want to be in a situation where the outcome of your AI system is misleading people and then this may have a, an impact on your brand, your credibility, your business finances, then you probably don't want to use an LLM at this stage, like where they're at. So this is where I think against an LLM, this is what people would do. On the other side. There's a lot of operational research tools that are available out there.

Optimization and operational research has been around for longer than statistical AI and they're very well known tools. I think there the problem becomes that using these operations research tools require a certain amount of skill, which is very scarce. You know, you need to understand these concepts, you need to understand maths, you need to know these libraries and know how to use them efficiently. And I think this is why like, you know, there is, you know, when you speak to an airline, for instance, they have their own operations research teams that are developing systems for them because this is so valuable for their business. But then there is a lot of other core of businesses which cannot afford to have these experts. So they are either making calls to consultancy firms which has these expertise or they don't do it. And I think this is where like Cogent comes in as a force for like, you know, democratizing this kind of access. One of our partners recently told me that like you're basically taking this PhD level skill and then bringing it to business analysts.

Wow. So this is one part of it to answer the first question, like what is it to become like, you know, fluent in Cogent is. So there. This is a platform which comes like, first of all, it's not just a language, it's a platform platform. The platform helps you capture your business knowledge. It generates APIs for, for being able to connect into your application and generates the Conversational interface so that you can power conversational experiences with user users. So it is, it is coming as a whole. So by, you know, adopting this platform you're not like, you know, you're also shortcutting a bunch of operations that you would do elsewhere and then you need to learn this language and be fluent in writing that language. One of the things that we are doing is that we actually, as I said, left part of the LLM sandwich in my mental model is acquiring cogent from natural language. So we have a chat based tool where you can say what we would like to do in free form English and it would translate it into this more precise form of English. And this is very interesting. It is somewhat similar to Copilot chat chat where you can say like I want to write this code and then it would return the Python code. Where it is really interesting when you experience this is that the output is also English. Like it is almost like a translation tool and it is very easy to look at what you said and then look at what the outcome is and then apply it or no and, or configure it. So we are building these facilitator tools for people to easily translate their business knowledge into, into, into cogent English. Right. I wanted to switch gears a little bit and

Seth Earley: just ask you about your journey and you know, who you are as a person. I see a musical instrument in the background, you're a musician and how did you kind of arrive at what you're doing and where you are? Yeah, I, I think like my journey was a

Erdem Özcan: lot like, you know, a lot of like right on time decision making. I actually, as my name indicates, I'm originally from Turkey. I grew up in Turkey and then, you know, went to, to France for my university studies with a little push from my family, which I'm very, you know, grateful about after the fact. And so I studied in France, I studied computer science and, and then did a PhD in computer science. And I actually had an opportunity to do AI at undergrad and wasn't attracted as much at the time and I preferred to lean into programming languages. Was really fascinating and was it, it's, it's my symbolic part in, in some sense like I, I, I found that the programming languages was one of the biggest inventions which allow all the, you know, computer programs that we have today and like this level of abstraction and productivity that they delivered. So I actually did a PhD in computer science focused on that for about 10 years as a researcher in the industry and then came around 2010 all these big Inventions around like Siri Watson, you know, like all these NLP powered applications and challenges. And so I increasingly became more and more interested and figured that everything that I did so far was very much applicable into those fields and sort of decided to change my expertise, joined a large company where it was actually working on Nuance, not IBM, sorry it was working on Watson and then started like I've been very privileged to be working on this lab landmark invention which did a geopardy and then we wanted to apply it to the help of medical doctors. I worked on that. I was very lucky to work with the World Star team on that and they taught me a lot. And then at some point something in me was always interested in product and application and not deep research. And so I started my own company with a few co founders. We built actually a neuro symbolic AI solution for marketing. You know, we built chatbots that help like people express their needs and find the best products for them and you know, grew that company and we got acquired and then you know, I took some break and then decided to join Elemental Cognition because like as you may see in the, in that narrative, like I've been always in between these symbolic and deep learning aspects and like in this company I am able to create stuff that is like really falling into this architecture that I deeply believe it. That's great.

Chris Featherstone: Oh yeah, with, with, with tons of investment you do a lot of neat, interesting things. Hey, let me ask you something Erdam. Within the last, last portion of, of this, you know, the time that we have, is there something that we could have asked you or should have asked you that we didn't or that, that you think we should? Well,

Erdem Özcan: I don't know, like one of the things could be maybe where this is all heading towards. Sure. Maybe a very big question and I think the, I mean I can share obviously like my perspective into this. I think that a lot of efforts are going into deep learning based methods and it is like you know, well justified in a lot of cases.

Now it also goes with a lot of money invested in that because developing deep learning systems is also a very extensive adventure. So I think that these systems are going to trend towards from great demos, they will have to get to great applications because at some point they need to justify themselves from a financial perspective. And I think that we are just living the era of this transition. A lot of LLM companies now are going around a phase where they need to deploy and then there are no very big use cases for them. Still if you put aside the creativity like helping to write something, et cetera, or generate images. There is still a lot of challenges in applying them in the larger core of enterprise. And I think that what is going to happen is again people are going to end up creating hybrid architectures to deal with them in real life situations and then hopefully this will get us to the next level where you know, these symbolic or like, you know, some people may not want to call it a symbolic but like programming tools, et cetera are going to be combined in more and more well architected ways with these deep learning systems. And I think this is where you know, like the AI, the grand challenge of AI will start converging because this pendulum have been shifting between statistical methods and symbolic methods like almost on a like 10 year kind of phase over the last, last like you know, 40, 50 years. And, and I, I, I, I would expect that there is going to be a convergence that is going to come out of that as we move on to the next decade. Less extreme

Seth Earley: pendulum swings and maybe it's going to start settling down as we. See normalizing a bit. Right, right. Yeah,

Erdem Özcan: that's cool. Artem, this has been great. I really appreciate you

Seth Earley: joining us. I appreciate your time and sharing your knowledge. So thank you so much for being on the show. Yeah, thanks a lot Chris said, and it's been

Erdem Özcan: a pleasure. Thanks for inviting me and thanks to

Seth Earley: our audience for listening in and of course share this if you found it interesting and then thank you to Liam and and Carolyn for their help behind the scenes. So thanks again.

Erdon has been a pleasure and appreciate your spending the time with us today.

Chris Featherstone: Thank you. Okay, appreciate it. Take care of them

Seth Earley: and we'll see you next time on the the next really AI podcast. Okay, that would be my pleasure.