Breaking Free from Dashboard Limitations: How 18-Dimensional Visualization Transforms AI Governance, Financial Analysis, and Machine Learning
Guest: Bob Levy, Founder and CEO at Immersion Analytics
Hosts: Seth Earley, CEO at Earley Information Science
Chris Featherstone, Sr. Director of AI/Data Product/Program Management at Salesforce
Published on: April 10, 2024
Seth Earley and Chris Featherstone are joined by special guest Bob Levy. Bob Levy, Founder and CEO of Immersion Analytics, brings a wealth of experience in technology and data visualization, having worked with top companies such as IBM, Rational Software, and Mathworks. He shares his profound insights on integrating multidimensional visualization technology using virtual and augmented reality to tackle complex data challenges.
Bob Levy is the Founder & CEO of Immersion Analytics. With extensive experience in R&D and product management at companies like IBM and Rational Software. Bob is an expert in AI and data visualization. He's been a speaker at prestigious events like MIT Technology Review’s EmTech Caribbean and Reilly Strata Data Conference and has won competitions like MIT’s Reality Virtually hackathon and Tableau’s DataDev Competition.
Key Takeaways:
- Organizations wrongly assume complexity is delegated successfully to others, yet those teams lack boardroom perspective, leading to oversimplified conclusions and mistaken outcomes.
- Traditional dashboards showing only four dimensions miss 3,060 unique relationships that 18-dimensional visualization reveals, multiplying discovery odds dramatically through combinatorics mathematics.
- Summary statistics alone are dangerously misleading as vastly different data sets can produce identical means, standard deviations, and correlations requiring visual verification.
- Large language model governance benefits from simultaneous visualization of vector embeddings, moderation factors, sentiment scores, and demographic fairness metrics for comprehensive oversight.
- Virtual and augmented reality devices triple the number of data points users can absorb compared to flat screens based on empirical research findings.
- Machine learning model evaluation requires examining individual data points rather than relying solely on error statistics to understand specific failure patterns.
- Financial derivatives traders discovered 20-minute arbitrage opportunities by visualizing theta anomalies in distant contract months that should never exhibit high time sensitivity.
Insightful Quotes:
"There's a wide belief that complexity is somebody else's job. Unfortunately, the people you've delegated it to don't have the same perspective as you would in the boardroom. Oversimplification leads to wrong conclusions and mistaken outcomes." - Bob Levy
"By seeing many dimensions at once, it doesn't matter if it's glowing or shimmering or what it's doing. It matters that it's doing something that you're able to see." - Bob Levy
Tune in to discover how multidimensional visualization technology transforms complex data understanding across AI governance, financial markets, and machine learning—enabling executives and regulators to grasp sophisticated systems without deep technical expertise.
Links
Materials referenced during episode:
LLM response vector embeddings and moderation API factors demo discussed
Immersive Analytics for AI Governance
Simplify Complexity for AI Governance - Slides
LinkedIn: https://www.linkedin.com/in/boblevy/
Website: https://www.immersionanalytics.com/
Ways to Tune In
Thanks to Our Sponsors
Podcast Transcript: Multidimensional Data Visualization, AI Governance, and the Future of Complex Analytics
Transcript introduction
This transcript captures an in-depth conversation between Seth Earley, Chris Featherstone, and Bob Levy exploring revolutionary multidimensional visualization technology that enables simultaneous viewing of up to 18 data dimensions, demonstrating practical applications in large language model governance, financial derivatives trading, machine learning model evaluation, and the transformative potential of virtual and augmented reality for data analysis.
Transcript
Seth Earley: Welcome to the Early AI Podcast. I'm Seth Early.
Chris Featherstone: And I'm Chris Featherstone. And we're really excited to introduce our guest today to discuss simplifying complex data visualization and understanding AI and data science and multiple dimensions around machine learning, model dimensionality reduction and visualization of vector embeddings. And for things like hate speech. And we'll talk about other examples, understanding the risks of oversimplifying complexity with organizational and financial context. Our guest today is founder and CEO of Immersion Analytics. He also serves in the Board of Directors for Liferaft Inc. And he has extensive experience in R and D and product management at companies like IBM and Rational Software, who's also with MathWorks. Our guest is an expert in AI and data visualization. He's been a speaker at prestigious events such as MIT Technology Review, O'Reilly, Strata Data Conference. Bob Levy, welcome to the show. Seth, great to meet
Bob Levy: you. Great. Great to be on the show. Happy to answer any questions. Sure. So why don't we
Seth Earley: start off with. I'd like to kind of get a sense of, you know, what are. What are. What are the misconceptions around machine learning and data and artificial intelligence that you come across?
Bob Levy: Yeah, the one, the big one we focus around is that there's a wide belief that complexity is somebody else's job. So if you're sitting in the boardroom, you assume you've delegated grappling with complexity. Unfortunately, the people you've delegated it to don't have the same perspective as you would in the boardroom. So oversimplification leads to wrong conclusions and mistaken outcomes. The view that I can just create a dashboard and people will pivot and look at the data from different perspectives, that they will actually find what they need. So these are some of the misconceptions we grapple with. And how does simplifying the
Seth Earley: complexity address the conceptions around AI being unexplainable in a black box? And especially with, you know, highly complex large language models.
Bob Levy: Right. So the ability. Now we have a technology for visualizing up to 18 dimensions at the same time, and an example of how that can be applied to large language model governance. I can show this example as well. And for those of you just listening in today, we
Seth Earley: do have a lot of visualizations and we're going to. So you can take a look at the YouTube video that we will be posting. So
Bob Levy: what's really exciting here is you Factors from moderation APIs. So looking at the response text of. So if you think of each data point as a prompt and response pair, there are a lot of things you can measure about those prompt response pairs in a large language model. So in this, in this visual, each of the data points again represents a prompt response pair. Now we have this technology of leveraging special effects to show different attributes of the data. Now Seth, are you able to see describe
Bob Levy: this? This is a three dimensional view of what look
Seth Earley: like spheres and the spheres are pulsing in different ways. There's some coloring, one of them is blowing. There's some smaller dots that are surrounding another sphere that also has a red circle around it. Just giving people a sense of what types of things we're looking at. So Seth, do you notice in the
Bob Levy: visual one of the. One of the data points is glowing very intensely, another has these orbiting particles, another is solid, whereas the remainder are translucent. Yeah. And what that means. So these are measuring different
Bob Levy: factors from a moderation API. So factors including self, harm, violence, explicit content, hate speech, all using standard moderation APIs. The technology can also be used to layer in sentiment analysis scores and natural language processing algorithms. Dictionary or neural net based topic categories you can layer in if you have anonymized user demographics for the purpose of making sure the model responses are level across all demographic groups. You can also layer in confidence levels and other quality assurance metrics. So what's really powerful here is you've noticed the glowing point that indicates hate speech. You've noticed the point with satellites and you've noticed the solid point with this. These points are organized in a special way. So we apply a standard method called principal components analysis to reduce the vector embeddings of the response. So this is sometimes 1000 or more dimensions. It turns out there are standard math techniques. So principal components analysis that identifies feature importance, it uses linear algebra to correlated components and it synthetically reduces this part of your data set so that it can be used to organize the data points on an X, y and z plane. So spatially organizing these, you can then click on the individual points and we're contemplating an application where by clicking the neighboring points to an anomaly, you can actually read the prompt, read the response. And even though the moderation API didn't flag it. What we've seen empirically is that you will then find potential paths of misalignment of the model so that you can broaden, appropriately broaden any guard railing efforts that those anomalies might.
Chris Featherstone: So Bob, just to ask a question. So where the sphere is in the X, Y and Z is it's. It's relevancy. And then where the other spheres are around it are their association via the vectors to that relevancy. Is that correct? Yeah.
Bob Levy: Think of the placement on XYZ as, as a representation of how similar the points are. Yeah. Based on their vector embedding. Yep. Yeah. Okay, perfect. And then the
Seth Earley: pulsing is what the, what does that represent? We were looking at
Bob Levy: pulse frequency was the absence of hate speech. Absence example. So more pulsing is less
Seth Earley: hate speech. More pulsing is less hate speech. And
Bob Levy: that reminds me, why don't I go ahead and actually invert that mapping. Okay, so we have. Yes, less pulsing is more hate speech. And then again the satellites, what did. Those
Bob Levy: represent in this, in this case the satellites are showing the intent for self harm. So that would be a response that is guided someone to. To harm themselves. Wow, very interesting. And. This glowing point that you see so glow is
Bob Levy: explicit content. And then there is a neighboring point that's also glowing. And we contemplate adding an interface where you can explore by clicking on the neighboring points to read the prompt and read the response. So you can see are there subtle threads of the misalignment this anomaly represents near it. And if, if so the proximity can be used as a way of informing your guard railing efforts. Yeah, this would be interesting against
Chris Featherstone: like a gaming, you know, context. Right. Where you have potential hate speech that's not necessarily bad, but it's associated with these things and associated with things like grooming techniques. Right. That often occur. So
Bob Levy: anything that you can measure can be, can be shown. We talked about principal components analysis earlier. This technique and it's the staple of the industry, IT or T sne, these dimensionality reduction techniques are all about taking a lot of attributes and boiling them down to using feature importance and matrix or linear algebra to identify correlated components. The problem with those techniques and the reason they can't be applied to the moderation factors like they were to the embeddings is because they no longer contain information about specific factors. So in other words, you'd see an anomaly, but you wouldn't know is it hate speech, is it explicit content, is it violence or otherwise? Right. Now how
Seth Earley: could this, could this also be used in a compliance application where someone is looking at, you know, regulatory compliance regulations and then looking at policies, or how would, how would something like that work? What would the application be? Or maybe there's another, there's another visualization for that. Absolutely. So this visual
Bob Levy: can be fed with real time streaming data. So you could envision a real time streaming view of prompts and responses coming into a system, an archive data set that could be then replayed over time. And the idea is that by seeing a wide swath of time, you'd be able to, you'd be able to explain to a regulator in what ways is the system behaving? Did it, did it, was it adapted to behave more effectively as the result of some corrective action that had been taken? So it's a good visual tool to explain that for a regulator who most likely is not an expert in AI technology at the deepest level.
Chris Featherstone: Let me ask you this then. Beyond some of these types of visualizations, how would this help in terms of evaluating model performance? Mm.
Bob Levy: So this can be used with. So why don't we look at a machine learning example? So this comes from Kaggle, common data science website. It's the stroke prediction challenge. And we look at, I'm sorry, the wet prediction challenge. It's to predict stroke, so patients who had an arrhythmia or stroke eventually. So in this training data set, the red points represent patients who had a stroke and the greens did not. There are a number of factors behind this we're able to see though, as we layer in each model, so we'll layer in linear regression as the orbiting satellites. And what we notice is that it identified only 2 of the data points, here and here. We would want to see that, that kind of model, or any model, correctly identify all of the red points and not identify the green points. Now we can layer in other model types as well. So if we layer in random forests as opacity, we'd want to see the red points stay solid and we want to see the green points go translucent. Now we see a much better fit. Now, in analyzing models, error statistics are often used. But then again, back to the point of why summary statistics are dangerous.
If you rely on summary statistics, you can't have a conversation about a specific data point where things went wrong. And what we found is that by looking at the individual data points with your team of experts, you can have thoughtful discussion about why did the model not get this one right, why did the model get these other ones right? And you can be specific about those elements as opposed to just Offering summary statistics. So isn't that a risk, no matter
Seth Earley: what you're doing, of losing the granularity and losing some of the factors? You know, you're using summary statistics and you have very complex multidimensional data. And the question is, you're always, there's always loss of data, right? There's always some loss. And so how are you, you know, is it a matter of trying different combinations or different approaches to the dimensionality reduction? How does that work?
Bob Levy: That's why our company exists. So think of it as a cognitive upgrade that expands your ability for if you think of wide data and also large data. So by rendering different effects around data points, up to 18 of them, this can be shown. But then what do you do if you have hundreds or thousands of dimensions? Right. So in those cases, we look for clever ways to use dimensionality reduction where appropriate, while preserving the dimensions that are like for example, in this case looking at did the model perform? Yes or no. In the last example, looking at the different moderation factors or quality assurance metrics.
Interesting. Now it really is incredible
Seth Earley: stuff. One of our advisors, he
Bob Levy: teaches at MIT, he quantified this use of effects to visualize 18 dimensions as multiplying your odds by 3,060 times that you will notice important relationships and the math behind that, if your audience is interested, it's the number of dimensions shown in a standard plot on a dashboard. So in Most cases that's XY size and color. So that's 4. Then Combinatorics 4 choose 18. So it's 3060 unique combinations that that one visual contains. We, we talked
Chris Featherstone: about it before the show, but you know, like I told you, I, I have that book that says how to lie with statistics, right? And I think you have another visualizations that show what aggregations can do, right. When you actually, you know, if you don't look at all the data, I don't know if you want to, you know, pull that chart or graph up and it's, it's really interesting, right. For those that are listening centered around, you know, we look at XY coordinates and doing summary statistics how you can actually see, you know, very similar statistics provide a wide range of outputs. But anyway, go ahead, Bob. Absolutely. Now this,
Bob Levy: this visual credited to the research team at Autodesk, they call it the data source. There's another famous French mathematician produced Anascombi's Quartet, which makes a similar point. But here we have some data and we have some summary statistics off to the right. The mean, the standard deviation, the correlation now we'll animate different data through this and as I do this, keep an eye on the summary statistics. You'll notice with these vastly different data sets, the summary statistics are identical. And this is one of the most important reasons why seeing all of the data points, seeing as much of the attributes, seeing the complexity is so important. So, so for those of you
Seth Earley: listening, one of the representations of the data is a Tyrannosaurus rex, so that's why they called it the data source. And the other representations, do you want to just describe one? One looks like concentric circles, one looks like a star. And again, what is it that is causing those renderings to appear as they. They are? So I don't have the
Bob Levy: algorithm the Autodesk team used to generate these, but it's, it's several visuals. Sorry, several data points. Yeah, that's okay. Yeah.
Bob Levy: So it's a number of different arrangements, be it horizontal lines, vertical lines, a picture of a dinosaur, a picture of other, other aspects, circles. The data points are arranged in such a way that the mean and the standard deviation of the correlation are very similar even as we transition from one to one to the other. Understood. Okay, so that's right. So. So
Seth Earley: these summary statistics are the same, but the actual data sets and the visualization is very different.
Bob Levy: Absolutely. Another interesting point would be. So why would people want to see more than a few dimensions in the first place? And this is a really good, really powerful visual analogy we found helps. So when you look at this image, it looks like we're viewing tic tac toe. It looks like Red Jacks just won the game that we think is tic tac toe. And when we add just one more dimension, we realize the game was actually three dimensional tic tac toe. The red piece that we thought led to a win, in fact did not. It's Blue's turn and Blue is about to place in, in the winning position. So the exact opposite outcome is, is truth. And being able to see that requires a greater fidelity of, of dimensions. Yeah, this is like when you
Chris Featherstone: always hear those folks say, well, you know, one plus one is always equal to two. Well, that's not the case. Right. When you change the math and you're, you know, one times one is always one. You know those types of scenarios, which is exactly what you're showing here, where if you add another dimension, it completely invalidates what you're looking at. Right.
Bob Levy: Oh, go ahead. No, I was just going to ask you. I would love to
Chris Featherstone: get an idea too, Bob, and maybe this is something we continue to think about. But when do you guys believe you and your colleagues believe that a lot of your work right now is done in the context of showing the visualizations of the data, how this is representing from the metadata, the perspective of the LLMs and things. When do you believe the LLMs will be able to produce these types of graphs and charts? So
Bob Levy: we do have a patent on the technology for visualizing the multiple dimensions. We are in discussion with a number of leading providers as to how they might use this technology to help their customers, for example in fine tuning efforts, in guardrail and governance efforts as well as I can't speak to specific time frames of when the technology will be integrated in some of those. It is architected as a component technology, can be used in a standard web application. It can even be used in virtual and augmented reality devices. Now those are completely optional, but there's really strong research showing that by using devices like that you actually triple the number of data points you can absorb versus on a flat screen. Interesting.
Seth Earley: So, so this is really providing lots of different insights that you know, traditional metrics and visualizations cannot provide. What does it take to leverage this? And again, what's the level of complexity for being able to build some of these visualizations?
Bob Levy: You know, I should show one of our examples comes from the financial industry. So this one imagines you're a portfolio manager. So this could be integrated with say your Bloomberg Terminal or other environment. So let's take someone who does not have a background in AI, does not have a quantitative research background. They're simply a fundamental investor like Warren Buffett. They're looking for value plays, bargains. So in this case we've the price earnings shown on the X axis, the investable universe. Here is the S&P 500. We've priced book as color. So blue. There are bluebergens to the left is a good way to think of it, but by layering. And what we found is by layering them one at a time using a technique we call stepwise storytelling. So I'll layer in price sales and our Bergen minded portfolio manager thinks they want to be toward the the front of this chart and the left. So they make a selection and they as we layer in more attributes so we can layer in book value as this shimmering effect. So the more intensely it's shimmering the greater the book values. Goldman Sachs and Google glowing in this case to show Ebitda. So Apple, Verizon AT&T. But then we layer earnings per share and we didn't know earnings per Share was relevant before the analysis. But by taking a look, what we see is that that AA point that our portfolio manager would have selected, it actually has a terrible earnings per share. And just like in that tic tac toe analogy, the point they wanted in this case was, was near it. It was a very different conclusion once they see the whole picture. So when you
Seth Earley: start thinking of large language models that have hundreds of thousands of dimensions. Right. I mean, what, what is that those orders of magnitude of greater dimensionality mean? Because that's what, that's what the large language model is about. Yeah. When you look inside of a large language model,
Bob Levy: you find a neural network, you find a lot of linear algebra, you find individual dimensions that are generated by the system that may or may not have human meaning. So when you're taking like say the vector embedding space is a really good example of this. You've thousands of dimensions. In those cases, we found techniques like principal components is really useful as a way of absorbing those thousands of dimensions into a way of visually organizing. But where those techniques fall short is if you then want to make specific observations like is it hate speech or is it a demographic element or so forth. So the combination of both of those turns out to be a really powerful way to, to address even the world's widest data sets. So, yeah, this is interesting.
Chris Featherstone: Bob. I appreciate you doing this. I mean, it's, it's, it's hard not to have a podcast without visualizations. If we're going to talk about visualizations. Right. And dive into it.
Bob Levy: We found. So explaining things like what our MIT professor advisor had helped quantify, explaining concepts around virtual and augmented reality and how those help triple capacity for data points, but there's nothing quite like seeing it. So, Bob, one of the things you
Chris Featherstone: brought up was the relevancy of seeing this, finally seeing this, and then bringing a lot of this data to life to answer more insightful questions, which, you know, you talked about the augmented reality and the virtual reality type of world. So it seems to me, and I'd love to get your take on it, that these, all these, these goggles and glasses and all these types of scenarios will now finally have some relevance in business to helping us instead of, you know, now, now, hey, everybody, put your goggles on. Let's look at the, the 10K. Right? You know, from an executive perspective. So go ahead. And I'd love to get your thoughts there, of course. And
Bob Levy: we're seeing this already in education. So professors at leading institutions demonstrating complex concepts to their students, we've won using the Microsoft Hololens. We are a partner with Microsoft or in there. It's a spatial computing device where you see the room around you and you also see the content as though it were holographic. And everything you just saw here on the flat screen that's represented holographically in space. You fill your room with the data, you walk around it and you're able to absorb much more. One of the papers in this from 1994, the researcher was Colin Ware. He was able to prove empirically that people were able to absorb three times as many data points in a pathfinding experiment around graph data than they were on a flat screen. Now that was with VR as it existed back in 94. It's come a long way since then. We were recently included. Apple hosted before they launched the Vision Pro. Finally allowed to talk about this, they hosted pre release Porting Labs in Cupertino. And I will say I've never seen a device with the level of visual fidelity this one has. If you have chance, you know, check it out at your local Apple store. They have fairly widely available demos. And it turns out that by seeing these things with such high fidelity, you're missing less. Now another partner of ours is zspace. So these folks make the world's first 3D laptop. So it's a standard looking laptop. You position your view right in front of the screen. There's special cameras that track your pupils and the screen is comprised of prisms. So one image to one eye, a slightly different image to the other eye and it results in an experience where it feels like the content literally is hovering over the keyboard of the thing. I have one of them sitting behind me. It is one of the more interesting devices out there, but then you need. You need something to visualize in three
Seth Earley: dimensions to make use of it. Right. And we,
Bob Levy: practical application have that in spades. Yes, yeah, yeah.
Chris Featherstone: Well, where does, where does compliance then start to play a role? Right. Because part of this is, you know, like I said, it's one thing to be able to pull a lot of this data in, you know, and make it, you know, simplistic or more simplistic. Right. I think that's one of the goals here is, yeah, we're dealing with really complex information and situations, but essentially all you're doing is making it more, more simple, more consumable. Right. And yet more understandable. Right. So Zephair,
Bob Levy: I think it's perfect the, you know, the view that if you can understand more, it's going to Be a lot easier to govern those more complex topics when you can bring a regulator into the room and explain to them why, you know, a specific thing had hate speech. You made a change to the model and the hate speech is gone. And you can show them that before, after, and the remediation steps that were taken.
The laws, the regulations around AI are of course still emerging, but it comes back to common sense. These regulators need to be able to understand things that they weren't trained to understand. Yeah, I think
Chris Featherstone: I read somewhere that there was close to 600 different compliance standards from 89 countries or 86 countries. And it's all over the board. There's no global standard. And I feel like, and I'd love to get your take here, where does this type of work then? Does it, does it take existing standards and compliances, you know, in term into consideration and, or does it beg for difference, you know, different ones and more, let's say, more thorough introspection. Right.
Bob Levy: So our focus as a company is around the visualization technology that makes it easier to understand these things from a, from a regulatory perspective though. Yes, these standards have to emerge. They'll emerge as most regulations do from case law, so trial and error essentially, which is very unnerving. I feel a lot better about a Future where my 10 year old daughter grows up into a world where she can actually understand what's going on in these systems and as opposed to hoping a regulator had the right checklist.
Seth Earley: Yeah. So Bob, maybe we could go through some of the other visualizations that you have up here to think more about what are the other corporate applications that we might have. So do you want to walk through some of the other ones? So we have a number in finance. This
Bob Levy: next one comes from the derivatives space. So think stock options. There's a standard view in the investment management world called a volume volatility smile. So we're looking at puts and calls. The reds are the puts and the greens are the calls. These are contracts for the ability to buy a stock at a certain price by a certain time. So the strike price is here on the X axis. The time to maturity or how long the options contract will last is, is on the Z axis and a metric that's essentially shows the implied volatility, the volatility that the given contract would imply. So we see the standard kind of smile when the price of the underlying stock is close to, is close to the price of the contract, strike price of the contract, that's where volatility will be the lowest. We go ahead and layer different derivative statistics. So this one being shown a shimmer, it's referred to as delta. So this is how much will the price of a contract change given a unit change of the price of the underlying. So we might be looking at Microsoft or Apple or others. How much will the price of each of these contracts change?
Now as we look at these, we're looking for things that are out of the ordinary and so far we don't see it. But we'll layer in another one. So these are called the Greeks. Theta is used as a way of showing how sensitive the price is to the passage of time. People will pay more for a contract. They have a lot more time to exercise than a contract that they must exercise immediately. So this glowing that you're seeing, it represents theta. And options math would tell us we expect to see high theta close to strike in contracts that are about to expire. What we notice is that theta fades out and then there's a hotspot of a couple of contracts in the most distant months. And this lasted for 20 minutes that are showing high theta. This never should happen. This is completely an arbitrage opportunity. Interesting. Many examples like this. So, so again, how does that.
Seth Earley: What's the arbitrage opportunity in that representation?
Bob Levy: Ah, yes. So the, the anomalous contracts are mispriced. Yep. So buying or selling to profit from the mispricing until the market comes back into alignment. And again, the theta represents the. It's the each contract sensitivity to the passage of an additional day. So how much will the price decrease each. Each day it is in motion. And it makes more
Seth Earley: sense if it's about to mature versus it's a long time up. Yeah, right. We'd expect to see the
Bob Levy: highest data at maturity. We'd expect to see the lowest beta distant from maturity. Anomalies show up in ways you'd never expect. Personal experience.
Personal experience by seeing many dimensions at once. It doesn't matter if it's glowing or shimmering or what it's doing. It matters that it's doing something that you're able to see. And that's right. That's what we have here. That's great. Yeah. Carry on,
Seth Earley: Carry on. If you want to show some other or explain some of the other. Because this gives us a sense of what those specific applications are. Well, I'm just, I'm just glad the discussion is, is truly
Chris Featherstone: multi dimensional. I mean. No, no pun intended. No, a pun was
Seth Earley: intended. Yeah. What
Chris Featherstone: else do you have, Bob? It's great. Okay, so this one comes
Bob Levy: from Another financial group. So we were looking at price. So height is showing us returns, a proxy for price. Looking at different companies in the technology space. Now there was some interest in understanding, well, why did some perform better than others. So if we look at debt capitalization as glowing, we see that encumbrance by debt is a contributor to low performance. These are the points that did not appreciate or the series that did not appreciate as much. Similarly, price sales ratio. So we find that one of the stocks is bright red, that's Adobe, and they have a substantially higher price sales ratio. And that resulted in. That seems to be a contributing factor or at least a correlating factor to increase price performance. So these are clues. I believe it was said correlation is not causation, but sometimes it's an interesting hint. And this is a way of seeing those hints woven together.
Chris Featherstone: That's fascinating. And what are the smaller.
Bob Levy: What are. The smaller pieces there that are kind of chained
Seth Earley: together? Yes, yes. So we're showing a financial time series of projects,
Bob Levy: prices each day. But companies report financial like fundamental metrics once a quarter. The bigger points are showing each quarterly report and the, the green points are showing us the, the daily prices.
Seth Earley: Gotcha. Okay. Exactly. Very good, very good. Next one.
Bob Levy: Great. We have had. Now this one works a lot better in virtual reality than it does on a flat screen. But what you're seeing here is an artificial neural network. So you're seeing both weights and biases. When you fill the room with data, you have it physically with you. You're able to visualize interestingly large neural networks. And you can see, for example, through a learning algorithm like back propagation at each training epoch. When you're training one of these, how do the weights and biases change? Wow. People have never been able to see that before. Wow. We're at the
Bob Levy: forefront of helping people understand the internals of these. Then walking through the data will have a
Seth Earley: whole different meaning. Like you say, if you fill the room with data and you're using virtual reality and you can walk around it and see it from different angles, that will be really something. And you could watch
Seth Earley: the training as it's happening. The fact that we can look at
Chris Featherstone: this because we always talk about, you know, in a two dimensional way, looking at a graph neural net for things like I deal with telco a lot, I deal with customer experience a lot, deal with those scenarios. And you're always trying to derive out the associations, of course, and then the propensities. Right. To predict what that looks like for specific things. And you know, it'd be really interesting to get in and just look at it from a true multidimensional perspective as opposed to just guessing what that relationship looks like. Again, man, you know the, the reason for everybody to get these, these, these glasses and goggles to actually look at, actually derive value from the data. You know, the pandemic forced us. Before the pandemic we
Bob Levy: were primarily focused on virtual and augmented reality. Because it is quite interesting, the pandemic forced us to. We. We have a component that if you have a web application, you can embed these kind of visuals in your web application as a component technology. So for example, if you have a platform where people build AI or govern AI systems, these visuals can be made a component of those platforms fairly easily.
We found that the wide data problem. So seeing the special effects works great on a flat screen. The large data problem, that's where the glasses really help. Or the 3D laptop. I'm sometimes asked do I see a future where you'll walk into a boardroom and people will actually have the goggles on? And I'm very happy to see devices like the ZSpace laptop, the 3D laptop. It does not look strange. It's the sort of thing you'd expect to see in a boardroom. And just for people who are listening, you have to watch
Seth Earley: the YouTube video on this because this is really amazing stuff. Do you want to explain the. We have a few more minutes. The turbine looking one or the one that looks like a knowledge graph. Ah yes. So we'll take the
Bob Levy: graph. This was graph data. So again we have this concept of a multi dimensional data point. We also have multi dimensional edges where different special effects are used. This is an analysis we had a Python script comb Reddit looking for different programming themes, which languages were related to which other languages. In this case it's an example of how knowledge graphs can be represented. But anything we have numeric measures to different attributes we control. Many of those proposals point as as well. Now this one and pair with me while I restart this.
So I have a Python script running in the background that is calling Coinbase's API. We're looking at the real time price of different cryptocurrencies which are very, very strong bull trend of late. What we're looking at here is each second so the relative change in Bitcoin versus Ethereum versus some of these other other coins. The color is showing us volume so we had a spike in volume before price increased on. On Bitcoin for example, it just had a spike in volume on Avax that spike corresponded to an increase in its price. Do this one visual that's giving you the same information of it would be 28 stock charts or crypto charts on a dashboard. If you think of the, the setup where you see lots of screens in front of these traders, they need a, a way to, to manage the cognitive overload. And
Seth Earley: one of those. This is, this is perfect. It's beautiful. So this is second by second real time pricing. That's right.
Bob Levy: I have a Python script running in the background that's connected to Coinbase's real time data API. It is then the Python script is feeding data into our visualization engine here and the technology is available to visualize anything you, you wish. You can use simple CSV data, you can use Python or other programming languages. You can introduce it to enterprise level applications and have the applications control the data flow as well. What, what does
Chris Featherstone: the, the overhead look like in terms of compute needs for this? Right. What GPU type visualizations?
Bob Levy: So this piece is a little bit of trade secret, but so my, my CTO has a background in rendering technology, so think the graphic processing unit we have benchmarked. Everything you're seeing today, I can show you on a circa 2014 Microsoft Surface Pro 4, which is not very powerful. All rendering is performed client side and with interesting level devices. So one of the machines here on loan from intel is a server core processor with a high end graphics card. We've visualized hundreds of thousands of data points with, with that one, it gets down to how much data, how many data points do you really want to visualize? The bottleneck there is the visual cortex. How much can we actually see? Yeah, so that's in the thousands to tens of thousands and at that scale you can even try. We do have a sample iPad app with some of these demos. If you type Immersion analytics into the App Store and you can see, you can see some of these running on a standard iPad. And the touch interface is, it's not as good as virtual reality, but it's better than a regular flat screen where you have no sense of control. People do get a broader sense of the data even with a standard iPad. So this has been
Seth Earley: fantastic. Bob, I really, really enjoyed this. I know we've had other conversations and this really went into a lot of depth. So I appreciate your time. And by the way, just so people can know a little bit more about you, what do you do for fun? What, what, what do you do in your spare time? I know you have kids and any, any hobbies, any leisure activities that you're interested in. Absolutely. So I am
Bob Levy: an avid technologist. I like tinkering with things, fixing things, but I also love sailing. I have been known to golf as well, so, and, and travel. Taking my daughter to D.C. next week actually. Wonderful. So you, you pay for
Chris Featherstone: frustration on the golf game. That's good. And if, hey, let me ask you one, one quick thing. If what was, what would be or what is something that we didn't ask you that we should have.
Bob Levy: Okay. I think we can leave with that
Chris Featherstone: because I think there's, you know, there's. This is amazing. We could keep going, but I always like to ask that. Oh, where to
Bob Levy: begin. If it's too long, no
Chris Featherstone: problem. Right. So we talked about the AI areas.
Bob Levy: We've talked about, I suppose, how something like this gets integrated into standard off the shelf applications. So for the technologists in the crowd who are comfortable, let's say visualization, you might think of learning GPU based technologies and writing potentially close to a million lines of code to make some of this work. Or we provide standard off the shelf SDKs where with in some cases a couple dozen lines of JavaScript you have one of these embedded in a regular website. So it turns out to be a powerful way whenever an ISV has conversations or customers who have complex data that they need to understand and haven't been. So in areas like cyber security or defense and intelligence areas of course, finances, we've discussed education,
AI governance of course, many areas we are I think still just scratching the surface of the applications where it fits. That's awesome. So Bob,
Seth Earley: where can people find you?
Bob Levy: Immersionanalytics.com that's I M M E R S I o n analytics.com if you want to reach out by email, Bob, immersion analytics.com will reach me directly. I also reply to responses on our contact form on the on the site. The site has lots of lot of videos that for those who weren't able to see some of the visualizations today, many videos throughout the website. Well, thank you
Seth Earley: Bob again for your time. This has been a pleasure. Thank you to our audience for listening and if you enjoyed this, please share the episode. Subscribe like all of those things and this has been another episode of the Early AI Podcast and we will see you next time. Thanks Bob, Appreciate it. Excellent.
Bob Levy: Thank you guys. That's great.