All Posts

    [Earley AI Podcast] Episode 27: Gordon Hart

    Machine Learning and Algorithms

    Guest: Gordon Hart

     

     

    About this Episode:

    Today’s guest is Gordon Hart, Co-Founder and Head of Product at Kolena. Gordon joins Seth Earley and Chris Featherstone and shares how ​​machine learning algorithms are a challenge from different perspectives. Gordon also discusses the core problem in his company before they turned it around. Be sure to listen in on Gordon giving his advice on how to validate models in order to have a successful product!

     

     

    Takeaways:

    • Gordon noticed that developing algorithms internally or buying from other model vendors, has really had a constant unexpected model behavior. It made him feel he couldn’t trust the models to behave sensibly. 
    • Gordon started his company because he noticed that time after time, he was getting blindsided. So he thought to himself that there has to be a better way to develop models and be able to validate what they were doing
    • The key challenge that Gordon and his team ran into was that when you have all the data when they were looking at that one number, they were looking at that aggregate metric computed across their entire benchmark.
    • Gordon expresses the importance of going through scenarios with your products. He found that when you break down your evaluation into these different scenarios, the test gives you an understanding of how this model improves in the aggregate over previous models and how are the failures distributed.
    • Gordon believes that testing data versus training data is more critical since your testing data is what you're using to decide if your new model has the behaviors it needs to have.
    • Testing the full pipeline from pre-processing through post-processing rather than testing the model component will oftentimes improve the visibility into how your product is actually going to work when you put it out there.



     

    Quote of the Show:

    • “Having your evaluation metrics align with the way that your system is going to be evaluated in the field is a key thing that you can do to get a better understanding of ‘is this model better for what I set out to do?’” (22:36)



    Links:



    Ways to Tune In:



    Thanks to our sponsors:

    Earley Information Science Team
    Earley Information Science Team
    We're passionate about enterprise data and love discussing industry knowledge, best practices, and insights. We look forward to hearing from you! Comment below to join the conversation.

    Recent Posts

    [RECORDED] Product Data: Insights for Success - How AI is Automating Product Data Programs

    Artificial Intelligence is changing the way businesses interact with their customers. From hyper-personalized experiences to chatbots built on Large Language Models, AI is driving new investment in digital experiences. That same AI and LLM can also be used to automate your product data program. From data onboarding and validation to generating descriptions and validating images, AI can help generate content faster and at a higher quality level to improve product findability, search, and conversion rates. In our second webinar in the Product Data Mastery series, we’re speaking with Madhu Konety from IceCream Labs to show exactly how AI and product data can work together for your business.

    AI’s Value for Product Data Programs

    By Dan O'Connor, Director of Product Data, Earley Information Science

    The Critical Role of Content Architecture in Generative AI

    What is Generative AI? Generative AI has caught fire in the industry – almost every tech vendor has a ChatGPT-like offering (or claims to have one). They are claiming to use the same technology – a large language model (LLM) (actually there are many Large Language Models both open source and proprietary fine-tuned for various industries and purposes) to access and organize content knowledge of the enterprise. As with previous new technologies, LLMs are getting hyped. But what is generative AI?