A talk by Hannes Kindbom from 9fin.
This session covers Improving complex RAG systems and achieving no regret lightning fast deployment iterations of LLMs.
In November 2022, ChatGPT took the world by storm and Large Language Models (LLMs) have been a hot topic ever since. However, their limitations such as outdated training data, restricted context windows, latency and API rate limits have become clear. Retrieval Augmented Generation (RAG) has grown popular as an approach to circumvent some of these challenges but RAG systems are complex and the user experience is hard to test offline, making prod deployments scary.
In this talk, you’ll learn how to tackle these problems to achieve safe and lightning fast deployment iterations of LLM based applications by deploying to “shadow” and feature flagging beta versions using AWS lambda aliases. We’ll dive into how to leverage the unique data this gives us to evaluate our system in production. Target audience include ML/software engineers as well as data scientists. Participants would benefit from having some coding experience with LLMs, RAG and AWS lambdas or equivalent.
Technical Level: Technical practitioner
This session was part of the Data Science Festival MayDay event 2024. Find out more at datasciencefestival.com/event...
The Data Science Festival is the place for data-driven people to come together, share cutting-edge ideas, and solve real-world problems. We run monthly events, meet-ups, and the biggest free-to-attend data festivals in the UK. Join the community at datasciencefestival.com/