Thanks for writing Rohan and the suggestion. Yeah was thinking to do a deep dive into this, but the setup is a little too fragile right now with open source models. Waiting for FastChat to release an update allowing GPTQ 4bit support (github.com/lm-sys/FastChat/pull/487) as this would enable running the Vicuna 7B 1.1 with OpenAI formatted API calls and really streamline things.

@@aidoks update on this?

yeah agree. No extra nonsense. Love it.

Thanks Ahmet, appreciate the feedback

Thanks Jacob. Was planning to do this shortly (see the comment to Rohan regarding the reason I've held back for now). In terms of the deep dive, anything specific you wanted me to focus on most?

Thank you for the feedback. Still planning to keep at it, just trying to tune my workflow a bit. Should have a new video on the channel shortly.

@@aidoks Glad to hear. I'm just getting into LLMs hoping to make it part of my ETL pipeline, but not sure how it will turn out since it's not deterministic.

@@Stubbelur Nice. Yeah, had an issue with this first hand. My work around was to provide the available options as context to the LLM, allow it to select the most appropriate option and then have a validator functioning double check if it really did select a viable answer (if not, re-run the LLM call).

It's possible now without needing OpenAi calls. However, a GPU with 12GB ram is needed for best results. Will publish a tutorial once it becomes more straightforward with a lower memory GPU

Any suggestion for the type of sheet you had in mind?

Thanks for the kind comment ☺️

I think you first search your local vector database of embeddings, and pass embeddings that match along with your query to ChatGPT. But I dunno.

You can find me on LinkedIn here: www.linkedin.com/in/vassiliz/