Breaking Down & Testing FIVE LLM Agent Architectures - (Reflexion, LATs, P&E, ReWOO, LLMCompiler)

Рет қаралды 8,717

17 күн бұрын

Large Language Model Agents have taken over LLM and Artificial Intelligence application design by storm, so this time we check out and simplify six main concepts and five popular papers documenting ways to set up language model based agents, as well as directly testing examples.
Resources -
@LangChain Agent Tutorials & Code - kzfaq.info/sun/PLfaIDFEXuae16n2TWUkKq5PgJ0w6Pkwtg
Overview on LLM Agents - kzfaq.info/get/bejne/fcCHea92z5a5oX0.html
Link to Miro Board PDF: drive.google.com/file/d/1ESnrIy4c5LPOhNHRnn87Cv7DU_i0-_J9/view?usp=sharing
Papers:
Reflexion Paper: arxiv.org/abs/2303.11366
LATs Paper: arxiv.org/abs/2310.04406
Plan-And-Execute Paper: arxiv.org/abs/2305.04091
ReWOO Paper: arxiv.org/abs/2305.18323
LLMCompiler Paper: arxiv.org/abs/2312.04511
LangSmith Traces:
Basic Reflection: smith.langchain.com/public/604a986b-a751-48ed-9dda-738bd2780ea5/r
Reflexion: smith.langchain.com/public/83b0f3a3-b788-471b-ae1b-57167ceef37c/r
LATs: smith.langchain.com/public/d4541960-3f1a-4e53-9ec7-d223736f64b7/r
Plan-And-Execute: smith.langchain.com/public/c13392c5-e058-4bbe-b25e-aac7214a28df/r
ReWOO: smith.langchain.com/public/ee636208-6914-4c97-9d29-64c572c6ecc0/r
LLMCompiler: smith.langchain.com/public/28857a8a-b9b1-45d5-8403-ded9f95fab6d/r
Chapters:
00:00 - Intro
01:08 - Basic Reflection
02:44 - Basic Reflection Testing
06:32 - Reflexion Actor
09:57 - Reflexion Action Testing
12:25 - Language Agent Tree Search (LATs)
17:04 - LATs Testing
20:54 - Plan And Execute
23:38 - Plan And Execute Testing
26:28 - Reasoning Without Observation (ReWOO)
29:26 - ReWOO Testing
31:11 - LLMCompiler
35:19 - LLMCompiler Testing
36:05 - Outro

Пікірлер: 37

@pinkmatter8488 14 күн бұрын

Your channel has been very valuable today to get me situated on how to get the hang of LLM use. I can now start thinking about project ideas to get some practice. Thank you very much !

@MekMoney79 17 күн бұрын

outstanding overview of key the agentic architectures, I learned a ton, prob one of the best out atm - Thanks

@PYETech 16 күн бұрын

That's an amazing work we have here, guys. Cheers to you, bro. Thanks!

@cmthimmaiah 15 күн бұрын

Very nicely done, thank you for such a good preseentation.

@kenchang3456 15 күн бұрын

This is really great info, thanks a bunch for sharing. What's really eye-opening is the run times and token counts.

@caokang4957 14 күн бұрын

Thank you for sharing! Great summary.

@TheFocusedCoder 16 күн бұрын

Really good break down for folks building,thanks for putting this out

@GriffinBrown-tq9jz 14 күн бұрын

Well done! Thank you, sir

@sanesanyo 12 күн бұрын

Great work, thanks for this🙏. There is another agentic approach which is called self discovery. Would be cool if you cover that as well 😊.

@user-gy7te1ql3g 15 күн бұрын

Good overview. It would be very interesting to see the answer quality benchmarks for these techniques. In a lot of real business cases the time and cost have much less importance than the quality.

@tyler-morrison 14 күн бұрын

This breakdown is insanely helpful 👏 I’ve been working as a Web Engineer for > 10 yrs and recently started learning about AI/ML. I began my career as a self-taught dev in the good ol’ jQuery days, but my lack of CS fundamentals is starting to come back an bite me. These architectural diagrams are incredibly useful for breaking down high-level concepts.

@AdamLucek 12 күн бұрын

Glad you found this helpful! Everything I record and share is all self-taught as well, I've got no formal CS background- I just think the topic is interesting and worth sharing!

@tk0150 11 күн бұрын

Would you share your slides? So helpful!

@MEvansMusic 10 күн бұрын

what is used for scoring?

@Jandodev 16 күн бұрын

We made a 7th with output focused recursive events at my company :)

@xollob 11 күн бұрын

Hi Adam, great work. I've been struggling trying to evaluate the different agent frameworks, autogen, crewai VRSEN and on and on. langchain etc. seems to be more logical as we can see what's happening and is more predictable. Would it be possible to get the Miro you built for this presentation? Greetings from France.

@AdamLucek 9 күн бұрын

Here you go! drive.google.com/file/d/1ESnrIy4c5LPOhNHRnn87Cv7DU_i0-_J9/view?usp=sharing

@xollob 3 күн бұрын

@@AdamLucek Thank you so much Adam.

@lavamonkeymc 11 күн бұрын

Question: If I have a data preprocessing agent that has access to around 20 preprocessing tools, what is the best way to go about executing them on a pandas data frame? Do I have the data frame in the State and then pass that input in the function? Does the agent need to have access to that data frame or can we abstract that?

@AdamLucek 11 күн бұрын

I imagine it could be abstracted out. A lot of the processing you can do with a langgraph setup similar to these doesn't necessarily need an LLM touch at the computation/function step- could use the LLM for logic based routing to the right node function that is already defined to affect a pre set dataframe

@geofffane5276 9 күн бұрын

Hey can you please share the miro board link? Or drop it into a high res pdf? AWESOME work btw 👍👍👍

@AdamLucek 9 күн бұрын

Here you go! drive.google.com/file/d/1ESnrIy4c5LPOhNHRnn87Cv7DU_i0-_J9/view?usp=sharing

@ricardoaltamiranomarquez753 12 күн бұрын

¿Puedes compartir con nosotros tu presentación de Miro?, Great Job

@AdamLucek 9 күн бұрын

Here you go! drive.google.com/file/d/1ESnrIy4c5LPOhNHRnn87Cv7DU_i0-_J9/view?usp=sharing

@ricardoaltamiranomarquez753 9 күн бұрын

@@AdamLucek thank you very much, you are very good

@linuszhu 16 күн бұрын

which one do you prefer for the recommendation

@AdamLucek 15 күн бұрын

I would say each have different applications, and are better used as parts of larger agent ecosystems. I.e. taking a reflection based approach to some end validation step would be useful, however a more plan-and-execute style approach to initial generation would likely be a better first step. As with most llm based apps, a lot depends on what data your using, the task/end goal you want, and your tolerance of processing time. Would more so apply the general concepts here rather than see them as strict end solutions 😁

@genXstream 3 күн бұрын

Which would you say is more crucial to analyzing the "correctness" of the language agent tree search result: "blah blah blah" or "yada yada yada"?

@AdamLucek 3 күн бұрын

Im more partial to yada yada yada, but I can see the benefits of blah blah blah. Really comes down to your use case and desired blah to yada ratio

@matthewpublikum3114 16 күн бұрын

Where's the code? It would be nice to know what is the smallest LLM capable of doing the planner/task decomposition and verification.

@AdamLucek 16 күн бұрын

The code comes from LangChain's series on LangGraph, linked in the description. Here's a direct link to their repo github.com/langchain-ai/langgraph/tree/main/examples

@PRColacino 14 күн бұрын

Great video! Could you share the code?

@AdamLucek 12 күн бұрын

Thanks! The code comes from LangChain's series on LangGraph, linked in the description. Here's a direct link to their repo github.com/langchain-ai/langgraph/tree/main/examples