微調代理的開源模型.pdf

1、2024 Databricks Inc.All rights reservedFINE-TUNING OPEN SOURCE MODELS FOR AGENTSTristan Zajonc-Continual.AIJune 11,20241Were entering the era of AI agentsConversational agentsGithub Copilot:https:/ agentsHex Magic:https:/hex.tech/product/magic-ai/Event-driven agentsCleric:https:/cleric.ioAutonomous

2、agentsGithub Copilot Workspace:https:/ do we build these types of agents?A G E N T S A R E S Y S T E M S N O T M O D E L SExternal ServicesAPIsData SourcesAgent SystemOrchestrationModelsToolsKnowledge BasesEvaluation,Monitoring,Feedback,and Fine-TuningIn-context agentsConversational agentsEvent-driv

3、en agentsAgent ExperienceAutonomous agentsLets talk about modelsOpen source models are laggingAgentBench:https:/llmbench.ai/agentSWE-Bench(Lite):https:/ UseReasoningKnowledgeTool UseReasoningKnowledgeIs there hope?TinyAgent:https:/bair.berkeley.edu/blog/2024/05/29/tiny-agent/How do we fine-tune a mo

4、del for agents?TinyAgent/LLMCompilerChoose how you want to call functionsOpenAIGenerate synthetic data using self-instruct or agent gymAgentGymOpenAI CookbookDont forgot to cover the full scope of user behaviorAgent-FLAN:https:/arxiv.org/abs/2403.12881Choose chat template suitable for LLM trainingRe

5、Act chat templateAgent-FLAN chat templateAgent-FLAN:https:/arxiv.org/abs/2403.12881Fine-tune your base modelAgentTuning:https:/thudm.github.io/AgentTuning/Consider embedding within a multi-agent systemMixture of Agents:https:/arxiv.org/abs/2406.04692Should you fine-tune open source models for AI age

6、nts?C O NProprietary models are still significantly ahead of open source models for agent use cases.Collecting agent trajectories for complex tasks like coding is non-trivial.Fine-tuning can easily degrade general performance and become a game of whack-a-mole.Scale effects are very real.Youre unlike

7、ly to beat frontier models for generalist agents.You will learn a lot and generate a lot of useful data.Self-instruct and agent gyms makes collection of trajectories feasible for many use cases.You can significantly increase performance of agents in specific domains,even beating frontier models.Open source frontier models are getting better just like proprietary models.It allows you to control your own destiny.P R OLearn how and give it a shot.Thank youhttps:/continual.aitristancontinual.ai