張寧豫_大模型智能體的自主規劃學習_watermark.pdf

1、大模型智能體的自主規劃學習張寧豫浙江大學 代表案例：AutoGPT，GPT-Engineer，Voyager,RT-2，什么是大模型驅動的自主智能體 定義：由大型語言模型驅動的自治代理，它們可以遵循語言指令并在真實世界或模擬環境中執行各種復雜任務背景1 A Survey on Large Language Model based Autonomous Agents大模型驅動自主智能體的飛速發展背景1 Artificial Intelligence:A Modern Approach2 Multiagent Reinforcement Learning:Rollout and Policy It

2、eration大模型驅動的自主智能體本質 智能體（Agent）的概念自人工智能領域誕生之日就存在，并同時受到面向對象系統、人機交互、分布式學習、強化學習等領域關注圖引用自1圖引用自2背景1 https:/lilianweng.github.io/posts/2023-06-23-agent/2 Reasoning with Language Model Prompting:A Survey大模型驅動的自主智能體本質圖引用自1圖引用自2 智能體（Agent）的概念自人工智能領域誕生之日就存在，并同時受到面向對象系統、人機交互、分布式學習、強化學習等領域關注 大模型驅動的自主智能體有什么區別？本質

3、的區別在于語言的運用 人工智能代理（Agent）將語言作為思維和溝通的工具，這是人類獨有的能力背景大模型驅動的自主智能體關鍵技術背景1 Graph of Thoughts:Solving Elaborate Problems with Large Language Models大模型驅動的自主智能體的任務規劃大模型驅動的自主智能體通過任務解耦和思維鏈（圖）的方式進行任務規劃智能體自主規劃大模型驅動的自主智能體的任務規劃智能體自主規劃1 Voyager:An Open-Ended Embodied Agent with Large Language Models9大模型驅動的自主智能體的任務規劃

4、智能體自主規劃智能體自主規劃 挑戰二：大模型智能體如何根據已有的推理路徑自主校準并優化規劃過程Towards A Unified View of Answer Calibration for Multi-Step Reasoning(2023)大模型智能體的任務自主規劃關鍵挑戰 挑戰一：大模型智能體如何根據任務的內容自主規劃并選擇和使用外部工具Making Language Models Better Tool Learners with Execution Feedback(2023)開源工具：Agents:An Open-source Framework for Autonomous La

5、nguage Agents(2023)智能體自主規劃Project:https:/zjunlp.github.io/project/TRICE/Code:https:/ Making Language Models Better Tool Learners with Execution Feedback大模型驅動的自主智能體的工具使用智能體自主規劃智能體使用工具的步驟:1.智能體什么時候候用？用什么工具?2.智能體需要提供工具哪些信息?3.智能體如何根據工具的結果做出反饋?大模型驅動的自主智能體通過外部工具和環境反饋實現自主規劃完成任務智能體自主規劃1 Making Language Mode

6、ls Better Tool Learners with Execution Feedback先基于指令微調學習如何使用工具，后基于工具執行反饋進一步習得何時使用工具的自主規劃能力智能體自主規劃1 Making Language Models Better Tool Learners with Execution Feedback智能體自主規劃1 Making Language Models Better Tool Learners with Execution Feedback智能體自主規劃1 Making Language Models Better Tool Learners with

7、Execution Feedback智能體自主規劃1 Making Language Models Better Tool Learners with Execution Feedback智能體自主規劃1 Making Language Models Better Tool Learners with Execution Feedback該方法可以在一定程度上降低智能體對工具依賴，讓智能體知道何時靠自己何時找幫手智能體自主規劃1 Making Language Models Better Tool Learners with Execution Feedback智能體自主規劃 挑戰二：大模型智

8、能體如何根據已有的推理路徑自主校準并優化規劃過程Towards A Unified View of Answer Calibration for Multi-Step Reasoning(2023)大模型智能體的任務自主規劃關鍵挑戰 挑戰一：大模型智能體如何根據任務的內容自主規劃并選擇和使用外部工具Making Language Models Better Tool Learners with Execution Feedback(2023)開源工具：Agents:An Open-source Framework for Autonomous Language Agents(2023)智能體自

9、主規劃1 Towards A Unified View of Answer Calibration for Multi-Step Reasoning智能體自主規劃大模型規劃路徑的自主校準問題：何時校準？校準到什么粒度？1 Towards A Unified View of Answer Calibration for Multi-Step Reasoning智能體自主規劃大模型智能體規劃路徑自主校準的統一視角，提供何時校準及如何校準的經驗性原理=0和=1是self-verification和self-self-consistency特例1 Towards A Unified View of A

10、nswer Calibration for Multi-Step Reasoning智能體自主規劃大模型智能體規劃路徑自主校準的統一視角，提供何時校準及如何校準的經驗性原理1 Towards A Unified View of Answer Calibration for Multi-Step Reasoning智能體自主規劃1 Towards A Unified View of Answer Calibration for Multi-Step Reasoning智能體自主規劃兩個閾值區間內智能體的規劃推理性能達到最優，驗證了所提出的統一視角的原理智能體自主規劃 挑戰二：大模型智能體如何根據

11、已有的推理路徑自主校準并優化規劃過程Towards A Unified View of Answer Calibration for Multi-Step Reasoning(2023)大模型智能體的任務自主規劃關鍵挑戰 挑戰一：大模型智能體如何根據任務的內容自主規劃并選擇和使用外部工具Making Language Models Better Tool Learners with Execution Feedback(2023)開源工具：Agents:An Open-source Framework for Autonomous Language Agents(2023)1 Agents:A

12、n Open-source Framework for Autonomous Language Agents智能體自主規劃Agents:An Open-source Framework for Autonomous Language Agents智能體自主規劃Controllability via Symbolic Control(SOP)An SOP is a graph of states each describing a sub-goal of the agent system.Language agents acts according to the specification of

13、 the state and transit according to the an LLM-based controller1 Agents:An Open-source Framework for Autonomous Language Agents智能體自主規劃In the Agents Framework,a(multi-)agent system is defined by the agents,the SOP,and the environment.All these components can be described in a single config file using

14、 natural language 1 Agents:An Open-source Framework for Autonomous Language Agents智能體自主規劃Running loop in the Agent framework is straightforward1 Agents:An Open-source Framework for Autonomous Language Agents智能體自主規劃Single AgentsMulti-Agents1 Agents:An Open-source Framework for Autonomous Language Age

15、nts智能體自主規劃Agents can help user generate a config file for a language agents system by simply inputting a short description of it.How?Retrieval-based generation using agent-hub that stores various config files.“The more you share,the more you can create”1 Agents:An Open-source Framework for Autonomou

16、s Language Agents展望 大模型智能體如何根據已有的推理路徑自主校準并優化規劃過程Towards A Unified View of Answer Calibration for Multi-Step Reasoning(2023)大模型智能體如何根據任務的內容自主規劃并選擇和使用外部工具Making Language Models Better Tool Learners with Execution Feedback(2023)開源工具：Agents:An Open-source Framework for Autonomous Language Agents(2023)大模

17、型智能體的自主規劃學習展望 Why multi-agents?Goodharts Law-The better on object A,the worse on many other objects B What do agents interact with?Knowledge boundary,Brain in a Vat What is the preferred method of communication among agents?Natural language or Code How to communicate between agents?Roles,Society,Behaviors1 Exploring Collaboration Mechanisms for LLM Agents:A Social Psychology View2023/11/3036ACCEPTMYENDLESSGRATITUDE敬請各位專家批評指正