大語言模型與交互式智能體：開放世界中的動態推理與規劃.pdf

1、DataFunSummit#2023大語言模型與交互式智能體：開放世界中的動態推理與規劃林禹臣-Allen Institute for AI-研究員Textual EnvironmentReal-world Situations for AgentsTask planning&execution interactive environmentALF Worldsample task:put A in Bmostly easy&shortaction space:very limitedScienceWorldComplex Setup-10 locations-25 action types-

2、200+object types-multiple states-random exceptions-30 high-level task typeshttps:/yuchenlin.xyz/swiftsage/FormulationBaseline methods:Reinforcement Learning DRRN:deep reinforcement relevance networkDRRN:deep reinforcement relevance networkAction State value for action i under state s at the time tKG

3、-A2C:add a dynamic graph to constrain the selection of actions+objects CALM:use a larger LM say GPT-2 to re-rank the action candidates after Q function is computed.Baseline methods:Imitation LearningBehavior Cloning with Transformer LMsBehavior Cloning with Transformer LMsDecision Transformer w/Caus

4、al Transformers Text Decision Transformers(Behavior Cloning)in the ScienceWorld paper Action History+Observations +Env t,t-1action t+1Oracle agent On training example tasks,I can search&generate golden paths for completing the tasks.Offline Training Data(in seq2seq mode)SayCanReflexionTaskTask:Your

5、task is to boil tin.For Each Timestep t:Action 1:go to kitchen you moved to kitchenAction 2:look around In this kitchen,you can see.Actiont-1:pick up metal pot metal pot in inventory nowDemoDemo:An oracle path for the task of boiling water.Kbest generations for Action tAction t:put metal pot on stov

6、eRerankingTaskTask:Your task is to boil tin.+the of previously failed trials in the last round.Action historyAction history(you moved to kitchenAction 2:look around In this kitchen,you can see.Action t-2:thinkthink:now I need to place the metal pot on a heater.OK.Actiont-1:pick up metal pot metal po

7、t in inventory nowDemoDemo:oracle path+manually annotated subgoalsLLMAction t:put metal pot on stoveFor Each Timestep t:LLMLLMSwiftSageTaskTask:Your task is to boil tin.ActionAction historyhistoryFor Each Timestep t:LLM Prompting Methods:baseline+oursLLM Prompting Methods:baseline+ours30 Tasks on Sc

8、ienceWorldEfficiencyEfficiencyCost-EffectivenessConclusionProject website:https:/yuchenlin.xyz/swiftsage/1.SwiftSage is a hybrid agent framework:smaller LM with imitation+LLM with two-stage prompting 2.Especially designed for embodied actions.Plan+Ground style for LLM prompting.3.Limitations:1)need an oracle agent for offline learning.2)still rely on closed LLM 3)need an interactive environment for feedback(world engine)4.Future directionsgeneralize to more complex tasks knowledge distillation real-world embodied robots(with vision inputs)more?感謝觀看