《ResearchInChina：2024年車載AI代理產品開發與商業化研究報告（簡版）（英文版）（15頁).pdf》由會員分享，可在線閱讀，更多相關《ResearchInChina：2024年車載AI代理產品開發與商業化研究報告（簡版）（英文版）（15頁).pdf（15頁珍藏版）》請在三個皮匠報告上搜索。

1、AutomotiveAIAgentProductDevelopmentandCommercializationResearchReport,2024A Automotive AI Agent product development:How to enable“cockpit endorser”viafoundation models?According to OPEN AIs taxonomy of AI(a total of 5 levels),AI Agent is at L3 in the AI development path:Limited by interaction modes

2、and tool usage capabilities,popular foundation models in 2023 can only reach L2(Reasoners)at most.In contrast,developingthe automotive AI system by building automotive agents is a more appropriate goal:Agent improves weak links of application of foundation models inscenarios by way of calling active

3、 intelligent features and multiple tools/foundation models,further improving cockpit intelligence Agent is the endorser of emotional cockpitsEmotional cockpit has been nothing new formultiple years,but actually realizing it still starts withthe introduction of foundation models in vehicles.Under spe

4、cific triggering conditions,voice assistantchats with the user through preset emotional corpus,but it cannot adapt to human dialogue logic in realchat scenarios.After being applied to vehicles,Agent integrated with multiple foundation modelbasescanrecognizetheenvironmentmoreaccurately,and more tool

5、library interfaces furtherenhances its generalization capability to cope withchat and Q&A in diversified scenarios,truly realizingthe warm companionship of the cockpit Agent Emotion Technology/Products of Some OEMs and Tier1sThedesignofmainstreamemotionalinteraction scenarios focuses on emotionrecog

6、nition,user memory,and behaviorarrangement.Some OEMs and Tier1s havealso launched technologies or products toenhance the emotional value of Agents: Steps to Build Xiaoai Tongxues Emotional Dialogue SystemFor example,Xiaoai Tongxues emotional dialogue system is built in three steps: The mixed strateg

7、y dredging model of XiaomiThemixedstrategydredgingmodeliscomposedofthreeimportantcomponents:mentalstate-enhancedencoder,mixed strategylearningmodule,andmulti-factor- Affective Reward-based Affectively FrameworkThe Institute of Digital Games at UniversityofMaltaproposestheAffectivelyFramework,establi

8、shesanemotionalmodel,and adopts behavior reward andaffective reward mechanisms in the trainingprocess to help Agents better understandhuman emotions and interact with humansmore Sore points that need to be solved to improve user experienceImagine that an intelligent cockpit can not only understandan

9、d execute instructions given by the car owner,but alsopredict the owners needs,just like a thoughtful personalassistant.Willthismakecarownersmoreexcited?Compared to buying a traditional car and having to exploreeach function on ones own,everyone wants a cockpitendorser which can help manage all cock

10、pit functions asthey just say a few words.Agent is a time-saving andtrouble-free solution.Currently most Agents introduced in vehicles still serve asan assistant and a companion listing functions for specificscenarios.Yet compared with foundation models,Agentsfeaturegreaterpotential,motivatedautonom

11、y,andoutstanding tool-using capabilities,more fit with the label ofactiveintelligence,andcanevenmakeupforthelimitations of foundation models in practical Sore points that need to be solved to improve user experienceThere is however still a long way to go in technology development to make automotive

12、agents truly active and intelligent and meetusers experience value.Agent needs to be more precise in active perception,data processing,state recognition,etc.,accuratelyunderstand the environment,judge real needs of people in the car,and then adopt corresponding strategies.Wherein,one of challengesli

13、es in Agents accurate judgment of user needs.Compared with passive interaction in normal circumstances,active intention recognitionlacks voice commands.In the process of environment/personnel/vehicle state recognition,it may not be possible to obtain a descriptionthat is extremely close to the curre

14、nt scenario through vector feature matching,and the preset solution may not satisfy the real intentionsof people in the car.At present,most recommended functions are just to execute preset instructions.This limits active and intelligent capabilities of Agentand leads to frequent sore points in the r

15、easoning process.For example,if Agent fails to accurately understand the current scenario,itmay not make recommendations as expected,for instance,recommending music or navigation at a wrong time.The final result is toaffect user experience and make the Agent become a guessing machine to users.In add

16、ition,Agent also has shortcomings in perception when receiving voice commands.According to ResearchInChinas incompletestatistics on sore points in automotive agent use cases of some car owners,the most frequent sore points are wake-up failure,recognition error,and false wake- Sore Points in Using Au

17、tomotive Agents based on Incomplete Statistics Among the 120 cases,wake-up failure,recognition error,and false wake-up arementioned 19,18,and 17 times respectively,namely,accounting for 16%,15%,and 14%.Other sore points also include unavailability of see-and-speak,semantic clarification andcontinuou

18、s commands,inability to recognize dialects,and delayed response,totaling 89sore points in voice link,or 74.2%of the total in this statistical survey.Furthermore,a range of problems caused by unreasonable Agent architecture/scenariodesign also include irrational scenario triggering conditions,seconda

19、ry wake-up offoundation models,failure of long/short-term memory,and recommended actions madeautonomously according to owners habits but failing to meet expectations,whichrespectively reflect limitations of Agent in scenario setting,architecture deployment,memory module,and reflection module.In summ

20、ary,sore points of users are concentrated in the perception and reasoning links:Perception:wake-up failure,false wake-up,recognition error,unavailability of see-and-speak,delayed response,etc.Reasoning:object recognition error,autonomous recommendation failing to meet userexpectations, Quick-respons

21、e multi-agent frameworkTo enable all the functions of the endorser in cockpit,it is very critical to design the service framework of Agent indiversified scenarios.Agent framework is relatively flexible in construction.The simplest receiver+executerarchitecture can be used,or a more complex multi-age

22、nt architecture can be built.Its design principle is very simple:as long as it can solve user problems in a specific scenario,it is a good framework design.As a qualified cockpitendorser,automotive Agent not only needs to act as an independent thinker,make decisions and solve problemson its own,but

23、also quickly and freely adopts human behavior patterns,acting as a human.A typical example is NIO Nomi.It uses a multi-agent architecture,calling different tools in different scenarios,andusing multiple agents with different functions to perform specific duties and jointly complete the process ofund

24、erstanding needs,making decisions,executing tasks,and reflecting on iterations.The multi-agent architectureallows Nomi to not only make quick response,but also react more naturally like a human.Its seamless integrationwith other vehicle functions brings smoother experiences.Compared with single-agen

25、t systems,multi-agent systems are more suitable for executing complex instructions.They are like a small community in which each agent has its own tasks,but can cooperate to complete morecomplex tasks.For example,one agent is responsible for understanding your instructions,another is responsiblefor

26、making a decision,and there are special agents to perform tasks.This design makes automotive AI Agentsystems more flexible and allows them to handle more diverse tasks.For example,the Commonwealth Scientificand Industrial Research Organization(CSIRO)of Australia proposed a multi-agent system that us

27、es bothcollaboration agents and execution agents:The entire Agent framework is divided into 6 modules,namely,Understanding&Interaction,Reasoning,Tool Use,Multi-Agent Collaboration,Reflection,and Alignment.It embraces mainstream Agent design patterns,and coversthe entire process from active perceptio

28、n,reasoning and decision,tool calling to generation and execution,reflection and iteration,and alignment with human values.This framework features a multi-agent system wheredifferent Agents can play different roles(distribution/decision/actuation)in the entire process,making best use ofeach Agent to

29、 improve task execution efficiency.Inaddition,indiversifiedscenarios,Agent deployment methods and toolcalling capabilities also affect whether ornot user needs can be quickly andaccurately executed.Take NIO Nomi asan example:Nomi Agents are deployed at the endand cloud sides.End-side model andNomiGP

30、T are deployed at the end andcloudsides,respectively.Deeplyintegrated with SkyOS,the end-sidemodel can call atomic capabilities intimeandscheduleresources(data,vehicle control hardware/software,etc.)across domains to speed up response.NomiGPT on the cloud connects morecloud tool resource interfaces

31、to furtherenhanceNomiAgentscapabilityofcalling tools.Nomi Agents architectureis arranged in SkyOS middleware layer.Combining with SkyOS,it makes theprocessofcallingatomicAPIs,hardware/softwareanddatamorenatural,coordinated,and Table of Content(1)1 Overview of Automotive AI Agent1.1 Definition of Age

32、nt1.2 Development History of Agent1.3 Foundation Models Regain Vitality Using the Agent Concept1.4 Differences between Foundation Models,Agents,and AIGC1.5 Automotive AI Agent Product Definition1.6 Automotive AI Agent based on Multi-agent System:Module Design1.6 Automotive AI Agent based on Multi-ag

33、ent System:Component Functions1.6 Automotive AI Agent based on Multi-agent System:Component Characteristics(1)1.6 Automotive AI Agent based on Multi-agent System:Component Characteristics(2)1.6 Automotive AI Agent based on Multi-agent System:Component Characteristics(3)1.6 Automotive AI Agent based

34、on Multi-agent System:Component Characteristics(4)1.6 Automotive AI Agent based on Multi-agent System:Component Characteristics(5)1.6 Automotive AI Agent based on Multi-agent System:Component Characteristics(6)1.6 Automotive AI Agent based on Multi-agent System:Component Characteristics(7)1.6 Automo

35、tive AI Agent based on Multi-agent System:Component Characteristics(8)1.7 Automotive AI Agent Reference Architecture(by Functional Module andComponent)1.7 Automotive AI Agent ReferenceArchitecture(by Deployment Level)1.8 Agent Architecture Case(1):Original Diagram of NIO(Nomi)Architecture1.8 Agent A

36、rchitecture Case(1):Original Diagram of NIO(Nomi)Deployment1.8 Agent Architecture Case(1):NIO(Nomi)Module Design1.8 Agent Architecture Case(1):NIO(Nomi)Module Design-Multimodal Perception1.8 Agent Architecture Case(1):NIO(Nomi)Module Design-Command Distribution1.8AgentArchitectureCase(1):NIO(Nomi)Mo

37、duleDesign-ScenarioCustomization and Creation Process1.8 Agent Architecture Case(1):Highlights of NIO(Nomi)(1)1.8 Agent Architecture Case(1):Highlights of NIO(Nomi)(2)1.8 Agent Architecture Case(1):Highlights of NIO(Nomi)(3)1.8 Agent Architecture Case(2):Original Diagram of Li Auto(Lixiang Tongxue)A

38、rchitecture1.8 Agent Architecture Case(2):Li Auto(Lixiang Tongxue)Module Design1.8 Agent Architecture Case(2):Li Auto(Lixiang Tongxue)Supporting Facilities-Data/Training Platform1.8 Agent Architecture Case(2):Li Auto(Lixiang Tongxue)Supporting Facilities-Reasoning Engine1.8 Agent Architecture Case(3

39、):Original Diagram of Xiaomi(Xiaoai Tongxue)Architecture1.8 Agent Architecture Case(3):Xiaomi(Xiaoai Tongxue)Module Design1.8 Agent Architecture Case(4):Zeekr Agent Module Design1.8 Agent Architecture Case(5):Original Diagram of Neta Agent ArchitectureDeployment1.8 Agent Architecture Case(5):Neta Ag

40、ent Module Design1.8 Agent Architecture Case(6):Original Diagram of BAIC Agent ArchitectureDeployment1.8 Agent Architecture Case(6):BAIC Agent Module Design1.8 Agent Architecture Case(7):Huawei(Pangu Agent)Module Design1.8 Agent Architecture Case(8):Original Diagram of AISpeech Agent ArchitectureDep

41、loyment1.8 Agent Architecture Case(8):AISpeechAgent Module D Table of Content(2)1.8 Agent Architecture Case(9):Original Diagram of Lenovo Agent ArchitectureDeployment1.8 Agent Architecture Case(10):Original Diagram of Zhipu Agent ArchitectureDeployment1.8 Agent Architecture Case(10):Zhipu Agent Modu

42、le Design1.8 Agent Architecture Case(11):Original Diagram of Tinnove Agent ArchitectureDeployment1.8 Agent Architecture Case(11):Tinnove Agent Module Design1.9 Agent Architecture Design Process:Framework Selection1.9 Agent Architecture Design Process:Tool Calling Method1.10 Comparison of Automotive

43、AI Agent Architecture2 Key Issues in Development of Automotive AI Agent Products-User SorePoints and Technical Difficulties2.1 Classification of Automotive AI Agent Scenario:Typical Commands in DifferentScenarios2.1 Classification of Automotive AI Agent Scenario:Case(1)NIO2.1 Classification of Autom

44、otive AI Agent Scenario:Case(2)Li Auto2.1 Classification of Automotive AI Agent Scenario:Case(3)Xiaomi2.2 Automotive AI Agent Scenario Design Case(1)Q&A Scenario2.2 Automotive AI Agent Scenario Design Case(2)Q&A Scenario2.2 Automotive AI Agent Scenario Design Case(3)Mobility Scenario2.2 Automotive A

45、I Agent Scenario Design Case(4)Chat Scenario2.2 Automotive AI Agent Scenario Design Case(5)Chat Scenario2.2 Automotive AI Agent Scenario Design Case(6)Chat Scenario2.2 Automotive AI Agent Scenario Design Case(7)Q&A/Office Scenario2.3 User Sore Points in Different Agent Usage Scenarios:Summary2.4 Use

46、r Sore Points(1):Vehicle Control Scenario2.4 User Sore Points(2):Mobility Scenario2.4 User Sore Points(3):Q&An Scenario2.4 User Sore Points(4):Entertainment Scenario2.5 Agent Technical Difficulties2.6 Agent Technology Case:Intent Recognition(Case 1)2.6 Agent Technology Case:Intent Recognition(Case 2

47、)2.6 Agent Technology Case:Intent Recognition(Case 3)2.6 Agent Technology Case:Intent Recognition(Case 4)2.6 Agent Technology Case:ReasoningAcceleration(Case 1)2.6 Agent Technology Case:ReasoningAcceleration(Case 2)2.6 Agent Technology Case:ReasoningAcceleration(Case 3)2.6 Agent Technology Case:Stre

48、aming Voice(Case 1)2.6 Agent Technology Case:Streaming Voice(Case 2)2.6 Agent Technology Case:Streaming Voice(Case 3)2.6 Agent Technology Case:Emotional Interaction(Case 1)2.6 Agent Technology Case:Emotional Interaction(Case 2)2.6 Agent Technology Case:Emotional Interaction(Case 3)2.7 Agent Technolo

49、gy Trends(1):Two Keys to Achieving Active Intelligence2.7 Agent Technology Trends(2):2.7 Agent Technology Trends(3):Two Mainstream Design Methods for EmotionalAnthropomorphism3 OEMsAI Agent Investment,Development,and Operation3.1 Comparison of Automotive AI Agent Development Support3.2 OEMs Planning

50、 for Automotive AI Agents3.3ComparisonbetweenThreeAutomotiveAIAgentDevelopmentModes:Advantages/Disadvantages3.3 Comparison between Three Automotive AI Agent Development Modes:Cost3.4 Position Setting of OEMsAI Agent T Table of Content(3)3.4 Case of OEMs AI Agent Team Position Setting(1):Positions Re

51、cruited byChery AI Agent Team3.4 Case of OEMs AI Agent Team Position Setting(2):Positions Recruited byGeely AI Agent Team3.4 Case of OEMs AI Agent Team Position Setting(3):Positions Recruited by LiAuto AI Agent Team3.4 Case of OEMs AI Agent Team Position Setting(4):Positions Recruited by NIOAI Agent

52、 Team3.4 Case of OEMs AI Agent Team Position Setting(5):Positions Recruited byXiaomi AI Agent Team3.5 AI Agent Development Cycle and Operation Mode3.6 AI Agent Business:OEMs Profit Model3.6 AI Agent Business:Suppliers Profit Model3.6 AI Agent Business:Suppliers Charging Standards3.7 Commercial Devel

53、opment Trends of Automotive AI Agents(1)3.7 Commercial Development Trends of Automotive AI Agents(2)4 Automotive AI Agent Suppliers and Their Supply Relationships4.1 Cockpit Base Foundation Model:Model Configurations4.1 Cockpit Base Foundation Model:Selection Reference Factors4.2 Cockpit Base Founda

54、tion Model Suppliers(1)4.2 Cockpit Base Foundation Model Suppliers(2)4.2 Cockpit Base Foundation Model Suppliers(3)4.2 Cockpit Base Foundation Model Suppliers(4)4.2 Cockpit Base Foundation Model Suppliers(5)4.2 Cockpit Base Foundation Model Suppliers(6)4.2 Cockpit Base Foundation Model Suppliers(7)4

55、.2 Cockpit Base Foundation Model Suppliers(8)4.2 Cockpit Base Foundation Model Suppliers(9)4.2 Cockpit Base Foundation Model Suppliers(10)4.3 Industry Chain of Vector Database Suppliers4.4 Comparison between Vector Database Products:Chinese Vector Databases4.4 Comparison between Vector Database Prod

56、ucts:Foreign Vector Databases4.5 Vector Database Supplier Cases(1)4.5 Vector Database Supplier Cases(2)4.5 Vector Database Supplier Cases(3)4.5 Vector Database Supplier Cases(4)4.5 Vector Database Supplier Cases(5)4.5 Vector Database Supplier Cases(6)4.5 Vector Database Supplier Cases(7)4.5 Vector D

57、atabase Supplier Cases(8)4.6 Comparison between Voice ASR Module Suppliers4.7 ASR Module Supplier Cases(1)4.7 ASR Module Supplier Cases(2)4.7 ASR Module Supplier Cases(3)4.7 ASR Module Supplier Cases(4)4.7 ASR Module Supplier Cases(5)4.7 ASR Module Supplier Cases(6)4.7 ASR Module Supplier Cases(7)4.

58、7 ASR Module Supplier Cases(8)4.7 ASR Module Supplier Cases(9)4.8 Cockpit Data Collection Sensors:Mainstream Configurations/Data CollectionRegulations4.9 Sensor Data Processing Cases(1)4.9 Sensor Data Processing Cases(2)4.9 Sensor Data Processing Cases(3)4.9 Sensor Data Processing Cases(4) ContactBeijing HeadquartersTEL:13718845418Email:Chengdu BranchTEL:028-68738514FAX:028-86930659Website:ResearchInChinaWeChat:Zuosiqiche