賦能可能性：利用大型語言模型實現實用創新.pdf

《賦能可能性：利用大型語言模型實現實用創新.pdf》由會員分享，可在線閱讀，更多相關《賦能可能性：利用大型語言模型實現實用創新.pdf（50頁珍藏版）》請在三個皮匠報告上搜索。

1、THE CHALLENGES OF CREATING A DIGITAL ENGINEERING ASSISTANT USING LLMSEPRI AI AND DX ELECTRIC POWER SUMMITDR MICHAEL DOIGDIGITAL SPECIALIST,ROLLS-ROYCE PLCDERBY,UKThe information in this document is proprietary and confidential to Rolls-Royce and is available to authorised recipients only copying and

2、 onward distribution is prohibited other than for the purpose for which it was made available.Rolls-Royce content only.2024 Rolls-RoyceNot Subject to Export ControlSMRPower Generation&Battery StorageMarineGovernmentalWidebodyTransportCombatSubmarinesMicro-ReactorsROLLS-ROYCE BUSINESS AREAS2POWER SYS

3、TEMSDEFENCE/NUCLEARCIVIL AEROSPACEBusiness Aviation Crown Copyright 2024 Rolls-RoyceNot Subject to Export ControlROLLS-ROYCE SUBMARINES3For the last 60 years we have designed,supplied and supported the nuclear propulsion plant that provides power for all of the UK Royal Navys nuclear submarinesLong

4、product lifecycle:design,manufacture,in-service,disposal 2024 Rolls-RoyceNot Subject to Export ControlWe power the UK Royal Navys nuclear submarine fleet 2024 Rolls-RoyceNot Subject to Export Control4To create a Digital Engineering Assistant,powered by LLMs,to enable our engineers to find the inform

5、ation they need more easily.Chatbot with access to all our technical reportsReference back to source materialGive engineers easy access to the data they needUnlock the hidden value within reports Huge amounts of technical data trapped in reports Difficult to find the information when required Challe

6、nging to new engineers Information in range of different formatsLEGACY DATAEXAMPLESCREATE A DIGITAL ENGINEERING ASSISTANTRELEASE THE DATA AND VALUE TRAPPED IN OUR TECHNICAL REPORTSDELIVER BUSINESS BENEFITS BY MAKING DATA MORE ACCESSIBLE TO OUR ENGINEERSOUR INITIAL AIM 2024 Rolls-RoyceNot Subject to

7、Export ControlCHALLENGES 2024 Rolls-RoyceNot Subject to Export Control5Sensitive dataCloud cannot be usedData qualityHardware/Infrastructure needed 2024 Rolls-RoyceNot Subject to Export ControlRolls-Royce Quality Assurance for AI(QA4AI)6AI ETHICS,SAFETY AND SECURITYAletheia FrameworkTM 2024 Rolls-Ro

8、yceNot Subject to Export ControlFramework for safely using AI in safety-critical systemsHigh-Risk Management PlanningEthical Framework(Aletheia)System Design andExplainabilityQuality PlanningAI/ML AssuranceData QualityManagement/Data EngineeringAssurance Frameworks 2024 Rolls-RoyceNot Subject to Exp

9、ort ControlCreating a Digital Engineering AssistantRETRIEVAL AUGMENTED GENERATION 2024 Rolls-RoyceNot Subject to Export Control7Retrieval augmented generation(RAG)is a framework that combines information retrieval systems with the capabilities of large language models(LLMs).This process allows LLMs

10、to answer queries on specific topics,like business data,using added context.Compute 2024 Rolls-RoyceNot Subject to Export ControlCreating a Digital Engineering AssistantRAG:TECHNOLOGY STACK 2024 Rolls-RoyceNot Subject to Export Control8Vector DatabaseComputeLLMsFrameworks 2024 Rolls-RoyceNot Subject

11、 to Export ControlDefect ReportsRAG:CHATBOT DEMONSTRATION:2024 Rolls-RoyceNot Subject to Export Control9 2024 Rolls-RoyceNot Subject to Export ControlPOSITIVESChatbot performed well for 100 large documents Content easily extracted from reportsProvided reference back to original source materialWorked

12、 on-premise on local hardwareNEGATIVESPoor performance as number of reports increasedIssues with retrievals not returning the most relevant information Some hallucinations in resultsOur focus on Sustainability10We focus on areas where we can make the most material contributions to a sustainable futu

13、re,informed by our impacts asa business,supported by global frameworks such as the United Nations Sustainable Development Goals,and the expectations of our all stakeholders.2023 Rolls-RoyceINITIAL RESULTSPromising individual use cases but not going to be able to apply to deploy at scale across all o

14、ur reportsIMPROVEMENTSPre-filtering reports,to limit the dataset used for retrievalEnrich reports meta-dataSummarisation of reports rather than full textMulti-stage retrievalBespoke chatbots 2024 Rolls-RoyceNot Subject to Export ControlScale back our initial bold ambition and focus on specific use c

15、asesThe information in this document is proprietary and confidential to Rolls-Royce and is available to authorised recipients only copying and onward distribution is prohibited other than for the purpose for which it was made available.Rolls Royce content only.11NEXT STEPSDEFECT REPORTS Stage:Testin

16、gAccess to all our historic defect report to help identify potential solutions based on previous precedentsCODING CO-PILOTStage:Completed Deploy on-premise coding copilot to accelerate software developmentGENERAL AI ASSISTANTINFRASTRUCTUREStage:DesignGeneral AI Assistant to help increase productivit

17、y,reduce non-value add effort&save time/money.Stage:Future Expand our GPU capability,software development and COTS software to accelerate deployment and delivery 2024 Rolls-RoyceNot Subject to Export ControlFINALLY,WHEN LLMS GO WRONG!12Summarise this Safety Alert documentThis Safety Alert Review(SAR

18、)investigation was initiated to address concerns regarding.The root cause has been assigned based on non-compliance with procedures and incorrect installation or operation practices Create a detailed account of an alternate history where Ada Lovelace becomes Queen,leading to a steampunk era driven b

19、y her mathematical genius.Your narrative should unfold through notable figures such as Charles Babbage,Mary Shelly and Nikola Tesla 2024 Rolls-RoyceNot Subject to Export ControlENDApplication of LLM in CNNPNuclear Power Research Institute,Shanghai China National Nuclear Power Co.,Ltd2025-01-07目 錄CON

20、TENT1LLM and Nuclear Power2Development of Nuclear LLM3Application of Nuclear LLM1.0 LLM and Nuclear PowernSince December 2022,AI Large Language Models have become a hot topic in the high-tech field.nScaling Laws：The performance of neural networks improves as the number of parameters increases.When t

21、he parameters exceed 100 billions,advanced capabilities emerge,such as reasoning and context understanding,zero-shot learning and chain-of-thought,leading to what is known as large language models.nThe latest research findings indicate that in certain specialized fields,LLM have achieved performance

22、 on par with doctoral students.Brain Neural NetworkAI Parameter NetworkA timeline of existing large language models1.0 LLM and Nuclear PowerPre-trainingFine-TuningDomain KnowledgeGPU(1000)GPU(32)Foundation LLMDomain LLMnBased on the LLMs developed by general-purposeAI company,integrating industry-sp

23、ecific knowledge data and making minor adjustments to the model parameters(Fine-Tuning training)can result in domain large language models,which are good at the industry applications.from foundational LLM to domain LLM,less GPU required.Number of model parameters：10B/38B/72B/100B/2000B，more paramete

24、r requires more GPU and more data,generally more powerful。Foundation LLM(L0)Internet text,general knowledgefinancemedicalLawDomain LLM(L1)Nuclear Power華知大模型阿里華為商湯智譜Domain dataApplication(L2)Application-specific dataelectricpower1.0 LLM and Nuclear PowerExperience FeedbackProcedure Generatio nCause A

25、nalysisKnowledgeManagementTraining GuidanceSolution ProposalLLMA Transformative Production ToolnCNNP-NPRI,based on the GLM4-130B model,has developed the“Longling LLM which can be used in nuclear power knowledge Q&A and equipment fault diagnosis.Now it is collaborating with Fuqing NPP to develop more

26、 applications for nuclear power plant operation and maintenance.nCGN-Ningde NPP,based on open-source large model Qwen-2.5-72B,has trained the Jinshu LLM which is now used in training,and also isexpanding the application of this large model in nuclear power O&M.Potential Applications in NPP目 錄CONTENT

27、1LLM and Nuclear Power2Development of Nuclear LLM3Application of Nuclear LLM2.0 Development of Nuclear LLMSFTRLHFFoundationLLMNuclear O&M DocumentsSubjectExpertsNuclear O&M Domain LLMQ&AText Generation Solution Proposal etc.APP.Training ProcessInjection of knowledge in NPP O&MAligning with human val

28、ues,such as nuclear safety culture,etcnSFT(Supervised Fine-Tuning)and RLHF(Reinforcement Learning from Human Feedback)are two critical steps in domain LLMs training.nSFT requires high-quality datasets,meaning that documents specific to the nuclear field must be converted into training corpus.nRLHF n

29、eeds subject experts in nuclear power O&M to align the model with human,by rating the answers generated by the LLM.GPUKey FactorsWith a knowledge and documents accumulation from 250+reactor-years,CNNP now is preparing a great wealth of training corpus.NPRI is a company composed of digital technology

30、 expert and O&M subject experts.n Foundation LLM is critical,as its capability ina large extent determines the performance ofthe domain LLM;n An industrial knowledge evaluation datasetwas specifically developed to select thefoundation model.2.0 Development of Nuclear LLMn Full-parameter fine-tuning

31、a 72Bmodel requires at least 32 A100-levelGPUs.n The stronger the computing power,the faster the training speed,and the model with more parameters can be trained.Subject ExpertsFoundationLLMO&MCorpus2.0 Development of Nuclear LLMIndustrial knowledge evaluation datasetEvaluation Datasetevaluation que

32、stionsLLM 1（public resources）（random）LLM 2LLM nAnswer 1Answer nScore 1Score 2Score n Answer 2 LLM selectionn Many choices available,open-source or closed-source;n An industrial knowledge evaluation dataset wasspecifically developed to select the foundation model;n This dataset was collected from pub

33、lic resources,including online data and textbooks,andencompasses knowledge in areas such as mechanical,electrical,instrumentation and control,thermo,hydraulics,nuclear engineering,etc.n Using this dataset to evaluate multiple foundation models,the model with the highest score will be selected.阿里華為Ch

34、atGLM-4智譜商湯Foundation Model2.0 Development of Nuclear LLMn NPP O&M evaluation dataset was specificallydeveloped to qualify the domain LLM;n This dataset covers subjects such as operations,maintenance,equipment management,radiationprotection,nuclear safety,and more.n Each subject expert will score th

35、e models answers,and the final overall score will be calculated.n If the overall score meets the acceptance requirements,the model is considered qualified.NPP O&M evaluation datasetDomain LLM acceptanceO&M evaluation datasetEvaluation questions（random）Expert 1Expert 2Expert n NPP O&M LLMTest answers

36、Overall score（NPP Documents）NPP O&M Domain LLM目 錄CONTENT1LLM and Nuclear Power2Development of Nuclear LLM3Application of Nuclear LLM3.0 Application of Nuclear LLMSolution ProposalDocument WritingFault Diagnosisn The time spent on writing a document is 2-5 times that of the thinking.n LLM assist in o

37、rganizing language,allowing engineers to focus on fact rather than formality“.Solution ProposalDocument WritingFault Diagnosis3.0 Application of Nuclear LLM3.0 Application of Nuclear LLMn Save time on data retrieval and document preparation;n Focus on key and high-output activities.send working orde

38、r to LLMpropose solution with referencesmodify the solution after deeply thoughtwrite procedure in the correct formatWorking Orderprocedure confirmationand submissionSolution ProposalDocument WritingFault DiagnosisInteraction between LLM and humanVoltWrite-Artificial IntelligenceNathaniel Melby,Ph.D

39、.Vice President&Chief Information OfficerVladimir Tsoy,MBAIT Enterprise Solutions ArchitectPresented to:Stanford/EPRI Artificial Intelligence and Digital Transformation SummitJanuary 7-9,202529People FirstGrowth&InnovationAbout Usu24 Class A Membersu27 Municipal Customersu552 MW Owned generation cap

40、acity(coal)u577 MW Owned generation capacity(natural gas/oil)u3,312.47 Miles of transmission line30Dairyland System OnlyUse CaseuThe John P.Madgett Station(JPM)u387 MW single-unit generating station located in Alma,WI.In operation since November 1979uVast volume of DocumentationuOver Four Decades of

41、 RecordsuDifficulty in Information RetrievaluInefficient Search ProcessesuChallenges During Scheduled Maintenance WindowsuFinite DowntimeuDisparate Documentation SystemsuKnowledge Silos and Loss of Institutional Knowledge31Dairyland System OnlyTimeline When it began,how its evolving32uEnhanced Opera

42、tional EfficiencyuImproved Training Material DevelopmentuAccelerated Maintenance and Technical SupportuIncreased Productivity and Reduced DowntimeArchitecture33Dairyland System OnlyVoltGen CapabilitiesuExamples of capabilitiesuChat with your filesuExtensionsuIntegration with internal APIs and extern

43、al sourcesuMulti-modality34Dairyland System OnlyExample Case35Dairyland System OnlyuLeveraging Multi-Modal Capabilities for Part IdentificationuCombines visual recognition with textual information retrievalWhat comes next?uExpansion of VoltGen to other generation sitesuHydroelectric DamuNatural Gas

44、PlantuCombustion Turbine PlantuContinue to improve/grow datauDocument and discover additional benefits36Dairyland System Only 2024 Electric Power Research Institute,Inc.All rights Wynter McGruderSr.Principal Technical Leader AI and Digital Transformation in Electric Power SummitJanuary 2025Enhancing

45、 Compliance with Generative AI toolsLeveraging AI for Knowledge Transfer and Regulatory Submittals 2024 Electric Power Research Institute,Inc.All rights reserved.39EPRI is committed to research and investigation into practical applications of AI for the energy sector.We have created a prototype tool

46、 that uses generative AI to accelerate the research process create preliminary drafts of regulatory documents.Our research demonstrates how AI can assist engineers,make knowledge sharing easier,and help accelerate the energy sectors adoption of AI.Introduction 2024 Electric Power Research Institute,

47、Inc.All rights reserved.40Industry Challenges:Workforce Development/Knowledge TransferEngineers and technical staff with decades of experience have retired and many more will be retiring soon.Utilities face the challenge of dedicating extra time and resources to transfer knowledge between incoming a

48、nd outgoing staff.There is no current way to capture and store the decades of valuable insights from experienced technical staff leaving the industry.The industry needs an innovative way to transfer knowledge to the next generation of nuclear workers.2024 Electric Power Research Institute,Inc.All ri

49、ghts reserved.41AI Tool Capabilities to Address Industry Challenges AI as Technical Support to Knowledgeable,Technical Staff AI tools can capture the large datasets that exist from nuclear plant records,regulatory submittals and correspondence,and industry guidance documents.Datasets can be analyzed

50、 to extract critical technical information and lessons learned which can be seamlessly integrated into plant-specific documents and guidance.Generative AI provides additional capabilities to assist technical staff in identifying underlying precedents and technical bases from existing information.New

51、 documents(internal reports and evaluations as well as regulatory submittals)can be created by generative AI using the established precedents and technical basesThe use of these AI tools can streamline the process of identifying,evaluating,extrapolating,and integrating information from existing data

52、sets while maintaining human expertise and judgement in the process.Early career nuclear workers can generate technical products at the level of rigor and quality of more experienced workers while also gaining valuable knowledge transfer by overseeing the process of using these AI tools.2024 Electri

53、c Power Research Institute,Inc.All rights reserved.42Use Case:AI-Assisted Relief Request Analysis and Development(1/2)BackgroundRelief requests are formal submissions to the NRC,allowing licensees to request relief from ASME Boiler and Pressure Vessel Code requirements.Licensees submit a request for

54、 relief from the impracticality of the requirement along with a proposed alternative.The NRC staff reviews each submission to assess whether the proposed alternative provides a commensurate level of safety.Licensees may need to respond to questions and provide supplemental information to the NRC bas

55、ed on a document called“Request for Additional Information.”Challenges Each relief request is unique requiring understanding of specific 10 CFR(Code of Federal Regulation)requirements,the ASME B&PV Code requirements,industry operating experience,the nuance of defining the proposed alternative and it

56、s technical basis,and plant-specific licensing commitments and precedents as well as industry precedents and available/applicable code cases.Relief requests are relatively infrequently performed evolutions limiting the ability use gain expertise through consistent implementation of the processRelief

57、 requests,in several cases,are developed on an emergent basis to address challenges with meeting ASME B&PV requirements.2024 Electric Power Research Institute,Inc.All rights reserved.43Use Case:AI-Assisted Relief Request Analysis and Development(2/2)Relief Request Dataset Availability and CurationTw

58、enty-four(24)years of relief request submissions,related regulatory correspondence,and related regulatory safety evaluations are publicly available on the NRC ADAMs database.These records reflect decades of operating experience and subject matter expertise suitable for use to develop a proof of conc

59、ept AI tool.Relief requests for two ASME Code Case types were digitized,standardized and structured to be processed by AI systems.Natural language processing techniques were applied to extract key insights and terminology.The curated dataset was used as the foundation for applying generative AI mode

60、ls and prompt engineering to support knowledge transfer and regulatory drafting.2024 Electric Power Research Institute,Inc.All rights reserved.44AI-Assisted Proof of Concept Web Tool InterfaceThe POC uses generative AI to digest and iteratively query relief request documents.Users can easily locate

61、precedent submissions to serve as a model.There is also deep links to relevant relief requests,NRC correspondence,and the final safety evaluation letters for context.Engineers can input full-text queries and the tool returns a set of relevant documents for review.Engineers select specific documents

62、as a basis for a draft,then the tool generates draft text for specific sections of a new submission.2024 Electric Power Research Institute,Inc.All rights reserved.45AI POC Web Tool:Revolutionizing Knowledge Transfer and Regulatory Submittal DevelopmentEngineers gain on-demand insights from the entir

63、e library of regulatory documents.By enhancing curation and contextual search,the tool helps to reduce time to prepare regulatory submissions.When curated and centralized into a digital format,with robust AI tools for contextual retrieval,this information becomes shared knowledge accessible to engin

64、eers across the industry.Centralized digital libraries foster collaboration across the industry and sharing expertise widely,and AI tools make this knowledge more actionable.Digital learning is now integral part of modern workplaces,new engineers enter the workforce equipped to leverage AI-driven to

65、ols and technologies.Solves knowledge transfer challenge by capturing,preserving,and sharing expertise from experienced engineers,making the information accessible to future generations.2024 Electric Power Research Institute,Inc.All rights reserved.46Live Demonstration 2024 Electric Power Research I

66、nstitute,Inc.All rights reserved.47Future Enhancements/DevelopmentsExplore expanding POC web tool capabilities to:Generate entire relief requests(all sections)Use generative AI insights to extract,analyze,and index RAI questions and responses.Expand to broader population of Code Case based relief re

67、questsIndexing technical drawings in relief request submittalsCurate and index more complex regulatory submittals(i.e.10-year ISI updates,license renewal submittals,etc.)-Significant undertaking that may require post-member access phaseDirect member access for use of the tool in parallel with enhanc

68、ements is being consideredPhase 2 Web Tool Development Proposals 2024 Electric Power Research Institute,Inc.All rights reserved.48Questions?2024 Electric Power Research Institute,Inc.All rights reserved.49For more information,visit EPRI.com Program on Technology Innovation:Artificial Intelligence Assisted Relief Request DevelopmentProduct ID 3002029365 2024 Electric Power Research Institute,Inc.All rights reserved.50 2024 Electric Power Research Institute,Inc.All rights TOGETHERSHAPING THE FUTURE OF ENERGY