利用生成式人工智能提高赫伯羅特的審計效率.pdf

《利用生成式人工智能提高赫伯羅特的審計效率.pdf》由會員分享，可在線閱讀，更多相關《利用生成式人工智能提高赫伯羅特的審計效率.pdf（31頁珍藏版）》請在三個皮匠報告上搜索。

1、2024 Databricks Inc.All rights reservedEnhancing Audit Enhancing Audit Efficiency at Efficiency at HapagHapag-Lloyd with Lloyd with Generative AIGenerative AIUlrich DanielUlrich DanielMichael ShtelmaMichael ShtelmaTania SennikovaTania Sennikova12024 Databricks Inc.All rights reservedUlrich DanielDir

2、ector of Corporate Audit Analytics at Hapag-Lloyd AGPresentersMichael Shtelma Lead Specialist Solutions Architect at DatabricksTania SennikovaSr.Solutions Architect at Databricks2024 Databricks Inc.All rights reservedAgenda1.Hapag-Lloyd company overview,challenges,and scale 2.Optimizing corporate au

3、dit at Hapag-Lloyd3.Generating findings&executive summary4.Chatbot for process documentation5.Whats next2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved5Optimizing Optimizing Corporate AuditCorporate Audit2024 Databricks Inc.All ri

4、ghts reserved2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved8Generating Generating findings&exec findings&exec summary summary 2024 Databricks Inc.All rights reservedProject stages9Deployment&application integrationModel deploymen

5、tChat interface developmentDefine ProblemDefine the business problemCreate evaluation datasetDefine metricsModel Adaptation&EvaluationData preparationChoose a base modelPrompt engineering Evaluating results2024 Databricks Inc.All rights reservedHigh Level Architecture10UsersDescribe Observations in

6、a documentScheduledPipelineFindings Generation ModelGenerate FindingsLLMSummary Writing ModelLLMStore in a Delta TableFew shot promptingQueryExample 1Example 2User supplied examplesStore Findings in a Table2024 Databricks Inc.All rights reservedModelingStarted MPT 30bFirst version used Llama 2 70bAf

7、ter that we have switched to MixtralNow using DBRX11Models we have tried2024 Databricks Inc.All rights reservedModelingDBRX is Databricks very own open source LLM DBRX is a transformer-based decoder-only LLM that was trained using next-token predictionDBRX was pretrained on publicly available online

8、 data sources It was trained on 12T tokens of carefully curated data and a maximum context length of 32k tokensDBRX Architecture:Fine-grained sparse mixture-of-experts(MoE)model architecture132B parameters and supports context up to 32K tokens While the model has 132B total parameters,only 36B of th

9、em are used for any given input when training,fine-tuning,or performing inference on the model12Introducing DBRX:2024 Databricks Inc.All rights reservedEvaluationWe have started with batch generation of the findingsDomain experts receive a big CSV with input and output.In our case with bullet points

10、 and recreated findings This approach is very time-consumingAt first we had no bullet points:We have generated them using an LLMNow moving to an automated approachLLM as a Judge uses another big LLM to evaluate the resultsSupported in MLflow:mlflow.evaluateWe can define custom metrics:we need to pro

11、vide several examples and prompts13Our evaluation journey2024 Databricks Inc.All rights reservedDeployment&Application integrationUsing Databricks Model ServingDeployed LLM using Databricks Foundational Model API Provisioned Throughput endpoints(GPU)The chain is deployed using classical Databricks M

12、odel Serving on CPU endpointWe are still using Gradio as a chat interface142024 Databricks Inc.All rights reservedAI AppApplication architecture:prompt engineeringUsersCreate Prompts2Send promptsResponse41Model Serving3QueryChoose and serve LLMTemplatesInstructionsExamplesPromptsCustom Models(CPU/GP

13、U)Web AppMonitoringLakehouse MonitoringInference TablesLog query,response,metrics5Foundation ModelsExternal ModelsModel HubModels in Unity CatalogHuggingFace HubModels in Marketplace2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedHere we are-Running our prototype 2.0202

14、4 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved18Chatbot for Chatbot for process process documentationdocumentation2024 Databricks Inc.All rights reservedProject definitionAuditors spend long time looking for a very specific pieces of information in different files:This c

15、an be manualsDifferent presentations,documents,etcThey need a simple querying interface supporting natural language queries that allow them to ask for the specific facts defined in the documentationIt should be possible to add new documents in runtime192024 Databricks Inc.All rights reservedRAG uses

16、 LLMs as reasoning engines,rather than as static models.Your data+an LLM“brain”Retrieval Augmented Generation(RAG)UsersQueryRAG chain“What is Spark Connect?”2Vector Database or Feature StoreRetrieverelevant info/data(context)“The Spark Connect client translates DataFrame”3PromptwithcontextAugmentpro

17、mpt with contextRespond to Qbased on D:Relevant docsQuestionInstruction-following LLM4Generateanswer from context“Spark Connect allows a decoupled client-server”2024 Databricks Inc.All rights reservedProject stages21Deployment&application integrationData pipelines deploymentModel deploymentChat inte

18、rface developmentDefine ProblemDefine the business problemCreate evaluation datasetDefine metricsData preparation&ModelingData preparationParsing filesChunking dataCalculating embeddingsIngesting into Vector DBChoose a base modelPrompt engineering Evaluating resultsRetrieval evaluationOverall evalua

19、tion2024 Databricks Inc.All rights reservedHigh-level chatbot architecture22UsersModel ServingMixtral 8x7BUser writes a query in Gradio UISends generated prompt containing relevant chunksQueries Vector Search to find relevant chunks of documentsData VolumeContexttablesProcesses files and ingests the

20、m into Vector Search IndexGenerates responseMosaic AI Vector SearchNotebook2024 Databricks Inc.All rights reserved Ingestion pipelines managed for you Indexes managed by Unity Catalog Also,APIs for Self-managed embeddings CRUD API upsert/deleteModel Serving Foundation Model API Custom model External

21、 modelVector SearchCreate auto-updating vector indexes,managed by Unity CatalogChoose your source tableCreate semantic search index via Unity Catalog UI or via APICall endpoint for real-time retrievalresult=index.similarity_search(query_text=What is Spark Connect?,columns=id,text,link,filters=doctyp

22、e:wiki)Choose any embedding model Integrate with LangChain,LlamaIndex,etc.Scale out endpoints as neededDocumentation:AWS,Azure2024 Databricks Inc.All rights reservedChains(and agents)Building pipelines to include context and complex reasoningVector Search lookupPrompt templateLLM generates responseE

23、xample(RAG)chainChains and agents can string together modular LLM features in a structured way,such as for RAG chains.Common frameworks include:LangChainLlamaIndexHugging Facemlflow.langchain.log_model(lc_model=llm_chain,)MLflow supports tracking and logging chains,agents,and models.Models can be re

24、gistered in the Unity Catalog for governance and lineage tracking.Built-in MLflow flavors include:LangChainOpenAITransformersSentence TransformersPyFunc(for any custom framework)DevelopmentDeployment and Tracking2024 Databricks Inc.All rights reservedApplication architecture:RAGVector SearchETLPrepa

25、re docs(cleanse,chunk,)IngestdocsFilesEmbeddingModelAutomatically sync with Delta tableChoose and load model(s)Compute embeddingsModel HubHuggingFace+Unity CatalogPreparation2024 Databricks Inc.All rights reservedAI ApplicationApplication architecture:RAGUsersConstruct PromptsSend prompts to LLM to

26、generate responseResponse61Instruction Following Model5QueryPromptsModel Serving3Search for related contentVector SearchRAG ChainQuery RAG modelMonitoringModel Serving4Prompt construction and execution22024 Databricks Inc.All rights reservedAI ApplicationApplication architecture:RAGUsersConstruct Pr

27、omptsSend prompts to LLM to generate responseResponse61Instruction Following Model5QueryPromptsModel Serving3Search for related contentVector SearchETLPrepare docs(cleanse,chunk,)IngestdocsFilesRAG ChainEmbeddingModelAutomatically sync with Delta tableChoose and load model(s)Compute embeddingsQuery

28、RAG modelMonitoringModel HubHuggingFace+Unity CatalogModel Serving422024 Databricks Inc.All rights reservedHere we are-Running Chat prototype2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved29Whats nextWhats next2024 Databricks Inc.All rights reservedNext Steps 301.The c

29、urrent solution is being tested by auditors of Hapag-Lloyd.There are plans to extend this solution and fine-tune a language model to help the Audit department better structure their reports.2.Improve and automate evaluation using Mosaic AI Agent Evaluation framework3.Various departments are increasi

30、ngly recognizing the value of Generative AI for business.They are exploring proper implementations for multiple use cases,including but not limited to chatbots,summarizing large documents,and providing code assistance.2024 Databricks Inc.All rights reservedThank you!Thank you!Questions?Questions?Author NameAuthor NameDateDate31