3134 Watson Assistant Retrieval Augmented Generation using Elasticsearch DBaaS.pdf

1、October 21-24,2024Mandalay Bay Convention CenterLas Vegas,NevadaSession Session 31343134Daniel PittnerIBM,STSM,Cloud Platform,Cloud Database ServicesWatsonx Assistant Retrieval Augmented Generation using Elasticsearch DBaaSAgenda0102030405RAG 101Watsonx Assistant&Search(Elastic)Search IntegrationFil

2、ter and Query configurationDeploying models to ElasticIBM TechXchange|2024 IBM Corporation#IBMTechXchange3What you will learn in this sessionIBM TechXchange|2024 IBM Corporation0102030405What is Retrieval Augmented Generation?Watsonx Assistant Search IntegrationWhat parts of RAG run directly in Elas

3、ticSearch methods in ElasticDeploying custom models to Elastic#IBMTechXchange4IBM TechXchange|2024 IBM Corporation“Can you build something like ChatGPT,but with access to my calendar,mail”My Mom,asking for something that could be done with RAGAgenda0102030405RAG 101Watsonx Assistant&Search(Elastic)S

4、earch IntegrationFilter and Query configurationDeploying models to ElasticIBM TechXchange|2024 IBM Corporation#IBMTechXchange6Retrieval Augmented Generation 101Question?AnswerLoadSplitPromptLLM(Granite)Vectorstore(Elastic)IBM TechXchange|2024 IBM CorporationEmbed(ELSER)#IBMTechXchangeRAG PipelineDat

5、a PreparationLoading and chunkingdocuments using themost effective strategyfor a business domainGenerate EmbeddingsTransforming chunkschunks and questionsquestions into vectorsSearch and RankingVarious search methodssearch methods(kNN,aNN)ReRankReRank for improved relevance7IBM TechXchange|2024 IBM

6、Corporation#IBMTechXchange8RAG with Watsonx Assistant&ElasticQuestion?AnswerLoadSplitPromptLLM(Granite)Vectorstore(Elastic)IBM TechXchange|2024 IBM CorporationEmbed(ELSER)Agenda0102030405RAG 101Watsonx Assistant&Search(Elastic)Search IntegrationFilter and Query configurationDeploying models to Elast

7、icIBM TechXchange|2024 IBM Corporation#IBMTechXchangeOmnichannelVirtual Agent(build once,deployacross voice,chat)Dialogs,train/testconnectorsOmnichannel Virtual AgentsAI Agent AssistOmnichannelHuman AgentMultipleAppsIncrease Agent Efficiency and Effectiveness DataAPIsKnowledgeArticlesUserJourneyRepo

8、rtingAnalyticsImprove Contact Center OperationsCustomerWatsonx AssistantDeliver consistent and intelligent customer careIBM TechXchange|2024 IBM CorporationIntegration with Watson Assistant for Conversational SearchIBM Cloud Databases for ElasticsearchWatson AssistantWatson Discovery,Elastic,Custom#

9、IBMTechXchangeConversational search offoffConversational search onon12Watsonx Assistant Search IBM TechXchange|2024 IBM CorporationAgenda0102030405RAG 101Watson Assistant&Conversational Search(Elastic)Search IntegrationFilter and Query configurationDeploying models to ElasticIBM TechXchange|2024 IBM

10、 CorporationSetting up Elasticsearch integrationConnect IBM Cloud Database Elasticsearch Platinum instance using a unique service credentialEnsure minimum of 16 GB RAM per node are metIBM TechXchange|2024 IBM CorporationSetting up Elasticsearch integrationChoosing an index strategy:Uploading documen

11、ts Uploading documents easiest to get started,just upload documents,Watsonx assistant uses ELSER to indexUse my index Use my index fine grained control over all aspects of indexing,custom chunking,custom model(e.g.german language support),custom index settingsIBM TechXchange|2024 IBM CorporationSett

12、ing up Elasticsearch integrationUploading documents Uploading documents upload English pdf documents to add a knowledge base to your assistantIBM TechXchange|2024 IBM CorporationSetting up Elasticsearch integrationIBM TechXchange|2024 IBM CorporationAgenda0102030405RAG 101Watson Assistant&Conversati

13、onal Search(Elastic)Search IntegrationFilter and Query configurationDeploying models to ElasticIBM TechXchange|2024 IBM CorporationFilter and Query ConfigurationFilterFilter Filter results from Elastic globally or with action specific context using Elastic Search DSLQueryQuery Define how your knowle

14、dge base will be searched by watsonx assistant,defaults to ELSERELSERKNN dense vectorHybridwith optional nestingIBM TechXchange|2024 IBM CorporationSparse Vectors ELSER model ELSER generates a collection of pairs(token,weights)for a better semantic representation of the given text IBM TechXchange|20

15、24 IBM CorporationELSER(Elastic Learned Sparse EncodeR)is a retrieval model trained by Elastic that enables semantic search to retrieve more relevant search resultsBoth sentences are not similar if encoded using dense vectors,as semantics are not captured How ELSER model works-Text Expansion Dog Dog

16、 chased chased t the he squirrsquirrelelCat Cat dislikes dislikes the the mousemouseIBM TechXchange|2024 IBM CorporationELSER expands the meaning of your domain to terms it“understands”Example:“Latitude”could have two meanings:geospatial but also could be a laptopA dense vector representation can no

17、t capture semanticsHow ELSER model works-Text Expansion Dog Dog chased chased t the he squirrsquirrelelCat Cat dislikes dislikes the the mousemousecaninepursued pet treeacornrodentExpandsExpandsExpandspetfelinerejectsdespisesrodentratExpandsExpandsExpandsIBM TechXchange|2024 IBM CorporationSimilarit

18、y algorithm matches only on the expansionexpansion How ELSER model works-Text Expansion+Weights Dog Dog chased chased t the he squirrsquirrelelCat Cat dislikes dislikes the the mousemousecaninepursued pet treeacornrodentExpandsExpandsExpandspetfelinerejectsdespisesrodentratExpandsExpandsExpands.9.5.

19、7.6.4.6.7.7.4.6.8.9IBM TechXchange|2024 IBM CorporationScore is calculated from matched termsHow ELSER model works-Text Expansion+Weights Dog Dog chased chased t the he squirrsquirrelelCat Cat dislikes dislikes the the mousemousecaninepursued pet treeacornrodentExpandsExpandsExpandspetfelinerejectsd

20、espisesrodentratExpandsExpandsExpands.9.5.7.6.4.6.7.7.4.6.8.9(0.5*0.7)(0.5*0.7)+(0.6*0.8)+(0.6*0.8)-Score=.83Score=.83 IBM TechXchange|2024 IBM CorporationDense Vectors 4 Authors with different perspectives on Mona LisaDense Vectors are a type of mathematical objects that represent dataKnown also as

21、“vector embeddings”Different models create different dense vectors.78,.35,.23 .45,.83,.24 45,.23,.24,.82,.32,.94,IBM TechXchange|2024 IBM CorporationkNN on dense vectorsK K-nearest neighbor search will find the nearest neighbor search will find the closestclosest k k results in an n dimensional vect

22、or spaceresults in an n dimensional vector spaceResults are sorted by their Results are sorted by their distancedistance from search from search vectorvectorThere are multiple ways to define distanceThere are multiple ways to define distanceEuclideanEuclideanDotDot-productproductCosineCosineMax inne

23、r ProductMax inner Product IBM TechXchange|2024 IBM CorporationdK=3Hybrid Reciprocal Rank FusionCombines multiple searches into a single ranked result to increase result relevanceAllows combining vector and NLP freelyCombining specialized embeddings Increases computational load and storage consumpti

24、on(generating multiple embeddings)IBM TechXchange|2024 IBM CorporationAgenda0102030405RAG 101Watson Assistant&Conversational Search(Elastic)Search IntegrationFilter and Query configurationDeploying models to ElasticIBM TechXchange|2024 IBM CorporationDeploying models to ElasticTransformer models tha

25、t conform to the standard BERT model interface and use the WordPiece tokenization algorithm should workEland provides a convenient mechanism to deploy a model to IBM Cloud Database ElasticsearchIBM TechXchange|2024 IBM CorporationDeploying models to ElasticSetup Kibana for convenient administrationA

26、fter deployment of a new model model synchronization is requiredCheck model status to validate youre ready to go!IBM TechXchange|2024 IBM Corporation#IBMTechXchange31Using a custom model for indexingQuestion?AnswerLoadSplitPromptLLM(Granite)Vectorstore(Elastic)IBM TechXchange|2024 IBM CorporationEmb

27、ed(ELSER)Using a custom model for indexingCreate an ingestion pipeline,including a chunking strategyConfigure a reference to the model in the inference sectionIn this case we define a max size of a chunk with 1024,since ELSER supports 512 token context windowIBM TechXchange|2024 IBM CorporationUsing

28、 a custom model for indexing/ChunkingOverlapping chunks by 25%is recommendedOptimal setting might depend on your documentsWatch out for layout sensitive information,like tables,diagrams,etc.IBM TechXchange|2024 IBM Corporation#IBMTechXchange34Q&AIBM TechXchange|2024 IBM CorporationDaniel PittnerIBM,

29、STSM,Cloud Platform,Cloud Data S#IBMTechXchange35Thank YouIBM TechXchange|2024 IBM Corporation#IBMTechXchangeNotices and disclaimersCertain comments made in this presentation may be characterized as forward looking under the Private Securities Litigation Reform Act of 1995.Forward-looking statements

30、 are based on the companys current assumptions regarding future business and financial performance.Those statements by their nature address matters that are uncertain to different degrees and involve a number of factors that could cause actual results to differ materially.Additional information conc

31、erning these factors is contained in the Companys filings with the SEC.Copies are available from the SEC,from the IBM website,or from IBM Investor Relations.Any forward-looking statement made during this presentation speaks only as of the date on which it is made.The company assumes no obligation to

32、 update or revise any forward-looking statements except as required by law;these charts and the associated remarks and comments are integrally related and are intended to be presented and understood together.2024 International Business Machines Corporation.All rights reserved.This document is distri

33、buted“as is”without any warranty,either express or implied.In no event shall IBM be liable for any damage arising from the use of this information,including but not limited to,loss of data,business interruption,loss of profit or loss of opportunity.Customer examples are presented as illustrations of

34、 how those customers have used IBM products and the results they may have achieved.Actual performance,cost,savings or other results in other operating environments may vary.Workshops,sessions and associated materials may have been prepared by independent session speakers,and do not necessarily refle

35、ct the views of IBM.Not all offerings are available in every country in which IBM operates.Any statements regarding IBMs future direction,intent or product plans are subject to change or withdrawal without notice.IBM,the IBM logo,and are trademarks of International Business Machines Corporation,regi

36、stered in many jurisdictions worldwide.Other product and service names might be trademarks of IBM or other companies.A current list of IBM trademarks is available on the Web at“Copyright and trademark information”at: TechXchange|2024 IBM Corporation36#IBMTechXchangeHow to continue your learning jour

37、ney127972797Technology BreakoutUsing watsonx Assistant with watsonx.ai for intelligent Assistants with Gen AI211011101LabAutomated generative AI deployments on IBM Cloud-Deployable Architecture327962796Technology BreakoutEnterprise grade Q&A with RAG with watsonx.ai+watsonx Discovery/Elastic Search423192319Technology BreakoutHarnessing Conversational Search for Business with IBM watsonx and Elasticsearch Vector DatabasesIBM TechXchange|2024 IBM Corporation37