利用 HeatWave GenAI 和 Vector Store 構建新應用程序.pdf

1、Building New Applications using HeatWave GenAI&Vector StoreNipun AgarwalSenior Vice President,MySQL HeatWaveCopyright 2024,Oracle and/or its affiliates1MySQL HeatWaveAnalyticsIn-database MLAutopilotOLTPMySQL HeatWaveAnalyticsIn-database MLAutopilotOLTPMySQL HeatWaveQueriesQueriesResultsResultsSocial

2、,eCommerce,IoT,gaming,fintech apps.Analytics and ML toolsObject StoreDatabase Database exportsexportsProcess ALL workloads with MySQL HeatWaveHeatWaveAnalyticsIn-database MLAutopilotOLTPDatabase exportsStreaming dataData SourcesEnterprise AppsWeb/SocialLog filesIoTMySQL storageScales from 16 GB to 5

3、12 TBLAKEHOUSE,REAL-TIME ANALYTICS,MACHINE LEARNING,TRANSACTION PROCESSING IN ONE DATABASE SERVICECopyright 2024,Oracle and/or its affiliates21.Vector store 2.Vector processing3.Generative AI 4.HeatWave Chat Object StoreDatabase Database exportsexportsMySQL HeatWaveAnalyticsIn-database MLGenAIGenAIO

4、LTPVector storeVector store(private preview)3Copyright 2024,Oracle and/or its affiliatesHeatWave Generative AI and Vector Store MySQL HeatWaveAnalyticsIn-database MLGenAIGenAIOLTPVector storeVector store(private preview)HeatWaveAnalyticsIn-database MLGenAIGenAIOLTPVector storeVector storeMySQL stora

5、geCopyright 2024,Oracle and/or its affiliates3Search on public and private enterprise dataSearch on unstructured data in vector store Retrieval Augmented GenerationGenerate insights from enterprise documentsGenerate blogs from pdf instruction manualsSummarize logs for root cause analysisContent gene

6、ration&summarizationNatural language interaction with proprietary unstructured data Personalized content retrieval and response back in natural languageNatural language interaction+Generative AI in HeatWave enables new use casesAugmented promptLLMProduce incident reports in natural language for an o

7、peratorAnomaly DetectionDetect anomalous sequences of logsHeatWave AutoMLUnsupervised Anomaly DetectionHeatWaveGenerative AIContinuously ingest unstructured text logsSummarize incidents from sequences of logsGenerate Incident summariesReport GeneratorReport Generation:HW AutoML+Summarization of anom

8、alous logsUsecase 1:Predictive maintenanceCONTENT GENERATION/SUMMARY5Copyright 2024,Oracle and/or its affiliatesAsk the question(What is the main problem in the following collection of logs.Provide a two-sentence summary.)“The main problem in this collection of logs is that the memory usage of a par

9、ticular process(with ID 8145)is consistently exceeding the defined memory threshold.This issue is repeatedly highlighted in the logs,and SIGTERM signals are sent to the main thread to terminate the process when it exceeds the memory threshold.Would you like me to help you with anything else regardin

10、g this collection of logs?Report Generation:Summarization of anomalous logs6Copyright 2024,Oracle and/or its affiliates Usecase 1:Predictive maintenance CONTENT GENERATION/SUMMARIZATION 6Copyright 2024,Oracle and/or its affiliatesUsecase 2:Financial fraud detectionSET context=(SELECT txn_description

11、 FROM open_transaction WHERE txn_id=mule_txn_id);SET query=Provided is a bank transaction information with field names and field values.Each field is separated by nn.Please provide a concise summary of this bank transaction.What is suspicious about this transaction?Your audience is a bank officer.;S

12、ELECT sys.ML_GENERATE(query,JSON_OBJECT(“context”,context);Transaction to Natural Language DescriptionHeatWaveInnoDBHeatWaveLLMDAT_TXN3/31/12COD_CC_BRN_TXN160COD_USERNO0CTR_BATCH_NO0REF_SYS_TR_AUD_NO0REF_SUB_SEQ_NOCOD_ACCT_NO1601000010862TXT_TXN_DESCBalance brought forwardDAT_VALUE3/31/12REF_CHQ_NOC

13、OD_DRCRCCOD_TXN_MNEMONIC5010COD_TXN_LITERALPBCAMT_TXN0FLG_PASBKUPDNRAT_CCY1COD_MSG_TYP200COD_PROC251000CTR_UPDAT_SRLNODAT_POST3/31/12AMT_TXN_TCY0COD_TXN_CCYRAT_CONV_TCLCY1COD_SCCOD_AUTH_IDREF_CARD_NOREF_TXN_NOREF_USR_NOTXT_ADDL_INFO1DAT_TS_TXN_INIT2015/02/18 15:38:58DAT_TS_SENT_FOR_AUTH2015/02/18 15

14、:38:58DAT_TS_AUTH2015/02/18 15:38:58DAT_TS_SUB_FOR_PROCESSING2015/02/18 15:38:58DAT_TS_SENT_TO_HOST2015/02/18 15:38:58DAT_TS_HOST_PROCESSING2015/02/18 15:38:58COD_ENTITY_VPD240The provided bank transaction information indicates that on March 31,2012,a balance was brought forward from the previous ac

15、counting period to the current one.The account number associated with this transaction is 1601000010862,and the amount of the transaction is zero.The flag indicating whether the transaction has been successfully processed is set to N,which means that the transaction was not successful.The currency c

16、ode for the transaction is 1,and the message type is 200,which indicates an error message.The processing code is 251000,which may indicate a specific error or issue with the transaction.The transaction has been sent to the host for processing,but it has not yet been processed.The entity VPD associat

17、ed with this transaction is 240.rnrnThere are several suspicious aspects of this transaction that require further investigation.Firstly,the fact that the amount of the transaction is zero raises questions about the purpose and validity of the transaction.Secondly,the flag indicating that the transac

18、tion was not successful suggests that there may have been an issue with the processing or authorization of the transaction.CONTENT GENERATION/SUMMARIZATION Copyright 2024,Oracle and/or its affiliates7Recommend,Retrieve and Generate description of dishes based on user preferenceOnline food deliveryPe

19、rsonalized Menu:HW AutoML+Retrieval Augmented Generation“Tofu Curry”Tofu Biryani”“Peas Curry”Restaurant menu8Copyright 2024,Oracle and/or its affiliatesUsecase 3:Personalization of Online DeliveryRETRIEVAL AUGMENTED GENERATION9Copyright 2024,Oracle and/or its affiliatesUsecase 3:Personalization of O

20、nline DeliveryRETRIEVAL AUGMENTED GENERATION9Copyright 2024,Oracle and/or its affiliates More accurate LLM results by filtering irrelevant data Faster LLM inference due to smaller search space10Copyright 2024,Oracle and/or its affiliatesHeatWaveInnoDBHeatWaveLLMVector StoreNatural language interacti

21、on with dataTraditional MLSynergy between HeatWave AutoML and Generative AI A DIFFERENTIATORCopyright 2024,Oracle and/or its affiliates10Users Contracts in PDF in Object storeRetrieval Augmented GenerationVector store ingestEmployee Assistant:Improve employee productivity 11Usecase 4:Natural languag

22、e interaction EMPLOYEE ASSISTANTCopyright 2024,Oracle and/or its affiliates1112HeatWave offers choice for running LLMsoIn-HeatWave LLMsNative execution within the HeatWave databaseRun smaller LLMs like Llama2-7B and Mistral-7B Secure,lower cost,guaranteed availability oOCI Generative AI service LLMs

23、Support larger models like Cohere-command and Llama2-70B and run on GPUsHigher quality,better performance Copyright 2024,Oracle and/or its affiliates12GenAI/Vector Store AvailabilityMulti-CloudDedicated RegionHybridDesigned for multi-cloud presence OCI,AWS,and AzureIn-database LLMs enables availabil

24、ity of GenAI capabilities in dedicated regionHybrid architecture enables processing of on-premise Vector data13Available in Public Cloud,Dedicated Region,Hybrid EnvironmentVECTOR STORE AND ABILITY TO RUN LLMS INSIDE HEATWAVE PROVIDES FLEXIBILITY OF DEPLOYMENT Copyright 2024,Oracle and/or its affilia

25、tes13 Introducing new Vector data type In-memory hybrid-columnar storage format for vector columnsVector Datatype Parallelize vector processing across the HeatWave nodes Processes at near memory bandwidth Vector Processing In-database parsing,parallel embedding generation Vector stored in the object

26、 store Vector Store14Copyright 2024,Oracle and/or its affiliatesVector SupportCopyright 2024,Oracle and/or its affiliates14Document DiscoveryParsingEmbedding GenerationInserting into Vector Store15All phases of creating a vector store done inside HeatWaveCopyright 2024,Oracle and/or its affiliates15

27、1.0,2.0,0.5,3.5,1.5,3.0,1.0,2.0,0.5,3.5,1.0,2.0,key1:val1,key1:val1,key1:val1,Automatically generate embedding for text from multiple file formatsParseTextTableImageVector EmbeddingsMetadataUnstructured dataGenerate Vector embeddingDifferent ML models used for different data modalitiesVector StoreHe

28、atWave parses and automatically creates embeddings for documentsCopyright 2024,Oracle and/or its affiliates16HeatWave StorageCustomer BucketHeatWave ClusterRead Encoder Persist embeddings as Heatwave chunksLakehouse Vector Store tableOIT Parser Read Encoder OIT Parser Read Encoder OIT Parser Table g

29、eneration Table generation Table generation Parse into segmentsCreate embeddingsdocument_namemetadatasegmentembeddingoci:/path/a.pdfkey1:val1,key2:val2“Hello World”1.0,2.0,.oci:/path/a.pdfkey1:val1,key2:val2“Program”0.5,3.5,.oci:/path/a.pdfkey1:val1,key2:val2“MySQL is.”0.5,3.5,.oci:/path/b.pdfkey1:v

30、al3,key2:val4“Quick brown.”1.0,2.0,.oci:/path/b.pdfkey1:val3,key2:val4“Game on”0.5,3.5,.oci:/path/c.pdfkey1:val5,key2:val6“Oracle is.”1.0,2.0,.oci:/path/c.pdfkey1:val5,key2:val6“Software.”0.5,3.5,.Read file from object storeVector Store creation with HeatWave is parallelizedPARSE SOURCE FILES AND CO

31、NCURRENT EMBEDDING GENERATION ACROSS NODESShuffle tasks(segments)across nodesCopyright 2024,Oracle and/or its affiliates17FileSizePagesParsing Time HW threads or nodesEncoding Time(sec)Vector Store creation time(s)SpeedupSingle PDF44 MB696316 sec1 thread84678588125 threads73884010.250 threads3704811

32、7.8Multiple PDFs 4 x 44 MB2785217 sec1 node2733288212 nodes147116911.74 nodes8079523.0Intra-documentInter-document18Vector Store creation in HeatWave scales outCopyright 2024,Oracle and/or its affiliates1819Vector Store can be queried by natural language or SQLAugmented promptLLMRetrieval AgentRecom

33、mender SystemVector Vector StoreStoreHeatWave AutoMLSQL QueryMySQL TablesQuery ResultsCopyright 2024,Oracle and/or its affiliates19Example of using HeatWave vector storeCreate Vector Store#Ingest documents from Object Store like any Lakehouse tableCALL sys.heatwave_load(“vector_store”,load_params);Q

34、uery Vector StoreNative SQL syntaxQuery Vector StoreML_RAG#Example:Find books semantically most similar to input and are in printSELECT id,titleFROM books b,books_in_print ipWHERE b.title=ip.titleORDER BY DISTANCE(b.segment_embedding,query_embedding,DOT)as distance DESC LIMIT 10;#Example:Answer ques

35、tions using data in documents ingested into Vector StoreCALL sys.ML_RAG(Which state has maximum carbon?,output);Copyright 2024,Oracle and/or its affiliates20Vector data type supportStandard SQL interface to create tables with vector columnsVector data storageoHeatWave:In-memory columnar formatoInnoD

36、B:BLOB 21mysql CREATE TABLE wikipedia(id INT,title VARCHAR(1024)page_data TEXT page_list TEXT,page_url TEXT,page_embedding VECTOR(1024)ENGINE_ATTRIBUTE=model:cohere)ENGINE=lakehouse,SECONDARY_ENGINE=rapid;Example distance functionsExample distance functionsL1/MANHATANL2/EUCLIDIANL12/MANHATAN_SQUARED

37、L22/EUCLIDIAN_SQUAREDCOSINEDOTHAMMINGCopyright 2024,Oracle and/or its affiliates21 For querying unstructured text documents in Lakehouse Allows for refinement of chat scope(querying documents in a specific folder,using different ML models)22HeatWave Chat NATURAL LANGUAGE INTERACTION Copyright 2024,O

38、racle and/or its affiliates22ClassificationPlayer churn predictionClassify warranty claimsAnomaly DetectionDetect anomalies in suppliesPredict assembly line jamDefective part identificationIdentify game hackersPredict when failure will occurIoT digital twin failure predictionPredict air pollutionRet

39、urn on advertising spend predictionUtilization demand forecastingTimeseries ForecastingIdentify similar usersRecommend movies to viewersSuggest substitute productsRecommend new productsRecommender SystemLoan default prediction Demand forecastingPredict flight delayLoan amount predictionRain fall amo

40、unt predictionRegressionHeatWave AutoMLIN DATABASE MACHINE LEARNING,FULLY AUTOMATED TRAINING,EXPLANATIONS,25X FASTER THAN REDSHIFT ML Copyright 2024,Oracle and/or its affiliates23Data Drift monitoring in HeatWave AutoMLTrain a data drift detector based on Autoencoder(AE)Use the detector to monitor d

41、ata drift in productionThe detector:computes a reconstruction error for new incoming samplesupdates the cumulative drift metricIf the metric exceeds a threshold,automatically recommends that the model should be retrained24Confidence Interval for forecasting in HeatWave AutoMLRepresents a range where

42、 future values are likely to fall,based on a certain confidence level(e.g.,95%)Helps users assess risk,make informed decisions,and understand the uncertainty in model predictions25New models for anomaly detection:GLOF,PCAPrincipal Component Analysis(PCA):The model reduces complex data into its main

43、components to more easily identify outliersGeneralized Local Outlier Factor(GLOF):An internally developed model,it detects global and clustered anomalies like GkNN,while also detecting local anomalies.26AUGMENTS THE EXISTING GKNN ALGORITHMAnomaly detection for logs 2024-03-05 13:28:59-2024-03-05 13:

44、29:20 Group replication-related failureGCS Failure reading from fd=from:2024-03-05 13:29:25-2024-03-05 13:29:27This server is not able to reach a majority of members in the group.This server will now block all updates.The server will remain blocked until contact with the majority is restored.It is p

45、ossible to use group replication force members to force a new group membership.2024-03-05 13:40:59-2024-03-05 13:41:21 Potential connection leak in group replicationGCS Old incarnation found while trying to add node 2024-03-05 17:26:31 Database was not shutdown normally!Starting crash recovery.Start

46、ing to parse redo log at lsn=Heatwave processes and generalizes incoming machine logs,then builds a tailored anomaly detection modelThis model helps in identifying anomalies in logs,enabling effective preventative maintenance and root cause analysis27Potential AnomaliesAnomaly ThresholdNOW TRAINED F

47、OR MYSQL LOGS SummaryIn-database vector store for querying unstructured text contentIn-database Generative AI in HeatWave brings power of LLMs to enterprise contentVector processing in HeatWave can be combined with other SQL operatorsContinued innovation in HeatWave AutoMLBest performance,price performance,scalability in the industry for querying data Single service for machine learning,GenAI,analytics,Lakehouse and OLTP 28Copyright 2024,Oracle and/or its affiliates