Data Warehouse and Lakehouse Analytics at the Speed of Thought with HeatWave [LRN3247].pdf

1、Data Warehouse and Lakehouse Analytics at the Speed of Thought with HeatWave LRN3247Abhinav AgarwalSenior Principal Product Manager,HeatWave DevelopmentGaurav ChadhaSenior Development Manager,HeatWave DevelopmentThis presentation is intended to outline our general product direction.It is intended fo

2、r information purposes only,and may not be incorporated into any contract.It is not a commitment to deliver any material,code,or functionality,and should not be relied upon in making purchasing decisions.The development,release,timing,and pricing of any features or functionality describedfor Oracles

3、 products may change and remains at the sole discretion of Oracle Corporation.Statements in this presentation relating to Oracles future plans,expectations,beliefs,intentions,and prospects are“forward-looking statements”and are subject to material risks and uncertainties.A detailed discussion of the

4、se factors and other risks that affect our business is contained in Oracles Securities and Exchange Commission(SEC)filings,including our most recent reportson Form 10-K and Form 10-Q under the heading“Risk Factors.”These filings are available on the SECs website or on Oracles website at http:/ infor

5、mation in this presentation is current as of September 2024 and Oracle undertakes no duty to update any statement in light of new information or future events.Some regulatory certifications or registrations to products or services referenced herein are held by Cerner Corporation.CernerCorporation is

6、 a wholly owned subsidiary of Oracle.Cerner Corporation is an ONC-certified health IT developer and a registered medical device manufacturer in the United States and other jurisdictions worldwide.The materials in this presentation pertain to Oracle Health,Oracle,Oracle Cerner,and Cerner Enviza which

7、 are all wholly owned subsidiaries of Oracle Corporation.Nothing in this presentation should be taken as indicating that any decisions regarding the integration of any EMEA Cerner and/or Enviza entities have been made where an integration has not already occurred.Oracle CloudWorld Copyright 2024,Ora

8、cle and/or its affiliates2The following is intended to outline our general product direction.It is intended for information purposes only,and may not be incorporated into any contract.It is not a commitment to deliver any material,code,or functionality,and should not be relied upon in making purchas

9、ing decisions.The development,release,timing,and pricing of any features or functionality described for Oracles products may change and remains at the sole discretion of Oracle Corporation.The materials in this presentation pertain to Oracle Health,Oracle,Oracle Cerner,and Cerner Enviza which are al

10、l wholly owned subsidiaries of Oracle Corporation.Nothing in this presentation should be taken as indicating that any decisions regarding the integration of any EMEA Cerner and/or Enviza entities have been made where an integration has not already occurred.Oracle CloudWorld Copyright 2024,Oracle and

11、/or its affiliates3Safe harbor statementMySQL is the#1 Open-Source DatabaseOracle CloudWorld Copyright 2024,Oracle and/or its affiliates4HeatWavediverse set of solutions5 In-database LLMs Automated,in-database Vector Store Scale-out Vector Processing HeatWave Chat Query data in Object Storage Unmatc

12、hed performance and price-performance Optionally combine with data in MySQL Accelerate MySQL queries by orders of magnitude Real-time analytics without ETL and/or separate analytics database Advanced security features In-database Machine Learning(ML)Automated pipeline to build Machine Learning(ML)mo

13、dels Train models using data in database and/or object storageHeatWaveAutopilot Machine Learning(ML)powered automation for HeatWave Automatically improves performance and price-performance Increases the productivity of DBAs and developersOracle CloudWorld Copyright 2024,Oracle and/or its affiliatesH

14、eatWave OLTP,OLAP,ML,Lakehouse,GenAI,Vector StoreOracle CloudWorld Copyright 2024,Oracle and/or its affiliates6Transactions,real-time analytics across data warehouse and data lake,machine learning,GenAI in one database serviceQueriesQueriesResultsResultsSocial,eCommerce,IoT,gaming,fintech apps Analy

15、tics/ML/GenAI toolsObject StorageDatabase Database exportsexportsDatabase exportsStreaming dataData SourcesEnterprise AppsWeb/SocialLogfilesIoTMySQL databaseHeatWaveOLAPOLAPOLTPOLTPAutopilotAutopilotVector storeVector storeGenAIGenAIAutoMLAutoMLMySQL HeatWave vs.Amazon RDS4 TB*Benchmark queries are

16、derived from TPC-H benchmark,but results are not comparable to published TPC-H benchmark results since they do not comply with TPC-H specification.RDS-MySQL(db.r5.24xlarge)MySQL HeatWave(10 E3 nodes)05,00010,00015,000GeoMean of Query Run Time$0$20,000$40,000$60,000Annual CostTime(Seconds)5400 xfaste

17、r8 sec11 hrs$54,393$34,0732/3the costOracle CloudWorldCopyright 2024,Oracle and/or its affiliates7HeatWave on AWSConsole,Control plane and Data plane are built and optimized to run natively in AWSOracle AWS accountControl planeData planeDataApplicationsAWS CloudCustomer AWS accountC9.0Oracle CloudWo

18、rld Copyright 2024,Oracle and/or its affiliates8TransactionProcessingTransactionProcessingAnalyticsGenAIAutoMLVector StoreAutopilotAlways FREE HeatWave instance available in Oracle CloudAlways free serviceRun small-scale applications on HeatWave at no costAvailable featuresOLTP,Autopilot,JavaScriptV

19、ector store,Vector processingMachine Learning(AutoML)Lakehouse(object store processing)Available now in all OCI accounts1 ECPU,16GB memory with HeatWave nodeMySQL storage:50 GB,Lakehouse storage:10 GBOne Always Free DB system free-of-charge in the home region of each commercial tenancyStart for free

20、 here:Oracle Cloud:https:/ HeatWave MySQL DB SystemsOracle CloudWorld Copyright 2024,Oracle and/or its affiliates9The challenges with growing dataManage costs of using separate cloud services for OLTP,analytics,ML,data lake,GenAIlicense,configure,deploy,manage,secure,upgradeExpensive and slow to ana

21、lyze growing data stored in filesHow to combine it with transactional dataLeverage machine learning and Gen AI on all datastructured,semi-structured,unstructuredWant flexibility to use multiple public cloudsOracle CloudWorldCopyright 2024,Oracle and/or its affiliates10Source data can be in multiple

22、formatsCSV Row major text formatParquet Hybrid columnar format Popular in data lakesAvro Row major binary format Popular in data lakesNDJSON Newline-delimited JSON Very popular in cloud applicationsDatabase exports Aurora Redshift MySQL.Oracle CloudWorldCopyright 2024,Oracle and/or its affiliates11O

23、f collected data may remain unused99.5%Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates12HeatWave OLTP,OLAP,ML,Lakehouse,GenAI,Vector StoreOracle CloudWorld Copyright 2024,Oracle and/or its affiliates13Transactions,real-time analytics across data warehouse and data lake,machine learning

24、,GenAI in one database service1.Query structured,semi-structured data in object storage2.Vector operations on unstructured data with structured data3.Data not copied to database4.No change to downstream applications5.Train,run inferences,explanations on Lakehouse data6.Combine object storage data wi

25、th OLTP dataHeatWave Lakehouse and AutopilotEasy interface for data in object store to be treated as(external)tableProvides Lakehouse-specific functionality with existing syntax and is extensible External source file locations specified in extensible JSON interfaceFiles can be distributed across mul

26、tiple object store buckets100%compliant with standard MySQL syntax14 CREATE TABLE tbl_name ENGINE=LAKEHOUSEENGINE_ATTRIBUTE=SECONDARY_ENGINE=RAPID;Oracle CloudWorldCopyright 2024,Oracle and/or its affiliatesMySQL AutopilotAuto Parallel Load in ActionDDL to create non-existing DBsDDL to create non-ex

27、isting tablesUsing inferred column typesLengthPrecisionSetting enginesSetting engine attributeCan extract column namesOracle CloudWorldCopyright 2024,Oracle and/or its affiliates15Load and Query performance across file formats100 TB TPCHCSVParquetAvroConfiguration100 Nodes100 Nodes100 NodesLoad Time

28、(hrs)2.82.82.74Geomean Query Time(sec)26.3326.2326.85Total Query Time(sec)921920933Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates16JavaScript support in HeatWaveOracle CloudWorldCopyright 2024,Oracle and/or its affiliates17Data movement is costlyData movement is costlyBuilding data pi

29、pelinesPipeline maintenanceCloud egress costNot always avoidableNot always avoidableFor example,data cleaning/data transformationMove complex dataMove complex data-intensive processing,close to intensive processing,close to data(database server)data(database server)Reduce costSimplify complex ETL EL

30、TIntegrate easily with other data processing featuresStored ProceduresStored FunctionsPrecompiledRoutinesMobile AppsMonitorsWeb AppsOperational SystemsDecision SupportData to Insights.Powered by HeatWave LakehouseOracle CloudWorld Copyright 2024,Oracle and/or its affiliates181.Data in object storage

31、2.Load data in HeatWave Lakehouse3.Oracle Analytics CloudConnect to HeatWave 4.DashboardHeatWave powers analytics dashboards with a single endpointUsing standard MySQLSame query performance when data inside MySQL or in object store14.214.2475979105020406080100120HeatWaveHeatWaveLakehouseSnowflakeRed

32、shiftGoogle BigQueryDatabricksQuery time(seconds)10 TB TPC-H Query performance*Benchmark queries are derived from the TPC-H benchmark,but results are not comparable to published TPC-H benchmark results since these do not comply with the TPC-H specifications.19Configuration:MySQL HeatWave Lakehouse:5

33、12 nodes;Snowflake:4X-Large Cluster;Databricks:3X-Large Cluster;Amazon Redshift:20-ra3.16xlarge;Google BigQuery:6400 slotsOracle CloudWorldCopyright 2024,Oracle and/or its affiliates19Same price-performance when data inside MySQL or in object store1.51.541.920.241.492.5020406080100HeatWaveHeatWaveLa

34、kehouseSnowflakeRedshiftGoogle BigQueryDatabricksPrice-Performance(cents)10TB TPC-H Price-Performance20*Benchmark queries are derived from the TPC-H benchmark,but results are not comparable to published TPC-H benchmark results since these do not comply with the TPC-H specifications.Configuration:MyS

35、QL HeatWave Lakehouse:512 nodes;Snowflake:4X-Large Cluster;Databricks:3X-Large Cluster;Amazon Redshift:20-ra3.16xlarge;Google BigQuery:6400 slotsOracle CloudWorld Copyright 2024,Oracle and/or its affiliates20TPC-DS 100TBHeatWaveSnowflake 3XLarge RedShift 10 ra3.16xlargeBigQuery3200 slotsDatabricks 2

36、XLargeHourly Cost($)56.4312886.0674.56103.39Load time(hrs)1.213.37.743.637.46HeatWave Load advantage 2.7x6.4x3x6.1xTotal Time(seconds)3,7195,3795,10811,69413,704Price-Perf($)58191122242394HeatWave price-perf advantage3.3x2.1x4.1x6.8xBest performance for query&load at the lowest priceTPC-DS 100TBBenc

37、hmark queries are derived from the TPC-DS benchmarks,but results are not comparable to published TPC-DS benchmark results since these do not comply with the TPC-DS specifications.Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates21Industry analysts about HeatWave Lakehouse“Organizations l

38、ooking for the best value in the cloud data lakehouse landscape must seriously consider MySQL HeatWave Lakehouse.”Carl Olofson,Research Vice President,Data Management Software“MySQL HeatWave demonstrates that Lakehouse performance can be identical to transaction query performanceunheard of and even

39、unthinkable.”Holger Mueller,VP and Principal Analyst“The ability of HeatWave to load and query data on such a massive number of nodes in parallel is the first in the industry.”Marc Staimer,Senior Analyst“MySQL HeatWave Lakehouse can simplify the life of data management professionals and should impro

40、ve the customer experience.”Matt Kimball,Vice President and Principal Analyst“Simply put:MySQL HeatWave Lakehouse enables you to stay ahead of the competition by taking swift action on meaningful business insights.”Steve McDowell,Principal Analyst&Founding PartnerOracle CloudWorld Copyright 2024,Ora

41、cle and/or its affiliates22Takashi KinoshitaChief Producer,e-Book DivisionNTT SOLMARE CORPORATION“HeatWave Lakehouse allows us to easily and quickly load data on object storage into HeatWave and combine it with MySQL data for analysis.”Oracle CloudWorldCopyright 2024,Oracle and/or its affiliates23Us

42、e-case:bank transactions offloadOracle CloudWorld Copyright 2024,Oracle and/or its affiliates24Oracle CloudTransactions10+years,millions of customersOffload to object storageNew records every 1 minuteLoad into HeatWaveCustomer reportsHigh concurrencyLow latencysub-second responseUse-case:video strea

43、ming service usage analyticsOracle CloudWorld Copyright 2024,Oracle and/or its affiliates25Oracle CloudLoad into HeatWave LakehouseContent and streaming quality statisticsDaily consumer streaming usage dataObject StoreDEMONSTRATION1Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates26INCRE

44、MENTAL REFRESH AND WRITEBACKChanges to user data are incrementally appliedProvides 1-to-1 mapping between user files and Lakehouse tablesOnly delta in user data is applied incrementally over existing Lakehouse tableIntegrated into Lakehouse Load API1-to-1 mappingUser BucketHeatWave Lakehouse TableSE

45、T options=JSON_OBJECT(refresh_external_tables,TRUE);CALL sys.heatwave_load(table_list,options);document_namedocument_namesegmentsegmentsession_idsession_idoci:/path/a.pdf“Hello World”LRN3247oci:/path/d.pdf“Quick brown.”SOL3724oci:/path/b.pdf“Game on”LRN3038oci:/path/c.pdf“Oracle is.”LRN2892oci:/path

46、/c.pdf“Software.”LRN2892Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates27Export transformed data to Object Store28SELECT INTO OUTFILEPREFIX FORMAT User BucketQuery resultsWrite query results to object storagenumber of rows exported3 1 9 6.Object StoreBuild Map Reduce applications with

47、HeatWaveOracle CloudWorldCopyright 2024,Oracle and/or its affiliates29Map ReduceHeatWaveConfigurationACIDApplication DevelopmentUser configures cluster and tasksHeatWave automatically tunes and advisesNo in-built ACID guaranteesMature database with ACID complianceExpressing data processing logic is

48、complex in non-SQL languages Manipulating data made easy by SQLHeatWave GenAIOracle CloudWorldCopyright 2024,Oracle and/or its affiliates30In-database,automated vector storeIn-database LLMsScale-out vector processingHeatWave ChatAutomate embedding generation without AI expertise or moving data to a

49、separate vector database.Instantly benefit from generative AI anywhere without integration issues,and help reduce infrastructure costs.Perform fast semantic searches on your organizations data to deliver rapid and accurate answers to questions.Engage in natural language conversations without complex

50、 manual operations.Easily refine your searches.Building GenAI applications with non-Oracle databases is complex Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates31Part 1Discover user documentsParse data from documentsExtract metadataSplit data into segmentsChoose embedding modelCreate ve

51、ctor embeddingsDesign vector storeInsert metadata+segments+embeddings into vector storeEnsure consistency of ML model when queryingCreate a vector storePart 2Use vector store with LLMsAsk a questionChoose embedding modelCreate query embeddingSelect Vector store to searchSelect search algorithmSelect

52、 search resultsCreate prompt with search results and guard railsSelect LLMGet resultsSQL call sys.heatwave_load(data_location,optional_params);SQL call sys.ML_RAG(What is HeatWave?,NL_response,optional_params)DEMONSTRATION2Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates32VECTOR STORE a

53、nd LAKEHOUSE Solving the challenges of growing dataManage costs of using separate cloud services for OLTP,analytics,ML,data lake,GenAIlicense,configure,deploy,manage,secure,upgradeExpensive and slow to analyze growing data stored in filesHow to combine it with transactional dataLeverage machine lear

54、ning and Gen AI on all datastructured,semi-structured,unstructuredWant flexibility to use multiple public cloudsBest performance and price-performance to query non-MySQL and MySQL workloadsFully automated ML for data in the database and in files.Gen AI with vector storeOne cloud service for OLTP,rea

55、l-time analytics,data lake,ML,GenAIAvailable in OCI,AWS,and AzureOracle CloudWorld Copyright 2024,Oracle and/or its affiliates33Join us forOracle CloudWorld Copyright 2024,Oracle and/or its affiliates34From Days to MinutesAutomate ML in Your Enterprise with HeatWave AutoMLWednesday,Sep 1211:30 AM-12

56、:15 PM PDTSandeep AgrawalConsulting Principal Member of Technical Staff,OracleMigrate to HeatWave for better performance,lower cost,and built-in AI/MLWednesday,Sep 114:45 PM-5:30 PM PDTMandy PangSenior Principal Product Manager,OracleAlways FREE HeatWave instance available in Oracle CloudAlways free

57、 serviceRun small-scale applications on HeatWave at no costAvailable featuresOLTP,Autopilot,JavaScriptVector store,Vector processingMachine Learning(AutoML)Lakehouse(object store processing)Available now in all OCI accounts1 ECPU,16GB memory with HeatWave nodeMySQL storage:50 GB,Lakehouse storage:10

58、 GBOne Always Free DB system free-of-charge in the home region of each commercial tenancyStart for free here:Oracle Cloud:https:/ HeatWave MySQL DB SystemsOracle CloudWorld Copyright 2024,Oracle and/or its affiliates35Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates36Online guide to all HeatWave sessions Subtitle goes hereThank Y CloudWorld Copyright 2024,Oracle and/or its affiliates37