利用生成式人工智能的數據結構自動化管理數據.pptx

《利用生成式人工智能的數據結構自動化管理數據.pptx》由會員分享，可在線閱讀，更多相關《利用生成式人工智能的數據結構自動化管理數據.pptx（41頁珍藏版）》請在三個皮匠報告上搜索。

1、Using Data Fabrics with Generative AI to Automate Data ManagementJeff FriedDirector,Platform Strategy&InnovationInterSystemsInterSystems Corporation.All Rights Reserved.GenAIData FabricsInterSystems Corporation.All Rights Reserved.Contact coordinates:JLongtime Data Management Nerd Director,InterSyst

2、ems CTO,BA Insight Senior PM,Microsoft VP,FAST Search SVP,LingoMotorsabout Jeff FriedPassionate About Data Management Search AI Big Data Text Analytics Information Strategyabout InterSystems2017,2018,2019,2022,2023,2024Handle 60%of NYStock Exchange trafficManage over 2 BillionPatient Records Worldwi

3、deTrack 20 Million Shipping ContainersInterSystems Impact in Healthcare,Financial Services,and Supply ChainUnparalleled performance,scalability,interoperability,and reliabilityData FabricsInterSystems Corporation.All Rights Reserved.DriversBusiness PerspectiveEnable less technical users to quickly f

4、ind,access,integrate and share dataAllows subject matter experts in the business to become a part of the data modeling processReduce the cycle time of accessing ready-to-use dataData Management PerspectiveProductivity advantages of automated data transformation and integration gives time back to IT

5、resources Cost optimization benefits of not having to buy multiple tools with redundant/overlapping capabilitiesAutomated optimization of data integration resulting in better price/performance and ROCEOrganizational PerspectiveImproved communication between data managers and data consumers creates a

6、 collaborative culture and a more agile,more resilient,more competitive organizationWhy Adopt Data Fabrics?“By 2024,data fabric“By 2024,data fabric deployments will deployments will quadruple quadruple efficiency in data useefficiency in data use,while while cutting human-driven cutting human-driven

7、 data management tasks in data management tasks in halfhalf”1 11 Gartner Insights,Striving to Become a Data-Driven Organisation?Start With 5 Key D&A Initiatives,https:/www.gartner.co.uk/en/information-technology/insights/data-and-analytics-essential-guidesGARTNER is a registered trademark and servic

8、e mark of Gartner,Inc.and/or its affiliates in the U.S.and internationally and is used herein with permission.All rights reserved.InterSystems Corporation.All Rights Reserved.OutcomesSuccessfully integrating 300 million data elements 20 million+patients18,000 different healthcare providers 8,000 dif

9、ferent healthcare facilities9 million real time alerts every monthRunning new aggregates and analyticsCreating and monetizing new data productsLargest US Health Information Exchange(HIE)Supporting 20M+Patients in New York StateRead case studyChallengeIntegrate massive sets of disconnected and dissim

10、ilar healthcare data Create new revenue generating data productsInterSystems Corporation.All Rights Reserved.Smart Data FabricBI/ANALYTICSNATURAL LANGUAGE PROCESSINGDATA EXPLORATIONANALYTIC SQLAI/ML/AUTOMLConnect and CollectReal-Time AnalyticsConsistent Data and MetadataInterSystems Corporation.All

11、Rights Reserved.SECURITYINTEGRATIONSEMANTIC LAYERPIPELININGNORMALIZATION&HARMONIZATIONLINEAGEINGESTIONCONNECTIVITYDATA WAREHOUSESRISK DATATHIRD PARTY APPLICATIONSCUSTOMER DATADATA LAKESMARKET DATABusiness UsersQuants/Data ScientistsApplications/APICustomers/PartnersSMART DATA FABRICDATA SOURCESDATA

12、CONSUMERSOutcomesAutomating internal portfolio,client and financial reportingDelivering previously monthly reports as on-demand intraday reportsImproving client satisfaction via dynamic dashboards and real time portfolio analyticsProviding data scientists and quants with data and secure processing w

13、orkspacesGained net new assets under management$100B+AUM Financial Services Asset Management FirmWatch VideoChallengesProvide the entire business and clients with current,accurate and actionable information while Lower the cost,effort and delays of working with enterprise-wide and external data“We d

14、eal with lots of data&every second matters.The data that is relevant now will not be relevant in five minutes.Ive been working with data for 25 years.We tried a few solutions&finally found something that works.”Jey Amalraj,CTOHarris AssociatesInterSystems Corporation.All Rights Reserved.SmartDataFab

15、ricSmartDataServicesDATA PLANECONTROL PLANECOMPOSABLE APPLICATIONS,PIPELINES&ANALYTICSComposable Smart ApplicationsInterSystems Corporation.All Rights Reserved.OutcomesImplemented a real-time,enterprise-wide smart data fabric Synchronizes data among dozens of applications Calculates real-time positi

16、ons and“on the fly”aggregations to meet sub-second SLAs Replaced multiple data management software products Gaining 9X performance improvement using only 30%of the infrastructureReal time smart data fabric connecting dozens of applicationsRead case studyChallenge Connect and synchronize dissimilar a

17、pplications,with real-time analyticsInterSystems Corporation.All Rights Reserved.OutcomesImplemented a next generation microservices-based wealth management platformReal-time microservices-based SaaS application leveraging the data fabric highly efficient“self-attention”for contextGenerative Pre-Tra

18、ined Transformers(GPT)Large-scale Language Model that is pre-trained on vast amounts of text data and fine-tuned for specific tasksGPT-4 has 1 trillion parametersGPT-5 expected to have 10-20 trillion parametersAccording to the Gartner Generative AI 2024 Planning Survey of 822 business leaders1 Surve

19、y respondents report:15.8%revenue increase 15.2%cost savings 22.6%productivity improvementGenerative AI Benefits1 Gartner Generative AI 2024 Planning Survey of 822 business leadershttps:/ Experimental Evidence on the Productivity Effects of Generative Artificial Intelligence,MIT.3 McKinsey Says“Abou

20、t Half”of Its Employees Are Using Generative AI,VentureBeat4 Will Generative AI Make You More Productive at Work?Yes,But Only If Youre Not Already Great at Your Job,Stanford University Human-Centered AI.5 Erik Brynjolfsson,Generative AI at Work|NBERWorker productivity improvements from GenAI:ChatGPT

21、 improves worker productivity by 37%.2 GenAI coding assistants improves worker productivity 7%to 55%3.3 GenAI conversational assistants improves customer service agents productivity 14%to 35%4,5 InterSystems Corporation.All Rights Reserved.GenAIData FabricsInterSystems Corporation.All Rights Reserve

22、d.Data fabrics and GenAI can complement each other well in managing,processing,and analyzing data.Heres how they work together:1.Data Integration and Management:Data fabrics provide a unified architecture to integrate data from various sources,including structured and unstructured data,streaming and

23、 batch data,etc.GenAI can then utilize this integrated data to train models effectively.For instance,GenAI can access data from different parts of the fabric seamlessly,ensuring that the models are trained on comprehensive datasets.4.Real-Time Data Processing:Many data fabrics support real-time data

24、 processing capabilities,enabling GenAI to ingest and analyze streaming data in real-time.This is particularly beneficial for applications such as real-time analytics,fraud detection,and recommendation systems,where immediate insights are crucial.3.Scalability and Performance:Data fabrics are design

25、ed to scale horizontally,allowing them to handle large volumes of data efficiently.This scalability aligns well with the requirements of GenAI,especially when training deep learning models that often require massive datasets and computational resources.GenAI can leverage the scalability of data fabr

26、ics to train models faster and more effectively.2.Data Accessibility and Governance:Data fabrics offer a centralized platform for data accessibility and governance,ensuring that data is available to GenAI models securely and in compliance with regulations.This accessibility streamlines the data prep

27、aration process for AI applications,allowing GenAI to focus more on model development and less on data wrangling.Ask ChatGPT:how do data fabrics and GenAI go together?InterSystems Corporation.All Rights Reserved.Data fabrics and GenAI can complement each other well in managing,processing,and analyzi

28、ng data.Heres how they work together:1.Data Integration and Management4.Real-Time Data Processing3.Scalability and Performance2.Data Accessibility and GovernanceInterSystems Corporation.All Rights Reserved.Ask ChatGPT:how do data fabrics and GenAI go together?“Your data is probably not AI-ready.Your

29、 data is probably not AI-ready.In the 2023 Gartner IT Symposium In the 2023 Gartner IT Symposium Research Super Focus Group,Research Super Focus Group,only 4%of respondents only 4%of respondents said their data is AI-readysaid their data is AI-ready”1 11 2023 Gartner IT Symposium Research Super Focu

30、s GroupGartner 2023 IT Symposium Research Super Focus Group(7 August 2023).n=72 CIOs and senior IT leaders from North America,the U.K.and Europe.More than half of respondents came from the C-suite and large enterprises.The data was collected through a live polling session as part of Gartners researc

31、h for the 2023 IT Symposium Opening Keynote.GARTNER is a registered trademark and service mark of Gartner,Inc.and/or its affiliates in the U.S.and internationally and is used herein with permission.All rights reserved.InterSystems Corporation.All Rights Reserved.Smart Data Fabric:Make Your Data GenA

32、I-ReadyBI/ANALYTICSNATURAL LANGUAGE PROCESSINGDATA EXPLORATIONANALYTIC SQLAI/ML/AUTOMLDeliver more data to the LLMReal-time dataConsistent,trusted,and governed dataInterSystems Corporation.All Rights Reserved.SECURITYINTEGRATIONSEMANTIC LAYERPIPELININGNORMALIZATION&HARMONIZATIONLINEAGEINGESTIONCONNE

33、CTIVITYDATA WAREHOUSESRISK DATATHIRD PARTY APPLICATIONSCUSTOMER DATADATA LAKESMARKET DATABusiness UsersQuants/Data ScientistsApplications/APIsLLMsSMART DATA FABRICDATA SOURCESDATA CONSUMERSHow can you use smart data fabricto be ready for genAI?Get to the data you need,handle it at scaleTransform and

34、 unify to make healthy data for AI useControl what you feed to LLM Models,and howEmerging Tech Impact Radar:Generative AI16 November 2023-ID G00791993 ByAnnette Zimmermann,Jim Hare,and 17 morehttps:/ DATATHIRD PARTY APPLICATIONSMARKET DATADATA WAREHOUSESCUSTOMER DATADATA LAKESDATA SOURCESDATA CONSUM

35、ERSBI/ANALYTICSDATA EXPLORATIONANALYTIC SQLAI/ML/AUTOMLSECURITYINTEGRATIONSEMANTIC LAYERPIPELININGNORMALIZATION&HARMONIZATIONLINEAGEINGESTIONCONNECTIVITYSMART DATA FABRICSmart Data Fabric with GenAI:Addressing the Top“Impact Radar”Use CasesBusiness UsersQuants/Data ScientistsApplications/APIsLLMs VE

36、CTOR/RAGInterSystems Corporation.All Rights Reserved.GenAI-Enabled Virtual AssistantsCreate Natural LanguageAssistants Using the Datain Data FabricsInterSystems Corporation.All Rights Reserved.EmbedGenAI intoExistingApplicationsEmbedded Gen-AI ApplicationsInterSystems Corporation.All Rights Reserved

37、.Create Custom Coding AssistantsAI Code GenerationPlease create a DTL that maps HL7 2.4 ADT_A0I to HL7 2.4 Set SendingApplication to Nicholai Set SendingFacility to ISC Set ReceivingApplication to ONBASECreate a loop around the source.DGI segment.InterSystems Corporation.All Rights Reserved.Emerging

38、 Tech Impact Radar:Generative AI16 November 2023-ID G00791993 ByAnnette Zimmermann,Jim Hare,and 17 morehttps:/ are trained on a WIDE variety of sourcesTraining on Cloud Clusters of 1000s of GPUsWeb Scraped DataOutbound Model usable on less hwLLMHuman Feedback Tuned DataManual Edits and“Alignment”Dat

39、a dumps of public sourcesBUT:-not trained on your data,and not up to date GPT-4 cutoff date is January 2022 -limited size for prompt+context+output GPT-4 limit is 8,192 tokens input+output;preview with 128K total/4K output Vector Search experimental feature in 2024.1Embedding VectorsHigh-dimensional

40、,dense arraysOutput of specialized embedding LLMsCapture semanticsBenefitsCompactGeneral,can represent imagesSearch is simply“nearest neighbors”Foundation for GenAI RAG applicationsChallengesRequire special indexingData managementCombining with other dataRetrieval Augmented Generation(RAG)PatternInn

41、ovating with generative AIVector Search,Retrieval-Augmented Generation,and AI OrchestrationAdd Semantics to your ApplicationsBuild AI-Powered Experiences with RAGCreate and Manage Composite genAI applicationsSummary:Data Fabrics and GenAI Can Supercharge Your Data ManagementData Fabrics Can Make You

42、r Data AI ReadyIncorporate more data from more sourcesHarmonized,accurate,cleansed,and consistentInclude real-time dataWith governance and controlIntegration of Vector and RAG Can Improve Accuracy,Usability and Efficiency of ApplicationsVector embeddings/search with RAG can augment your existing app

43、lications and analyticsLook for Data Platforms That Incorporate Analytics,Vectors,and LLM Integration Directly Within the Data FabricEliminates latency,data duplication,and data consistency issuesEasier to implement GenAI-enabled applications Faster time-to-value,simpler operations,less complexity a

44、nd lower total cost of ownershipInterSystems Corporation.All Rights Reserved.38 2024 Gartner,Inc.and/or its affiliates.All rights reserved.Gartner is a registered trademark of Gartner,Inc.and its affiliates.Gartner Predicts,2024:Data management solutions embrace generative AIBy 2028,the data managem

45、ent markets will converge into“a single market”around data ecosystems enabled by data fabric and GenAI reducing technology complexity.Thank YouDisclaimer:This presentation may contain descriptions or depictions of AI projects,capabilities,or features that should not be construed to represent availab

46、le products or services.InterSystems Corporation.All Rights Reserved.Global leader in data management,integration,and analyticsUnparalleled performance,scalability,interoperability,and reliabilityGlobal customers and partners across Healthcare,Financial Services,Supply Chain,and other ecosystemsAbout InterSystemsHandle 60%of NYStock Exchange trafficManage 2BPatient Records WorldwideTrack 20M containers continuallyOur Impact in Healthcare,Financial Services,Supply Chain