GenAI 平臺作為民主化使用和維護治理的一種方式.pdf

1、2024 Databricks Inc.All rights reserved1GENERATIVE AI PLATFORM GENERATIVE AI PLATFORM AS A WAY TO DEMOCRATIZE AS A WAY TO DEMOCRATIZE USE AND MAINTAIN USE AND MAINTAIN GOVERNANCEGOVERNANCEMaria Dela RosaMaria Dela Rosa202420242024 Databricks Inc.All rights reserved2Maria Maria Eugenia Dela Eugenia D

2、ela RosaRosaSr.Manager BradescoPhysicistMasters degree in BiotechnologyLive in So Paulo/Brazil with my wifeLove videogamesLinkedIn:2024 Databricks Inc.All rights reserved32024 Databricks Inc.All rights reserved42024 Databricks Inc.All rights reservedOne of the largest financial groups in Latin One o

3、f the largest financial groups in Latin AmericaAmericaUS$3.283.28US$402.2402.2(Billion)US$176.4176.4US$1.81.8US$72.672.6US$82.682.6US$12.112.12024 Databricks Inc.All rights reservedLarge Large distributiondistribution network in network in BrazilBrazil andand abroadabroad2024 Databricks Inc.All righ

4、ts reservedTechnologicalTechnological andand innovativeinnovativeIn the 2023:App App IndividualsIndividuals+CompaniesCompanies:3.6 3.6 billionbillionfinancial transactions18 18 millionmilliona accessesccesses per day to the App per day to the App(23 monthly accesses per client)INNOVATIONComplete Com

5、plete innovattioninnovattion ecosystemecosystemStrategicinnovation2024 Databricks Inc.All rights reservedBroadlyBroadly diversifieddiversified portfolioportfolioProducts and services in all modalities,with digital with digital journeys journeys and artificial intelligence(BIA):artificial intelligenc

6、e(BIA):2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedBIA:Bradesco Artificial InteligenceBIA:Bradesco Artificial Inteligence15,315,3MiMiUsers in Mobile+13%+13%s/1T23Warning of Warning of Suspected Suspected Fraud for Fraud for Corporate Corporate ClientsClientsFor For

7、EnterprisesEnterprisesFor For peoplepeople8787%ResolutenessMar/244,84,8MiMiUsers in WhatsApp+26%+26%s/1T232,32,3BiBiInteractions since implementation+192+192%Transactions withWhatsApp s/1T231T241T241T242024 Databricks Inc.All rights reserved10WHEREWHERETOTOSTART OUR START OUR GEN AI GEN AI JOURNEY?J

8、OURNEY?2024 Databricks Inc.All rights reserved11IDEATHONIDEATHONp paving the organization towards generative AIaving the organization towards generative AIPEOPLEPEOPLEIDEASIDEASTECHNOLOGYTECHNOLOGY2024 Databricks Inc.All rights reserved5 5Use Cases Use Cases 6363IdeiasIdeiasIDEATHONIDEATHONSupportSu

9、pport in in resolvingresolvingprotestsprotestsJudicial Judicial DemandsDemands,obligationsobligations andandoficial oficial letterslettersBIA Sales BIA Sales CopilotCopilotCustomerCustomerRelationshipRelationshipCenterCenterCopilotCopilot for for issuingissuing advisoryadvisoryopinionsopinions|3737M

10、entoredMentoredAreasAreas+60+60Training Training hourshours+200+200ParticipantsParticipants2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedIDEATHON WINNER CASESIDEATHON WINNER CASES13Discovering learning,applications and preparing infrastructureDiscovering learning,appl

11、ications and preparing infrastructureBackofficeBackofficeSumarizationClassificationEntity TabulationOCROmbudsmanOmbudsmanSumarizationClassificationEntity ExtractionTextGenerationConsortiumConsortiumQ&AClassificationSemanticSearchTextGenerationCreditCredit CardCardQ&AUploadSemanticSearchTextGeneratio

12、nLegalLegalQ&AUploadSemanticSearchTextGenerationJudicial Judicial DemandsDemands,obligationsobligations andandoficial oficial letterslettersSupportSupport in in resolvingresolvingprotestsprotestsBIA Sales BIA Sales CopilotCopilotCustomerCustomer RelationshipRelationshipCenterCenterCopilotCopilot for

13、 for issuingissuingadvisoryadvisory opinionsopinions2024 Databricks Inc.All rights reserved15ModularizedModularized Use CasesUse CasesSpecific development by use caseSpecific development by use caseBackofficeBackoffice6 63 32 26 63 32 24 4OmbudsmanOmbudsman1 14 43 3Sales Sales CopilotCopilot1 14 4Cu

14、stomerCustomer RelationshipRelationship1 14 4LegalLegal3 32 24 45 51 16 6ExperimentsExperiments1 QA2 Entity Tabulation3 Classification4 Text Generation5 Image Generation6-Sumamarization2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedPlatformPlatformMotivatorsMotivatorsG

15、overnanceSecurity MonitoringBest Practices2024 Databricks Inc.All rights reserved17IntegratedIntegrated GenerativeGenerative AI AI PlatformPlatformUse case agnosticUse case agnosticQ&AQ&AEntityEntity TabulationTabulationClassificationClassificationTextText GenerationGenerationImageImage GenerationGe

16、nerationSumarizationSumarizationG GO OV VE ER RN NA AN NC CE ES SE EC CU UR RI IT TY YM MO ON NI IT TO OR RI IN NG GBackofficeBackofficeOmbudsmanOmbudsmanSales Sales CopilotCopilotCustomerCustomer RelationshipRelationshipLegalLegalUse Case AUse Case AUse Case BUse Case B2024 Databricks Inc.All right

17、s reserved18HOW?HOW?2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform19Architecture OverviewArchitecture Overview2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedIntegratedIntegr

18、ated GenerativeGenerative AI PlatformAI Platform20Architecture OverviewArchitecture OverviewRuntimeRuntimeGovernanceGovernanceServiceServices s andandData Data SourcesSourcesData Data PreparatioPreparation nStakeholdersStakeholdersData Data LakeLake2024 Databricks Inc.All rights reserved2024 Databri

19、cks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform21Data Data SourcesSourcesData SourcesThey They cancan bebe transactionaltransactionalsystems,systems,databasesdatabases,file,file servers,servers,messagesmessages,etc.,etc.2024 Databricks Inc.All rights reser

20、ved2024 Databricks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform22Data Data PreparationPreparation3.StorageRepositoriesRepositories ofof variousvarioustypestypes thatthat willwill bebe usedusedfor training for training andandexecutingexecuting AI ModelsAI Mo

21、dels2.ProcessingApplication Application ofofstrategiesstrategies for data for data preparationpreparation thatthat willwill bebeconsumedconsumed andand usedused byby AI.AI.ExampleExample:SentenceSentence ororparagraphparagraph fragmentationsfragmentations(chunkingchunking)1.CaptureTransmissionsTrans

22、missionsandand file capture file capture flowsflows2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform23GovernanceGovernanceAdministration and ControlAdministrativeAdministrative tool for tool for managingmana

23、ging thethe AI Platform.AI Platform.ThroughThrough it,it,thethe followingfollowing isisexpectedexpected toto bebe establishedestablished:-ProfilesProfiles-Complex workflows StepsComplex workflows Steps-ParameterParameter definitiondefinition-ApprovalsApprovals andand securitysecuritysupportsupport-F

24、eedback Feedback analysisanalysis(via logs)(via logs)2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform24RuntimeRuntimeRuntimeSet Set ofof capabilitiescapabilities thatthatreceivereceive requestsrequests anda

25、nd processprocessthemthem throughthrough orchestrationorchestration2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform25API API RequestsRequests1.Applications and UsersSystem System integrationsintegrations fo

26、r for sedingseding requestsrequests toto prepre-trainedtrained AI ModelsAI Models2.SecurityDomainDomain-basedbased.OrganizedOrganized bybygroupsgroups.ToTo use APIs use APIs andand accessaccessdata,data,authorizationauthorization withinwithin na na authorizedauthorized group group isis requiredrequi

27、red.MandatoryMandatory for for UsersUsers andandApplicationsApplications3.InterfaceExposedExposed via via APIsAPIs4.MicroservicesSpecializedSpecialized in in receivingreceivingrequestsrequests andand injectinginjectingthemthem in in thethe appropriateappropriateformatformat for for processingprocess

28、ing ononthethe platformplatform2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform26RecognitionRecognition2.EventsNa Na eventevent isis generatedgenerated bybyinput services input services totosequencesequence

29、 thethe processingprocessingofof thethe requestrequest1.First RequestSynchronousSynchronous executionexecutionservices services thatthat utilize AI utilize AI Models for Models for strategystrategyclassificationclassification2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reser

30、vedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform27AlternativeAlternative FlowFlow2.EventsAnAn eventevent isis generatedgeneratedfor for sequencingsequencing thetheprocessingprocessing ofof thetherequestrequest1.Alternative FlowDependingDepending onon thethescenarioscenario,thereth

31、ere isis a a possibilitypossibility ofof invokinginvokingintegrationintegration services services withwith transactionaltransactionalsystems systems toto obtainobtaininformationinformation withwith lowerlowerlatencylatency for prompt for prompt compositioncomposition2024 Databricks Inc.All rights re

32、served2024 Databricks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform28GenerativeGenerative AIAI2.OpenAI APIsThe specialized services are programmedto consume APIs from AI ServicesCurrently:Azure OpenAI1.Specialized AI ServicesAs per the execution workflow,spe

33、cialized services are triggeredthrough event consumption:-Summarization-Entities-Classification-Text Generation-SemanticSearch-Indexing3.PreparedDatabasesPreprocessed data isstructured so that data fragments can be correlatedTokenizationand Embedding2024 Databricks Inc.All rights reserved2024 Databr

34、icks Inc.All rights reservedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform29GenerativeGenerative AIAI2.Return ServiceFor For resultresult retrievalretrieval,thethe returnreturn serviceserviceconsumes consumes thethe outcomesoutcomesandand stores stores themthem in a in a databaseda

35、tabase for later for later queryingquerying byby thetherequestingrequesting system.system.3.Application and UsersThe The requestingrequesting Application Application consumes consumes thethe resultresult bybypresentingpresenting na na identifieridentifierprovidedprovided atat thethe beginningbeginni

36、ng ofofthethe requestrequest.1.ResultsThe The outcomeoutcome obtainedobtainedafterafter executingexecuting thetheworkflow workflow isis insertedinsertedintointo thethe messagemessage broker broker for for consumptionconsumption2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights res

37、ervedIntegratedIntegrated GenerativeGenerative AI PlatformAI Platform30GenerativeGenerative AIAINew Features SupprtsSupprts additionaladditional andandcustomizedcustomizedfunctionalitiesfunctionalities2024 Databricks Inc.All rights reserved31Generative AI Generative AI requires a requires a change i

38、n the change in the dynamics of dynamics of content content governancegovernance2024 Databricks Inc.All rights reserved_Document management is done in adecentralizeddecentralized wayway_CurationteamworksworksononAIAItrainingtrainingandcreatingquestionsand answers_Communicationbetweenfrontsisnotnot c

39、entralizedcentralized_AIAIknowledgeknowledgebasebaseoutoutofofsyncsyncwith corporate regulatory portal_CurationTeamactsinthegovernancegovernanceofofknowledgeknowledgebasebasecontentcontentClassic Classic ModelModelBIA BIA regular AIregular AIQ&A referenceInternalknowledgedatabaseBusiness Business te

40、amsteamsTechTechsquadsquadGovernanceGovernancePERFORMANCE PERFORMANCE MANAGEMENTMANAGEMENTCONTENT CONTENT MANAGEMENTMANAGEMENTAI MODEL AI MODEL MANAGEMENTMANAGEMENT2024 Databricks Inc.All rights reserved_Documentmanagementfocusedonbusinessbusiness areasareas_AItrainingisdoneonlyonlybybythethetechnic

41、altechnical squadsquad_Governanceteamfocusedonperformanceperformance managementmanagement_MaingovernanceactivityisAIAIperformanceperformance managementmanagement_AIknowledgebasereflectsreflectsregulatoryregulatory portalportal_Curationteamdoesnotactindocument governanceProposed ModelProposed ModelGe

42、nerative AIGenerative AIInternalInternalknowledgeknowledgedatabasedatabaseBusiness Business teamsteamsTechTechsquadsquadCurationCurationTeamTeamPERFORMANCE PERFORMANCE MANAGEMENTMANAGEMENTCONTENT CONTENT MANAGEMENTMANAGEMENTAI MODEL AI MODEL MANAGEMENTMANAGEMENT2024 Databricks Inc.All rights reserve

43、dVulnerabilities Vulnerabilities TOP 5 OWASP*TOP 5 OWASP*Ensuring Ensuring secutirysecutiry1 12 23 34 45 5BRADESCO CONTROL*Open Web Application Security PrPROMPT INJECTIONINFORMATION LEAKNON-ISULATED ENVIRONMENTUNAUTHORIZED CODE EXECUTIONINAPPROPRIATE RESPONSESUntrusteddatainsertioncontaining malici

44、ous commandsAttacks that seek to manipulate orinduce systems to exploit confidentialinformationOperateinanenvironmentthatallowsimproperaccesstoapplications and system informationAttackerscreatecommandsthattriggertheexecutionofmaliciouscodeInadequacyofresponsesrelating to human valuesThere is an orch

45、estrator layer thatcontrols the actions and commandsexecutedAccess to data follows the policy ofeach application,containing logsThe environment used is isolated andthedevelopmentshaverulestocontrol responsesOrchestratorlayercontrolsexecuted commandsCreativity control and response scoperestriction vi

46、a prompt engineer2024 Databricks Inc.All rights reserved35ACHIEVEMENTSACHIEVEMENTS2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved36GovernanceCustomizableUnique architectureFlexibleBradescoIA GenPlatformScalability gainDemocratization of AI2024 Databricks Inc.All rights

47、 reserved37SkillsGenerative AIEntitytabulationClassificationTextGenerationImageSummarizeQ&A2024 Databricks Inc.All rights reservedSearches from natural language in databases.Allows the business area to consult information without the intermediary of a specialized analyst4040%reduction%reduction in d

48、ata extraction time for analysisAssistant for developers,based on Bradesco documentation and standards.Agility in consulting documentation and reducing development timeSummaries and comparisons of COPOM minutes and announcements.Hawk-Dove Indicator Chart(Monetary Policy Bias):Hawks:a more aggressive

49、 stance.Doves:Likely to keep interest rates low.Open Finance Q&ADevOps ChatbotMonetary Policy ComOpen Finance Q&ADevOps ChatbotMonetary Policy Commmittee ittee minute meetingsminute meetings38AchievementsAchievementsGenerative AI Applications BradescoGenerative AI Applications Bradesco2024 Databrick

50、s Inc.All rights reserved39HackathonDatabricks&Bradesco+50+50P Participantsarticipants+20+20I Ideasdeas1111SelectedSelectedcasescases5 5Cases Cases approvedapproved.Vector Vector SearchSearchModel Model ServingServingVisionVisionRAGRAGEntityEntity tabulationtabulationSumarizationSumarizationSummariz

51、ation/description of images Summarization/description of images for prospecting banking correspondent for prospecting banking correspondent establishmentsestablishmentsAgribusiness Virtual Agribusiness Virtual AssistantAssistantAnalysis of customer complaints with Analysis of customer complaints wit

52、h debit and credit card products and debit and credit card products and contact via digital channels for contact via digital channels for resolutionresolution2024 Databricks Inc.All rights reserved40BIG NUMBERSBIG NUMBERS2024 Databricks Inc.All rights reserved41+90+90Mapped UsecasesMapped Usecases24

53、0%240%More More speed to reach productionspeed to reach productionIncreaseIncreaseUser satisfactionUser satisfaction2024 Databricks Inc.All rights reservedSimpleSimplequestionsInformalInformalquestionsQuestion withtypostyposImpreciseImprecisequestions93%93%97%97%1 1Hi,can you tell me where I can fin

54、d the phone numbers for Bradescosadvisors?how do i find the pone number for bradescoadvisorHow do I find the phone number for those bank advisors?ExemplesExemplesCategoriesCategories85%85%98%98%57%57%93%93%40%40%93%93%2 23 34 4PercentagePercentage ofof usefuluseful responses responses toto thethe us

55、ersusersHowHow GenerativeGenerative AI improve AI improve BIAsBIAsresponse response toto contextcontextHow can I find the telephone numbers of Bradescos advisors?2024 Databricks Inc.All rights reservedBIA|BIA|AchievementsAchievementsCompared to the Old BIA,would you say the experience with the new B

56、IA Agencies:(n=19)79%of managers point out that the New BIA has evolvedNPS reached 77 for Nova BIA among relationship managersOn a scale of 0 to 10,how likely are you to recommend the new BIA to a colleague?15x more15x moreE EqualqualImprovedImprovedWorseWorseE EqualqualPromoterPromoterDetractorDetractor2024 Databricks Inc.All rights reserved4490%Accuracy greater than for all difficulty levels/BIA with Generative AI attacks managers main pain point and performs better when the degree of complexity increasesBIA|BIA|AchievementsAchievements