AI Microservices with Containers, Kubernetes, and Oracle Database 23ai [LRN3337].pdf

《AI Microservices with Containers, Kubernetes, and Oracle Database 23ai [LRN3337].pdf》由會員分享，可在線閱讀，更多相關《AI Microservices with Containers, Kubernetes, and Oracle Database 23ai [LRN3337].pdf（52頁珍藏版）》請在三個皮匠報告上搜索。

1、Modern AI Apps using Kubernetes,Microservices,and Oracle Database 23aiWei HuSenior Vice PresidentDatabase DevelopmentSeptember 10,2024Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates1The following is intended to outline our general product direction.It is intended for information purpos

2、es only,and may not be incorporated into any contract.It is not a commitment to deliver any material,code,or functionality,and should not be relied upon in making purchasing decisions.The development,release,timing,and pricing of any features or functionality described for Oracles products may chang

3、e and remains at the sole discretion of Oracle Corporation.The materials in this presentation pertain to Oracle Health,Oracle,Oracle Cerner,and Cerner Enviza which are all wholly owned subsidiaries of Oracle Corporation.Nothing in this presentation should be taken as indicating that any decisions re

4、garding the integration of any EMEA Cerner and/or Enviza entities have been made where an integration has not already occurred.Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates2Safe harbor statementGlobally Distributed DatabaseOver 50 SQL simplificationsTrue CacheLock-FreeReservationsSQL

5、 FirewallPropertyGraphsJSON/Relational DualityRead-Only Per-PDB StandbyPriority TransactionsDeveloper RoleContainers,Kubernetes OperatorReal-time SQL Plan ManagementRolling PatchingAI Vector SearchMicroservice SupportOracle Database 23ai Oracle Database 23ai Developers,Mission Critical,AIDevelopers,

6、Mission Critical,AI3Oracle CloudWorld Copyright 2024,Oracle and/or its affiliatesLock-Free long-running transactionsData Use CaseDomainsETAGHow do Kubernetes&Microservices relate to AI?Everyone knows about Oracle Database 23ai vector search capabilitiesEqually important,Oracle Database 23ai supports

7、 AI development through its support for Kubernetes and microservicesHow so?Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates4Microservices are Essential for AIMost AI development will involve augmenting existing apps with AI,instead of writing new apps from scratchFastest way to get bene

8、fits of AIMicroservices allow you to do this in a clean,modular mannerWhen I say Microservices,bthink in terms of MiniservicesExperience has shown that using dozens to hundreds of microservices is not the right approach.Instead,think in terms of a few(e.g.,Millers Law:7+2)Miniservices provides the b

9、est of both monolithic and microservices providing flexibility,independence without the extreme complexity of microservicesOracle CloudWorld Copyright 2024,Oracle and/or its affiliates5Many Microservice Technologies Benefit AI AppsContainers for isolation,portability,and integration with CI/CD pipel

10、inesKubernetes to streamline deployment,scaling,and management of containerized appsEvent Mesh for composability of microservicesSagas and Transactions for correct outcomes under concurrency and failuresProven design patterns to ensure successPowerful frameworks such as Spring Boot to streamline Jav

11、a app development Converged Database(with Vector Search)with native container supportGenerative development for the enterprise(GenDev)enables ACID MicroservicesOracle CloudWorld Copyright 2024,Oracle and/or its affiliates6AI Apps Also Have Unique RequirementsPolyglot:AI apps tend to use Python.This

12、means your app will likely have microservices written in both Java and PythonThis also requires polyglot event mesh support to allow event producers and consumers to be written in different languagesGreater technology diversity:vector database,RAG,Greater need for modularity and abstraction.New&impr

13、oved models,tools,and frameworks becoming available at a fast pace.Must be properly abstracted so that you can swap in the best LLM of the dayNeed to experiment with different technologies and approaches AI SandboxOracle CloudWorld Copyright 2024,Oracle and/or its affiliates7The Basic Building Block

14、sContainers and KubernetesOracle CloudWorld Copyright 2024,Oracle and/or its affiliates8Containerized Oracle Database 19c and 23aiSingle instance,RAC(on-prem only)23ai Free Globally Distributed Database,True Cache23ai support on Oracle Cloud Native Environment(OCNE),Oracle Kubernetes Engine(OKE),and

15、 Podman on Oracle CloudDocker on Oracle Linux 7,Podman on Oracle Linux 8Coming23ai on-prem with RACLinux 9-Oracle Linux Release 9 and RHCK(Red Hat Compatible Kernel)Control Group(Cgroup)V2 process containers to control resource utilizationOracle CloudWorld Copyright 2024,Oracle and/or its affiliates

16、9 Roadmap Next 6-12 months Oracle Container Based Runtime Environments Kubernetes 23ai on Oracle Cloud Native Environment(OCNE)in Azure and AWSOracle Cloud Native Environment(OCNE)Release 1.9 and OL9Support for OCNE Storage Classes(Ceph&Rook)Oracle CloudWorld Copyright 2024,Oracle and/or its affilia

17、tes10Kubernetes Operator for Oracle DatabaseOne operator,multiple backend controllersOracle CloudWorld Copyright 2024,Oracle and/or its affiliates11Supported configurations:1.Oracle Database in Docker/Podman container Single Instance DB(SE,EE,XE,23c Free)Globally Distributed Database2.Autonomous Dat

18、abases(Shared and Dedicated)3.Base Database on OCI4.On-Premises DBsPhysical Infrastructure(e.g.,Exadata)Oracle DB OperatorKubernetes Clusterkubectl Containerized Oracle DBsAutonomous DBsOn-Premises DBs1243Base DBsOracle Cloud InfrastructureSelected enhancements in this releaseEnhanced security:scope

19、d deployment to namespace&OKE credentials for strong access control23ai Free support Automatic deployment from the Oracle Kubernetes Engine(OKE)ConsoleOraOperator v1.1.0 Production(May 2024)Operator features comingPublish OraOperator on operatorhub.ioOption to support Helm Chart deploymentInclude Or

20、aOperator in official OpenShift catalogInstallable via Operator Lifecycle Manager RAC supportOracle CloudWorld Copyright 2024,Oracle and/or its affiliates12Review Containers&Kubernetes for AI and Enterprise AppsContainerized Oracle Database makes it easy to incorporate Oracle Database in your contai

21、nerized applicationsSimplify adding Oracle 23ai features such as vector database to your CI/CD pipeline along with AI-oriented languages(Python),libraries(PyTorch,TensorFlow,SpaCy,)while supporting enterprise Java microservices in other containersOraOperator makes it seamless to manage Oracle Databa

22、ses in KubernetesNext:How do the microservices talk to each other?Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates13Microservice-to-Microservice CommunicationThe Event MeshOracle CloudWorld Copyright 2024,Oracle and/or its affiliates14Event-driven and REST REST is synchronous,stateless,

23、and works well for services offered over the internet(e.g.,accessing public LLMs)However,synchronous means tight coupling between microservices can result in slow response or hangsEvent-driven architectures are asynchronous,thereby more resilient in the presence and delays and failures,and scale bet

24、ter.Better for intra-company microservices Drawbacks of event-driven architectures are:complexity,consistency,monitoring and trouble-shootingBest approach:Use REST as the public interface,internally use event-driven architectureOracle CloudWorld Copyright 2024,Oracle and/or its affiliates15Event Mes

25、h for Enterprise and AI AppsEnterprise Apps require correct and consistent results dont bill customers twice for a shipmentExisting event systems such as Kafka make it hard to achieve this because they offer weak(eventual)consistencyOracle Database includes a built-in event mesh called Transactional

26、 Event QueuesSimplify development with built-in transaction supportPolyglot(multiple languages and message types JSON,JMS,RAW,ADT)Highly scalable and availableCompatible and interoperable with KafkaIncluded Free as part of Oracle DatabaseOracle CloudWorld Copyright 2024,Oracle and/or its affiliates1

27、6Transactional Event Queues(TxEventQ)Free event mesh built into the Oracle Database and continuously enhanced for 20 years(previously called AQ)and used by tens of thousands of customersTxEventQ simplifies microservices with built-in Transactional Outbox supportProduction/consumption of events is tr

28、ansactionally integrated with data changesHandles difficult cases like events that change database state and produce more eventsProvides exactly-once semantics without 2-phase commit and idempotency headachesSecurity,backup,recovery,etc.of events and data are automatically synchronizedTxEventQ has a

29、 scale-out architecture that handles over a million events per secondSource/sink connectors(Oracle or Confluent)to ship events between Kafka and TxEventQOracle CloudWorld Copyright 2024,Oracle and/or its affiliates17Transactional Event Queues(TxEventQ)New FeaturesSupport for Kafka Java API&Confluent

30、 Kafka REST APIKafka Connector for bi-directional event propagation between TxEventQ and Kafka12X better throughput than Confluent connectorHigher performanceAdaptive in-memory cachingMulti-process queue-to-queue propagationParallel consumption from single event stream and morePrometheus and Grafana

31、 support to make it simple to monitor and troubleshootOracle CloudWorld Copyright 2024,Oracle and/or its affiliates18Sample TxEventQ Kafka Java Clientokafka.jar on Maven-https:/ org.oracle.okafka.clients.producer.KafkaProducer;import org.apache.kafka.clients.producer.Producer;import java.util.Proper

32、ties;public class SimpleProducerOKafka public static void main(String args)try Properties props=new Properties();props.put(bootstrap.servers,database:1521);props.put(key.serializer,mon.serialization.StringSerializer);props.put(value.serializer,mon.serialization.StringSerializer);Producer producer=ne

33、w KafkaProducer(props);Future lastFuture=null;int msgCnt=20000;for(int i=0;imsgCnt;i+)ProducerRecord producerRecord=new ProducerRecord(TxEventQ,i+,Test message#+i);lastFuture=producer.send(producerRecord);System.out.println(Produced+msgCnt+messages.);lastFuture.get();producer.close();catch(Exception

34、 e)System.out.println(Exception in Main +e);e.printStackTrace();Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates19Import Kafka Producer interfaceRest of the code is unchangedChange the broker:portHandling Concurrency and FailuresSagas,Transactions,and GenDevOracle CloudWorld Copyright 2

35、024,Oracle and/or its affiliates20Unfortunately,Things Go Wrong in the Real World Transactional Event Queues ensure your events are consistent in the presence of crashes and network failuresReal apps need to handle other types of failures:Concurrency&race conditionsOne microservice failed while the

36、others completedMulti-step business processes that must be completed together or not at all(transfer money)Not handling these failures means data is no longer reliable or consistent(accounts dont balance)Transactions are meant to solve these problemsIntroducing MicroTxOracle CloudWorld Copyright 202

37、4,Oracle and/or its affiliates21MicroTx(Transactional Manager for Microservices)Simplifies application development by coordinating cross-microservice consistency Supports REST-based Try-Confirm/Cancel,Long-Running Sagas,and XAEasy to use and deploySimple API Integrated with and deployable in Kuberne

38、tesSupports popular languages(Java,TypeScript,Python,),frameworks(SpringBoot,Helidon,Micronaut,Express/JS,),and databases(Oracle,MySQL,PostgreSQL,MS SQL Server)Highly available and scalable to thousands of transactions/secondBoth Free and Enterprise Edition(available as part of GoldenGate for Distri

39、buted Applications and Analytics)Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates22MicroTx(Transactional Manager for Microservices)ComponentsOracle CloudWorld Copyright 2024,Oracle and/or its affiliates23Transaction Coordinator and ConsoleMicroservices deployed as app microservicesMicro

40、Tx Library Language-specific,used by each microserviceREST-based Transaction context automatically generated and passed with incoming and outgoing REST calls No additional requirements on application microservice Can coordinate transactions with apps deployed in ORDS,WebLogic Server,Tuxedo,and Oracl

41、e Blockchain Platform Back to SagasXA is the gold standard for distributed transactions.It is widely supported and strongly consistent.Mandatory for anything dealing with moneyWorks best when the microservices are deployed close together(same machine or data center)When your application is more deco

42、upled or long-running,then Sagas may work betterSagas also work better with event-driven architectures or workflows.Sagas provide eventual consistencyOracle supports Sagas in MicroTx and natively in the Database(similar to database support for XA)Oracle CloudWorld Copyright 2024,Oracle and/or its af

43、filiates24How Do Sagas Work?Sagas allow you to perform multiple operations while allowing for error handling in case some of the operations failUnlike XA that commits after everything is done,the individual steps(microservices)of a Saga commit as they complete This means that if a step fails,you nee

44、d to undo the effects of previously committed steps writing these undo or compensating actions can be challengingOracle CloudWorld Copyright 2024,Oracle and/or its affiliates25Example:Sagas for Distributed TransactionBooking flight,hotel,and rental car for a tripBegin SAGA1).Book flight If failed Ca

45、ncel2)Book hotel If failed Cancel3)Book car If failed Cancel 4)If all 3 succeed Commit Saga Id;Else Abort Saga Id (Compensation)End SAGACancel Car(Compensate Tx 3)Book Flight(Transaction 1)Cancel Flight(Compensate Tx 1)Book Car(Transaction 3)Book Hotel(Transaction 2)Cancel Hotel(Compensate Tx 2)Trav

46、el Agency AppOracle CloudWorld Copyright 2024,Oracle and/or its affiliates26Oracle Database 23ai Simplifies the Saga PatternStandard LRA APIs extended to support messaging/async communication for scale1.Built-in Saga coordinator in the database Supports Java and PLSQL microservices running inside th

47、e database or externally in containers or bare-metal No need to install and configure a separate Saga coordinator2.Auto-compensating columns(Lock-free Reservations)Changes to these columns are automatically undone if the saga were rolled back Supports numeric types(number,float,integer)where compens

48、ation logic is well-defined No need to write compensation code3.Oracle TxEventQ as the event mesh for Saga coordination The exactly-once semantics of TxEventQ makes this reliable No need to set up and manage event streams among Saga initiators&participantsOracle CloudWorld Copyright 2024,Oracle and/

49、or its affiliates27Saga Code Annotation ExampleOracle CloudWorld Copyright 2024,Oracle and/or its affiliates28Participant(name=Airline)public class Airline extends SagaParticipant Request(sender=TravelAgency)-5 public String handleTravelAgencyRequest(SagaMessageContext info)/*Perform all DML with th

50、is connection to ensure everything is in a single transaction*/FlightService fs=new FlightService(info.getConnection();fs.bookFlight(info.getPayload(),info.getSagaId();return response;-6 /*Local commit is automatically performed*/*status is returned back to the initiator*/Compensate-10 public void c

51、ompensate(SagaMessageContext info)fs.deleteBooking(info.getPayload(),info.getSagaId();Complete-10 public void complete(SagaMessageContext info)fs.sendConfirmation(info.getSagaId();Participant(name=TravelAgency)public class TravelAgencyController extends SagaInitiator LRA(end=false)public jakarta.ws.

52、rs.core.Response booking(HeaderParam(LRA_HTTP_CONTEXT_HEADER)URI lraId -3,BookingDto bookingPayload)Saga saga=this.getSaga(lraId.toString();try saga.beginSagaTransaction(mySession,myPublisher);saga.sendRequest(Airline,-4 bookingPayload.getFlight().toString();saga.sendRequest(”Car,-4 bookingPayload.g

53、etCarRental().toString();saga.endSagaTransaction();mySmit();catch(SagaException e)mySession.rollback();Single DB Transaction(can be performed in different threads/sessions)GenDev Generative Development for the EnterpriseThis afternoon Juan Loaiza will introduce GenDev model for building enterprise a

54、pps in a generative AI worldGenDev enables developers,assisted by AI,to accelerate the creation of enterprise appsGenDev provides better independence between app modulesIndependence is critical because we cant depend on AI to flawlessly manage complex or implicit dependencies between microservicesOr

55、acle CloudWorld Copyright 2024,Oracle and/or its affiliates29GenDevGenDev:JSON Relational Duality Views Oracle 23ai JSON Relational Duality Views separates data access format from the storage formatEach app module can simply GET/PUT the data it needs even while part of the data is shared o Transpare

56、ntly stores shared data once as a row in a tableo Concurrency and consistency of shared data are automatically handledJSON Relational Duality provides data consistency that does not lock out concurrent users or modules without compromising app module independenceOracle CloudWorld Copyright 2024,Orac

57、le and/or its affiliates30Duality Views enables better AI MicroservicesDuality Views expose APIs,with ACID consistency for data shared by multiple modulesACID consistency hugely simplifies app dev and generationThis means you can share the data instead of making copies of the data that you need to k

58、eep consistentACID MicroservicesImplemented with Duality ACID Consistency+=Evolvability ofMicroservices GenDev is also a Design PatternGenDev features in Database 23ai are useful even for human-written appsGenDev provides better independence between app modulesThis simplifies the task that AI needs

59、to do to generate appsThis independence also makes it easy for humans to write correct apps in the absence of AIGenDev design pattern:by making it easier for AI,we also make it easier even for the average(non-superhuman)developerOracle CloudWorld Copyright 2024,Oracle and/or its affiliates32GenDevRe

60、view:Transactions,Sagas,and GenDev Essential for Mission-Critical AIIf an AI microservice is going to be part of your application or workflow,you must handle failure conditionsfor example,if a credit card application uses AI to do fraud detection,you must handle case where the AI fraud module fails

61、Transactions,Sagas,and GenDev make it easier for you to incorporate AI into your real mission-critical applicationsGenDev uniquely provides ACID Microservices that share dataAI microservices will work with business data and financial information need data consistency when deployed with enterprise ap

62、plicationsOracle CloudWorld Copyright 2024,Oracle and/or its affiliates33Design Patterns and AI MicroservicesSpring Boot,Spring AI,and AI Microservices SandboxOracle CloudWorld Copyright 2024,Oracle and/or its affiliates34Design Patterns are ImportantDeveloping microservices is still hardTo increase

63、 your chance of success,adopt what worksWeve mentioned several design patterns Transactional Outbox,Reliable Event Mesh,Sagas,GenDev,etc.Weve identified 12 proven design patterns for AI MicroservicesWe are also providing a platform that provides all the components required to implement these pattern

64、sOracle CloudWorld Copyright 2024,Oracle and/or its affiliates35The 12 Proven Patterns for Microservices SuccessGo to:https:/bit.ly/microservices_dev_guide for details Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates361.Bounded Context&Loose Coupling5.GenDev9.Command Query Responsibilit

65、y Segregation2.Transactional Outbox6.Polyglot microservices10.CI/CD Pipelines for Microservices3.Reliable Event Mesh7.Unified Observability11.AI Microservices Sandbox4.Sagas8.Event Aggregation 12.Backend(as-a-Service)Spring Boot is the most popular Java frameworkOracle CloudWorld Copyright 2024,Orac

66、le and/or its affiliates37Used for building microservices and cloud-native applicationsSimplifies Spring Development:Spring Boot provides an opinionated approach to configuring and setting up Spring applications,greatly reducing the boilerplate code and configuration requiredSource:https:/ Backend f

67、or Spring Boot&MicroservicesOracle CloudWorld Copyright 2024,Oracle and/or its affiliates38Simplifying to three steps Get Starters,Build&Deploy AI microservicesOracle DatabaseApp ContainerApp ContainerApp ContainerDevelopersGet Spring Boot Starters from Spring Cloud OracleOracle AI Microservices Pla

68、tform Services ContainersDeploy MicroservicesIn App Containers12DevOpsBuild Spring Boot AI App in IDE(VS Code,IntelliJ)3KubernetesOracle Backend for Spring Boot&MicroservicesOracle CloudWorld Copyright 2024,Oracle and/or its affiliates39Simplifying to three steps Get Starters,Build&Deploy AI microse

69、rvicesOracle DatabaseApp ContainerApp ContainerApp ContainerDevelopersGet Spring Boot Starters from Spring Cloud OracleDeploy MicroservicesIn App Containers12DevOps3Spring Ops Center Apps DashboardWorkflowsService DiscoverySecurityAuth&Access NetworkingMicroTxExternal Saga/LRACoordinator KubernetesO

70、racle Backend for Spring Boot&Microservices is PolyglotGreat support for JavaGreat support for Python,JavaScript,and other languages in App Containers(Docker/Podman)These microservices can all work together and interoperate via Transactional Event Queues,Transactions,and SagasOracle CloudWorld Copyr

71、ight 2024,Oracle and/or its affiliates41Building AI Microservices on Oracle Backend for Spring Boot&MicroservicesSpring AI is a set of extensions to Spring to build AI applicationsMany developers will want to build Spring Boot Microservices applications using Spring AI for RAG with Oracle Database 2

72、3aiRAG=Retrieval Augmented Generation Spring Boot+Spring AI+RAG+Oracle Database 23aiOracle CloudWorld Copyright 2024,Oracle and/or its affiliates42Oracle Backend for Spring Boot&AI Microservices v1.3Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates43Working with AI in Microservices Get S

73、tarters,Build,Optimize,&DeployOracle DatabaseApp ContainerApp ContainerAI DevelopersGet Spring AI and LangChain starters,Get LLM API KeysOracle AI Microservices Platform Services ContainersDeploy AI microservicesIn App Containers45DevOpsBuild AI microservices with REST endpoints in IDE/Python6App Co

74、ntainerOracle AI Microservices SandboxOptimize AI microservicesin AI Sandbox7KubernetesLLM services OpenLLM OLLAMA.TEI-HFReality:AI is Evolving RapidlyBy the time we reach this slide,things have changedAny AI app must be designed to evolve and uptake the latest innovationsA good approach is to provi

75、de Sandbox whereby you can rapidly swap in different components(e.g.,LLMs,embedding models,weights,etc.)and see the effectsWe are working towards an AI Microservices SandboxThe initial focus is building ChatbotsChatbots are examples of AI Inference use case.Other examples include image recognition,d

76、rug discovery,medical diagnosis,fraud detection,The AI Microservices Sandbox can be applied to other AI Inference use casesOracle CloudWorld Copyright 2024,Oracle and/or its affiliates50Oracle AI Microservices Sandbox for ChatbotsOpen source integration to help experiment with RAG+techniquesRAG with

77、 Scalable AI Vector SearchAny LLM Model,Execute AnywhereTune Model ParametersLoad,Split,Embed DocumentsAutomated Testing PlatformAgents and specialized Inference EnginesHuman feedback and fine-tuningAll with Oracle Database 23aiGiskardOracle CloudWorld Copyright 2024,Oracle and/or its affiliates51Yo

78、u:How do I determine the accuracy of my vector indexes?Bot:To determine the accuracy of your vector indexes,you can calculate various evaluation metrics like precision,recall,and F1-scoreYou:How do I do it in Oracle Database 23ai?Bot:Im sorry,but it seems you mentioned Oracle Database 23ai.As of my

79、last update,the latest version of Oracle Database is Oracle Database 19cYou:In Oracle Database 23ai,how do I determine the accuracy of my vector indexes?Bot:To determine the accuracy of your vector indexes in Oracle Database 23ai,you can use the DBMS_DATA_MINING.DETECT_INDEX procedureYou:Please give

80、 me more details.Bot:Of course!More details about what specifically?Feel free to provide more information so I can better assist you.An Example of improving a Chatbot application(Before)Knowledge CutoffHallucinationsNo Memorygenerated on gpt-3.5-turbo July 2024Oracle CloudWorld Copyright 2024,Oracle

81、 and/or its affiliates52You:How do I determine the accuracy of my vector indexes?Bot:To determine the accuracy of your vector indexes you can utilize the index accuracy reporting feature provided by the DBMS_VECTOR package.You:How do I do it in Oracle Database 23ai?Bot:In Oracle Database 23ai,you ca

82、n determine the accuracy of your vector indexesYou:In Oracle Database 23ai,how do I determine the accuracy of my vector indexes?Bot:In Oracle Database 23ai,you can determine the accuracy of your vector indexes using the DBMS_VECTOR.INDEX_ACCURACY_QUERY procedureYou:Please give me more details.Bot:Th

83、is information is based on the documentation in the oracle-ai-vector-search-users-guide.pdf regarding index accuracy reporting features in Oracle Database 23aiExample:Improving a Chatbot Application(After)Memory/ContextGrounded ResponseRAG Attributiongenerated on gpt-3.5-turbo July 2024Oracle CloudW

84、orld Copyright 2024,Oracle and/or its affiliates53Live KnowledgeOptimizing AI Microservices in the SandboxIn this example1.Run with RAG or no RAG2.Model choice for embedding;e.g.HF-GTE-Base,OpenAI-TE-3 3.Chunking strategy and size;e.g.256,81914.LLM choice and LLM parameters;e.g.GPT-3.5-turbo,Llama-3

85、,GPT-4o5.Use of Memory and context window size;e.g.choice is based of the LLM,shows what chunk size to use6.Choice of data format to chunk/embed;e.g.pdf/html/JSON experiment with attribution accuracy7.Choice of vector similarity algorithm;e.g.similarity search,maximum marginal reference,etc.8.Distan

86、ce metrics;e.g.Cosine-similarity,Euclidean,Dot-product,Jaccard max inner-product9.Prompt engineering;e.g.ground your facts in the document,admit if you dont have enough info10.Re-ranking of chunks;e.g.TopK chunks to give best attribution11.Temperature;e.g.scale from low to high(0 to 2);tunes creativ

87、ity of the LLM;Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates54Example:Improving a Chatbot Application(in Sandbox)Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates55SummaryModern applications are best built as AI MicroservicesOracle Database 23ai with Containers,Kubernetes

88、,and Spring Boot simplifies development and deployment of AI MicroservicesIve given a brief overview,but you can find details in deep dive sessionsOracle CloudWorld Copyright 2024,Oracle and/or its affiliates561.Developers view of microservices development and 12 proven patterns in action2.Oracle Ba

89、ckend for Spring Boot and Microservices CloudBank example3.Kubernetes Operator and Container deployments 4.Data&Event Consistency with transactions across microservices/Sagas,Event Mesh5.AI Microservices Sandbox and deploying Customized Inference MicroservicesBring your teams microservices architect

90、ure to the platformBuild,Deploy,Manage multi-cloud AI Microservices Deep Dive Session(after OCW)Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates57Request a deep-dive for your teamEmail:obaas_1.LRN3525 TxEventQ and AQ Update in Database 23aiBuilding Asynchronous AI Microservices -Sanjay

91、Goil&Mahesh Girkar2.THR3327 0 to Prod in 30 Minutes with Spring Boot Microservice on Oracle Converged Database-Mark Nelson&Anders Swanson 3.PRSH3011 Develop Microservices with Oracle DatabaseBuild a Video Game and Online Store Paul Parkinson&Kuassi Mensah4.LRN1194 Oracle APEX with AIBuilding Next-Ge

92、n AI-Powered Apps Monica Godoy5.HOL2961 Use REST APIs for Data Consistency with Microservices Apps Brijesh Deo6.LRN3410 Getting Started with Kubernetes Operators for Databases Kuassi Mensah and Sanjay SinghCheck your schedule for exact day/timesAI Microservices Sessions at OCW24Oracle CloudWorld Copyright 2024,Oracle and/or its affiliates58Modern AI Apps Using Kubernetes,Microservices,and Oracle Database 23aiThank YouWei.HOracle CloudWorld Copyright 2024,Oracle and/or its affiliates59