用 NVIDIA Jarvis 來構建你的語音助理.pdf

1、CONVERSATIONAL AI IS TRANSFORMING INDUSTRIESVIDEOCONFERENCECALL CENTERSMARTSPEAKERSRETAILASSISTANTSIN-CAR ASSISTANTSCC,TRANSLATIONTRANSCRIPTION500M Calls per Day75M NewCars par Year12M Retail Stores150M Sold perYear200MMeetingsporDay2百KALDOSfnstiS#page#THE RACE TO CONVERSATIONAL Al0.5 sec on GPU“wha

2、t is the best Greek100001restaurant nearby？”“KokkariSpeech RecognitionLanguageModelingWav2letterBERT14sec onCPUNVIDIA Jasper（ms）VoiceEncoderText to Speechav2letteWavenetTacotron2NVIDIAWaveglowTacotron2#page#1.Introduce Our Use Case#page#PROBLEM STATEMENTSolve a Real Problem with Conversational AlRea

3、ltime video conferencing application for telemedicineCustomer feedback: challenging to remember everything discussed with physicianCan we highlight key information for the participantsConnection口6YourcameraPeercameraYourID:7894Yournaetodisplay38SPeor IDto callCall#page#JARVIS CONTACT EXAMPLEPeer-to-

4、Peer Video ConferencingAuthorizationAuthenticatonAeryidcochaSession ManagemenAnalyticsPePeer-to-peenegotiatioWebCentChat Demo Serven#page#Simple UI Code for our Baseline ApplicationWeb-based Video Calling AppSimple Node-based application serverResponsible for:hosting assetsestablishing peer-to-peer

5、connectionOne HTML pageOne client-side JavaScriptOne server-side JavaScript#page#2.Building a baselineappplication using Nvidia Jarvis#page#WHAT IS JARVISA Platform to Develop and Deploy Al for Multi-Modal Conversation Al Interaction at ScaleGesture，Image 8 SpeechGaze f Speechhttps:/ AND CLIENT LIBR

6、ARY DESIGNPython client library wraps all gRPC communicationFully asynchronous and streamingy uses modern idiomatic Python constructsAL computation runs as microservice in a container- you choose where to hostUses Transfer Learning Toolkit CTLT） for customization and deploying a custom model as aser

7、vice#page#CHALLENGESBarriers to Integrating Conversational AlDeploymentReal TimeMultiple sensorsCustom modelsHigh accuracyVarying latency andCustomization can bememory footprintsDifficult to useaudio，limited orcostlyLow latency requiredacross multiple serversNeed state-of-the-artvideo，andtextData so

8、vereigntyfor natural interactionsEfficiently deployingalgorithms and modelsefficiently for naturalconcernsandscaling tosupportinteractionsmany clientsnvIDL#page#JARVIS BENEFITSDeploymentReal TimeMultiple sensorsHigh accuracyCustom modelsSimplify fusion acrossStart from a baseMicro-service approachBe

9、st-in-industryEnd to end inferencemodalities，i.e.model and train withdesigned for K8salgorithmson GPUS optimized forspeech，vision andyour data on yourSimple APls easy toDirect access tominimal latencytext， for more naturalinfrastructureintegratecutting-edge researchinteractionsnVIDL#page#JARVIS FEAT

10、URES 8 PRE-TRAINED MODELSNLPASRSLLNamed Entity RecognitionBatch ModeBatch ModeDomain ClassifierStreaming ModeStreaming ModeIntent and SlotPunctuationModelsslapowQuestion-AnsweringJasperTacotron2QuartzNetWaveglowModelsBERTMegatron#page#START JARVIS LOCALLY FOR DEVELOPMENTgRPC endpoints available out

11、of the box with 3 commands in less than an hourConfigure DeploymentLaunch Example ClientsStart Servers1.Select models and server1.Launch containerLaunch Jarvis Skillspreferences（config.sh）（./jarvis_start_client.sh）2.Download，optimize，andserver2.Test example clientsprepare models（./jarvis_start.sh）3E

12、xplore notebooks（./jarvis_initt.sh）#page#Demo - Jarvis Setup#page#JARVIS CONTACT EXAMPLEPeer-to-Peer Video ConferencingAuthorizationAuthenticatonAeryidcochaSession ManagemenAnalyticsPePeer-to-peenegotiatioWebCentChat Demo Serven#page#JARVIS CONTACT EXAMPLEPeer-to-Peer Video ConferencingStreaming ASR

13、Jasper/uartzNeservicecallstreaming ASRAerDoarmapecificNLP service calyidcochatAnalyticsNamed Entity Recognition0U02D=NVIDIAJarvisSkillsCallnegotiatoauthentcation，eWebCentChat Demo Serven#page#INTEGRATE JARVISClient:Send audio to serverCreate location to display transcript/other metadataServerImport

14、Jarvis APl using Protocol BuffersWrite basic convenience functions around APIMediate data to/from client and send to Jarvis functions#page#Demo- BaselineApplication withJarvis#page#3.Fine-tuning models forimproved performance#page#TRANSFER LEARNINGA key component of the many SOTA ASR and NLP recipes

15、Natural Language ProcessingAutomatic Speech RecognitionTask1Task2Jasper Decoder Vocab1LanguageModelSpeech Encode（e.8.BERT）（e.g.QuartzNetJasper Decoder Vocab2TaskNModularity is natural for transfer learningnVID#page#FINE-TUNING JARVIS MODELSTLTNEMOhttps:/ Transfer Learning ToolkitOaStrtedNVIDIA NeMoI

16、ntroductionNeMotoclktmakesitpossilefoMJCSDR2WIDASTesorCosadsleRequiemsNMswh1.Pythcn3.6or372Pytorch1.6crabowenVID1#page#Demo- Fine tuneddomain specificappllication Jarvis#page#4.Deploy to Production#page#DEPLOY MODELS AS REAL-TIME SERVICESOne Click to Create High-Performance Services from SOTA Models

17、JARVIS SERVICESNLUDeploy models toservices in the cloud，dataVisioncenter，and at the edgeSpeechSingle command to set up and run the entire Jarvis具One click deployment駕application through Helm charts on Kubernetes clusterTensorRTCustomization of Helm charts for your setup and useServcaseJanisAPI Serve

18、rNLU&w8SHelmcommand to deploy models to productiior#page#CLOUD DEPLOYMENT FOR PRODUCTIONDownload Helm ChartInstall on KubernetesConfigure DeploymentModifyvalues.yamlhelm fetch ngc/jarvisswith your custom models andhelm install jarvis-apideploymentspecifics#page#Demo- Deploy to Production#page#JARVIS

19、 PUBLIC BETA#page#Jarvis Public BetaJarvis Public Beta will be available for access Early 2021Your early access application wil automatically transition to Jarvis Beta and you will receive an emailonce available for downloadResource Links:Jarvis Product pageJarvis Intro BlogNvidia Conv AI Youtube Playlisto GTC Sessions:口 Building Your First Conversational Al Appm Jarvis: The NVIDIA Platform for Conversational AlTLT Product page#page#Thank youl