NVIDIA 助力構建安全和高效的推薦系統.pdf

1、RECOMMENDERS - THE ENGINE OF THE INTERNETBilions of Users- Trilions of Items100Sof millions of etail items Amazon 8 Alibabarecommenders10003 of movies Netflix recommenderslapuauuoal saunll B AyHods sSuos Jo suoyu Jo S.0L10s of millions of books - Amazon recommenderBillions of Tik Toka YT videos- TT&

2、 YTrecommenderBillions of websites Google search rankSo much news!Il s Google 8 FB news recommendersShMDL#page#DEEP LEARNING RECOMMENDERS ARE A DIRECT PATHTO INCREASED REVENUESGather DataModel ComplexityTraining FrequencyClick-Through-RateRevenueLARGER DATASETSDEEP LEARNING MODELSFRESHERMODELSBETTER

3、 RECOMMENDATIONSIMPROVEOUTPERFORMINCREASESIMPROVESRECOMMENDATION QUALITYTRADITIONAL METHODSPROBABILITY OF PURCHASEENGAGEMENTAND RETENTIONnWIDIA#page#INTRODUCING DGX A100The Universal AI System- Data Analytics， Training and Inference18o0NVSwcNVME SSD#page#NEW MULTI-INSTANCE GPU (MIG）Optimize GPU Util

4、ization Expand Access to More Users with Guaranteed Quality of ServiceUp To 7 GPU Instances In a Single A100: DedicatedAmberSM，Memory，L2 cache，Bandwidth for hardwareQoS8isolationSimultaneous Workload Execution With GuaranteedQuality Of Service: AIL MIG instances run inPUGPUSTCCPUGPJparallel with pre

5、dictable throughput 8 latencyGPUMomGPU MomGPU MSmGPU MamGFUMmGPUMamGPUMomRight Sized GPU Allocation:Different sized MIGinstances based on targetworkloadsFlexibility to run any type ofworkload on a MGinstanceDiverse Deployment Environments: Supported withBare metal，Docker Kubernetes，Virtualized Env.S

6、ee MIG in action Running Al Inference and Mixed WorkloadsShMbr#page#NEED FOR INFERENCING AT THE EDGEREAL-TIMEAIAT THE EDGECORE DATACENTERBILLIONSOFSENSORSTHE NVIDIAEGXPLATFORMRETRAININGAND REPORTINGFromJe13INSICHTSANOMALYALER#page#MODERN APPS DEMAND ADVANCED NETWORKINGMonolithicArchitectureMicroserv

7、ice ArchitectureKubernetes typically runs modern workloads: data-driven，real-Data Access Layertime and highly distributedBusineLogcMicroservices run on multiple，arbitrary serversEach microservice runs multiple timesMicroservices generate intensive east-west data movementsHigh-throughput，low-latency

8、isimperativeMassivCommunicatio#page#DATA-CENTER SECURITY CHALLENGESThe Perimeter is BrokenLack of Visibility and ControlAttack SophisticationAttack SurfaceNew Cyber Regulation#page#EGX A100 Family Hardware Support for End to End SecuritySystem Level Security Benefits of IntegrationBenefits ofnVIDIAI

9、ntegrationDPURDMA and GPUDirectSecure ApplicationNew Security Engine forL4 FirewallConfidential AlSecure DataTLS Crypto EngineSecure/AuthenticatedSecure PlatformBootIPsec Crypto EngineHardware Root of Trust10nVIDIA#page#BLUEFIELD-2XDATA PROCESSING UNITAI-Powered DPU200 Gb/s BlueField-2 augmented by

10、Ampere GPUEnhanced the DPU with AI capabilitiesScale out computing performance with GPUDirect and CUDATighter security across the PCle busApply Al to real time network traffic-Anomaly detection f automatedresponseTraffic shaping/steeringDynamic security orchestration#page#INTRODUCINGNVIDIA DOCAINFRA

11、STRUCTURE APPLICATIONSData Center Infrastructure-on-a-ChipSoftware-definedSoftwaredefinedInfrastructureSoftware-definedStorageSecurityNetworkingManagementArchitectureDOCA SDKSDK for BlueField DPUsStorageSecurityNetworkingPDKDPDK/PASPDKOpen source APIs- DPDK,SPDK，P4TelemetryManaCRVPJOROTCertified ref

12、erence apps 8 3r party solutionsSupport for multipple OS#page#INTRODUCING THE DATA PROCESSING UNITSoftware Defined DataaCenterr Infrastructure-on-a-Chip委團ned5toragNVIDIA NICNVIDIA DPUwith Arm Cores f AcceleratorsiraleratonEnsiesTo Software Defined InfrastructureTo Software Defined InfrastructureFrom

13、 Hardwareon CPUon DPUAppliancesnVIDIA#page#SOFTWARE-DEFINED，HARDWARE-ACCELERATEDSoftware Defined NetworkingSoftware Defined StorageSoftware Defined Security（1161）二品VMsvRoutervSwitchNAT/LOaTelco/NFNVMe-oFDistributed IDS/IPSRootOJWWDDOSDataElasticCompressioDeDupContainersVStorage Direct EncryptionStor

14、ageSegmentation PreventioranNG FirewallOfBalancerTrustDPUDPiDPUDPUDPUDPUnviDIA#page#SECURITY BIGGER PICTUREModern Data Center煙nternetNorth-SouthxEdgeNorth-South3IDSNGFWAnti-MalwareVPN親Security Services8苗8話VMVMCTR CTRVMVMYMHG6APSmartNIVVSmartNICSmartNICSmartNICSmartNICSmartNIC6Storage1SmartNICEast-We

15、stBare Metal ServersPublic Cloud ServersWeb Servers15nVIDIA#page#DPU SECURITYIntegrated Security for modern data center needsSECUREDADVANCED L4-L7PROGRAMMABILTYCRYPTOHARDWARESECURITY8 ISOLATIONACCELERATIONNG stateful firewallHardened IsolationData-in-motion enc.Secure FW upgradeDeep Packet Inspectio

16、nData-at-restenc.Micro-SegmentationRoot-of-TrustPublic KeyAccelerationHostintrospectionProgrammable algo.Armtrust zonenVIDIA#page#BLUEFIELD-2 IS THE MOST SECURE DPUTrust Shifts to the DPUCPUDistrbutedRootofDDOSIDS/IPSMcroTrustNGFirewallPreventioatiorIsolated Security Control PlaneFullIsolationDeep P

17、acket Inspectionfrom theNetworkHostTrafficStateful FirewallInline Crypto AcceleratorsRoot-of-TrustGPUnVIDIA#page#BLUEFIELD-2 DPU SOFTWARE COMPONENTSBlueField-2 DPUBootloader-UEFl，ATF (Arm Trusted FW），ACPArmLinux Distro Centos reference drivers. Ubuntu commercial OSDrversOFEDMFTBluoField2utsDiag.utiM

18、ellanox Drivers: OFED driver ASAP2，NVME SNAPLinuxDistro(Centos.Ubuntu）DrsSecure Boot and Secure Firmware UpgradeBMBootloaderH3nACPIPXEBOOtNC-SBooROMred BootOpenBMC for BMC ManagementIPMPCle SwitchConnectx-6 DxNICFirmwreOpenBMCConnectX-6Dx firmware binary fileBlucField2 SmartNIC/Storage ControllerVIO

19、IUE#page#ADAPTERS SECURITY INNOVATIONHostProgrammabilityAdvanced SecurityArmwith IsolationRegular Expression Pattern MatchingCryptoAccelerationEngineHardware Accelerated Public KeySecured NICNVIDIAHardware Secured Key ManagementHardware Accelerated StatefulBLUEFIELD-2 DPUUp to 200Gbps Accelerated In

20、ineCryptographyAccelerated HardwareSteering andFilteringHardware Root-of-TrustTURN ZERO-TRUST TOHERO-TRUST19nviDIA#page#100G DPUIPSEC TCPSame performance as Plain Text with 1/3 of the commodity NIC CoresCPU UtilizationThroughput100TCP Plain TextIPsec SW & Commodity NICBlueField-29090908070565656#ofC

21、ores#ofCores#OFCore5品湖Commodity NIC used45coresoutof56cores403020100SW+Commodity NICBlueField-2TCP Plain TextBLUEFILEDncrypted Data20nViDIA#page#Security Must Be AcceleratedDenial of Service (DoS） and Firewalls= SW protection cant scale= HW acceleration to the rescueApplication throughput during DoS

22、 attack6.5Mpps attackAtackHwDosMitatApplication performance total collapse duringApplication performance unaffected duringattachattackHWmitigation solutionSoftware mitigation is worthless21nvIDIA#page#SECURITY IS MOVING FROM THE PERIMETER TO THE SERVERCloud ServerCloud Server8IDS陽WorkloadWorkloadWor

23、kloadpeo1MOMDevOpsDevOpsL4-L7 InspectionNGFWWorkloadWorkloadFirewall/ Micro-segmentation中WITH DPUEncryption （Software）HOUT DPUAnti-MalwareWorkloadWorkloadSoftwareSoftware DefineddefinedNetworking (SDN）Storage5D5）5olatioOptionalL4-L7 InspectionITOpsUSegmentationNGFW/ CrvptoSDNSDSNICS8S822nVIDIACore D

24、ata Center Perimete#page#BLUEFIELD-2 DELIVERS HIGHEST APPLICATION EFFICIENCYEquivalent additional CPUs to match a single DPUVIDEO STREAMINGIPSEC ENCRYPTIONMALWARE PATTERN MATCHINC5010X15XELASTIC BLOCK STORAGECLOUD OVERLAY NETWORKINGNG STATEFUL FIREWALL15030X2.5XX23nViDIA#page#SECURE CLOUD WITH THE B

25、EST PERFORMANCEShifting the trust to NVIDIA DPUBetter SecurityBetter PerformanceBetter TCOSmaller attack surfaceOpex savingsHardware acceleratedCapex savingsReal-time reaction to threatsInlinenetworking8storageaccelerationSecurity in every hostaworkloadSeamless integrativesecurityFulvisibilityTransp

26、arent cryptographynVIDIA#page#THE DATA CENTER IS THE NEW UNIT OF COMPUTINGAcceleratedDisaggregatedInfrastructure（ADI）Accelerated ComputingSoftware defined，GPU:Amachine learningHardware-acceleratedGpUcriticalforAlmachinelearningDPUessentialtodisagaregatNVIDIA Networking25nViDIA#page#REINVENTING THE DATA CENTER21st Century Unit of ComputingGPUDPUCPU26nviDIA#page#THANK YOUNVIDIA