NVIDIA 網絡打造超低時延證券解決方案.pdf

1、NVIDIANVIDIA網絡打造超低時延證券解決方案陳龍，高級市場開發經理2020.December#page#構建最快的交易系統低時延網卡低時延交換機connectsLinkyou toXmorel低時延線纜降低擁塞時延#page#時延分布ServerSwitchDOWN-ModuleQUEUEMAC+PHYCPUPassive Copper0.3nsDDRSerdes100-200nsDDR DIMTCP/IP5-15USActive Copper 1ns10-100GbpFiber 10ns-1Tbps2nsSwitch64-128Gbps200-400n510-100Gbp10-100

2、GbpainpowainpowMAC+PHY2nsPassive Copper 0.3nsPassive Copper0.3ns100-200nsSINSerdesActive Copper insActive Copper ins100-200nsFiber 10nsFiber 10nsPropagationLatency#page#高速轉發和時延關系（1）10G/25G/100G128GbpsStoreForward Latency100080600h4bb.h4002000212102425Gbps100Gbps10Gbps#page#高速轉發和時延關系（2）目由于在線交易時，最易發生的

3、“微爆流”，使網絡交換機由于擁堵延退傳輸甚至高達2毫秒Congestion_Latency(us）鎖剪麗領題物鋼加的住B88838BBR#page#三種低時延方案NVIDIAVMA-兼容性好平衡性高RoCEInfiniBand極致低時延#page#VMA硬件卸載降低轉發延時Standard ApplicationStandard Applicationwithout NVIDIA VMAwith NVIDIA VMA跳過Kernel層User ApplicationUser ApplicationUser SpaceVMA運行在用戶態，精簡了協議棧Socket APISocket API的處理

4、，帶來處理時延的大幅降低VMATCPUDPKernelBypassNIC Driver應用透明用于上層軟件開發不感知VMA，開發與TCP/IP無差別對端無需做任何修改即可通信#page#NVIDIAVMA-追求極致低時延KERNELVSVMACONNECTX-5UDP LATENCY蘇0Kernel2.943VMA1.0261432512102464173256sLatency Ping-Ping vs.#connections Multicast with VMA Socketxtreme1.6861.6631.6321.581501.361.3871.361.3331591.1311002

5、01010002030nVIDL#page#NVIDIAVMA-一舉多得對于金融交易系統，VMA帶來眾多的好處不僅使得時延大幅降低，而且穩定，同時釋放了寶貴的CPU資源，提升服務器的應用價值，葆護好用戶的投資LowerLower Latency80%LATENCY*LatencyLessThroughput85%r CPU utilizationLowerCORESLOWERLess90%Lower JitterJITTER*LOWEHGb/sHighest throughput100PER PORTHIGHERComparing to Kernel performance#page#ROCE

6、進一步釋放CPU資源Server-InitiatorServer-InitiatorServer-TargetServer-TargetApplicationApplicationbaforApplicationApplicationSocketsSocketsNIC DriverNIC DriverRoCERNICSRNICNICNIC武一市#page#ROCE保證低時延的同時進一步釋放CPU資源ROCEWithout RDMAWith RDMA and Offload25G100GRoCELatency天73253%CPU-88%CPUUtilizationUtilization27060

7、1747%CPU-12%CPUOverhead/dleOverhead/ldle1.20.70.8108Connectx-5B2BWrite#page#INFINIBAND打造最低時延INFINIBANDTRADEASSOCIATIOIHigh BandwidthLowLatencyS&PDOWJONESINDICESScalability/FlexibilityCPU OffloadsT_TYPICALUSECHVESimplifedManagementQuality ofservice256128512102420484096HDR B2BWritelatencynViO#page#SPE

8、CTRUMCUTTHROUGH架構確保時延穩定一致 Top of rack topologyTop of Rack Topology= 25 servers talking toeach other Spectrum: 300ns latency=Consistent cut-through latency25GbE Spectrum is the only low latency switch 10GServerLinks（SPl25GbE to 25GbE Latency Test Results3.334ns3500300082500ny2000FIFOLater150010005000

9、128MellanoxSpectrumCompetitionFrameSize（bytes）#page#SPECTRUM打造25G平臺最低時延交換機Lowest Port-to-Port LatencyLowest Packet Delay Variance Oitter）MELANOMELLANOXSpectrumSWITCH6ITCHSWITCHSWITCHSWITCHSWITCSWITCH1.5ndependent IEEE1588（PTP)Snake test#page#256字節以上時延全面優于友商產品NVIDIA SwichCompetitor switch3.rsn024RoCE

10、write測試包含交換機、網卡m#page#100G方案時延全面優于友商2920nsNVIDIA100GSwitchNVIDIA25GSwitchCompetitor 10G Switch2.34340ns210ns2.011.961.841.381.41.311.281.261.221.211.211.2163241282565121024#page#證券極速交易場景-以太方案10G10G/25G/100G以太交換機以太交換機交易所交易前置風險控制網關交易核心報盤電安BS181.5512-3-4-5一價VMA1000+3001000+3001000+300VMARoCE52004600RoC

11、E800+300800+300800+300#page#證券極速交易場景-INFINIBAND方案-融合方案InfiniBand交換機InfiniBand交換機以太交換機交易所交易前置風險控制報盤網關交易核心品量181.551-2-3一25-InfiniBand600+90600+90600+901000+3003370總時延（ns）#page#INFINIBAND可以避免流量突發造成時延劇烈抖動Congestion_Latency Comparation120010001000ethernet alliance800B%3PPP72660040200INFINIBANDOTAADEASSOC

12、IATIONCongestion_Latency(us）口10G-Eth口25G-Eth陽陽PP口100G-Eth口InfiniBand-EDR#page#單模光纖可以節省傳輸時延1150ns550s640ns600ns家樂藥220ns180nsLng5ns1005mwOgMnviDIA#page#測試環境10G交換方案訂單管理TCP/IP網關服務器F核心交易VRDMA報盤nVIDIA Spectrum SN2410TGW提合系統FT25G交換方案#page#客戶時延實測Other測試交易筆數NVIDIASave200081901825000187176118000197762110000193186Latency Comparison200251952019015185180101755170016550008000200010000口Cisco 93180SpectrumSN2410Save#page#測試結果對比10G交換方案訂單管理187-197us8網關服務器核心交易V3H176-186us報盤nVIDIA Spectrum SN2410TGW最合系統三0T25G交換方案24#page#NVIDIA