SNIA-SDC23-Vyas-Mesnier-Hardware-Accelerated-Data-Integrity-Check-on-a-CSD.pdf

1、1|2023 Intel Labs&Solidigm.All Rights Reserved.Virtual ConferenceSeptember 28-29,2021Hardware Accelerated Data Integrity Check on a CSDVinit Vyas,SolidigmMichael Mesnier,Intel Labs2|2023 Intel Labs&Solidigm.All Rights Reserved.DisclaimersAll product plans,roadmaps,specifications,and product descript

2、ions are subject to change without notice.Nothing herein is intended to create any express or implied warranty,including without limitation,the implied warranties of merchantability,fitness for a particular purpose,and non-infringement,or any warranty arising from course of performance,course of dea

3、ling,or usage in trade.Contact your Solidigm representative or your distributor to obtain the latest specifications before placing your product order.For copies of this document,documents that are referenced within,or other Solidigm literature,please contact your Solidigm representative.All products

4、,computer systems,dates,and figures specified are preliminary based on current expectations,and are subject to change without notice.Solidigm.“Solidigm”is a trademark of SK hynix NAND Product Solutions Corp(d/b/a Solidigm).“Intel”is a registered trademark of Intel Corporation.Other names and brands

5、may be claimed as the property of others.Some results have been estimated or simulated using internal Solidigm analysis or architecture simulation or modeling,and provided to you for information purposes only.Any differences in your system hardware,software or configuration may affect your actual pe

6、rformance.3|2023 Intel Labs&Solidigm.All Rights Reserved.OutlineUse case description Why use CS?Software stack CS implementation Distributed processing and scalabilityFuture workConclusion4|2023 Intel Labs&Solidigm.All Rights Reserved.Use Case and Problem Statement2023 Solidigm,Flash Memory Summit.A

7、ll Rights Reserved.Data integrity check is:Compute intensiveRead intensive(SSD&PCIe bus)Memory intensive(host)Not scalable5|2023 Intel Labs&Solidigm.All Rights Reserved.Why Use CS For This Use Case?Off-load the hostThe host is only interested in the data integrity check resultsReduces PCIe trafficNo

8、 need to consume bandwidth and power to move the raw data to the hostReduces host memory footprintAll data required for processing is contained in the driveScalable with storagePerformance increases as drives are added2023 Solidigm,Flash Memory Summit.All Rights Reserved.6|2023 Intel Labs&Solidigm.A

9、ll Rights Reserved.Software Stack2023 Solidigm,Flash Memory Summit.All Rights Reserved.Application LayerDevice DriversFilesystem Layer(ext3,ext4,xfs,)File:“vid.mpg”NVMeVendor Unique(VU)Read Cmdwith LBA listObject Storage(MINIO,CEPH,SWIFT,)NVMeTCP/IPData integrity validation of the object shard is de

10、coupled from data transfer.Data integrity hash calculation is done by the CSD.The object storage node validates the result.CSDShard 1Shard 2Shard 3012345678910 11121314 15LBA list7|2023 Intel Labs&Solidigm.All Rights Reserved.CSD Implementation2023 Solidigm,Flash Memory Summit.All Rights Reserved.Fi

11、le:vid.mpgLBA listSize 10 MBNANDFlashNVMeHW accel.(CRC32/64,SHA256,etc.)buffercomputeVendor Unique(VU)Read Cmdwith LBA list1:VU Read Cmd3:Read from NAND to buffer4:Data integrity acc.on NAND data cmd5:Data integrity result transfer6a:Data integrity result added to VU cmd completionNVMe plane2:NAND R

12、ead Cmd:LBA list6b:Deallocate bufferApplication LayerDevice DriversFilesystem Layer(ext3,ext4,xfs,)Object Storage(MINIO,CEPH,SWIFT,)NVMeTCP/IP8|2023 Intel Labs&Solidigm.All Rights Reserved.CSD Implementation Gen5 off-the-shelf product Single ASIC controller Low cost High energy efficiency High perfo

13、rmance Off-the-shelf NVMe driver Ready to support TP4091&TP41312023 Solidigm,Flash Memory Summit.All Rights Reserved.9|2023 Intel Labs&Solidigm.All Rights Reserved.Performance and Scalability(1)2023 Solidigm,Flash Memory Summit.All Rights Reserved.Presented by Intel Labs at FMS 202210|2023 Intel Lab

14、s&Solidigm.All Rights Reserved.Performance and Scalability(2)2023 Solidigm,Flash Memory Summit.All Rights Reserved.Modeling results indicate a high degree of scalability ideal for CPU offload*Based on Solidigms internal analysis.CSD theoretical performance based upon modeling.11|2023 Intel Labs&Soli

15、digm.All Rights Reserved.Future WorkAlign with NVMe TP4091 Enhance implementation to leverage the Computational Programs Command Set TP4091 commands can activate and execute the data integrity checkAlign with SNIA CS API Leverage the SNIA CS API to standardize the user libraryIntroduce dynamic resou

16、rce allocation Leverage existing FW architecture and CSD programming model2023 Solidigm,Flash Memory Summit.All Rights Reserved.12|2023 Intel Labs&Solidigm.All Rights Reserved.Dynamic Compute Resource Allocation 2023 Solidigm,Flash Memory Summit.All Rights Reserved.HostResource AllocationCS Resource

17、 FactorSSD ResourceFactorResource Adjudicator(FW/OS)NVMe Admin Cmd (host hint)CSD Compute Resources ThrottlingSSD FWNANDFlashHW accelerationbuffercompute1:VU Read Cmd3:Read from NAND to buffer4:Data integrity acc.on NAND data cmd5:Data integrity result transfer6a:Data integrity result added to VU cm

18、d completionNVMe plane2:NAND Read Cmd:LBA list6b:Deallocate buffer13|2023 Intel Labs&Solidigm.All Rights Reserved.Conclusion Computational Storage is ideal for processing meta-data tasks on-driveUtilizes existing HW accelerators and SW solutions,no new work requiredOperates on SHARDED dataMajor valu

19、e add to customers concerns of data localityScales across multiple CSDsWorks independently,but brings overall increased performance to systemOur PoC demonstrates linear scaling performance with additional drives This use-case is adaptable to the latest Computational Storage standardsCan become fully

20、 compliant with TP4091/4131 and Architectural and API specsDoes not restrict Host from using resources for other Computational work on drive2023 Solidigm,Flash Memory Summit.All Rights Reserved.14|2023 Intel Labs&Solidigm.All Rights Reserved.Please take a moment to rate this session.Your feedback is important to us.