用于任務關鍵型邊緣 AI 的 MENTIUM 混合計算與 SYNOPSYS 合作.pdf

1、Mentium Technologies All Rights Reserved 9/13/20241 2024 Mentium Technologies Inc.MENTIUM HYBRID COMPUTATION FOR MISSION-CRITICAL EDGE AI,PARTNERING WITH SYNOPSYSMirko Prezioso,Co-founder&CEO,Mentium TechnologiesSeptember 10,2024 2024 Mentium Technologies Inc.Mentium Technologies History Spin-off fr

2、om UCSB In 2015,we demonstrated the first DNN in-memory computing in an integrated array.Mentium,an AI Co-processors Startup,was founded in 20172 2024 Mentium Technologies Inc.Cloud-level AI on Edge Systems for mission-critical AI applicationsSub 1W regionHybrid architecture allows 10 x larger model

3、s AI capabilities(accuracy,complexity,reliability)Enhancing existing systems by easy integration No competition with SoC manufacturersSaves customers millions of dollars to re-use existing system softwareRad-hard option developed with NASA no other solution Military&Police BodycamsHigh-EndSmart Secu

4、ritySpace EdgeRoboticsADASAR/VRGenAIMobileExpanding toDronesMentium Today3 2024 Mentium Technologies Inc.References:a)COCO test-dev Benchmark(Object Detection)|Papers With Codeb)docs/en master weixin_47013828/myrep GitCodec)How Hugging Face achieved a 2x performance boost for Question Answering c)wi

5、th DistilBERT in Node.js The TensorFlow Blogd)1810.00736(arxiv.org)a)c)d)Neural Network Capacity“Inference quality”is proportional toNeural Network sizeAI Inference is only as good as your Neural Networka)b)d)4 2024 Mentium Technologies Inc.Analog in-memory computing is greatirrriiLarge DNN model is

6、suesAnalog in-memory advantagesSolved?On-chip storageGreat storage density advantage over SRAM Memory BottleneckStorage and computing in the same physical device ThroughputMassively parallel So,problem solved,right?No!neuroni+wi1wi2wijri1i2i3xxx5 2024 Mentium Technologies Inc.but not perfect.SRAMPro

7、cessor(s)Digital ProcessingAnalog in-Memory ProcessingData transfer between memory and processing:Memory bottleneckirrriiA to DD to AData conversion between D to A and A to D:Conversion bottleneck!Both Digital and Analog in Memory Computation have Bottlenecks6 2024 Mentium Technologies Inc.hybrid is

8、 best!DigitalProcessorLayer InputKernelLayer InputXKernel=OutputAnalog In-MemoryLayer InputKernelOutputOutputCONS:Lot of time/energy spent on Kernel transfer:Memory BottleneckPRO:No ADC and DAC overheadCONS:Lot of time/energy spent on D/A/D conversion:Conversion BottleneckPRO:No time or energy spent

9、 on Kernel transferDNNs operation(VMM):7 2024 Mentium Technologies Inc.DigitalProcessorLayer InputKernelLayer InputXKernel=OutputAnalog In-MemoryInputKernelOutputOutputCONS:Lot of time/energy spent on Kernel transfer:Memory BottleneckPRO:No ADC and DAC overhead,much higher freqBEST:Small Kernels,lot

10、 of repetitions CONS:Lot of time/energy spent on D/A/D conversion:Conversion BottleneckPRO:No time or energy spent on Kernel transferBEST:Large Kernels x Small inputs,few repetitionsDNNs operation(VMM):We use the best of both worlds!hybrid is best!8 2024 Mentium Technologies Inc.Example:Lets look at

11、 Resnet 501101001000100001000000.0010.010.11101001 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53RepetitionsInput size/kernel sizeLAYERInput size/kernelrepetition and repetitions vsLayerDigitalProcessingAnalog In-MemoryProcessingI/K ratio&RepetitionsSmall Kernelsmany repet

12、itionsKernel SizeLarge Kernelsfew repetitions9 2024 Mentium Technologies Inc.Unique Hybrid ArchitectureFast Accurate EfficientDVMAVMDigital Computing(DVM Core)10X better than any other digital design Co-designed with AVM On par with AVM efficiency and speed Best solution for sparsely connected layer

13、sAnalog in-memory computing(AVM Core)No more memory bottleneck 40 x storage density advantage Massively parallel Unmatched on densely connected layersBased on two equally important partsResult:AIM architecture,a co-designed synergistic system Delivering best-in-class efficiency at high throughput wi

14、th large DNNs10 2024 Mentium Technologies Inc.Products road-mapSimple Bolt-on solutions.Complete and expand AI capabilities of existing systems by helping their host processors:Easy hardware and software integration across multiple platforms and BUs Re-use of customers DNN deployment toolchain11Plea

15、se feel free to contact Mirko Prezioso,CEO if you are interested in our products and/or our Roadmapmpreziosomentium.tech 2024 Mentium Technologies Inc.Software integration pipelineModel TrainingPyTorch,TensorflowDatasetONNX modelDatasetsubsampleMentium CompilerPerformance ProfilingHardware-aware qua

16、ntizationAnd optimizationOptimized modelTraining and compiling DNNsTesting DNNs on Dev kitm.2 PCBwith Mentium Contributed APIsMentium APIs+DeployingOn Mentiums equipped ProductsTested model12 2024 Mentium Technologies Inc.Synopsys Cloud for Chip DevelopmentChallengesEDA execution unoptimized due to

17、hardware and OS configured for multipurpose useInstabilities due to HW bottleneck when critical EDA tool flow stages require extra resources(memory,CPU)Resource conflicts in multi-user environment as team expandedServer maintenanceNetwork infrastructure downtimeChallenges with EDA software installat

18、ion and configurationBenefits of moving to Synopsys Benefits of moving to Synopsys Cloud Cloud SaasSaasSetup and operational in 2 days Optimized environment for EDA workloads(compute/storage)Flexible,scalable,self-serve license management(on-demand tool access)Streamlined onboarding and EDA tool sup

19、port Built in job scheduler for effective resource management13First iteration of Mentium DVM accelerator was done using Synopsys EDA products on Mentiums on-prem servers 2024 Mentium Technologies Inc.Synopsys Cloud SaaS:End-to-End SoC Design EnvironmentMentium teamRole BasedAccess ControlBROWSERorC

20、ITRIX CLIENTFILE TRANSFERPORTALUser Management-Compute Configuration-License Purchase-AnalyticsArchive and Transfer Project SSDPersistent Primary Compute ServerOn-Demand Compute ServerSubmit ServerJob SchedulingSynopsys Full Analog and RTL-to-GDS EDA toolsReference Design FlowsMixed 3/6/12 month and

21、 Pay-Per-Use Licensinggit,python,vscode&other open-source toolsSOC 2 Type IIAE Support 2024 Mentium Technologies Inc.Development and Production Board KitSoftware SDKHardware m.2 PCIe-PCBMentium Model OptimizerNeural Net AnalysisQuantizingCompilingProfilingPython APIsLinux Driver+Able to run main neu

22、ral networks benchmarks and calculate the power consumed.Can be connected to any platform running Linux through an m.2 socket or PCIe adaptor.Accepts standard AI-model file formats such as ONNX,Tensorflow,PyTorch+To request:productsmentium.tech or follow the link15Mentium Technologies All Rights Reserved 9/13/202416 2024 Mentium Technologies Inc.#1 IN MISSION-CRITICAL AI AT LOWEST EDGE POWERSIMPLE INTEGRATION,COMPLEMENTARY TO ALL SOCs&MCUsWrap-upUNIQUE HYBRID APPROACHMirko Prezioso mpreziosomentium.techLinkedIn: