AI for All Path towards an open AI Infrastructure.pdf

1、Engineering DirectorMetaOla TrudbakkenMetas OCP Engagement1Board Member3Steering Committees12Project Leaders146Specs Contributed20112012201320142015201620172018201920212022TripletRackDataCenterFreedom ServersBattery CabinetSpitfire Server(AMD)PowerSupplyWatermark(AMD)Windmill(Intel)Mezzanine Card V1

2、WinterfellKnoxOpen Rack V1Group HugMezzanine Card V2Open Rack V2Cold StorageMicro Server(Panther)LeopardBluRayWedgeHoney BadgerBig SurWedge 100YosemiteSix PackBackpackLightningBryce CanyonWedge 100SYosemite V2Tioga PassBig Basin100G CWSM4-OCPTwin LakeBig Basin V2OCP NIC 3.0FAV3MinipackOpen Accelerat

3、or ModuleMinilakeYosemite V3Delta LakeWedge 400Minipack 2400GOpen Rack V3Meta OCP Contributions2023Grand TetonWedge 400CCrate LakeZionAI-enabled creation toolsText-to-image generationsurrealist paintingLarge language models(LLMs)+173%ARTIFICIAL INTELLIGENCESource:Meta for Business.Culture Rising:202

4、3 Trends Report.2023.Conversation topic growth on InstagramMeta AI is used for diverse casesLlama European use casesARTIFICIAL INTELLIGENCEAutomotive sales assistantFully Ventures,a Germany based developer,has used Llama 2 as a conversational AI sales agent that provides a recommendation regarding t

5、he most suitable car to buy.Country:GermanyLive demo:fully.ventures/company/fordAnalysing customer complaintsRuter,a Norwegian transportation info app provider,co-owned by Oslo Municipality and a regional council,fine tuned Llama 2 to analyze customer complaints.Country:NorwayLive demo:Teaching abou

6、t LLMsSkoleGPT is an LLM developed for teachers to use in classrooms when teaching kids about AI and LLMs.The agent is in Danish and designed to be a safe and secure educational resource.Country:DenmarkLive demo:skolegpt.dk GenAI runs on Large Languages ModelsTotal Compute(PF/s)400Memory Capacity(TB

7、)10Llama-2 65B2023Training Scale(GPUs)4kand we are not done yet.Llama-22023Llama-32024and we are not done yet.Llama-22023Llama-32024Text1x TokensText7-8x Tokenstowards Multi-ModalityLlama-Next202xAudioImagesVideosLlama-22023Llama-32024Text1x TokensText7-8x Tokens6,000202216,0002023600,00020242024AI

8、Cluster Size 202620282030Number of connected accelerators10 xNumber of connected accelerators10 x1xReticle2x2.5D8x2.5D+3D512xScaleupAI Silicon Evolution20242030100s kW15kW Rackscale System DesignHigher TDP by Advanced PackagingTightly Coupled Scaleup DomainsServiceability&AvailabilityUnified Managem

9、entScaleup DomainsHeterogeneous HardwareWhile we expand our fleet,we will also need to support heterogeneous hardware Chassis&Rackscale Architecture Liquid Cooling Designs&Blind mate ConnectorsManagement,Tooling&Telemetry Time to Production AI Cluster Design RequiresStandardUniversalCommonTime to Pr

10、oduction AI Cluster Design Requires ModularitySwitch BankAccelerator Bank Accelerator Bank Rack managerORv3 RackPower ShelvesIntra-Rack ConnectivitySwitch BankAccelerator Bank Accelerator Bank Intra-Rack ConnectivitySwitch BankAccelerator Bank Accelerator Bank 200G CopperTime to Production AI Cluste

11、r Design Requires ModularitySwitch BankAccelerator Bank 2OU#44OU#44Accelerator Bank 1Rack managerORv3 FRONTPower ShelvesTime to Production AI Cluster Design Requires ModularityORv3 BACKInterconnectBackplaneManifold(In/Out)Bus BarSwitch BankAccelerator Bank 2OU#44OU#44Accelerator Bank 1Rack managerPo

12、wer ShelvesORv3 FRONTBlind Mate Quick ConnectStandardize Flow Rates 30 CLiquid Cooling Design+PowerAI System Hardware ManagementCommon rack level view for ease of management and future growthAccelerator management,telemetry and RAS featuresIncludes liquid cooling signal aggregation and leak detectio

13、n responseManagement Feature ParityMulti FRU CapabilitySystem ViewThe Infra for AI isSoftware FrameworksAI ModelsHardware SystemsAI MetaOPEN!The open model is workingSoftware Llama ModelsOpen AI SystemsRack CommonsData CenterCall To ActionManagementRack CommonsData CenterCall To ActionManagementOCP Open AI Systems InitiativeAn Interactive Supercomputer