利用 SLM 結合邊緣設備構建 AIoT Agent-盧建暉.pdf

1、利用 SLM 結合邊緣設備構建 AIoT Agent盧建暉 微軟高級云技術布道師模型發展模型繼續發展文本語音視頻圖片向量閉源模型 云端 開源模型 本地企業最終選擇Cloud LLMLocal SLMs混合模型GPUNPUCPU云+本地的算力架構微調微調RAG行業數據SLM什么是 SLM(Small Language Model)LLMSLM廠商的 SLM 角力模型選擇 LLM vs SLM性能1.更少的算力需求2.部署在更小的設備甚至邊緣計算場景上無障礙1.更多開發者和組織可以使用2.具備一定的業務能力，便于企業開發人員使用定制化1.針對特定領域和任務進行微調2.所有權不同的開源小模型應用場景

2、Qianwen-chatOpenELMGemma-2bPhi3知識/文本/聊天NanoDBjina-embeddings-v2-base-cn向量模型圖片，代碼LLaVAAudioCraftNanoVLM數據算法評估部署如何組織數據企業內部多個數據源的整理如何有效組織數據數據安全Qlora vs Lora參數調整計算本地或云端模型性能提示工程有效性效能存儲邊緣設備的部署模型壓縮回應迭代影響構建行業模型的四大要素Azure AIAzure AI Content SafetyAzure OpenAI ServiceAzure AI TranslatorAzure AI DocumentIntell

3、igenceAzure AI VisionAzure AI SearchAzure AI LanguageAzure AI SpeechAzure AI ServicesPre-trained,turnkey solutions for intelligent applicationsCutting-Edge ModelsAccess to the latest foundation and open-source modelsAzure AI StudioOne place for building and deploying AI solutionsResponsible AI Model

4、Fine TuningModel TrainingAzure Machine LearningFull-lifecycle tools for designing and managing responsible AI modelsPrompt FlowOrchestrationModel CatalogSolutionEvaluationModelBenchmarkingContinuous MonitoringCode-First ExperienceAzure AI InfrastructureState-of-the-art silicon and systems for AI wor

5、kloadsAzure Maia SiliconMicrofluidic CoolingHigh-Bandwidth NetworkingMicrosoft FabricUnified data platformLake House can build data based on businessCloud+local data integrationPrompt flowResult EvaluationCompare modelsbusiness flow integrationONNX RuntimeCross-platformResponseInt4,float32,float16 m

6、ulti-format compatibleAzure Machine LearningFull-lifecycle tools for designing and managing responsible AI modelsPrompt FlowOrchestrationDeploymentDatastoreModel CatalogComputeNvidiaSLM+Azure MLSLM OpsSLMSLMF Fineine-tuning-tuningModelModeldatasetEvaluationDeploymentAzure Machine Learning/Azure AI S

7、tudioFull-lifecycle tools for designing and managing responsible AI modelsMicrosoft OliveMicrosoft OliveMicrosoft Olive是一款非常易用的開源模型優化工具，既可以涵蓋Generative AI領域的微調，也可以引用。只需簡單的配置，結合使用開源SLM和相關運行時環境（AzureML/本地GPU、CPU、DirectML），通過自動優化即可完成模型的微調或參考，讓您找到 最佳模型部署到云或邊緣設備。Windows AI Studio 上引入了 Microsoft Olivemicr

8、osoft/Olive:Olive is an easy-to-use hardware-aware model optimization tool that composes industry-leading techniques across model compression,optimization,and compilation.()Microsoft Olive makes SLMOps engineering easier for development teamsMicrosoft Olive-Making the SLMOps process easy混合解決方案混合解決方案

9、SLMSLMF Fineine-tuning-tuningModelModeldatasetEvaluationDeploymentSLMOpsComputeEndpointOlive.config as ServiceOlive.config Model Catalog Olive.config-Compute SettingsOlive.config Qlora&Lora Olive.config ConvertONNX Model Convert（Float16，Float32，Int4）Provides a base for the Copilot StackDemoRunning Phi3 in Jetson+Running Phi3 in AIPCRunning Phi3 in Mobile