Immuta：使用自動數據訪問控制構建端到端 MLOps 工作流.pdf

《Immuta：使用自動數據訪問控制構建端到端 MLOps 工作流.pdf》由會員分享，可在線閱讀，更多相關《Immuta：使用自動數據訪問控制構建端到端 MLOps 工作流.pdf（21頁珍藏版）》請在三個皮匠報告上搜索。

1、Building an End-to-End MLOps Workflow with Automated Data Access ControlsDatabricks2023AgendaBuilding an End-to-End MLOps Workflow with Automated Data Access ControlsData and MLOps Approach at WorldQuant PredictivePutting Data at the Center-Data Access Controls with ImmutaPutting it all together at

2、WorldQuant PredictiveWorldQuant Predictive IntroductionSignal Factory finds predictive signals in the data with networks of ensemble models.WorldQuant Predictive delivers ready-made predictive AI solutions,trained on the worlds data.New York,NY Global Team of Data Scientists&EngineersGlobal Research

3、 NetworkExpands our expertiseWhat We DoWho We AreWe scout&curate differentiated data,which are derived from public,commercial and non-traditional sources.Quanto,our AI platform,enables anyone to access the models to immediately predict outcomes,simulate scenarios and optimize decisions.WorldQuant Pr

4、edictive Data and MLOps Workflows OverviewData EngineersData AnalystsData ScientistsML Ops Business SMEsCustomersPersonas,Use Cases,Tech StackData IngestionData QualityData ExplorationData Governance/SecurityFeature ExtractionML Model ExperimentationML Testing/ValidationML DeploymentDatabricksSnowfl

5、ake+SnowparkImmutaMLFlowAirflowDBTGitWQPs BigFeat PersonasUse CasesTech Stack(Our Toolkit)Law of Conservation of Complexity:Removing complexity from user experience moves it to system setupOur ApproachWe want to hide complexity of environments and tools from usersKeep it simple-use consistent toolki

6、t and building blocksEverything as code-use Git as promotion mechanismTrust in policies to provide appropriate data when neededWorldQuant Predictive Data and MLOps Workflows OverviewIngest through Databricks+ImmutaTransform with DBTExplore data with SnowflakeTrain in DatabricksStore models in MLFlow

7、Serve models in AWSPromotion of code from Dev to ProdData flow from Prod to DevSlicing of data access based on policiesProvide compliance and security Maximize productivityHide environment complexityEmpower self-serviceUsers CanSystems ManageOur Goals&ChallengesCloud Data Security Dilemma Speed and

8、InnovationvsInfosec and ComplianceIdealized Data Governance and AccessData Access is at the center of ML WorkflowsUsers see what they need via Data and Subscription PoliciesData sets are updated with new data safely and easily through Schema EvolutionCentralized audit of policy,access,and changesusi

9、ng DetectPolicies are applied across disparate platforms without redundancy with ImmutaPersonas+Purpose+Policies Data EngineersData AnalystsData ScientistsML Ops Business SMEsCustomersOrder or Chaos?Data IngestionData ExplorationML Model ExperimentationML Testing/ValidationML InferenceNo PII and No

10、PHI by defaultNever share holdout set to data scientistsUse fuzzy/synthetic data during Data ExplorationRemove access to expiring data sourcesPersonasPurposePoliciesSpeed of access is EMPOWERED by policy enforcement Immuta Policy as CodeCreate rulesets for each data source Create complex rules often

11、 as custom viewsLimit access until absolutely neededCreating a centralized process for managing organizations policiesGroup data sources by project and apply policies at onceExpress policies in intuitive policy builderApply policies on sensitive or confidential data by default across all data source

12、sCodify your access control policy configurationManage governance policies across availability zones,regions,and cloud data platformsCreate auditable,reproducible governance configurations,allowing change management,rollbacks,and blue and green testingTraditional SolutionsWith ImmutaAdd Policy as Co

13、deImmuta Policy as CodeOne common code-drive repositoryManage enterprise-wide policies and complianceIntegrate with DevOps and DevSecOps toolchainPutting it all together in QuantoQuanto for Consumer GoodsMitigate uncertainty,tap into competitive insights&intelligently adapt your strategyimmediate pr

14、edictionsValuable insight into how to grow vs.your competitors on Day 1-no data or advanced analytics resources required.persistent accuracyRegain control with accuracy thats driven by premium syndicated data and powerful models that are constantly refreshed.empowered decisionsDont just see the futu

15、re,but discover the keys to reaching your goals with industry-leading simulation and optimization Databricks ML Flow+DBT&Airflow+ImmutaWorldQuant Predictive Data and MLOps Workflows OverviewUse the same blocks and approach for Data Science and Data Analytics WorldQuant Predictive Data and MLOps Work

16、flows OverviewUse the same blocks and approach for Data Science and Data Analytics Final ThoughtsLeveraging consistent building blocks and patterns to simplify our environment and toolkitTreating everything as code helps with reliability and repeatability across environmentsData is everywhere,and so is the need data governance and protectionBecome an expert in prediction accuracy!All The Ways To Learn More About WorldQuant PredictiveUseful ReferencesImmutahttps:/ Predictivehttps:/