精益數據工廠：數據智能平臺的力量.pdf

《精益數據工廠：數據智能平臺的力量.pdf》由會員分享，可在線閱讀，更多相關《精益數據工廠：數據智能平臺的力量.pdf（20頁珍藏版）》請在三個皮匠報告上搜索。

1、2024 Databricks Inc.All rights reservedA LEAN DATA A LEAN DATA FACTORYFACTORYKythera LabsKythera LabsJune 2024June 20241Optimizing the Power of a Data Optimizing the Power of a Data Intelligence PlatformIntelligence Platform2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserv

2、ed2WE WE LOVELOVEDATADATAMATT RYANMATT RYANKythera Labs Co-Founder,CTOJOSUE BOGRANJOSUE BOGRANKythera LabsSolutions Architect Manager2024 Databricks Inc.All rights reserved Introductions Addressing the 7 Wastes in a Data Intelligence PlatformWhat is a Data Intelligence PlatformThe 7 wastes defined A

3、pplying these principles The Future Q&A3AGENDAAGENDA2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved4INTRODUCTION INTRODUCTION-MANUFACTURING ANALOGYMANUFACTURING ANALOGY2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved5WHAT IS A DATA INTELLI

4、GENCE PLATFORMWHAT IS A DATA INTELLIGENCE PLATFORMData Intelligence as Data FactoryData Intelligence as Data FactoryOur use of Unity Catalog Transport,Inventory,Motion,Waiting,DefectsBuilt-On Waiting,Production,OverprocessingOur Common Data Model/Analytics Assets Waiting,Overproduction,Overprocessin

5、g,DefectsServerless Waiting,Over Processing2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved6DATA INTELLIGENCE PLATFORMDATA INTELLIGENCE PLATFORMIs in a Cloud data centerIs in a Cloud data center62024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reser

6、ved7Not in a Cloud Not in a Cloud The big decision The big decision-Kytheras original data centerKytheras original data center72024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved8LEAN MANUFACTURINGLEAN MANUFACTURINGA Fundamental Concept In Reducing Waste in ManufacturingA

7、Fundamental Concept In Reducing Waste in ManufacturingLean manufacturing is a systematic approach to identifying and eliminating waste through continuous improvement.The primary objective of Lean Manufacturing is to enhance efficiency,reduce costs,and improve overall production quality.Waste in manu

8、facturing refers to any activity or process that consumes resources without adding value to the product or service.Excessive waste leads to increased production costs,longer lead times,and lower customer satisfaction.Recognizing and addressing waste is crucial for streamlining processes,maximizing p

9、roductivity,and maintaining a competitive edge2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved9THE 7 WASTES OF LEAN MANUFACTURINGTHE 7 WASTES OF LEAN MANUFACTURINGDate Intelligence as a Data FactoryDate Intelligence as a Data Factory9TransportOver Over ProcessingProcess

10、ingInventoryMotionDefectsOverInventoryMotionDefectsOverProductionProductionWaitingWaiting2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved10LEAN PROCESSING IN DATA FLOWSLEAN PROCESSING IN DATA FLOWSAddressing the 7 wastesAddressing the 7 wastes10Batch PipelineBatch Pipel

11、ineBuilt OnBuilt OnUnity Catalog/DistributionUnity Catalog/Distribution2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved11BATCH PIPELINEBATCH PIPELINEAirflowAirflow11TransportMotionDefects2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved12BUI

12、LT ONBUILT ONKythera WayfinderKythera WayfinderKythera spins up Workspaces on behalf of clients under Kythera AWS and DatabricksLeverage Unity Catalog for sharing of Kytheras large data setsAbility for clients to access/process data with a wide variety of industry tools(including their own Databrick

13、s)Unified cost reportingOverProductionOver ProcessingMotion2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved Each client has their own workspace and catalog Only have access to views to centrally stored tables-not to underlying tables Shared volume for access to Wayfinde

14、r features API Leverage Hive Partitioning/Liquid Clustering for efficient joining of client specific patient dimension13UNITY CATALOGUNITY CATALOGMultiple workspaces,central locationMultiple workspaces,central location13TransportInventoryMotionDefectsWaiting2024 Databricks Inc.All rights reserved202

15、4 Databricks Inc.All rights reservedLanguage FlexibilityOrganized Complex CodeFamiliar Modern Coding Experience14Traditional SQL ExperienceEnhanced CapabilitiesData Exploration with EaseLeverages the SQL Serverless WarehousesCODE EDITORSCODE EDITORSMinimizing defects and motion across diverse coding

16、 environmentsMinimizing defects and motion across diverse coding environments14NotebooksNotebooksSQL EditorSQL Editor2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedAllAll-Purpose ComputePurpose Compute15COMPUTECOMPUTESQL Without Management OverheadFast Performance,4 Se

17、cond StartupLow Cost For Recurring WorkloadsVersatility&InteroperabilityScaling Out&Scaling In As NeededReducing waiting,overproduction,and excess processing in compute.Reducing waiting,overproduction,and excess processing in compute.15Job ComputeJob ComputeSQL Serverless WarehousesSQL Serverless Wa

18、rehouses2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reservedEasy to UseHigh-Impact16Chat-Like InterfaceEasy Debugging Enriched By Unity Catalog MetadataAIAICutting transportation,motion,and over processingCutting transportation,motion,and over processing16SQL AISQL AIDatabr

19、icks AI AssistantDatabricks AI Assistant2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved Embedded Dictionaries With Comments Enriched Data Experience in DatabricksUnity CatalogUnity Catalog17Schedule&File-Arrival TriggersCost SafeguardsTask DependenciesPipeline Traceabi

20、lityGOVERNANCE&ORCHESTRATIONGOVERNANCE&ORCHESTRATIONOptimizing inventory while decreasing overOptimizing inventory while decreasing over-processing,motion,and defectsprocessing,motion,and defects17WorkflowsWorkflows2024 Databricks Inc.All rights reserved2024 Databricks Inc.All rights reserved Kyther

21、a features in workspaces outside of Built On(Python in Serverless)More and more data!Sigma dashboards Databricks default storage Data rooms and AI libraries18THE FUTURE(COMING SOON)THE FUTURE(COMING SOON)2024 Databricks Inc.All rights reserved19RESOURCESRESOURCESImproving Data Quality Through Improv

22、ing Data Quality Through the Medallion Architecture the Medallion Architecture Serverless SQL Compute:Time Serverless SQL Compute:Time Is Money In a Very Real WayIs Money In a Very Real WayTame the Costs of Cloud Tame the Costs of Cloud Computing With the Right ToolsComputing With the Right Tools2024 Databricks Inc.All rights reserved20Thank YouThank You