如何創建、治理和管理數據產品.pptx

1、 Eckerson Group 2023 Twitter:eckersongroup Wayne EckersonPresidentEckerson GroupHow to Create,Govern,and Manage Data Products Eckerson Group 2023 Twitter:eckersongroup About Wayne EckersonBest in class thought leadership reports,articles,and webinarsStrategy and design work to create modern data pro

2、grams and platforms.Go-to-market services for software vendors.President,Eckerson Group30 year thought leader in data and analyticsSought-after speaker and consultantAuthor of hundreds of articles and reportsAuthor of two bestselling books Eckerson Group 2023 Twitter:eckersongroup Data GovernanceDat

3、a StrategyModern Data ArchitectureSelf-Service AnalyticsData Literacy DataOpsData ScienceData Program DesignConsulting AreasEckerson Group ConsultingCarlos BossySenior Consultant Data ArchitectDave WellsSenior ConsultantData ArchitectSean HewittSenior Consultant Data GovernanceKurt ThearlingSenior C

4、onsultantData ScienceJosh Reid Senior ConsultantData TransformationRich FoxSenior ConsultantAnalytics ExecutiveWayne EckersonPresident200 years of data and analytics experience Eckerson Group 2023 Twitter:eckersongroup Consulting Clients Eckerson Group 2023 Twitter:eckersongroup Data Products are Ho

5、t!They solve three main problems plaguing data users:1.Users cant find the data2.Users cant access the data 3.Users dont trust the data4.Users cant share the data Eckerson Group 2023 WHAT IS A DATA PRODUCT?DatasetDashboardSQL queryCodeML modelData componentsAnalytic componentsYES!Eckerson Group 2023

6、 TARGET USER TYPE OF DATA PRODUCTData Engineer/Developer Reusable code componentsData Analyst Reusable queries,metrics,and data setsData Scientist Reusable features,models,and data setsBusiness Analyst Standard metrics,dimensions,cohortsBusiness Executives Standard dashboards&reportsCustomer or Part

7、ner Product dashboards and data setsTarget Audience Eckerson Group 2023 DashboardCodeData AssetsData ProductsDefinitionKey Characteristics Targeted to an audience Designed for reuse Designed for broad sharing Packaged with rich metadata Contains a real or implied“contract”Systematically governed Man

8、aged by a product team Continuously funded Continuously improved Published in a data marketplace Eckerson Group 2023 Lets Be Clear!What a Data Product is Not A one-off project Built by a solo developer For an individual And one-time use Without much oversight Or data governance.What a Data Product I

9、s A continuously funded program Built by a team of developers For a target user group That is continually enhanced With product manager oversight And rigorous data governance Eckerson Group 2023 Twitter:eckersongroup The State of Data Products26%31%43%We are trying to understand themWe are trying to

10、 implement themWe have already implemented themWhat is the status of data products in your organization?74%From Wayne Eckerson,“Data Products Revolution,”Eckerson Group,2024.)Eckerson Group 2023 Twitter:eckersongroup Drivers of Data Products69%63%51%37%26%20%3%20%Need for trustworthy dataNeed to sha

11、re data internallyNeed for reusable componentsNeed to share data externallyDelivery of a commercial productData mesh implementationWere not implementing data productsOther(please specify)What prompted your organization to adopt data products?From Wayne Eckerson,“Data Products Revolution,”Eckerson Gr

12、oup,2024.)Eckerson Group 2023 Twitter:eckersongroup Challenges Implementing Data Products57%54%37%37%34%29%29%29%23%17%17%11%6%6%6%Creating a data product mindsetDefining data productsPromoting data productsGaining adoption for data productsGoverning data productsDeveloping data productsCreating a d

13、ata product organizationScaling the initiativeSustaining fundingCreating and enforcing contractsRecruiting/training data products managersSecuring fundingSharing data productsDoes not applyOther(please specify)What challenges have you faced implementing data products?From Wayne Eckerson,“Data Produc

14、ts Revolution,”Eckerson Group,2024.)Eckerson Group 2023 The Role of a Data MarketplaceData Market 3.Sub-scriptions3.Defines who can consume what products,for how long,at what frequency,and through which channels and delivery methods.2.Defines upfront who can view,access,and subscribe to data product

15、s.2.Access Control1.Search&Sample1.Makes it easy for data consumers to browse,search,and evaluate relevant content.6.Auto-Delivery6.Supports automatic delivery to a desired target via a preferred channel in a specified format.5.Charge ($)5.Specifies the price or cost(i.e.,chargeback)of the product,i

16、f any.4.Terms of useand service (contract)4.Defines what consumers can do with the product;and what producers must deliver.A data marketplace unlocks the true value of data productsAll large organizations will have a data marketplace in 3-5 years.Automates frictionless data sharing Eckerson Group 20

17、23 And Offers Rich Additional ValueFor Product Owners:Saves time fulfilling manual requests for data assetsFor Business Executives:Creates a culture of data sharing and analytics.For Data Leaders:Bridges data silos and turbocharges data governance.For Product Consumers:Saves time waiting for permiss

18、ion to access data assets;guarantees data quality.For Product Managers:Makes it easy to publish,track,and retire data products.DATA PRODUCTS+DATA MARKET Eckerson Group 2023 Data Product Platform Connectivity Connect,Access,Query,Ingest,Stream,Store Transform Clean,Master,Deduplicate,Transform,Model

19、Orchestrate/ObserveSchedule,Test,Execute,Monitor,Alert,RemediateData&Metadata GovernanceDATA MARKETPLACE Data Assets With Descriptions,Schema,Statistics,Quality,Lineage Product StudioProduct Store Create products Publish products Define access/terms Track usage Field questions Browse products Subscr

20、ibe to products View terms Sample/manipulate data Ask questionsData ExchangeSource Systems and Data Data ConsumersProduct ManagersData DevelopersTarget Systems Many data products will be virtual.Data FabricData Product GovernanceBuild,Buy,or Mix&Match Eckerson Group 2023 Whos shipping a DPP?Data Pro

21、duct PlatformData IntegrationInformatica,Nexla,RightData Data CatalogsZeenea,Alation(soon)Data CommerceHarbr Data,Revelate,Narrative.io,Dawex Data AnalyticsCoginiti,PromethiumMaster DataTamr,Informatica Eckerson Group 2023 Data Catalog vs Data Product PlatformData CatalogData Product PlatformContent

22、Data assetsData productsBusiness purposeDiscovery&governanceData sharingTarget usersData analysts,stewards,developersData producers&data consumersContent ManagerData curatorProduct managerUser scopeInternal Internal&externalData accessUpon requestPredefinedMetadataBusiness&technical Plus,subscriptio

23、ns,terms of service&usage,delivery methodsData delivery NoneAutomatedUse casesDiscovery,Governance,Data Migration,Impact AnalysisDiscovery,Access,Data Sharing,Monetization Eckerson Group 2023 HOW TO CREATE AND GOVERN A DATA PRODUCT PRODUCT PACKAGINGBusiness&technical metadataPRODUCT GOVERNANCERoles,

24、review,and certificationPRODUCT PROVISIONINGPublish to data marketPRODUCT LIFECYCLE MANAGEMENTMonitor,iterate optimize,retireBusiness CustomersProduct TeamPRODUCT DEVELOPMENTDevelopment,architecture,tools PRODUCT DEFINITIONDefine target customers and requirementsCUSTOMER TRAINING&SUPPORTCUSTOMER FEE

25、DBACK&USAGE Eckerson Group 2023 Product ManagerAdapted from Lennys NewsletterConduct market researchDefine user needsCreate product planPresent to stakeholdersSecure funding Submit to reviewDevelop tech specsCoordinate with dev teamCreate product version Operationalize productPublish to data storeMa

26、rket and promote the productMonitor product usageSolicit customer feedback Adapt product plan Manage product versions(Not a project manager!)(Not a BI/data developer!)(Not a data analyst or data scientist!)Manage the productKey Role Eckerson Group 2023 Product Organization Product teamEnterprise Dat

27、a MarketplaceProduct teamBusiness DomainProduct teamDecentralized Model(Data Mesh)Every business domain has its own data product team and local data market which feeds an enterprise data market.Corporate Data Product TeamBusiness DomainEnterprise Data MarketBusiness domainBusiness DomainBusiness Dom

28、ainCentralized ModelBusiness domains feed requirements to a corporate data product team which publishes to an enterprise data market.Pros Local autonomy Less overhead No bottlenecksCons Lack of product mgrs.Product misalignment Unserved domainsPros 1 PM team only Product alignment All domains served

29、Cons Loss of local autonomy More overhead Central bottleneck Eckerson Group 2023 Where most organizations will end upCorporate Data Product TeamProduct TeamEnterprise Data MarketProduct TeamBusiness DomainBusiness DomainHybrid ModelA corporate team serves domains that dont have local data product te

30、ams and sets standards to align data products.Pros Product alignment All domains served Local autonomy Few bottlenecksCons More overhead Eckerson Group 2023 Product TeamProduct OwnerProduct ManagerProduct MarketerProduct IdeasProduct SpecsProduct Review BoardDo customers want this?Do we have resourc

31、es?Does it align with what exists?What are the success criteria?Shall we fund this?CustomersConsume/UseReview/RateProvide feedback Technical Review BoardProduct SubmissionDoes it deliver as promised?Does it adhere to standards?Is it secure?Is it private?Does it perform/scale?Dev/Ops TeamData Archite

32、cts Data/Analytics DevelopersOperations*(Marketing,Sales,Operations,Legal,Security,Compliance,HR)IT Operations Monitor deployment Track/fix errors Send alerts&reports Monitor usageGovernance ReviewsPublish to Data StoreData Product GovernanceTo Prod Team Eckerson Group 2023 Data ContractsAdapted fro

33、m“Data Mesh Manager,“What is a Data Contract?”Terms of UseUsers,duration,volume,cost,approved uses,limitationsTerms of ServiceVersion,data model,freshness,schema,quality score,quality checksContracts should be produced in machine-readable form(e.g.,YAML)so they can be automatically enforced.Descript

34、ionModelSchemaQuality Checks Eckerson Group 2023 Data Product Characteristics60%60%53%50%47%43%40%40%23%23%23%20%20%20%3%Targeted to a specific audienceShared across internal domainsManaged by product manager or ownerContains business&technical metadataStored in a product library or data storeSuppor

35、ts versioningUpdated automaticallyComes with support SLAsUsers can subscribe to the data productsShared externallyContains a contract or proof of qualityCertified by a review boardComes with a projected ROI or costSpecifies terms of useOther(please specify)Which characteristics apply to your data pr

36、oducts?From Wayne Eckerson,“Data Products Revolution,”Eckerson Group,2024.)Eckerson Group 2023 Product Groups55%30%24%A data product team that gathers requirements,publishes products,and monitors usageA technical review board that evaluates data products prior to implementationA product review board

37、 that reviews and approves data product proposalsWhich product groups exist in your organization?From Wayne Eckerson,“Data Products Revolution,”Eckerson Group,2024.)Eckerson Group 2023 Product Management RigorTARGET USER TYPE OF DATA PRODUCTData Engineer/Developer Reusable code componentsData Analys

38、t Reusable queries,metrics,and data setsData Scientist Reusable features,models,and data setsBusiness Analyst Standard metrics,dimensions,cohortsBusiness Executives Standard dashboards&reportsCustomer or Partner Product dashboards and data setsHighLowDegree of Product Management Rigor Eckerson Group

39、 2023 Best Practices in their own words“Track adoption and usage.”“Educate people to think like product managers.”“Create well-defined use cases”“Create a dedicated central team supported by cross-functional business heads.”“Get customer feedback early in the design process.”“Find executives who are

40、 passionate about data-driven decision-making and will continuously fund a data product.”“Deliver high-quality releases.”“Keep it simple and integrate it into user workflows.”“Train a small number of creators,iterate,and build out as you go.”“Show value and ROI.”“Know the business problem and users

41、for your product.”Assign a governance group to decide the functionality of a data product and its enhancements.”“Seed your team with experienced product managers.”“Be sure everyone understands what a data product actually is.”“Make it work as a service.”BEFORE YOU STARTWHEN YOU DESIGNONCE YOU BUILD“

42、Promote your products heavily.”Eckerson Group 2023 Summary01Data Products Are Hot!02Data Products Arent Data Assets 03The Product Organization is Key04A Data Market Unlocks Full Value05Data Product Platforms are ComingOrganizations want trustworthy data,data sharing,and data reuseTargeted to an audi

43、ence,managed by a product team,reusable and shared,continuously funded and improvedCreate a product organization that educates,curates,governs,promotes,and secures continuous funding.Simplifies and automates the exchange of data products across organizational boundariesA majority of large organizations will adopt a DPP in 3-5 years