構建彈性數據團隊和可擴展解決方案：來自現場的經驗教訓.pdf

《構建彈性數據團隊和可擴展解決方案：來自現場的經驗教訓.pdf》由會員分享，可在線閱讀，更多相關《構建彈性數據團隊和可擴展解決方案：來自現場的經驗教訓.pdf（15頁珍藏版）》請在三個皮匠報告上搜索。

1、Hosted by:Welcome!The presentation will begin shortly If your badge was not scanned at the door,please use the self check-in feature in the session details in the Schedule.Building Resilient Data Teams And Scalable SolutionsLessons From The Field Of Data And Analytics EngineeringGreat Lakes Data,AI&

2、Analytics SummitApr 10,2025PresentersBrittany SpencerDirector,Analytics EngineerAAA LifeHarini RajagopalManager,Data EngineeringAAA LifeAgendaEngineering is done when Data is UseableGathering Business Requirements into Technical SpecsScalable Solutions as Opposed to One and DoneCoordination for Prod

3、 Support on the Data PipelineSelf-Serve Analytics&Single Unified Source of TruthInvestment in Data Tech InfrastructureData and Analytics Engineering Not all Data Teams are the SameTruly empowered data driven organizations require support across every stage of the data pipelineEngineering is done whe

4、n Data is UseableDelivering raw data isnt the finish line of a projectTimelines should reflect the work it takes to make it consumableRobust project plans include dependencies for data deliveryBusiness Requirements!=Technical SpecsData is a natural bridge builder across business units Bridging the g

5、ap between technical specs for implementation and business requirements is much harderFocus on simple questions to view the issue from a stakeholder perspective:What questions do you need to be able to answer at the end of this project?How are you measuring the success of this launch?How people need

6、 to see the data is just as important as how to get it to themScalable Solutions vs Build for Distant FutureDesignFor smaller incrementsBuild modular pipelines with reusable componentsTerraform,Airflow,dbt and AWSStart somewhereFocus onOn Automation and MonitoringCI/CDFalse Positives monitoringPrior

7、itizeData Integrity and GovernanceEstablish robust data quality checksCounts and consistence ChecksTrust on data pipeline delivery SLAsCoordination for Prod Support on the Data PipelineData is interwoven at almost every stage of a business or product pipeline so support cant happen in a vacuumSiloed

8、 thinking doesnt serve well when trying to keep data production lines stable and efforts have to be coordinated across multiple teamsSelf-Serve Analytics and the Need for TruthA consumption layer with a shared source of truth,unified across source systems is critical in the world of self-serve analy

9、ticsStandardization around metrics and definitions drives consistency in resultsInvestment in Data Tech InfrastructureAdopt tools for CapabilitiesAdopt tools for CapabilitiesLook for capabilities in tools Data quality checks and Data Catalog vs Data GovernanceGrow for Patterns and Trust Grow for Pat

10、terns and Trust Implement automated testing,CI/CD pipelines,and robust monitoring systems to enhance the resilience and efficiency of the data infrastructure.Prioritize Integration and Prioritize Integration and InteroperabilityInteroperabilityBuild a tech stack that supports seamless integration wi

11、th existing tools and systems,ensuring smooth data flow and reducing time-to-value for stakeholders.Q&AHosted by:Thank you for attending!We value your feedback!Please rate your speaker in the conference evaluation provided at the end of the event.Infinite BacklogsREQUESTS FOR DATA TEAMS NEVER ENDPRI

12、ORITIZATION IS KEY TO THE SUCCESS OF YOUR PEOPLE INDIVIDUALLY AND AS A TEAMTYING OKRS AND PRIORITIES BACK TO COMPANY INITIATIVES CAN BE TRICKY BECAUSE SOME GROUPS ARE UNDERSERVED BUT DRIVING TOWARD CORPORATE GOALS EMPOWERS DELIVERYSELF-SERVE ANALYTICS CAN HELP(OR ADD LOAD)Stakeholder Investment in D

13、ata Team SustenanceCommunicate Value Through Communicate Value Through ImpactImpactRegularly showcase how data initiatives drive measurable business outcomes,emphasizing their role in decision-making,cost savings,and revenue growth to secure long-term stakeholder buy-in.Involve Stakeholders in Plann

14、ingInvolve Stakeholders in PlanningCollaborate with stakeholders to align priorities,ensuring that their needs are addressed while building shared accountability for the success of the data teams projects.Advocate for Continuous Advocate for Continuous InvestmentInvestmentHighlight the importance of modernizing tools,infrastructure,and skills to maintain a competitive edge,advocating for sustained funding and support for the data teams growth and innovation.