how-to-manage-database-clusters-without-a-dedicated-operator-nanoxi-operatorzha-fa-lia-zhong-shi-shanshan-ying-apecloud-shun-ding-china-mobile-cloud-1.pdf

《how-to-manage-database-clusters-without-a-dedicated-operator-nanoxi-operatorzha-fa-lia-zhong-shi-shanshan-ying-apecloud-shun-ding-china-mobile-cloud-1.pdf》由會員分享，可在線閱讀，更多相關《how-to-manage-database-clusters-without-a-dedicated-operator-nanoxi-operatorzha-fa-lia-zhong-shi-shanshan-ying-apecloud-shun-ding-china-mobile-cloud-1.pdf（41頁珍藏版）》請在三個皮匠報告上搜索。

1、1How to Manage Database Clusters Without a Dedicated OperatorAboutUsShanshanYing-KubeBlocks Maintainer,Apecloud-Senior Engineer,AlibabaCloud DatabaseGroupShuDing-KubeBlocks Contributor-Senior Systems Architect,China Mobile Cloudkubeblocks.io3Manage Databaseskubeblocks.io4Manage Various Databaseskube

2、blocks.io5Manage VariousDatabaseson Kuberneteskubeblocks.io6ManagePROVISIONINGSCALINGBACKUP ANDRESTORECONFIGURATION(ESPECIALLYPARAMETERS)UPGRADINGVOLUMEEXPANSIONMONITORkubeblocks.io7ManagePROVISIONINGSCALINGBACKUP ANDRESTORECONFIGURATION(ESPECIALLYPARAMETERS)UPGRADINGVOLUMEEXPANSIONMONITOR A Team Co

3、nsists of Database Experts Cloud-Native Experts Operators are delivered in different maturity levels kubeblocks.io8ManagePROVISIONINGSCALINGBACKUP ANDRESTORECONFIGURATION(ESPECIALLYPARAMETERS)UPGRADINGVOLUMEEXPANSIONMONITOR A Team Consists of Database Experts Cloud-Native Experts Operators are deliv

4、ered in different maturity levels kubeblocks.io9Recap:BackupandRecover Backup Methods Regular Full Backups Adopting Database Specific Backup Tools Taking Volume Snapshot Continuous Archive Logs Backup Scheduler Hourly,daily,or Weekly,or(at user-specified scale)Backup Repository a storage location wh

5、ere backup files are kept(local PV,or objectstorage Recovery how to restore data(restore data before/after Pods start)kubeblocks.io10Recap:BackupandRecover Backup Methods Regular Full Backups Adopting Database Specific Backup Tools Taking Volume Snapshot Continuous Archive Logs Backup Scheduler Hour

6、ly,daily,or Weekly,or(at user-specified scale)Backup Repository a storage location where backup files are kept(local PV,or objectstorage Recovery how to restore data(restore data before/after Pods start)kubeblocks.io11IntroductiontoKubeBlocks Open Source,Cloud-Neutral Mitigatethe risk ofvendor lock-

7、in 2k stars,1 year Database OperatorFramework(database-type agnostic)Extensible Flexible Pluggablekubeblocks.io12IntroductiontoKubeBlocks Open Source,Cloud-Neutral Mitigatethe risk ofvendor lock-in 2k stars,1 year Database OperatorFramework(database-agnostic)Extensible-UnifiedAPIs provides the Add-o

8、n mechanism for integrating engines Flexible customizeand composeyour cluster topology pickcomponentsfrom the Add-on market Pluggable-Modular design Backup&Restore,Monitoring,Configuration arepluggablekubeblocks.io13AConciseModelingofDatabase(1/3)Component Refers to an indivisible database system or

9、 service.kubeblocks.io OrchestratorMySQL14AConciseModelingofDatabase(1/3)Cluster A Collection of Components,thatareinter-connected,worked-together,tohandleatask Component Refers to an indivisible database system or serviceCluster OrchestratorMySQLkubeblocks.io15AConciseModelingofDatabase(1/3)Cluster

10、 A Collection of Components,thatareinter-connected,worked-together,tohandleatask Component Refers to an indivisible database system or service InstanceSet:A Role-Aware Workload Mange instances inspecific Role orderPrimary-Secondary,Leader-Follower-Learner.Instance POD,and associated ResourcesCluster

11、ComponentInstanceSetInstanceK8s APIPodPVCConfigmapServiceSecretkubeblocks.io16Cluster DefinitionComponent DefinitionComponent VersionComponentClusterDatabase ExpertKubeBlocks DeveloperDatabase UsersTopologiesProvisionorders EngineSpecificBehaviorsImagesService VersionsCombabilityComponentsResourcesS

12、erviceVersionsAConciseModelingofDatabase(2/3)kubeblocks.io17LifecycleActions-Database-I(3/3)Role-Probetriggered periodically to check Role of each replicaSwitchovertransition of leadership to a new replicaMember-Leaveremove a replica from the replication group,invoked on scale-inMember-Joinadd a new

13、 replica to the replication group,invoked on scale-outData-Dumpprocedure for exporting the data from a replica.Data-Loadprocedure for importing data into a replicaPre-Terminatehook to be executed prior to terminating a componentPost-Provisionhook to be executed after a components creationHorizontal-

14、ScalingIn/OutRole-BasedManagementComponent-Levelkubeblocks.io18KubeBlocks OperatorCluster ControllerComponent ControllerInstanceSet ControllerClusterComponentInstanceSetInstanceK8s APIPodPVCConfigmapServiceSecretKubeBlocks APIClusterDefintionComponentDefinition ComponentVersionAddon APIkbcliKubeBloc

15、ks CloudKubeBlocks AddonsDatabase-Type Agnostic Operatorkubeblocks.io19KubeBlocks OperatorCluster ControllerComponent ControllerInstanceSet ControllerClusterComponentInstanceSetInstanceK8s APIPodPVCConfigmapServiceSecretKubeBlocks APIClusterDefintionComponentDefinition ComponentVersionAddon APIkbcli

16、KubeBlocks CloudKubeBlocks AddonsDatabase-Type Agnostic Operatorkubeblocks.io20KubeBlocks OperatorCluster ControllerComponent ControllerInstanceSet ControllerClusterComponentInstanceSetInstanceK8s APIPodPVCConfigmapServiceSecretKubeBlocks APIClusterDefintionComponentDefinition ComponentVersionAddon

17、APIkbcliKubeBlocks CloudKubeBlocks AddonsDatabase-Type Agnostic Operatorkubeblocks.io21Conclusion Support Various Databases Unified APIs for Clusters Database-I for Lifecycle Actions KubeBlock 1.0 will be Released Soon!Shardings more actions Join our communitykubeblocks.io22kubeblocks.io23Recap:Back

18、upandRecover Backup Methods Regular Full Backups Adopting Database Specific Backup Tools Taking Volume Snapshot Continuous Archive Logs Backup Scheduler Hourly,daily,or Weekly,or(at user-specified scale)Backup Repository a storage location where backup files are kept(local PV,or objectstorage Recove

19、ry how to restore data(restore data before/after Pods start)kubeblocks.io24BackupPolicyTemplateBackup-ActionSetBackupRepoBackupScheduleBackupPolicyDatabase ExpertKubeBlocks DeveloperDatabase UsersBackupMethodsPairs of Actions toBackupand RestoreEngine SpecificBehaviorsRecap:BackupandRecoverkubeblock

20、s.io25How to Manage Database Clusters Without a Dedicated OperatorChina Mobile Cloud CaseShun Ding China Mobile CloudAbout China Mobile Cloud DBaaSComprehensive Product LineTransactional,Analytical&Search,NoSQL,etc.Open Source,3rd Party,In-House DB Engines35K+Customers in 9 Key Industries130K+Cloud

21、Database Instances in 15 Lv.1 regions&31 Lv.2 regionsMost run in K8s Clusters!Current ChallengesIneffective Developer AllocationLack of a unified DBaaS architecture across different productsDevelopers unable to be reused across productsHigh Requirements for DevelopersDevelopers need to know operator

22、 frameworks AND DB engine knowledges Unable to Rapidly DeliverFor new DB engines,need to develop a whole new dedicated K8s operatorUnable to reuse logics for DB common operations How KubeBlocks Solves Our ProblemsUniversal Operator Framework Single operator/CRD set to maintain Sharable knowledge acr

23、oss developers Developers re-usable on different productsLow-Code Development Model Writing Add-On rather than operator Developers only need to know DB engine stuffs Less code,accelerating deliverySpecially Designed for DB Workloads Operator covers most common operations for any DB engines Easy to e

24、xtend Successful Adoption Case:H-DBIntroduction to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation of Computation&StorageSuccessful Adoption Case:H-DBIntroduction to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation

25、of Computation&StorageClusterDefinition&ClusterVersion ScaffoldSuccessful Adoption Case:H-DBIntroduction to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation of Computation&StorageClusterDefinition&ClusterVersion ScaffoldBootstraping Scripts&ConfigMap Templat

26、eSuccessful Adoption Case:H-DBIntroduction to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation of Computation&StorageClusterDefinition&ClusterVersion ScaffoldBootstraping Scripts&ConfigMap TemplateBackup-Recovery Scripts&ActionSetSuccessful Adoption Case:H-D

27、BIntroduction to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation of Computation&StorageClusterDefinition&ClusterVersion ScaffoldBootstraping Scripts&ConfigMap TemplateBackup-Recovery Scripts&ActionSetConfigConstraintSuccessful Adoption Case:H-DBIntroduction

28、 to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation of Computation&StorageClusterDefinition&ClusterVersion ScaffoldBootstraping Scripts&ConfigMap TemplateBackup-Recovery Scripts&ActionSetConfigConstraintHigh Availability&Role DetectionSuccessful Adoption Ca

29、se:H-DBIntroduction to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation of Computation&StorageClusterDefinition&ClusterVersion ScaffoldBootstraping Scripts&ConfigMap TemplateBackup-Recovery Scripts&ActionSetConfigConstraintHigh Availability&Role DetectionObs

30、ervability SidecarsSuccessful Adoption Case:H-DBIntroduction to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation of Computation&StorageClusterDefinition&ClusterVersion ScaffoldBootstraping Scripts&ConfigMap TemplateBackup-Recovery Scripts&ActionSetConfigCons

31、traintHigh Availability&Role DetectionObservability SidecarsMore ClusterVersionsSuccessful Adoption Case:H-DBIntroduction to H-DB(placeholder name)Fully In-House DevelopedCloud Native Distributed DB EngineSeparation of Computation&StorageFinish DBaaS System in 2 months from scratchFirst successful K

32、ubeBlocks integration case in China Mobile Cloud ClusterDefinition&ClusterVersion ScaffoldBootstraping Scripts&ConfigMap TemplateBackup-Recovery Scripts&ActionSetConfigConstraintHigh Availability&Role DetectionObservability SidecarsMore ClusterVersionsKubeBlocks Add-On vs.Dedicated OperatorKubeBlock

33、s Add-OnDedicated OperatorDeveloper Resource Invested2 person-months6 person-monthsCodes Written2000+7000+Code ContentsCR Objects YAMLFunctional ScriptsDesign the CRDOperator Golang CodeFunctional ScriptsDeveloper Pre-RequirementDedicated DB Engine KnowledgeScripting KnowledgeDedicated DB Engine Kno

34、wledgeGolang KnowledgeK8s Operator Framework(Client-Go/Controller-Runtime)Scripting KnowledgeGoal:Unified Cloud-Native DBaaS PlatformComprehensive restructuring based on KubeBlocksIntegrate ALL DB engines as KB Add-OnsUnified multi-cloud architectureServerless K8sUnified database orchestration&management platformReduce development resoruces by 50%