提升您的數據使用ClickHouse在Kubernetes上進行實時分析.pdf

1、 2024 Altinity,Inc.Elevate Your DataReal-Time Analytics on Kubernetes with ClickHouseRobert HodgesData Summit Boston-May 9th,20241 2024 Altinity,Inc.2024 Altinity,Inc.Lets make some introductionsClickHouse support and services including Altinity.Cloud Authors of Altinity Kubernetes Operator for Clic

2、kHouse and other open source projectsRobert HodgesDatabase geek with 30+years on DBMS systems.Day job:Altinity CEOAltinity EngineeringDatabase geeks with centuries of experience in DBMS and applications2 2024 Altinity,Inc.Understands SQLRuns on bare metal to cloudShared nothing architectureStores da

3、ta in columnsParallel and vectorized executionScales to many petabytesIs Open source(Apache 2.0)ClickHouse is a real-time analytic databaseIts a popular engine for real-time analyticsClickHouseEvent StreamsELTObject StorageInteractive GraphicsDashboardsAPIs3 2024 Altinity,Inc.What can we do with a d

4、atabase like ClickHouse?Enable Fast,Cost-Efficent End User Access to Trillion-Row Datasets4Consistent,sub-second response that scales linearly with resourcesDeliver query results at costs that are low and predictableMarket TICK data,DNS queries,weblogs,network flow logs,service logs,CDN telemetry,re

5、al-time ad bids,2024 Altinity,Inc.Why not just use PostgreSQL?5ClickHouseRead only selected columnsRows minimally or not compressedColumns highly compressedPostgreSQL,MySQLRead all columns in row 2024 Altinity,Inc.ClickHouse is often 1000 x faster on analytic queries659 GB(100%)21 MB(.035%)2.6 MB(.0

6、044%)1.7 GB(3%)Read 109 columns Read 3 columns from 109Read 3 compressed columnsRead 3 compressed columns over 8 threads 2024 Altinity,Inc.2024 Altinity,Inc.Kubernetes,Operators,and ClickHouse7 2024 Altinity,Inc.VMKubernetes manages container-based applications8ClickHouseContainerBlock Storage 2024

7、Altinity,Inc.Kubernetes maps resource definitions to infrastructure9Stateful SetPersistent VolumePodPersistent Volume ClaimKubernetesResourcesContainerAWS EBS StorageRuntime AbstractionsVolumeProcess running on hostPhysical Infrastructure 2024 Altinity,Inc.ClickHouse is usually a little more complic

8、ated!ClickHouse Servershard1Analytic applicationKeeper ServerClickHouse Servershard1ClickHouse Servershard2ClickHouse Servershard2Zookeeper ServerZookeeper ServerAvailability ZoneAvailability ZoneAvailability Zone10 2024 Altinity,Inc.kube-system namespaceOperators make complex databases work on Kube

9、rnetesAltinity ClickHouse Operatoryour-favorite namespaceApache 2.0 source,distributed as Docker imagekubectl-f apply demo.yamlBest practice deployment“Adjust reality”ClickHouse Installation CRD11 2024 Altinity,Inc.2024 Altinity,Inc.Setting up your first ClickHouse cluster in Kubernetes12 2024 Altin

10、ity,Inc.kubectl apply-f https:/ ClickHouse operator from GitHubDefines the ClickHouse CRDInstalls operator in kube-systemAnd other good stuff 2024 Altinity,Inc.Get Zookeeper stateful set definition:wget https:/ Zookeeper.kubectl create ns zoo1nskubectl apply-f zookeeper-1-node.yaml-n zoo1nsSet up Zo

11、oKeeper14Dev only 2024 Altinity,Inc.Define your cluster(cluster configuration,page 1)apiVersion: layout:replicasCount:1 shardsCount:1 templates:volumeClaimTemplate:storage podTemplate:replica zookeeper:nodes:-host:keeper15Shards and replicasDefinitions for pods and storageWhere is ZooKeeper?2024 Alt

12、inity,Inc.templates:podTemplates:-name:replica spec:containers:-name:clickhouse image:altinity/clickhouse-server:23.8.8.21.altinitystable16Define your cluster(pod definition,page 2)Server version 2024 Altinity,Inc.volumeClaimTemplates:-name:storage#Do not delete PVC if installation is dropped.reclai

13、mPolicy:Retain spec:accessModes:-ReadWriteOnce resources:requests:storage:50Gi17Protect storage from deletionStorage size Define your cluster(storage definition,page 3)2024 Altinity,Inc.18DEMO TIME!2024 Altinity,Inc.How does the operator manage adding replicas?VMVMVMVMUpdate cluster metadataAdd stat

14、eful setCopy schema to new replicaData replication(ClickHouse does this)Provisioner adds VM 2024 Altinity,Inc.2024 Altinity,Inc.Scaling up to production on Kubernetes20 2024 Altinity,Inc.Choosing a Kubernetes distribution21*Used for demo/test only 2024 Altinity,Inc.Your Kubernetes setup needs to get

15、 a few things right22Master NodeWorker NodeWorker NodePodPodPodPodClickHouse PodClickHouse PodProvision a clusterAdd/remove nodes as needed(auto-scaling)Manage cloud block storage for data 2024 Altinity,Inc.Provided you have the right magic configuredPodPersistent VolumePersistent Volume ClaimKarpen

16、terProvisionerStorageClassEBS VolumeWorker Node VMsWorker Node VMsWorker Node VMsEBS VolumeEBS VolumeCreates PVs in response to claimsAdjusts VMs to matchAllocates storage to match PVsWatches pod resource requests 2024 Altinity,Inc.We can control VM type,zone,and storage typeNetwork-Attached Block S

17、torageCPURAMVMm6i.largeus-west-2a/data150Gb gp2IOPS 2024 Altinity,Inc.templates:podTemplates:-name:server spec:containers:-name:clickhouse image:altinity/clickhouse-server:23.3.13.7.altinitystable nodeSelector:node.kubernetes.io/instance-type:m6i.large25Assign pod to a VM typeVM type“Well-known”labe

18、l 2024 Altinity,Inc.templates:podTemplates:-name:server spec:containers:-name:clickhouse image:altinity/clickhouse-server:23.3.13.7.altinitystable nodeSelector:node.kubernetes.io/instance-type:m6i.xlarge26Then make the VM bigger!New VM type 2024 Altinity,Inc.IP 10-0-1-30 in us-west-2 Affinity vs.ant

19、i-affinity chi-demo-cl-0-0IP 10-0-2-151 in us-west-2 Anti-AffinityAffinity27chi-demo-cl-0-1chi-demo-cl-0-1 2024 Altinity,Inc.apiVersion: layout:shards:-replicas:-templates:podTemplate:replica-in-zone-us-west-2a -templates:podTemplate:replica-in-zone-us-west-2b28Assign replicas to particular availabi

20、lity zonesSeparate pod template for each AZTwo replicas for 1 shard 2024 Altinity,Inc.templates:podTemplates:-name:replica-in-zone-us-west-2a zone:values:-us-west-2a podDistribution:-type:ClickHouseAntiAffinity scope:ClickHouseInstallation spec:containers:-name:clickhouse image:altinity/clickhouse-s

21、erver:23.8.8.21.altinitystable29Add affinity rules for each podPod must be scheduled in us-west-2aKeep pods on different hosts 2024 Altinity,Inc.2024 Altinity,Inc.Building complete analytic stacks30 2024 Altinity,Inc.How can we deploy analytic stacks in a cloud native way?31Working Kubernetes Cluste

22、rZooKeeperCluster ConsensusClickHouseAnalytic DBPrometheusOperational MetricsClickHouseResourcesGrafana ResourcesPrometheus ResourcesDefinitions in GitGrafanaDashboardsZooKeeper Resources 2024 Altinity,Inc.ArgoCD maps deployments from GitHub to K8s32KubernetesKubernetes ManifestKustomize+ManifestsHe

23、lm ChartApp ResourcesApp ResourcesApp ResourcesArgoCD 2024 Altinity,Inc.(Install Argo CD)argocd app create clickhouse -repo https:/ -path apps/clickhouse -dest-server https:/kubernetes.default.svc -dest-namespace defaultargocd app sync clickhouse33Keep your configuration in GitHub and install anywhe

24、rehttps:/ 2024 Altinity,Inc.Advanced GitOps using GitHub,ArgoCD,and Kubernetes34APIRepository ServiceApplication Controllerargocd namespacech namespaceClickHouseapplicationPrometheus applicationZooKeeperapplicationhttps:/ pushargocd app createargocd app syncargocd app deleteLocal k8s clusterProd k8s

25、 cluster 2024 Altinity,Inc.2024 Altinity,Inc.Final thoughts 35 2024 Altinity,Inc.How to get started with ClickHouse on KubernetesThe Altinity Kubernetes Operator manages ClickHouse clusters Try it out on Minikube or other dev versions of ClickHouseMost people use managed Kubernetes services for prod

26、uction systemsBut OpenShift,Rancher,KOPS are OK tooEmbroider features on the manifest to deliver production clustersAuto-scaling of compute and storageSpreading servers across availability zonesScale up shards and replicasUse Argo CD(or your favorite GitOps tool)to build an entire stack36 2024 Altin

27、ity,Inc.More information!Altinity Kubernetes Operator for ClickHouse on GitHubhttps:/ Altinity documention(https:/)Altinity blog(https:/ docs(https:/kubernetes.io/docs/home/)EKS,GKE,and AKS documentation37 2024 Altinity,Inc.38Thank you!Questions?https:/rhodges at Contact me on LinkedInAltinity.CloudAltinity Kubernetes Operator for ClickHouseWere hiring!2024 Altinity,Inc.