2019年構建強一致性分布式數據庫TiDB.pdf

1、構建強一致性分布式數據庫 TiDBR&D Engineer PingCAP目錄What is TiDB?How to test?What is TiDB?Single PointSingle Point of FailureAsynchronous ReplicationA=1B=2A=1MasterSlaveAsynchronous ReplicationA=1B=2A=1B=?MasterSlaveSynchronous ReplicationA=1B=2A=1B=2MasterSlaveSynchronous ReplicationA=1B=2A=1B=2MasterSlaveServi

2、ng?Slave-Master?Huge Amounts of DataMasterSlaveMasterSlaveMasterSlaveOpen-source NewSQL database High availability Strong consistency Horizontal scalability Support the MySQL wire protocol https:/ wired protocolSQLNoSQLtidb-servertikv-serverNo single point of failure Automatic failoverHigh Availabil

3、ity at ScaleSnapshot isolation TiDB Linearizability TiKVStrong Consistency at ScaleRaftClientA=1B=2State MachineA=1B=2LogRaft ModuleA=1B=2State MachineA=1B=2LogRaft ModuleA=1B=2State MachineA=1B=2LogRaft ModuleLeader election Voting for leader Network isolation/node failure tiggers election Log repl

4、ication Leader handles client requests,append logs Replicate logs to followers Constraints Quorum:log replication,leader election Up to date:leader electionRaft:Key PointsLazy Peer Cross DC,cross regions deploymentLearner Cross DC,cross regions backupJoint consensus Safe membership changingRaft:New

5、FeaturesRaft:Lazy Peer3-DC in 2 regions deploymentRaft:LearnerFollowerFollowerLeaderLearnerReplace nodes?Node2Node1Node3Node2Node1Node3Node4Node2Node1Node4Raft:Joint consensusNode2Node1Node3Node2Node1Node4How to test?Distributed Multiple Components Strong guarantee on weak foundation 2-4%of disk dri

6、ves fail each year N x 2-4%Broken optic cable Its complicated!Unit tests Integration tests Fuzz testing libFuzzer AFL Fault injection Failpoint,https:/www.freebsd.org/cgi/man.cgi?query=fail Jepsen,https:/ framework for distributed systems verification,with fault injectionFound bugs Redis etcd Cassan

7、dra JepsenJepsen:How does it work?N1N2N3N4N5ControlTiDB ClusterRequests&NemesisNetwork Clock ProcessNemesisNetwork:Partition OneN1N2N3N4N5Network:Partition RingN1N2N3N4N2N3N4N5N3N4N5N1N2N5N1Clock:Bump&Strobekill-9kill-SIGSTOPProcess:Kill&PauseSI,the default isolation level for TiDB Bank test CREATE

8、TABLE accounts(id INT NOT NULL,balance BIGINT NOT NULL);UPDATE accounts SET balance=balance-5 WHERE id=1000;UPDATE accounts SET balance=balance+5 WHERE id=2000;SI:Bank TestVerification SELECT SUM(balance)FROM accounts;SI:Bank TestSI:Bank TestWrite skew is valid in Snapshot Isolation Solution?SELECT

9、FOR UPDATE SI Write skew?Append unique numbers to key 0 and key 1 Example:SELECT val FROM tb WHERE id=0;UPDATE tb SET val=1,3,5,6,7 WHERE id=1;Write skew:Append TestWrite skew w/o read-lock T1:f:txn,:value:append 0 28:r 1 1 3 7 5 6 11 12 2 10 8 17 18 19 15 16 22 23 24 25 21 28,:process 22 T2:f:txn,:

10、value:append 1 29:append 1 30:r 0 1 7 8 9 12 13 14 15 16 17 2 3 19 21 22 18 20 24 27 4,:process 6Write skew w/o read-lockAppend unique numbers to key 0 and key 1 Example:SELECT val FROM tb WHERE id=0 FOR UPDATE;UPDATE tb SET val=1,3,5,6,7 WHERE id=1;Write skew gone w/read-lockWrite skew gone w/read-lock