李海翔-分布式數據庫與分布式事務.pdf

《李海翔-分布式數據庫與分布式事務.pdf》由會員分享，可在線閱讀，更多相關《李海翔-分布式數據庫與分布式事務.pdf（21頁珍藏版）》請在三個皮匠報告上搜索。

1、分布式數據庫與分布式事務處理技術演講人：李海翔CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CONTENT目錄分分布布式式事事務務型型數

2、數據據庫庫的的理理念念與與背背景景分布式事務處理技術第三代分布式數據庫強一致性+高性能=魚和熊掌兼得CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek

3、 20231.1 分布式數據庫的基礎高可靠（high reliability）定義為一個服務連續無故障運行的時間，無故障運行的時間越長，可靠性就越高。99.999%高可用定義為在足夠長的時間里，一個服務可用的時間，服務可用時間越長越好高性能低隔離別+高性能可串行化+低性能易用性例例如如：1 MySQL可可串串行行化化，性性能能差差2 MySQL是是單單機機系系統統推推測測：分分布布式式的的可可串串行行化化，是是不不是是性性能能更更差差？CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek

4、 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.1 分布式數據庫的基礎數據的正確性并發的性能事事務務處處理理領領域域的的怪怪現現象象1 有有多多少少個個數數據據異異常常？-不不知知道道2 為為什什么么性性能能變變低低？-不不知知道道事事務務處處理理技技術術的的兩兩個個核核心心問問題題1 沒沒有

5、有任任何何數數據據異異常常2 并并發發導導致致性性能能變變差差事事務務處處理理技技術術CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.

6、2 怪現象-強一致+低性能弱弱一一致致性性強強一一致致性性缺點不能100%確保數據一致性性能低（額外判斷，耗時，如SSI）開發者需要在數據庫上層消除數據不一致認知8種有限個數的不一致性現象（數據異常）需要認知所有的不一致性現象優點100%確保數據的準確性，用戶編程簡單用戶/開發者編程簡單主流觀點夠用了，Oracle的RC是日常選擇因性能不好而被棄用新問題如果性能足夠好，將極大簡化用戶的開發負擔隔隔離離級級別別，與與一一致致性性的的關關系系？-不不知知道道陷陷入入如如下下具具體體的的問問題題，就就事事論論事事CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWe

7、ek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.2 怪現象-強一致+低性能用戶：要求高性能開發者：用弱隔離級別用戶：要求數據正確開發者：主動加鎖數據庫：性能降低SelectFor Update數數據據庫庫使使用用怪怪圈圈靈

8、靈魂魂問問題題：Q1：我們能精準地說出，什么情況下應該用For Update、什么情況下不用嗎？現現實實問問題題：Q2：為什么我們需要精準地知道什么情況下應該用For Update、什么情況下不用？CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2

9、023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.2 怪現象-強一致+低性能測試目的：驗驗證證用用戶戶主主動動加加鎖鎖、不不加加鎖鎖的的性性能能差差異異背景：TPC-C，數據集只有30GB，MySQL 的page全部緩存在buffer pool中R RC C隔隔離離級級別別不加鎖用戶加鎖并并發發數數tpmctpmc加鎖影響1 16 6109012104995-3.68%32140952130118-7.69%64148549134400-9.52%128153807136824-11.04%256147817126601-14.35

10、%512144511118096-18.28%1024132213104113-21.25%RR（可可重重復復讀讀）不加鎖用戶加鎖并并發發數數tpmctpmc加鎖影響1 16 6111853105088-6.05%32140714127591-9.33%64146152131044-10.34%128153687134165-12.70%256148323120713-18.61%512139814107418-23.17%102412827598620-23.12%用用戶戶主主動動加加鎖鎖，有有性性能能損損失失CoreJavaWeek 2023CoreJavaWeek 2023CoreJav

11、aWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.2 怪現象-強一致+低性能測試目的：驗驗證證用用戶戶主主動動加加鎖鎖、不不加加鎖鎖的的性性能能差差異異背景：測試數據集只有30GB，MySQL 的page全部緩存在bu

12、ffer pool中，TPC-C性能差異并發數RC+用戶加鎖RR+用戶加鎖S16104995105088104708321301181275911280656413440013104413202512813682413416513434725612660112071312659251211809610741811295910241041139862098637用用戶戶主主動動加加鎖鎖，性性能能接接近近于于可可串串行行化化CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Cor

13、eJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20231.2 怪現象-強一致+低性能測試目的：驗驗證證用用戶戶主主動動加加鎖鎖、不不加加鎖鎖的的性性能能差差異異背景：Percona Server 8.0.18-9：驗驗證證用用戶戶主主動動加加鎖鎖、不不加加鎖鎖的的性性能能差差異異背景：測試的recordcount設置為

14、4000w，數據實際容量為90GB，稍大于buffer pool的大小，YCSB的只讀場景只讀場景，用戶加鎖性能損失很大只讀場景，系統內部加鎖，性能損失小CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Co

15、reJavaWeek 2023CoreJavaWeek 20231.2 怪現象-強一致+低性能測試目的：驗驗證證用用戶戶主主動動加加鎖鎖、不不加加鎖鎖的的性性能能差差異異Pg14tpmCRC164.9W可串行化67W可串行化+For Update40.4W結結論論：【可可串串行行化化】優優于于【低低隔隔離離級級別別+用用戶戶主主動動加加鎖鎖】1 基基于于數數據據庫庫的的開開發發變變得得簡簡單單 2 數數據據庫庫的的性性能能有有保保證證3 分分布布式式數數據據庫庫應應該該更更加加慎慎用用鎖鎖CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Co

16、reJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CONTENT目錄分分布布式式事事務務型型數數據據庫庫的的理理念念與與背背景景分分布布式式事事務務處處理理技技術術第三代分布式數據庫強一致性+高性能=魚和熊掌兼得CoreJavaWeek 2023

17、CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事務處理技術序值獲取異常消除（執行階段）異常消除（提交前沖突判定）日志（WAL）日志分散（分布式）原子提交/回滾（

18、2PC）提交序值（可見性依據）全全局局讀讀一一致致性性C全全局局寫寫一一致致性性C+弱弱隔隔離離級級別別I全全局局寫寫一一致致性性C+可可串串行行化化I原原子子性性A持持久久性性D與與一一致致性性C持持久久性性D與與一一致致性性C與與原原子子性性A恢復全全局局讀讀一一致致性性CCoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJava

19、Week 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事務處理技術全局讀一致性全局寫一致性提交的原子性編編號號各各種種方方案案缺缺點點案案例例1全局事務管理器非去中心化、低效Pg XC2基于封鎖的并發訪問控制算法+全局可串行化低效某些系統SS2PL+MVCC3全局可串行化+線性一致性所有事件全序排序=所有事務全局排序，低效SpannerSS2PL+MVCC4全局可串行化+混合邏輯時鐘+全局事務提交標志數據是否可讀，需要通過全局事務提交狀態驗證，增加通訊

20、次數CockroachDBSSI+MVCC52次讀Scalable atomic visibility with ramp transactions增加了通訊輪數，且只能解決讀半已提交數據異常學術界的解決方式CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaW

21、eek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事務處理技術全局讀一致性全局寫一致性提交的原子性寫一致性執行階段寫一致性驗證階段快快照照技技術術止止步步于于此此可可串串行行化化需需要要驗驗證證階階段段確確保保CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Cor

22、eJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事務處理技術分分布布式式事事務務：S SS SI I，基基于于M MV VC CC C的的可可串串行行化化開開始始階階段段：要獲取全局快照（全局讀一致性）執執行行階階段段：不要全局加鎖（WW沖突通常采取鎖機制）驗驗證證階階段段：需需要要進進行行全全局局可可串串行行化化驗驗證證（可可串串行行化化的的關關鍵鍵）提提交交階階段段：要獲取全局的提交時間戳 2 2P PC C提提交交結結束束階階段段

23、：不需要釋放全局鎖（WW的寫鎖是局部的）分分布布式式事事務務:快快照照隔隔離離開開始始階階段段：要獲取全局快照（全局讀一致性）執執行行階階段段：不要全局加鎖（WW沖突通常采取鎖機制）驗驗證證階階段段：不需要可串行化驗證提提交交階階段段：要獲取全局的提交時間戳 2 2P PC C提提交交結結束束階階段段：不需要釋放全局鎖（WW的寫鎖是局部的）CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWe

24、ek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20232 分布式事務處理技術分分布布式式事事務務提提交交階階段段，保保證證提提交交操操作作的的原原子子性性全局讀一致性全局寫一致性提交的原子性CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023Core

25、JavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CONTENT目錄分分布布式式事事務務型型數數據據庫庫的的理理念念與與背背景景分布式事務處理技術第三代分布式數據庫強強一一致致性性+高高性性能能 =魚魚和和熊熊掌掌兼兼得得CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2

26、023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20233 第三代分布式數據庫極簡的易用性只能做強一致性的正確性可控時延的“永遠在線”的高可用性CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20

27、23CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20233 第三代分布式數據庫CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20

28、23CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20233 第三代分布式數據庫SpannerCRDBOBxxDB架構SN+文件級共享存儲SNSNSN+日志級共享存儲事務強一致性嚴格可串行化因果+可串行化SI嚴格可串行化、可串行化多寫架構-可串行化多寫SI多寫可串行化多寫可擴展-擴展性較弱千臺級規模擴展存儲無限擴展計算千臺級擴展高可用數據多副本提

29、供系統級的高可用，事務級瞬時閃斷數據可靈活分布，數據多副本提供系統級的高可用數據多副本提供系統級的高可用數據級高可用：日志服務器；內存級高可用：主主互備、RO切換高可靠-更資料截?。–hange Data Capture，CDC）、備份與恢復、管理API，以及99.99%正常運行時間數據備份和日志備份庫、表級備份恢復日志級備份恢復數據塊級備份恢復可控時延超大時延超低性能高時延（全球級、時延未做特定設計）低性能高時延、擴展節點獲高性能低時延高性能、擴展節點獲更高性能CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 20

30、23CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023感謝聆聽演講人：嘉賓名CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023CoreJavaWeek 2023