（許鵬）WiredTiger引擎實現探秘（17頁）.pdf

1、WiredTiger實現探秘許鵬摘要-無鎖化規避線程間競爭-頁面布局(memory layout)-緩存和文件壓縮整體架構內存管理 頁面管理 BTree內存管理 葉子結點初始化寫入 數據沒有落入磁盤$97=u=intl=parent_ref=0 x0,split_gen=0,_index=0 x0,row=0 x0,fix_bitf=0 x0,col_var=col_var=0 x0,repeats=0 x0,entries=0,type=7 a,flags_atomic=0 000,unused=000,read_gen=201,memory_footprint=746,dsk=0 x0,m

2、odify=0 x6933a0,cache_create_gen=1,evict_pass_gen=0$99=intl=root_split=0 x693080,column_leaf=append=0 x693080,update=0 x0,split_recno=0,row_leaf=insert=0 x693080,update=0 x0p(WT_CURSOR_BTREE*)cursor).btree.root.page.u.intl._index.index0.page.modify.u2p*(WT_CURSOR_BTREE*)cursor).btree.root.page.u.int

3、l._index.index0.pageRow Leaf-葉子結點的內存信息$61=u=intl=parent_ref=0 x69fe30,split_gen=0,_index=0 x0,row=0 x69fe30,fix_bitf=0 x69fe30 277,col_var=col_var=0 x69fe30,repeats=0 x0,entries=2,type=7 a,flags_atomic=2 002,unused=000,read_gen=201,memory_footprint=555,dsk=0 x69edd0,modify=0 x69fe50,cache_create_gen

4、=1,evict_pass_gen=0-從磁盤加載數據-修改已經存在的key-關注如下兩個成員變量-dsk-Modify-row表示從dsk位置開始的偏移量p*(WT_CURSOR_BTREE*)cursor).btree.root.page.u.intl._index.index0.pageGDB調試指令頁面釋放 Hazard Pointer1.建立一個全局數組，數組容量為線程數目，每個線程只能修改自己的數組元素，而不允許修改其他的數組元素，但可以讀別的數組元素。2.當線程嘗試訪問一個關鍵數據節點時，先把該節點指針賦給自己的數組元素（即不要釋放這個節點）。3.每個線程自己維護一個私有鏈表，當

5、線程準備釋放掉某個節點時，將該節點放入到鏈表中。當鏈表內的數目達到一個設定的數目后，遍歷該鏈表用于釋放鏈表內所有節點。4.當釋放節點時，需要檢查全局數組，確定沒有任何一個線程的數組元素與當前指針相同時，就釋放該節點。否則仍然滯留在自己的鏈表中。數據持久化-Checkpoint-Occur at an interval of 60 seconds-Journal log-Sync to disk every 50 milliseconds文件壓縮-默認使用Snappy壓縮算法-緩存中是解壓后的格式,占用內存空間遠大于磁盤并發控制訪問struct _wt_rwlock/*Read/write lo

6、ck*/volatile union uint64_t v;/*Full 64-bit value*/struct uint8_t current;/*Current ticket*/uint8_t next;/*Next available ticket*/uint8_t reader;/*Read queue ticket*/uint8_t readers_queued;/*Count of queued readers*/uint32_t readers_active;/*Count of active readers*/s;u;int16_t stat_read_count_off;/

7、*read acquisitions offset*/int16_t stat_write_count_off;/*write acquisitions offset*/int16_t stat_app_usecs_off;/*waiting application threads offset*/int16_t stat_int_usecs_off;/*waiting server threads offset*/WT_CONDVAR*cond_readers;/*Blocking readers*/WT_CONDVAR*cond_writers;/*Blocking writers*/;d

8、b.serverStatus().wiredTiger.concurrentTransactions性能指標監控 db.server.status().wiredtiger db.coll.stats()Notes:根據上述統計信息計算出mongotop和mongostat的結果緩存命中率function missrate(intervalMs)var statsStart=db.serverStatus().wiredTiger.cache;var startTime=new Date();sleep(intervalMs);var endTime=new Date();var statsE

9、nd=db.serverStatus().wiredTiger.cache;var logicalReads=statsEndpages requested from the cache-statsStartpages requested from the cache;var physicalReads=statsEndpages read into cache-statsStartpages read into cache;var elapsedTime=endTime-startTime;var missRate=physicalReads*100/logicalReads;var log

10、icalReadRate=Math.round(logicalReads*1000*100/elapsedTime)/100;var physicalReadRate=Math.round(physicalReads*1000*100/elapsedTime)/100;print(Elapsed time(ms),elapsedTime);print(logical Read Rate IO/s,logicalReadRate);print(physical Read Rate IO/s,physicalReadRate);print(wiredTiger miss rate,Math.rou

11、nd(missRate*100)/100);-wiredTiger.cache-pages requested from cache-pages read into cache-unmodified page evicted參數設置db.createCollection(users,storageEngine:wiredTiger:configString:leaf_page_max=64kb,leaf_value_max=64MB )Demo程序#include wiredtiger.h“#include int main(int argc,char*argv)char*home=/tmp/

12、wt;WT_CONNECTION*conn;WT_SESSION*session;WT_CURSOR*cursor;const char*key,*value;int ret;wiredtiger_open(home,NULL,create,&conn);conn-open_session(conn,NULL,NULL,&session);/create a tablesession-create(session,table:demo,key_format=S,value_format=S);/open the tablesession-open_cursor(session,table:de

13、mo,NULL,NULL,&cursor);cursor-set_key(cursor,name);cursor-set_value(cursor,peng);cursor-insert(cursor);cursor-set_key(cursor,iris);cursor-set_value(cursor,teacher);cursor-insert(cursor);cursor-reset(cursor);while(ret=cursor-next(cursor)=0)cursor-get_key(cursor,&key);cursor-get_value(cursor,&value);pr

14、intf(Got Record:%s,%sn,key,value);conn-close(conn,NULL);Demo程序運行和調試gcc-g-I.-o/tmp/wt_demo.o-c/working/wt_demo.cgcc-L.libs/$PWD/.libs/libwiredtiger-3.1.0.so/tmp/wt_demo.o-o/tmp/wt_demoLD_LIBRARY_PATH=$PWD/.libs/tmp/wt_demo編譯調試set print prettybr wt_demo.c:37rset scheduler-locking on編譯wiredtigergit clone https:/ wiredtigersh autogen.sh./configure&makeQ&A