[{"data":1,"prerenderedAt":196},["ShallowReactive",2],{"content:\u002Fprojects\u002Fmydb\u002Fmydb8":3,"surround:\u002Fprojects\u002Fmydb\u002Fmydb8":185},{"id":4,"title":5,"body":6,"categories":162,"date":164,"description":165,"draft":166,"extension":167,"image":168,"meta":169,"navigation":171,"path":172,"permalink":168,"published":168,"readingTime":173,"recommend":168,"references":168,"seo":178,"sitemap":179,"stem":180,"tags":181,"type":183,"__hash__":184},"content\u002Fposts\u002Fprojects\u002Fmydb\u002Fmydb8.md","MYDB 8. 索引管理",{"type":7,"value":8,"toc":155},"minimark",[9,21,25,28,31,34,37,40,51,54,57,65,68,74,77,80,86,89,92,98,101,107,110,116,119,122,125,128,131,134,140,143,149,152],[10,11,12,13,20],"p",{},"本章涉及代码都在 ",[14,15,19],"a",{"href":16,"rel":17},"https:\u002F\u002Fgithub.com\u002FCN-GuoZiyang\u002FMYDB\u002Ftree\u002Fmaster\u002Fsrc\u002Fmain\u002Fjava\u002Ftop\u002Fguoziyang\u002Fmydb\u002Fbackend\u002Fim",[18],"nofollow","backend\u002Fim"," 中。",[22,23,24],"h3",{"id":24},"前言",[10,26,27],{},"IM，即 Index Manager，索引管理器，为 MYDB 提供了基于 B+ 树的聚簇索引。目前 MYDB 只支持基于索引查找数据，不支持全表扫描。感兴趣的同学可以自行实现。",[10,29,30],{},"在依赖关系图中可以看到，IM 直接基于 DM，而没有基于 VM。索引的数据被直接插入数据库文件中，而不需要经过版本管理。",[10,32,33],{},"本节不赘述 B+ 树算法，更多描述实现。",[22,35,36],{"id":36},"二叉树索引",[10,38,39],{},"二叉树由一个个 Node 组成，每个 Node 都存储在一条 DataItem 中。结构如下：",[41,42,47],"pre",{"className":43,"code":45,"language":46},[44],"language-text","[LeafFlag][KeyNumber][SiblingUid]\n[Son0][Key0][Son1][Key1]...[SonN][KeyN]\n","text",[48,49,45],"code",{"__ignoreMap":50},"",[10,52,53],{},"其中 LeafFlag 标记了该节点是否是个叶子节点；KeyNumber 为该节点中 key 的个数；SiblingUid 是其兄弟节点存储在 DM 中的 UID。后续是穿插的子节点（SonN）和 KeyN。最后的一个 KeyN 始终为 MAX_VALUE，以此方便查找。",[10,55,56],{},"Node 类持有了其 B+ 树结构的引用，DataItem 的引用和 SubArray 的引用，用于方便快速修改数据和释放数据。",[41,58,63],{"className":59,"code":61,"language":62,"meta":50},[60],"language-java","public class Node {\n    BPlusTree tree;\n    DataItem dataItem;\n    SubArray raw;\n    long uid;\n    ...\n}\n","java",[48,64,61],{"__ignoreMap":50},[10,66,67],{},"于是生成一个根节点的数据可以写成如下：",[41,69,72],{"className":70,"code":71,"language":62,"meta":50},[60],"static byte[] newRootRaw(long left, long right, long key)  {\n    SubArray raw = new SubArray(new byte[NODE_SIZE], 0, NODE_SIZE);\n    setRawIsLeaf(raw, false);\n    setRawNoKeys(raw, 2);\n    setRawSibling(raw, 0);\n    setRawKthSon(raw, left, 0);\n    setRawKthKey(raw, key, 0);\n    setRawKthSon(raw, right, 1);\n    setRawKthKey(raw, Long.MAX_VALUE, 1);\n    return raw.raw;\n}\n",[48,73,71],{"__ignoreMap":50},[10,75,76],{},"该根节点的初始两个子节点为 left 和 right, 初始键值为 key。",[10,78,79],{},"类似的，生成一个空的根节点数据：",[41,81,84],{"className":82,"code":83,"language":62,"meta":50},[60],"static byte[] newNilRootRaw()  {\n    SubArray raw = new SubArray(new byte[NODE_SIZE], 0, NODE_SIZE);\n    setRawIsLeaf(raw, true);\n    setRawNoKeys(raw, 0);\n    setRawSibling(raw, 0);\n    return raw.raw;\n}\n",[48,85,83],{"__ignoreMap":50},[10,87,88],{},"Node 类有两个方法，用于辅助 B+ 树做插入和搜索操作，分别是 searchNext 方法和 leafSearchRange 方法。",[10,90,91],{},"searchNext 寻找对应 key 的 UID, 如果找不到，则返回兄弟节点的 UID。",[41,93,96],{"className":94,"code":95,"language":62,"meta":50},[60],"public SearchNextRes searchNext(long key) {\n    dataItem.rLock();\n    try {\n        SearchNextRes res = new SearchNextRes();\n        int noKeys = getRawNoKeys(raw);\n        for(int i = 0; i \u003C noKeys; i ++) {\n            long ik = getRawKthKey(raw, i);\n            if(key \u003C ik) {\n                res.uid = getRawKthSon(raw, i);\n                res.siblingUid = 0;\n                return res;\n            }\n        }\n        res.uid = 0;\n        res.siblingUid = getRawSibling(raw);\n        return res;\n    } finally {\n        dataItem.rUnLock();\n    }\n}\n",[48,97,95],{"__ignoreMap":50},[10,99,100],{},"leafSearchRange 方法在当前节点进行范围查找，范围是 [leftKey, rightKey]，这里约定如果 rightKey 大于等于该节点的最大的 key, 则还同时返回兄弟节点的 UID，方便继续搜索下一个节点。",[41,102,105],{"className":103,"code":104,"language":62,"meta":50},[60],"public LeafSearchRangeRes leafSearchRange(long leftKey, long rightKey) {\n    dataItem.rLock();\n    try {\n        int noKeys = getRawNoKeys(raw);\n        int kth = 0;\n        while(kth \u003C noKeys) {\n            long ik = getRawKthKey(raw, kth);\n            if(ik >= leftKey) {\n                break;\n            }\n            kth ++;\n        }\n        List\u003CLong> uids = new ArrayList\u003C>();\n        while(kth \u003C noKeys) {\n            long ik = getRawKthKey(raw, kth);\n            if(ik \u003C= rightKey) {\n                uids.add(getRawKthSon(raw, kth));\n                kth ++;\n            } else {\n                break;\n            }\n        }\n        long siblingUid = 0;\n        if(kth == noKeys) {\n            siblingUid = getRawSibling(raw);\n        }\n        LeafSearchRangeRes res = new LeafSearchRangeRes();\n        res.uids = uids;\n        res.siblingUid = siblingUid;\n        return res;\n    } finally {\n        dataItem.rUnLock();\n    }\n}\n",[48,106,104],{"__ignoreMap":50},[10,108,109],{},"由于 B+ 树在插入删除时，会动态调整，根节点不是固定节点，于是设置一个 bootDataItem，该 DataItem 中存储了根节点的 UID。可以注意到，IM 在操作 DM 时，使用的事务都是 SUPER_XID。",[41,111,114],{"className":112,"code":113,"language":62,"meta":50},[60],"public class BPlusTree {\n    DataItem bootDataItem;\n\n    private long rootUid() {\n        bootLock.lock();\n        try {\n            SubArray sa = bootDataItem.data();\n            return Parser.parseLong(Arrays.copyOfRange(sa.raw, sa.start, sa.start+8));\n        } finally {\n            bootLock.unlock();\n        }\n    }\n\n    private void updateRootUid(long left, long right, long rightKey) throws Exception {\n        bootLock.lock();\n        try {\n            byte[] rootRaw = Node.newRootRaw(left, right, rightKey);\n            long newRootUid = dm.insert(TransactionManagerImpl.SUPER_XID, rootRaw);\n            bootDataItem.before();\n            SubArray diRaw = bootDataItem.data();\n            System.arraycopy(Parser.long2Byte(newRootUid), 0, diRaw.raw, diRaw.start, 8);\n            bootDataItem.after(TransactionManagerImpl.SUPER_XID);\n        } finally {\n            bootLock.unlock();\n        }\n    }\n}\n",[48,115,113],{"__ignoreMap":50},[10,117,118],{},"IM 对上层模块主要提供两种能力：插入索引和搜索节点。向 B+ 树插入节点和搜索节点的算法和实现，不再赘述。",[10,120,121],{},"这里可能会有疑问，IM 为什么不提供删除索引的能力。当上层模块通过 VM 删除某个 Entry，实际的操作是设置其 XMAX。如果不去删除对应索引的话，当后续再次尝试读取该 Entry 时，是可以通过索引寻找到的，但是由于设置了 XMAX，寻找不到合适的版本而返回一个找不到内容的错误。",[22,123,124],{"id":124},"可能的错误与恢复",[10,126,127],{},"B+ 树在操作过程中，可能出现两种错误，分别是节点内部错误和节点间关系错误。",[10,129,130],{},"当节点内部错误发生时，即当 Ti 在对节点的数据进行更改时，MYDB 发生了崩溃。由于 IM 依赖于 DM，在数据库重启后，Ti 会被撤销（undo），对节点的错误影响会被消除。",[10,132,133],{},"如果出现了节点间错误，那么一定是下面这种情况：某次对 u 节点的插入操作创建了新节点 v, 此时 sibling(u)=v，但是 v 却并没有被插入到父节点中。",[41,135,138],{"className":136,"code":137,"language":46},[44],"[parent]\n    \n    v\n   [u] -> [v]\n",[48,139,137],{"__ignoreMap":50},[10,141,142],{},"正确的状态应当如下：",[41,144,147],{"className":145,"code":146,"language":46},[44],"[ parent ]\n       \n v      v\n[u] -> [v]\n",[48,148,146],{"__ignoreMap":50},[10,150,151],{},"这时，如果要对节点进行插入或者搜索操作，如果失败，就会继续迭代它的兄弟节点，最终还是可以找到 v 节点。唯一的缺点仅仅是，无法直接通过父节点找到 v 了，只能间接地通过 u 获取到 v。",[10,153,154],{},"今天是 12 月 25 日，圣诞节。Happy Xmas！",{"title":50,"searchDepth":156,"depth":156,"links":157},4,[158,160,161],{"id":24,"depth":159,"text":24},3,{"id":36,"depth":159,"text":36},{"id":124,"depth":159,"text":124},[163],"项目","2021-12-24 21:01:00","MYDB 基于 B+ 树实现了聚簇索引。通过 IM 直接与数据管理（DM）交互，省略了版本管理（VM）层，确保索引数据直接写入数据库文件。章节中详细描述了二叉树索引的结构，涵盖节点的基本组成元素，包括叶子标记、键数量及兄弟节点标识等，为实现索引查找提供了基础框架。",false,"md",null,{"slots":170},{},true,"\u002Fprojects\u002Fmydb\u002Fmydb8",{"text":174,"minutes":175,"time":176,"words":177},"7 min read",6.1,366000,1220,{"title":5,"description":165},{"loc":172},"posts\u002Fprojects\u002Fmydb\u002Fmydb8",[62,182],"mydb","tech","biHSUoDmmysmkJc_qcUO-UxdHm4lT8thKJeu0rGCHlw",[186,191],{"title":187,"path":188,"stem":189,"date":190,"type":183,"children":-1},"MYDB 7. 死锁检测与 VM 的实现","\u002Fprojects\u002Fmydb\u002Fmydb7","posts\u002Fprojects\u002Fmydb\u002Fmydb7","2021-12-23 21:20:00",{"title":192,"path":193,"stem":194,"date":195,"type":183,"children":-1},"MYDB 9. 字段与表管理","\u002Fprojects\u002Fmydb\u002Fmydb9","posts\u002Fprojects\u002Fmydb\u002Fmydb9","2021-12-25 15:44:00",1787554445610]