[{"data":1,"prerenderedAt":303},["ShallowReactive",2],{"content:\u002Fprojects\u002Fmydb\u002Fmydb7":3,"surround:\u002Fprojects\u002Fmydb\u002Fmydb7":292},{"id":4,"title":5,"body":6,"categories":269,"date":271,"description":272,"draft":273,"extension":274,"image":275,"meta":276,"navigation":278,"path":279,"permalink":275,"published":275,"readingTime":280,"recommend":275,"references":275,"seo":285,"sitemap":286,"stem":287,"tags":288,"type":290,"__hash__":291},"content\u002Fposts\u002Fprojects\u002Fmydb\u002Fmydb7.md","MYDB 7. 死锁检测与 VM 的实现",{"type":7,"value":8,"toc":261},"minimark",[9,21,25,28,31,34,37,48,51,54,57,67,70,78,81,84,87,93,96,102,105,111,114,117,123,126,132,135,141,144,148,151,157,164,170,176,182,188,194,197,203,209,215,221,227,233,239,242,245],[10,11,12,13,20],"p",{},"本章涉及代码都在 ",[14,15,19],"a",{"href":16,"rel":17},"https:\u002F\u002Fgithub.com\u002FCN-GuoZiyang\u002FMYDB\u002Ftree\u002Fmaster\u002Fsrc\u002Fmain\u002Fjava\u002Ftop\u002Fguoziyang\u002Fmydb\u002Fbackend\u002Fvm",[18],"nofollow","backend\u002Fvm"," 中。",[22,23,24],"h3",{"id":24},"前言",[10,26,27],{},"本节将收尾 VM 层，介绍一下 MVCC 可能导致的版本跳跃问题，以及 MYDB 如何避免 2PL 导致的死锁，并将其整合为 Version Manager。",[22,29,30],{"id":30},"版本跳跃问题",[10,32,33],{},"说到版本跳跃之前，顺便提一嘴，MVCC 的实现，使得 MYDB 在撤销或是回滚事务很简单：只需要将这个事务标记为 aborted 即可。根据前一章提到的可见性，每个事务都只能看到其他 committed 的事务所产生的数据，一个 aborted 事务产生的数据，就不会对其他事务产生任何影响了，也就相当于，这个事务不曾存在过。",[10,35,36],{},"版本跳跃问题，考虑如下的情况，假设 X 最初只有 x0 版本，T1 和 T2 都是可重复读的隔离级别：",[38,39,44],"pre",{"className":40,"code":42,"language":43},[41],"language-text","T1 begin\nT2 begin\nR1(X) \u002F\u002F T1 读取 x0\nR2(X) \u002F\u002F T2 读取 x0\nU1(X) \u002F\u002F T1 将 X 更新到 x1\nT1 commit\nU2(X) \u002F\u002F T2 将 X 更新到 x2\nT2 commit\n","text",[45,46,42],"code",{"__ignoreMap":47},"",[10,49,50],{},"这种情况实际运行起来是没问题的，但是逻辑上不太正确。T1 将 X 从 x0 更新为了 x1，这是没错的。但是 T2 则是将 X 从 x0 更新成了 x2，跳过了 x1 版本。",[10,52,53],{},"读提交是允许版本跳跃的，而可重复读则是不允许版本跳跃的。解决版本跳跃的思路也很简单：如果 Ti 需要修改 X，而 X 已经被 Ti 不可见的事务 Tj 修改了，那么要求 Ti 回滚。",[10,55,56],{},"上一节中就总结了，Ti 不可见的 Tj，有两种情况：",[58,59,60,64],"ol",{},[61,62,63],"li",{},"XID(Tj) > XID(Ti)",[61,65,66],{},"Tj in SP(Ti)",[10,68,69],{},"于是版本跳跃的检查也就很简单了，取出要修改的数据 X 的最新提交版本，并检查该最新版本的创建者对当前事务是否可见：",[38,71,76],{"className":72,"code":74,"language":75,"meta":47},[73],"language-java","public static boolean isVersionSkip(TransactionManager tm, Transaction t, Entry e) {\n    long xmax = e.getXmax();\n    if(t.level == 0) {\n        return false;\n    } else {\n        return tm.isCommitted(xmax) && (xmax > t.xid  t.isInSnapshot(xmax));\n  }\n}\n","java",[45,77,74],{"__ignoreMap":47},[22,79,80],{"id":80},"死锁检测",[10,82,83],{},"上一节提到了 2PL 会阻塞事务，直至持有锁的线程释放锁。可以将这种等待关系抽象成有向边，例如 Tj 在等待 Ti，就可以表示为 Tj --> Ti。这样，无数有向边就可以形成一个图（不一定是连通图）。检测死锁也就简单了，只需要查看这个图中是否有环即可。",[10,85,86],{},"MYDB 使用一个 LockTable 对象，在内存中维护这张图。维护结构如下：",[38,88,91],{"className":89,"code":90,"language":75,"meta":47},[73],"public class LockTable {\n\n    private Map\u003CLong, List\u003CLong>> x2u;  \u002F\u002F 某个 XID 已经获得的资源的 UID 列表\n    private Map\u003CLong, Long> u2x;        \u002F\u002F UID 被某个 XID 持有\n    private Map\u003CLong, List\u003CLong>> wait; \u002F\u002F 正在等待 UID 的 XID 列表\n    private Map\u003CLong, Lock> waitLock;   \u002F\u002F 正在等待资源的 XID 的锁\n    private Map\u003CLong, Long> waitU;      \u002F\u002F XID 正在等待的 UID\n    private Lock lock;\n\n    ...\n}\n",[45,92,90],{"__ignoreMap":47},[10,94,95],{},"在每次出现等待的情况时，就尝试向图中增加一条边，并进行死锁检测。如果检测到死锁，就撤销这条边，不允许添加，并撤销该事务。",[38,97,100],{"className":98,"code":99,"language":75,"meta":47},[73],"\u002F\u002F 不需要等待则返回 null，否则返回锁对象\n\u002F\u002F 会造成死锁则抛出异常\npublic Lock add(long xid, long uid) throws Exception {\n    lock.lock();\n    try {\n        if(isInList(x2u, xid, uid)) {\n            return null;\n        }\n        if(!u2x.containsKey(uid)) {\n            u2x.put(uid, xid);\n            putIntoList(x2u, xid, uid);\n            return null;\n        }\n        waitU.put(xid, uid);\n        putIntoList(wait, xid, uid);\n        if(hasDeadLock()) {\n            waitU.remove(xid);\n            removeFromList(wait, uid, xid);\n            throw Error.DeadlockException;\n        }\n        Lock l = new ReentrantLock();\n        l.lock();\n        waitLock.put(xid, l);\n        return l;\n    } finally {\n        lock.unlock();\n    }\n}\n",[45,101,99],{"__ignoreMap":47},[10,103,104],{},"调用 add，如果需要等待的话，会返回一个上了锁的 Lock 对象。调用方在获取到该对象时，需要尝试获取该对象的锁，由此实现阻塞线程的目的，例如：",[38,106,109],{"className":107,"code":108,"language":75,"meta":47},[73],"Lock l = lt.add(xid, uid);\nif(l != null) {\n    l.lock();   \u002F\u002F 阻塞在这一步\n    l.unlock();\n}\n",[45,110,108],{"__ignoreMap":47},[10,112,113],{},"查找图中是否有环的算法也非常简单，就是一个深搜，只是需要注意这个图不一定是连通图。思路就是为每个节点设置一个访问戳，都初始化为 -1，随后遍历所有节点，以每个非 -1 的节点作为根进行深搜，并将深搜该连通图中遇到的所有节点都设置为同一个数字，不同的连通图数字不同。这样，如果在遍历某个图时，遇到了之前遍历过的节点，说明出现了环。",[10,115,116],{},"实现很简单：",[38,118,121],{"className":119,"code":120,"language":75,"meta":47},[73],"private boolean hasDeadLock() {\n    xidStamp = new HashMap\u003C>();\n    stamp = 1;\n    for(long xid : x2u.keySet()) {\n        Integer s = xidStamp.get(xid);\n        if(s != null && s > 0) {\n            continue;\n        }\n        stamp ++;\n        if(dfs(xid)) {\n            return true;\n        }\n    }\n    return false;\n}\n\nprivate boolean dfs(long xid) {\n    Integer stp = xidStamp.get(xid);\n    if(stp != null && stp == stamp) {\n        return true;\n    }\n    if(stp != null && stp \u003C stamp) {\n        return false;\n    }\n    xidStamp.put(xid, stamp);\n\n    Long uid = waitU.get(xid);\n    if(uid == null) return false;\n    Long x = u2x.get(uid);\n    assert x != null;\n    return dfs(x);\n}\n",[45,122,120],{"__ignoreMap":47},[10,124,125],{},"在一个事务 commit 或者 abort 时，就可以释放所有它持有的锁，并将自身从等待图中删除。",[38,127,130],{"className":128,"code":129,"language":75,"meta":47},[73],"public void remove(long xid) {\n    lock.lock();\n    try {\n        List\u003CLong> l = x2u.get(xid);\n        if(l != null) {\n            while(l.size() > 0) {\n                Long uid = l.remove(0);\n                selectNewXID(uid);\n            }\n        }\n        waitU.remove(xid);\n        x2u.remove(xid);\n        waitLock.remove(xid);\n    } finally {\n        lock.unlock();\n    }\n}\n",[45,131,129],{"__ignoreMap":47},[10,133,134],{},"while 循环释放掉了这个线程所有持有的资源的锁，这些资源可以被等待的线程所获取：",[38,136,139],{"className":137,"code":138,"language":75,"meta":47},[73],"\u002F\u002F 从等待队列中选择一个 xid 来占用 uid\nprivate void selectNewXID(long uid) {\n    u2x.remove(uid);\n    List\u003CLong> l = wait.get(uid);\n    if(l == null) return;\n    assert l.size() > 0;\n    while(l.size() > 0) {\n        long xid = l.remove(0);\n        if(!waitLock.containsKey(xid)) {\n            continue;\n        } else {\n            u2x.put(uid, xid);\n            Lock lo = waitLock.remove(xid);\n            waitU.remove(xid);\n            lo.unlock();\n            break;\n        }\n    }\n    if(l.size() == 0) wait.remove(uid);\n}\n",[45,140,138],{"__ignoreMap":47},[10,142,143],{},"从 List 开头开始尝试解锁，还是个公平锁。解锁时，将该 Lock 对象 unlock 即可，这样业务线程就获取到了锁，就可以继续执行了。",[22,145,147],{"id":146},"vm-的实现","VM 的实现",[10,149,150],{},"VM 层通过 VersionManager 接口，向上层提供功能，如下：",[38,152,155],{"className":153,"code":154,"language":75,"meta":47},[73],"public interface VersionManager {\n    byte[] read(long xid, long uid) throws Exception;\n    long insert(long xid, byte[] data) throws Exception;\n    boolean delete(long xid, long uid) throws Exception;\n\n    long begin(int level);\n    void commit(long xid) throws Exception;\n    void abort(long xid);\n}\n",[45,156,154],{"__ignoreMap":47},[10,158,159,160,163],{},"同时，VM 的实现类还被设计为 Entry 的缓存，需要继承 ",[45,161,162],{"code":162},"AbstractCache\u003CEntry>","。需要实现的获取到缓存和从缓存释放的方法很简单：",[38,165,168],{"className":166,"code":167,"language":75,"meta":47},[73],"@Override\nprotected Entry getForCache(long uid) throws Exception {\n    Entry entry = Entry.loadEntry(this, uid);\n    if(entry == null) {\n        throw Error.NullEntryException;\n    }\n    return entry;\n}\n\n@Override\nprotected void releaseForCache(Entry entry) {\n    entry.remove();\n}\n",[45,169,167],{"__ignoreMap":47},[10,171,172,175],{},[45,173,174],{"code":174},"begin()"," 开启一个事务，并初始化事务的结构，将其存放在 activeTransaction 中，用于检查和快照使用：",[38,177,180],{"className":178,"code":179,"language":75,"meta":47},[73],"@Override\npublic long begin(int level) {\n    lock.lock();\n    try {\n        long xid = tm.begin();\n        Transaction t = Transaction.newTransaction(xid, level, activeTransaction);\n        activeTransaction.put(xid, t);\n        return xid;\n    } finally {\n        lock.unlock();\n    }\n}\n",[45,181,179],{"__ignoreMap":47},[10,183,184,187],{},[45,185,186],{"code":186},"commit()"," 方法提交一个事务，主要就是 free 掉相关的结构，并且释放持有的锁，并修改 TM 状态：",[38,189,192],{"className":190,"code":191,"language":75,"meta":47},[73],"@Override\npublic void commit(long xid) throws Exception {\n    lock.lock();\n    Transaction t = activeTransaction.get(xid);\n    lock.unlock();\n    try {\n        if(t.err != null) {\n            throw t.err;\n        }\n    } catch(NullPointerException n) {\n        System.out.println(xid);\n        System.out.println(activeTransaction.keySet());\n        Panic.panic(n);\n    }\n    lock.lock();\n    activeTransaction.remove(xid);\n    lock.unlock();\n    lt.remove(xid);\n    tm.commit(xid);\n}\n",[45,193,191],{"__ignoreMap":47},[10,195,196],{},"abort 事务的方法则有两种，手动和自动。手动指的是调用 abort() 方法，而自动，则是在事务被检测出出现死锁时，会自动撤销回滚事务；或者出现版本跳跃时，也会自动回滚：",[38,198,201],{"className":199,"code":200,"language":75,"meta":47},[73],"private void internAbort(long xid, boolean autoAborted) {\n    lock.lock();\n    Transaction t = activeTransaction.get(xid);\n    if(!autoAborted) {\n        activeTransaction.remove(xid);\n    }\n    lock.unlock();\n    if(t.autoAborted) return;\n    lt.remove(xid);\n    tm.abort(xid);\n}\n",[45,202,200],{"__ignoreMap":47},[10,204,205,208],{},[45,206,207],{"code":207},"read()"," 方法读取一个 entry，注意判断下可见性即可：",[38,210,213],{"className":211,"code":212,"language":75,"meta":47},[73],"@Override\npublic byte[] read(long xid, long uid) throws Exception {\n    lock.lock();\n    Transaction t = activeTransaction.get(xid);\n    lock.unlock();\n    if(t.err != null) {\n        throw t.err;\n    }\n    Entry entry = super.get(uid);\n    try {\n        if(Visibility.isVisible(tm, t, entry)) {\n            return entry.data();\n        } else {\n            return null;\n        }\n    } finally {\n        entry.release();\n    }\n}\n",[45,214,212],{"__ignoreMap":47},[10,216,217,220],{},[45,218,219],{"code":219},"insert()"," 则是将数据包裹成 Entry，无脑交给 DM 插入即可：",[38,222,225],{"className":223,"code":224,"language":75,"meta":47},[73],"@Override\npublic long insert(long xid, byte[] data) throws Exception {\n    lock.lock();\n    Transaction t = activeTransaction.get(xid);\n    lock.unlock();\n    if(t.err != null) {\n        throw t.err;\n    }\n    byte[] raw = Entry.wrapEntryRaw(xid, data);\n    return dm.insert(xid, raw);\n}\n",[45,226,224],{"__ignoreMap":47},[10,228,229,232],{},[45,230,231],{"code":231},"delete()"," 方法看起来略为复杂：",[38,234,237],{"className":235,"code":236,"language":75,"meta":47},[73],"@Override\npublic boolean delete(long xid, long uid) throws Exception {\n    lock.lock();\n    Transaction t = activeTransaction.get(xid);\n    lock.unlock();\n\n    if(t.err != null) {\n        throw t.err;\n    }\n    Entry entry = super.get(uid);\n    try {\n        if(!Visibility.isVisible(tm, t, entry)) {\n            return false;\n        }\n        Lock l = null;\n        try {\n            l = lt.add(xid, uid);\n        } catch(Exception e) {\n            t.err = Error.ConcurrentUpdateException;\n            internAbort(xid, true);\n            t.autoAborted = true;\n            throw t.err;\n        }\n        if(l != null) {\n            l.lock();\n            l.unlock();\n        }\n        if(entry.getXmax() == xid) {\n            return false;\n        }\n        if(Visibility.isVersionSkip(tm, t, entry)) {\n            t.err = Error.ConcurrentUpdateException;\n            internAbort(xid, true);\n            t.autoAborted = true;\n            throw t.err;\n        }\n        entry.setXmax(xid);\n        return true;\n    } finally {\n        entry.release();\n    }\n}\n",[45,238,236],{"__ignoreMap":47},[10,240,241],{},"实际上主要是前置的三件事：一是可见性判断，二是获取资源的锁，三是版本跳跃判断。删除的操作只有一个设置 XMAX。",[10,243,244],{},"今天是 2021 年 12 月 24 日，平安夜。",[246,247,248],"blockquote",{},[10,249,250,251,254,255,257,258,260],{},"愿你有一个灿烂的前程",[252,253],"br",{},"\n愿你有情人终成眷属",[252,256],{},"\n愿你在尘世获得幸福",[252,259],{},"\n我只愿面朝大海，春暖花开",{"title":47,"searchDepth":262,"depth":262,"links":263},4,[264,266,267,268],{"id":24,"depth":265,"text":24},3,{"id":30,"depth":265,"text":30},{"id":80,"depth":265,"text":80},{"id":146,"depth":265,"text":147},[270],"项目","2021-12-23 21:20:00","VM 需要处理 MVCC 导致的版本跳跃及死锁问题。通过简单的标记方式，MYDB 能轻松撤销或回滚事务，确保 aborted 事务的数据不会影响其他事务。这种设计使得事务在处理并发时更为高效和可靠，避免了传统 2PL 方法常见的死锁风险，提升了整体系统的稳定性和性能。",false,"md",null,{"slots":277},{},true,"\u002Fprojects\u002Fmydb\u002Fmydb7",{"text":281,"minutes":282,"time":283,"words":284},"11 min read",10.195,611700,2039,{"title":5,"description":272},{"loc":279},"posts\u002Fprojects\u002Fmydb\u002Fmydb7",[75,289],"mydb","tech","BoAAZqpA59mLTFbEE5c0D_KVs2JbJgxc2vudc1XgHg4",[293,298],{"title":294,"path":295,"stem":296,"date":297,"type":290,"children":-1},"MYDB 6. 记录的版本与事务隔离","\u002Fprojects\u002Fmydb\u002Fmydb6","posts\u002Fprojects\u002Fmydb\u002Fmydb6","2021-12-18 14:58:00",{"title":299,"path":300,"stem":301,"date":302,"type":290,"children":-1},"MYDB 8. 索引管理","\u002Fprojects\u002Fmydb\u002Fmydb8","posts\u002Fprojects\u002Fmydb\u002Fmydb8","2021-12-24 21:01:00",1787554445709]