[{"data":1,"prerenderedAt":223},["ShallowReactive",2],{"content:\u002Fprojects\u002Fmydb\u002Fmydb9":3,"surround:\u002Fprojects\u002Fmydb\u002Fmydb9":212},{"id":4,"title":5,"body":6,"categories":189,"date":191,"description":192,"draft":193,"extension":194,"image":195,"meta":196,"navigation":198,"path":199,"permalink":195,"published":195,"readingTime":200,"recommend":195,"references":195,"seo":205,"sitemap":206,"stem":207,"tags":208,"type":210,"__hash__":211},"content\u002Fposts\u002Fprojects\u002Fmydb\u002Fmydb9.md","MYDB 9. 字段与表管理",{"type":7,"value":8,"toc":181},"minimark",[9,27,31,34,38,41,44,55,66,73,76,79,82,85,91,94,100,103,106,114,117,123,126,132,135,150,153,156,162,166,169,175,178],[10,11,12,13,20,21,26],"p",{},"本章涉及代码都在 ",[14,15,19],"a",{"href":16,"rel":17},"https:\u002F\u002Fgithub.com\u002FCN-GuoZiyang\u002FMYDB\u002Ftree\u002Fmaster\u002Fsrc\u002Fmain\u002Fjava\u002Ftop\u002Fguoziyang\u002Fmydb\u002Fbackend\u002Fparser",[18],"nofollow","backend\u002Fparser"," 与 ",[14,22,25],{"href":23,"rel":24},"https:\u002F\u002Fgithub.com\u002FCN-GuoZiyang\u002FMYDB\u002Ftree\u002Fmaster\u002Fsrc\u002Fmain\u002Fjava\u002Ftop\u002Fguoziyang\u002Fmydb\u002Fbackend\u002Ftbm",[18],"backend\u002Ftbm"," 中。",[28,29,30],"h3",{"id":30},"前言",[10,32,33],{},"本章概述 TBM，即表管理器的实现。TBM 实现了对字段结构和表结构的管理。同时简要介绍 MYDB 使用的类 SQL 语句的解析。",[28,35,37],{"id":36},"sql-解析器","SQL 解析器",[10,39,40],{},"Parser 实现了对类 SQL 语句的结构化解析，将语句中包含的信息封装为对应语句的类，这些类可见 top.guoziyang.mydb.backend.parser.statement 包。",[10,42,43],{},"MYDB 实现的 SQL 语句语法如下：",[45,46,51],"pre",{"className":47,"code":49,"language":50},[48],"language-text","\u003Cbegin statement>\n    begin [isolation level (read committedrepeatable read)]\n        begin isolation level read committed\n\n\u003Ccommit statement>\n    commit\n\n\u003Cabort statement>\n    abort\n\n\u003Ccreate statement>\n    create table \u003Ctable name>\n    \u003Cfield name> \u003Cfield type>\n    \u003Cfield name> \u003Cfield type>\n    ...\n    \u003Cfield name> \u003Cfield type>\n    [(index \u003Cfield name list>)]\n        create table students\n        id int32,\n        name string,\n        age int32,\n        (index id name)\n\n\u003Cdrop statement>\n    drop table \u003Ctable name>\n        drop table students\n\n\u003Cselect statement>\n    select (*\u003Cfield name list>) from \u003Ctable name> [\u003Cwhere statement>]\n        select * from student where id = 1\n        select name from student where id > 1 and id \u003C 4\n        select name, age, id from student where id = 12\n\n\u003Cinsert statement>\n    insert into \u003Ctable name> values \u003Cvalue list>\n        insert into student values 5 \"Zhang Yuanjia\" 22\n\n\u003Cdelete statement>\n    delete from \u003Ctable name> \u003Cwhere statement>\n        delete from student where name = \"Zhang Yuanjia\"\n\n\u003Cupdate statement>\n    update \u003Ctable name> set \u003Cfield name>=\u003Cvalue> [\u003Cwhere statement>]\n        update student set name = \"ZYJ\" where id = 5\n\n\u003Cwhere statement>\n    where \u003Cfield name> (>\u003C=) \u003Cvalue> [(andor) \u003Cfield name> (>\u003C=) \u003Cvalue>]\n        where age > 10 or age \u003C 3\n\n\u003Cfield name> \u003Ctable name>\n    [a-zA-Z][a-zA-Z0-9_]*\n\n\u003Cfield type>\n    int32 int64 string\n\n\u003Cvalue>\n    .*\n","text",[52,53,49],"code",{"__ignoreMap":54},"",[10,56,57,58,61,62,65],{},"parser 包的 Tokenizer 类，对语句进行逐字节解析，根据空白符或者上述词法规则，将语句切割成多个 token。对外提供了 ",[52,59,60],{"code":60},"peek()","、",[52,63,64],{"code":64},"pop()"," 方法方便取出 Token 进行解析。切割的实现不赘述。",[10,67,68,69,72],{},"Parser 类则直接对外提供了 ",[52,70,71],{"code":71},"Parse(byte[] statement)"," 方法，核心就是一个调用 Tokenizer 类分割 Token，并根据词法规则包装成具体的 Statement 类并返回。解析过程很简单，仅仅是根据第一个 Token 来区分语句类型，并分别处理，不再赘述。",[10,74,75],{},"虽然根据编译原理，词法分析应当写一个自动机去做的，但是又不是不能用",[28,77,78],{"id":78},"字段与表管理",[10,80,81],{},"注意，这里的字段与表管理，不是管理各个条目中不同的字段的数值等信息，而是管理表和字段的结构数据，例如表名、表字段信息和字段索引等。",[10,83,84],{},"由于 TBM 基于 VM，单个字段信息和表信息都是直接保存在 Entry 中。字段的二进制表示如下：",[45,86,89],{"className":87,"code":88,"language":50},[48],"[FieldName][TypeName][IndexUid]\n",[52,90,88],{"__ignoreMap":54},[10,92,93],{},"这里 FieldName 和 TypeName，以及后面的表明，存储的都是字节形式的字符串。这里规定一个字符串的存储方式，以明确其存储边界。",[45,95,98],{"className":96,"code":97,"language":50},[48],"[StringLength][StringData]\n",[52,99,97],{"__ignoreMap":54},[10,101,102],{},"TypeName 为字段的类型，限定为 int32、int64 和 string 类型。如果这个字段有索引，那个 IndexUID 指向了索引二叉树的根，否则该字段为 0。",[10,104,105],{},"根据这个结构，通过一个 UID 从 VM 中读取并解析如下：",[45,107,112],{"className":108,"code":110,"language":111,"meta":54},[109],"language-java","public static Field loadField(Table tb, long uid) {\n    byte[] raw = null;\n    try {\n        raw = ((TableManagerImpl)tb.tbm).vm.read(TransactionManagerImpl.SUPER_XID, uid);\n    } catch (Exception e) {\n        Panic.panic(e);\n    }\n    assert raw != null;\n    return new Field(uid, tb).parseSelf(raw);\n}\n\nprivate Field parseSelf(byte[] raw) {\n    int position = 0;\n    ParseStringRes res = Parser.parseString(raw);\n    fieldName = res.str;\n    position += res.next;\n    res = Parser.parseString(Arrays.copyOfRange(raw, position, raw.length));\n    fieldType = res.str;\n    position += res.next;\n    this.index = Parser.parseLong(Arrays.copyOfRange(raw, position, position+8));\n    if(index != 0) {\n        try {\n            bt = BPlusTree.load(index, ((TableManagerImpl)tb.tbm).dm);\n        } catch(Exception e) {\n            Panic.panic(e);\n        }\n    }\n    return this;\n}\n","java",[52,113,110],{"__ignoreMap":54},[10,115,116],{},"创建一个字段的方法类似，将相关的信息通过 VM 持久化即可：",[45,118,121],{"className":119,"code":120,"language":111,"meta":54},[109],"private void persistSelf(long xid) throws Exception {\n    byte[] nameRaw = Parser.string2Byte(fieldName);\n    byte[] typeRaw = Parser.string2Byte(fieldType);\n    byte[] indexRaw = Parser.long2Byte(index);\n    this.uid = ((TableManagerImpl)tb.tbm).vm.insert(xid, Bytes.concat(nameRaw, typeRaw, indexRaw));\n}\n",[52,122,120],{"__ignoreMap":54},[10,124,125],{},"一个数据库中存在多张表，TBM 使用链表的形式将其组织起来，每一张表都保存一个指向下一张表的 UID。表的二进制结构如下：",[45,127,130],{"className":128,"code":129,"language":50},[48],"[TableName][NextTable]\n[Field1Uid][Field2Uid]...[FieldNUid]\n",[52,131,129],{"__ignoreMap":54},[10,133,134],{},"这里由于每个 Entry 中的数据，字节数是确定的，于是无需保存字段的个数。根据 UID 从 Entry 中读取表数据的过程和读取字段的过程类似。",[10,136,137,138,141,142,145,146,149],{},"对表和字段的操作，有一个很重要的步骤，就是计算 Where 条件的范围，目前 MYDB 的 Where 只支持两个条件的与和或。例如有条件的 Delete，计算 Where，最终就需要获取到条件范围内所有的 UID。MYDB 只支持已索引字段作为 Where 的条件。计算 Where 的范围，具体可以查看 Table 的 ",[52,139,140],{"code":140},"parseWhere()"," 和 ",[52,143,144],{"code":144},"calWhere()"," 方法，以及 Field 类的 ",[52,147,148],{"code":148},"calExp()"," 方法。",[10,151,152],{},"由于 TBM 的表管理，使用的是链表串起的 Table 结构，所以就必须保存一个链表的头节点，即第一个表的 UID，这样在 MYDB 启动时，才能快速找到表信息。",[10,154,155],{},"MYDB 使用 Booter 类和 bt 文件，来管理 MYDB 的启动信息，虽然现在所需的启动信息，只有一个：头表的 UID。Booter 类对外提供了两个方法：load 和 update，并保证了其原子性。update 在修改 bt 文件内容时，没有直接对 bt 文件进行修改，而是首先将内容写入一个 bt_tmp 文件中，随后将这个文件重命名为 bt 文件。以期通过操作系统重命名文件的原子性，来保证操作的原子性。",[45,157,160],{"className":158,"code":159,"language":111,"meta":54},[109],"public void update(byte[] data) {\n    File tmp = new File(path + BOOTER_TMP_SUFFIX);\n    try {\n        tmp.createNewFile();\n    } catch (Exception e) {\n        Panic.panic(e);\n    }\n    if(!tmp.canRead()  !tmp.canWrite()) {\n        Panic.panic(Error.FileCannotRWException);\n    }\n    try(FileOutputStream out = new FileOutputStream(tmp)) {\n        out.write(data);\n        out.flush();\n    } catch(IOException e) {\n        Panic.panic(e);\n    }\n    try {\n        Files.move(tmp.toPath(), new File(path+BOOTER_SUFFIX).toPath(), StandardCopyOption.REPLACE_EXISTING);\n    } catch(IOException e) {\n        Panic.panic(e);\n    }\n    file = new File(path+BOOTER_SUFFIX);\n    if(!file.canRead()  !file.canWrite()) {\n        Panic.panic(Error.FileCannotRWException);\n    }\n}\n",[52,161,159],{"__ignoreMap":54},[28,163,165],{"id":164},"tablemanager","TableManager",[10,167,168],{},"TBM 层对外提供服务的是 TableManager 接口，如下：",[45,170,173],{"className":171,"code":172,"language":111,"meta":54},[109],"public interface TableManager {\n    BeginRes begin(Begin begin);\n    byte[] commit(long xid) throws Exception;\n    byte[] abort(long xid);\n\n    byte[] show(long xid);\n    byte[] create(long xid, Create create) throws Exception;\n\n    byte[] insert(long xid, Insert insert) throws Exception;\n    byte[] read(long xid, Select select) throws Exception;\n    byte[] update(long xid, Update update) throws Exception;\n    byte[] delete(long xid, Delete delete) throws Exception;\n}\n",[52,174,172],{"__ignoreMap":54},[10,176,177],{},"由于 TableManager 已经是直接被最外层 Server 调用（MYDB 是 C\u002FS 结构），这些方法直接返回执行的结果，例如错误信息或者结果信息的字节数组（可读）。",[10,179,180],{},"各个方法的具体实现很简单，不再赘述，无非是调用 VM 的相关方法。唯一值得注意的一个小点是，在创建新表时，采用的时头插法，所以每次创建表都需要更新 Booter 文件。",{"title":54,"searchDepth":182,"depth":182,"links":183},4,[184,186,187,188],{"id":30,"depth":185,"text":30},3,{"id":36,"depth":185,"text":37},{"id":78,"depth":185,"text":78},{"id":164,"depth":185,"text":165},[190],"项目","2021-12-25 15:44:00","表管理器（TBM）的职责是管理字段和表结构。通过对类 SQL 语句的结构化解析，Parser 能将语句信息封装为相应的类，从而简化后续操作。章节内容也包括 MYDB 使用的 SQL 语句语法，为理解整个管理过程提供了基础。",false,"md",null,{"slots":197},{},true,"\u002Fprojects\u002Fmydb\u002Fmydb9",{"text":201,"minutes":202,"time":203,"words":204},"8 min read",7.395,443700,1479,{"title":5,"description":192},{"loc":199},"posts\u002Fprojects\u002Fmydb\u002Fmydb9",[111,209],"mydb","tech","iHrzu-ENF7T55WF3EcQGLu1KDPfXPVL4T3SzDo96YU8",[213,218],{"title":214,"path":215,"stem":216,"date":217,"type":210,"children":-1},"MYDB 8. 索引管理","\u002Fprojects\u002Fmydb\u002Fmydb8","posts\u002Fprojects\u002Fmydb\u002Fmydb8","2021-12-24 21:01:00",{"title":219,"path":220,"stem":221,"date":222,"type":210,"children":-1},"MYDB 10. 服务端客户端的实现及其通信规则","\u002Fprojects\u002Fmydb\u002Fmydb10","posts\u002Fprojects\u002Fmydb\u002Fmydb10","2021-12-25 18:26:00",1787554445583]