[{"data":1,"prerenderedAt":1316},["ShallowReactive",2],{"content:\u002Fen\u002Ffiddling\u002Fopnsense-transparent-proxy":3,"series:content_en":607},{"id":4,"title":5,"authorship":6,"body":7,"categories":577,"date":579,"description":580,"draft":581,"extension":582,"image":583,"meta":584,"navigation":586,"path":587,"permalink":588,"published":588,"readingTime":589,"recommend":588,"references":588,"seo":594,"seoDescription":595,"seoTitle":588,"sitemap":596,"stem":597,"tags":598,"type":605,"__hash__":606},"content_en\u002Fposts\u002Ffiddling\u002Fopnsense-transparent-proxy.md","Transparent Proxying and Traffic Routing with OPNsense","human-only",{"type":8,"value":9,"toc":559},"minimark",[10,15,27,45,48,51,54,64,67,71,76,79,91,104,114,117,125,128,143,147,158,164,172,183,187,190,196,202,205,249,252,285,288,338,341,345,348,351,354,357,363,366,369,373,376,379,395,401,408,418,422,430,436,442,448,454,462,465,471,475,480,488,494,498,501,508,515,518,522,525,528,531,537,544,547,550,553,556],[11,12,14],"h3",{"id":13},"introduction","Introduction",[16,17,18,19,26],"p",{},"My original transparent proxy and traffic-routing setup used iKuai as the main router and OpenWrt as a side router, the standard approach in most online tutorials. Later I read that iKuai might ",[20,21,25],"a",{"href":22,"rel":23},"https:\u002F\u002Fwusiyu.me\u002F2022-ikuai-non-cloud-background-activities\u002F",[24],"nofollow","generate unrequested traffic and report information in the background",". Being a closed-source system developed in China also left me with security concerns.",[16,28,29,30,34,35,39,40,44],{},"I switched to ",[20,31,33],{"href":32},"\u002Fen\u002Ffiddling\u002Fdebian-as-bypass-router","OpenWrt as the main router and Debian as the side router",", then moved away from the side-router design. I tried ",[20,36,38],{"href":37},"\u002Fen\u002Ffiddling\u002Ffake-ip-based-transparent-proxy","FakeIP-based routing"," and then ",[20,41,43],{"href":42},"\u002Fen\u002Ffiddling\u002Fmore-accurate-chnroute","BGP-based routing",", finally settling on BGP.",[16,46,47],{},"Recently I discovered the OPNsense firewall\u002Frouter system. It is pretty much my dream router OS: free and open source, a beautiful UI, comprehensive features, and even GUI support for automatically updating IP lists used in routing. I decided to migrate my current design to it, integrating Clash into the main router instead of keeping a separate software router for censorship circumvention.",[16,49,50],{},"There were not many tutorials online, and some had aged out of usefulness. After falling into a few traps, I decided to document the full setup.",[16,52,53],{},"The required features are:",[55,56,57,61],"ul",{},[58,59,60],"li",{},"Forward DNS to Clash for centralized resolution.",[58,62,63],{},"Once traffic enters OPNsense, route it according to a list, sending selected traffic into Clash.",[16,65,66],{},"I will skip the basic OPNsense installation; plenty of material already covers it.",[11,68,70],{"id":69},"install-clash","Install Clash",[72,73,75],"h4",{"id":74},"binary-and-configuration-file","Binary and Configuration File",[16,77,78],{},"Both DNS resolution and traffic processing depend on Clash, so install it first.",[16,80,81,82,86,87,90],{},"SSH into OPNsense—how do you enable SSH? STFW—and create ",[83,84,85],"code",{"code":85},"\u002Fusr\u002Flocal\u002Fclash"," for the binary, configuration, and supporting files. I recommend transferring the binary with scp, since the main router cannot yet circumvent censorship and direct downloads are slow. Upload the binary and configuration there, naming them clash and ",[83,88,89],{"code":89},"config.yaml"," respectively.",[16,92,93,94,99,100,103],{},"Download the latest core from the Mihomo ",[20,95,98],{"href":96,"rel":97},"https:\u002F\u002Fgithub.com\u002FMetaCubeX\u002Fmihomo\u002Freleases",[24],"releases page",". Choose the FreeBSD build and your architecture: 386, amd64, or arm64. If you use amd64 and encounter the following error when running Clash, download ",[83,101,102],{"code":102},"amd64-compatible"," instead.",[105,106,112],"pre",{"className":107,"code":109,"language":110,"meta":111},[108],"language-shell","This PROGRAM can only be run on _AMD64 processors with v3 microarchitecture_ support.\n","shell","",[83,113,109],{"__ignoreMap":111},[16,115,116],{},"Use your usual configuration file, but change these settings:",[105,118,123],{"className":119,"code":121,"language":122,"meta":111},[120],"language-yaml","mixed-port: 7890\n\ndns:\n  listen: 127.0.0.1:5353\n\ntun:\n  enable: false\n","yaml",[83,124,121],{"__ignoreMap":111},[16,126,127],{},"DNS listens on port 5353 as the upstream for OPNsense’s built-in DNS. Disable TUN so Clash does not capture traffic itself; OPNsense will select and feed it traffic. mixed-port serves SOCKS, HTTP, and HTTPS proxy functions together.",[16,129,130,131,134,135,138,139,142],{},"Run ",[83,132,133],{"code":133},"pw user add clash -c \"Clash\" -s \u002Fusr\u002Fsbin\u002Fnologin"," to create a clash user that cannot log in, then grant ownership with ",[83,136,137],{"code":137},"chown clash:clash \u002Fusr\u002Flocal\u002Fclash",". Test with ",[83,140,141],{"code":141},"\u002Fusr\u002Flocal\u002Fclash\u002Fclash -d \u002Fusr\u002Flocal\u002Fclash"," and check that it runs successfully.",[72,144,146],{"id":145},"register-the-clash-service","Register the Clash Service",[16,148,149,150,153,154,157],{},"Create ",[83,151,152],{"code":152},"\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Fclash"," and ",[83,155,156],{"code":156},"\u002Fusr\u002Flocal\u002Fopnsense\u002Fservice\u002Fconf\u002Factions.d\u002Factions_clash.conf"," to register Clash as a system service.",[105,159,162],{"className":160,"code":161,"language":110,"meta":111},[108],"#!\u002Fbin\u002Fsh\n# $FreeBSD$\n\n# PROVIDE: clash\n# REQUIRE: LOGIN cleanvar\n# KEYWORD: shutdown\n\n# Add the following lines to \u002Fetc\u002Frc.conf to enable clash:\n# clash_enable (bool): Set to \"NO\" by default.\n# Set to \"YES\" to enable clash.\n# clash_config (path): Clash config dir.\n# Defaults to \"\u002Fusr\u002Flocal\u002Fetc\u002Fclash\"\n\n. \u002Fetc\u002Frc.subr\n\nname=\"clash\"\nrcvar=clash_enable\n\nload_rc_config $name\n\n: ${clash_enable:=\"NO\"}\n: ${clash_config=\"\u002Fusr\u002Flocal\u002Fclash\"}\n\ncommand=\"\u002Fusr\u002Flocal\u002Fclash\u002Fclash\"\n#pidfile=\"\u002Fvar\u002Frun\u002Fclash.pid\"\nrequired_files=\"${clash_config}\"\nclash_group=\"clash\"\nclash_user=\"clash\"\n\ncommand_args=\"-d $clash_config\"\n\nrun_rc_command \"$1\"\n",[83,163,161],{"__ignoreMap":111},[105,165,170],{"className":166,"code":168,"language":169},[167],"language-text","[start]\ncommand:\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Fclash onestart\ntype:script\nmessage:starting clash\n\n[stop]\ncommand:\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Fclash stop\ntype:script\nmessage:stoping clash\n\n[status]\ncommand:\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Fclash statusexit 0\ntype:script_output\nmessage:get clash status\n\n[restart]\ncommand:\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Fclash onerestart\ntype:script\nmessage:restarting clash\n","text",[83,171,168],{"__ignoreMap":111},[16,173,174,175,178,179,182],{},"Make it executable with ",[83,176,177],{"code":177},"chmod +x \u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Fclash",", then activate the configuration with ",[83,180,181],{"code":181},"service configd restart",".",[72,184,186],{"id":185},"start-clash-at-boot","Start Clash at Boot",[16,188,189],{},"Next is starting Clash at boot, but there is a catch:",[191,192,193],"blockquote",{},[16,194,195],{},"When launched as a system service, Clash does not automatically move into the background after starting. During boot, the system therefore gets stuck at Clash, which stays in the foreground and prevents later services from starting.",[16,197,198,199,182],{},"A roundabout solution is to use OPNsense’s built-in Monit service monitor to start Clash and watch its status. Enable it under ",[83,200,201],{"code":201},"Services → Monit",[16,203,204],{},"Add two Service Tests under Service Test Settings. The first starts Clash:",[206,207,208,221],"table",{},[209,210,211],"thead",{},[212,213,214,218],"tr",{},[215,216,217],"th",{},"Setting",[215,219,220],{},"Value",[222,223,224,233,241],"tbody",{},[212,225,226,230],{},[227,228,229],"td",{},"Name",[227,231,232],{},"Clash",[212,234,235,238],{},[227,236,237],{},"Condition",[227,239,240],{},"failed host 127.0.0.1 port 7890 type tcp",[212,242,243,246],{},[227,244,245],{},"Action",[227,247,248],{},"Restart",[16,250,251],{},"The second prevents an endless restart loop:",[206,253,254,262],{},[209,255,256],{},[212,257,258,260],{},[215,259,217],{},[215,261,220],{},[222,263,264,271,278],{},[212,265,266,268],{},[227,267,229],{},[227,269,270],{},"RestartLimit4",[212,272,273,275],{},[227,274,237],{},[227,276,277],{},"5 restarts within 5 cycles",[212,279,280,282],{},[227,281,245],{},[227,283,284],{},"Unmonitor",[16,286,287],{},"Finally, add this under Service Settings:",[206,289,290,298],{},[209,291,292],{},[212,293,294,296],{},[215,295,217],{},[215,297,220],{},[222,299,300,306,314,322,330],{},[212,301,302,304],{},[227,303,229],{},[227,305,232],{},[212,307,308,311],{},[227,309,310],{},"Match",[227,312,313],{},"clash",[212,315,316,319],{},[227,317,318],{},"Start",[227,320,321],{},"\u002Fusr\u002Flocal\u002Fsbin\u002Fconfigctl clash start",[212,323,324,327],{},[227,325,326],{},"Stop",[227,328,329],{},"\u002Fusr\u002Flocal\u002Fsbin\u002Fconfigctl clash stop",[212,331,332,335],{},[227,333,334],{},"Tests",[227,336,337],{},"Clash,RestartLimit4",[16,339,340],{},"Save, wait a little, and check Monit → Status to see whether Clash is running normally.",[11,342,344],{"id":343},"dns-resolution","DNS Resolution",[16,346,347],{},"I set OPNsense’s built-in Unbound DNS to use Clash at 127.0.0.1:5353 as its upstream, but resolution kept failing. Very strange.",[16,349,350],{},"After failing to figure it out, I disabled Unbound DNS and went back to AdGuard Home as the default DNS server, taking over port 53.",[16,352,353],{},"AdGuard Home is not in OPNsense’s default plugin repository, so add the community repository manually.",[16,355,356],{},"SSH into OPNsense and run:",[105,358,361],{"className":359,"code":360,"language":110,"meta":111},[108],"$ fetch -o \u002Fusr\u002Flocal\u002Fetc\u002Fpkg\u002Frepos\u002Fmimugmail.conf https:\u002F\u002Fwww.routerperformance.net\u002Fmimugmail.conf\n$ pkg update\n",[83,362,360],{"__ignoreMap":111},[16,364,365],{},"In the web GUI, go to System → Firmware → Plugins, search for adguard, and install os-adguardhome-maxit. Enable AdGuard Home under Services → Adguardhome. Its web interface is on port 3000. I will skip initialization, but make sure DNS listens on port 53, making AdGuard Home the default DNS server on the OPNsense machine.",[16,367,368],{},"Under AdGuard Home’s Settings → DNS Settings, set upstream DNS to 127.0.0.1:5353, Clash’s listening address.",[11,370,372],{"id":371},"routing-chinese-and-overseas-ips","Routing Chinese and Overseas IPs",[72,374,75],{"id":375},"binary-and-configuration-file-1",[16,377,378],{},"OPNsense includes Squid for proxying, but it only handles HTTP\u002FHTTPS, not general TCP or UDP traffic. That makes it rather incomplete as a proxy. I took an indirect route instead, using tun2socks to feed TCP\u002FUDP traffic into Clash.",[16,380,149,381,384,385,390,391,394],{},[83,382,383],{"code":383},"\u002Fusr\u002Flocal\u002Ftun2socks"," for the binary and configuration. Download the latest FreeBSD binary from ",[20,386,389],{"href":387,"rel":388},"https:\u002F\u002Fgithub.com\u002Fxjasonlyu\u002Ftun2socks\u002Freleases",[24],"GitHub Releases"," into it and rename it tun2socks. Create ",[83,392,393],{"code":393},"\u002Fusr\u002Flocal\u002Ftun2socks\u002Fconfig.yaml",":",[105,396,399],{"className":397,"code":398,"language":122,"meta":111},[120],"# debug \u002F info \u002F warning \u002F error \u002F silent\nloglevel: info\n\n# URL format: [protocol:\u002F\u002F]host[:port]\nproxy: socks5:\u002F\u002F127.0.0.1:7890\n\n# URL format: [driver:\u002F\u002F]name\n# TUN 设备名称，避免使用 tun0\ndevice: tun:\u002F\u002Fproxytun2socks0\n\n# Maximum transmission unit for each packet\nmtu: 1500\n\n# Timeout for each UDP session, default value: 60 seconds\nudp-timeout: 120s\n",[83,400,398],{"__ignoreMap":111},[16,402,403,404,407],{},"Set ",[83,405,406],{"code":406},"proxy"," to the address of Clash’s SOCKS5 port.",[16,409,130,410,413,414,417],{},[83,411,412],{"code":412},".\u002Ftun2socks -config .\u002Fconfig.yaml"," inside ",[83,415,416],{"code":416},"\u002Fusr\u002Flocal\u002Ftun2socks\u002F"," to test the configuration.",[72,419,421],{"id":420},"register-the-service","Register the Service",[16,423,149,424,153,427,182],{},[83,425,426],{"code":426},"\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Ftun2socks",[83,428,429],{"code":429},"\u002Fusr\u002Flocal\u002Fopnsense\u002Fservice\u002Fconf\u002Factions.d\u002Factions_tun2socks.conf",[105,431,434],{"className":432,"code":433,"language":110,"meta":111},[108],"#!\u002Fbin\u002Fsh\n\n# PROVIDE: tun2socks\n# REQUIRE: LOGIN\n# KEYWORD: shutdown\n\n. \u002Fetc\u002Frc.subr\n\nname=\"tun2socks\"\nrcvar=\"tun2socks_enable\"\n\nload_rc_config $name\n\n: ${tun2socks_enable:=no}\n: ${tun2socks_config:=\"\u002Fusr\u002Flocal\u002Ftun2socks\u002Fconfig.yaml\"}\n\npidfile=\"\u002Fvar\u002Frun\u002F${name}.pid\"\ncommand=\"\u002Fusr\u002Flocal\u002Ftun2socks\u002Ftun2socks\"\ncommand_args=\"-config ${tun2socks_config} > \u002Fdev\u002Fnull 2>&1 & echo \\$! > ${pidfile}\"\n\nstart_cmd=\"${name}_start\"\n\ntun2socks_start()\n{\n    if [ ! -f ${tun2socks_config} ]; then\n        echo \"${tun2socks_config} not found.\"\n        exit 1\n    fi\n    echo \"Starting ${name}.\"\n    \u002Fbin\u002Fsh -c \"${command} ${command_args}\"\n}\n\nrun_rc_command \"$1\"\n",[83,435,433],{"__ignoreMap":111},[105,437,440],{"className":438,"code":439,"language":169},[167],"[start]\ncommand:\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Ftun2socks start\nparameters:\ntype:script\nmessage:starting tun2socks\n\n[stop]\ncommand:\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Ftun2socks stop\nparameters:\ntype:script\nmessage:stopping tun2socks\n\n[restart]\ncommand:\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Ftun2socks restart\nparameters:\ntype:script\nmessage:restarting tun2socks\n\n[status]\ncommand:\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Ftun2socks status; exit 0\nparameters:\ntype:script_output\nmessage:request tun2socks status\n",[83,441,439],{"__ignoreMap":111},[16,443,149,444,447],{},[83,445,446],{"code":446},"\u002Fetc\u002Frc.conf"," and add:",[105,449,452],{"className":450,"code":451,"language":169},[167],"tun2socks_enable=\"YES\"\n",[83,453,451],{"__ignoreMap":111},[16,455,174,456,459,460,182],{},[83,457,458],{"code":458},"chmod +x \u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Ftun2socks",", then run ",[83,461,181],{"code":181},[16,463,464],{},"Start tun2socks manually as well:",[105,466,469],{"className":467,"code":468,"language":110,"meta":111},[108],"\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Ftun2socks start\n",[83,470,468],{"__ignoreMap":111},[72,472,474],{"id":473},"start-at-boot","Start at Boot",[16,476,149,477,182],{},[83,478,479],{"code":479},"\u002Fusr\u002Flocal\u002Fetc\u002Frc.syshook.d\u002Fearly\u002F60-tun2socks",[105,481,486],{"className":482,"code":484,"language":485,"meta":111},[483],"language-bash","#!\u002Fbin\u002Fsh\n\n# Start tun2socks service\n\u002Fusr\u002Flocal\u002Fetc\u002Frc.d\u002Ftun2socks start\n","bash",[83,487,484],{"__ignoreMap":111},[16,489,490,491,182],{},"Make the file executable: ",[83,492,493],{"code":493},"chmod +x \u002Fusr\u002Flocal\u002Fetc\u002Frc.syshook.d\u002Fearly\u002F60-tun2socks",[72,495,497],{"id":496},"create-an-interface-and-gateway","Create an Interface and Gateway",[16,499,500],{},"Under Interfaces → Assignments in OPNsense, add an interface using proxytun2socks0, the device named in the configuration, and save.",[16,502,503,504,507],{},"Open its configuration, enable it, set Description to TUN2SOCKS, IPv4 Configuration Type to Static IPv4, and IPv4 Address to ",[83,505,506],{"code":506},"10.0.3.1\u002F24",", then save.",[16,509,510,511,514],{},"Under System → Gateways → Configuration, create a gateway named TUN2SOCKS_MIHOMO. Select the TUN2SOCKS interface and enter ",[83,512,513],{"code":513},"10.0.3.2"," as the IP address, leaving the remaining settings at their defaults.",[16,516,517],{},"We now have a gateway that forwards any traffic entering it to 127.0.0.1:7890 for Clash to proxy.",[72,519,521],{"id":520},"configure-chineseoverseas-ip-routing","Configure Chinese\u002FOverseas IP Routing",[16,523,524],{},"The most useful OPNsense feature for me is Firewall → Aliases. You can define an IP list and reuse it directly in rules. Lists can be entered manually or fetched dynamically by subscribing to a URL.",[16,526,527],{},"Open Firewall → Aliases and create two aliases.",[16,529,530],{},"The first, InternalAddress, represents LAN address ranges. Choose Network(s) as the type and enter:",[105,532,535],{"className":533,"code":534,"language":169},[167],"0.0.0.0\u002F8\n127.0.0.0\u002F8\n10.0.0.0\u002F8\n172.16.0.0\u002F12\n192.168.0.0\u002F16\n169.254.0.0\u002F16\n224.0.0.0\u002F4\n240.0.0.0\u002F4\n",[83,536,534],{"__ignoreMap":111},[16,538,539,540],{},"The second, CN_V4, represents IP ranges within China. Choose URL Table (IPs) and enter a URL containing all Chinese network ranges, such as ",[20,541,542],{"href":542,"rel":543},"https:\u002F\u002Fraw.githubusercontent.com\u002Fgaoyifan\u002Fchina-operator-ip\u002Frefs\u002Fheads\u002Fip-lists\u002Fchina.txt",[24],[16,545,546],{},"Next, add two rules at the top of Firewall → Rules → LAN. Their top-to-bottom order matters.",[16,548,549],{},"The first uses InternalAddress as its destination, with everything else left at default. LAN destinations will follow normal routing.",[16,551,552],{},"The second uses CN_V4 as its destination, with Destination \u002F Invert checked and TUN2SOCKS_MIHOMO selected as the gateway. This sends non-Chinese destinations to that gateway.",[16,554,555],{},"The remaining default rules go below. All other traffic follows them, so Chinese IPs connect directly.",[16,557,558],{},"When you visit Google, AdGuard Home receives the DNS query and forwards it to Clash, which resolves Google’s real address. Traffic to that address then matches the second firewall rule, goes to TUN2SOCKS_MIHOMO, and enters Clash through its SOCKS5 port. You are through the wall.",{"title":111,"searchDepth":560,"depth":560,"links":561},4,[562,564,569,570],{"id":13,"depth":563,"text":14},3,{"id":69,"depth":563,"text":70,"children":565},[566,567,568],{"id":74,"depth":560,"text":75},{"id":145,"depth":560,"text":146},{"id":185,"depth":560,"text":186},{"id":343,"depth":563,"text":344},{"id":371,"depth":563,"text":372,"children":571},[572,573,574,575,576],{"id":375,"depth":560,"text":75},{"id":420,"depth":560,"text":421},{"id":473,"depth":560,"text":474},{"id":496,"depth":560,"text":497},{"id":520,"depth":560,"text":521},[578],"fiddling","2025-01-16 23:09:00","OPNsense is an open-source firewall and router with an attractive interface and a comprehensive feature set. After trying several routing setups, I came to appreciate its potential for transparent proxying and traffic routing. Combining it with BGP-based routing offers better security and stability, while its automatically updated IP lists make network management more convenient.",false,"md","https:\u002F\u002Fblog-img.774352199.xyz\u002FxA8C1E.webp",{"slots":585},{},true,"\u002Ffiddling\u002Fopnsense-transparent-proxy",null,{"text":590,"minutes":591,"time":592,"words":593},"7 min read",6.47,388200,1294,{"title":5,"description":580},"Configure OPNsense with mihomo, AdGuard Home, and tun2socks for transparent proxying, using IP-list aliases and firewall rules to route selected traffic.",{"loc":587},"posts\u002Ffiddling\u002Fopnsense-transparent-proxy",[599,600,601,602,603,604],"OPNsense","FreeBSD","mihomo","tun2socks","Transparent proxy","Traffic routing","tech","AOwrbaWZkfXZbmBzSa4wSsCsAopEizNgE-7ObaON6xQ",[608,626,643,661,678,696,714,730,748,764,779,794,810,826,843,861,876,893,907,923,940,957,961,978,994,1013,1029,1046,1064,1078,1096,1113,1129,1145,1159,1176,1190,1205,1220,1234,1248,1262,1277,1291,1302],{"categories":609,"date":611,"description":612,"image":613,"path":614,"readingTime":615,"recommend":588,"tags":620,"title":625,"type":605},[610],"daily","2024-09-01 22:03:10","Chronic gastritis has been a long, uneven journey: stomach trouble throughout childhood, a surprising reprieve at university, then a return of symptoms after years of late nights and drinking. Frequent nausea and reflux eventually became too much to live with. After repeated examinations and some reflection, I finally began taking the recovery process seriously.","https:\u002F\u002Fblog-img.774352199.xyz\u002F0Hr9l5.webp","\u002Fdaily\u002Fanti-chronic-gastritis",{"text":616,"minutes":617,"time":618,"words":619},"4 min read",3.08,184800,616,[621,622,623,624],"Chronic gastritis","Medical care experiences","Gastroscopy","Health journal","My Journey with Chronic Gastritis",{"categories":627,"date":628,"description":629,"image":630,"path":631,"readingTime":632,"recommend":588,"tags":637,"title":642,"type":605},[610],"2022-04-11 00:13:13","In The Three-Body Problem, Liu Cixin uses an immense historical canvas to explore the relationship between individuals and the collective. His doubts about Western democracy and the majority’s ability to determine its own fate run through the trilogy. Heroes struggle and sacrifice, only to see their work swept aside by history. Beginning with the Cultural Revolution, the story sets up a conflict between exceptional individuals and the ordinary masses, raising uncomfortable questions about humanity and society.","https:\u002F\u002Fblog-img.774352199.xyz\u002FLTzbFP.webp","\u002Fdaily\u002Fpeople-in-three-body",{"text":633,"minutes":634,"time":635,"words":636},"9 min read",8.635,518100,1727,[638,639,640,641],"The Three-Body Problem","Liu Cixin","Science fiction","Reading notes","The “Majority” in The Three-Body Problem",{"categories":644,"date":645,"description":646,"image":647,"path":648,"readingTime":649,"recommend":588,"tags":654,"title":660,"type":605},[610],"2024-01-04 02:06:51","My learning goals for 2024: reach N2 in Japanese, finish SICP and TAPL, build a kernel with full POSIX support, and update the blog theme. A challenging list, but a clear direction for the year.","https:\u002F\u002Fblog-img.774352199.xyz\u002FV3rSQC.webp","\u002Fdaily\u002Fplan2024",{"text":650,"minutes":651,"time":652,"words":653},"1 min read",0.15,9000,30,[655,656,657,658,659],"Annual planning","Learning Japanese","SICP","TAPL","Operating systems","My Learning Plan for 2024",{"categories":662,"date":663,"description":664,"image":665,"path":666,"readingTime":667,"recommend":588,"tags":672,"title":677,"type":605},[610],"2024-09-28 16:26:00","Three years at ByteDance have made time feel strangely accelerated. Amid the changes, I have still managed to hold on to some mental balance. Looking back at the new graduate gazing at distant mountains from the balcony of our Hangzhou office, I can trace how curiosity, workplace challenges, and shifting expectations taught me to find a rhythm of my own.","https:\u002F\u002Fblog-img.774352199.xyz\u002FhnCaht.webp","\u002Fdaily\u002Fwork-for-3-years",{"text":668,"minutes":669,"time":670,"words":671},"10 min read",9.74,584400,1948,[673,674,675,676],"ByteDance","Software engineering careers","Career retrospective","Mental health","Working at ByteDance for Three Years and Staying Somewhat Sane Is Not Entirely Impossible",{"categories":679,"date":680,"description":681,"image":682,"path":683,"readingTime":684,"recommend":588,"tags":689,"title":695,"type":605},[578],"2025-05-28 22:09:00","After moving my blog to Astro, the usual Google Analytics integration no longer fit its performance goals. Adding event-reporting JavaScript directly to the head works, but hurts page performance. I used partytown to move the scripts off the main thread so they would not interfere with loading. With a few adjustments to the example code, Google Analytics finally worked, balancing performance with analytics.","https:\u002F\u002Fblog-img.774352199.xyz\u002FQ0w4RN.webp","\u002Ffiddling\u002Fastro-google-tag-manager",{"text":685,"minutes":686,"time":687,"words":688},"2 min read",1.785,107100,357,[690,691,692,693,694],"Astro","Google Analytics","Google Tag Manager","Partytown","Web analytics","Adding Google Analytics to Astro with Tag Manager",{"categories":697,"date":698,"description":699,"image":700,"path":701,"readingTime":702,"recommend":707,"tags":708,"title":713,"type":605},[578],"2023-04-08 13:16:36","Designing a new programming language is challenging and fun. Setting aside complicated compiler theory and implementation details to focus on where code runs helps clarify how languages are built. Starting from the RISC-VI instruction set, this discussion explores the underlying architecture, layered computer systems, and virtual-machine model, reflecting on the nature of programming languages as well as their implementation.","https:\u002F\u002Fblog-img.774352199.xyz\u002FuO420B.webp","\u002Ffiddling\u002Fchitchat-about-programming-language",{"text":703,"minutes":704,"time":705,"words":706},"15 min read",14.275,856500,2855,2,[709,710,711,712],"Programming language design","Type systems","Compiler design","RISC-V","Some Thoughts on Programming Languages",{"categories":715,"date":716,"description":717,"image":718,"path":719,"readingTime":720,"recommend":560,"tags":725,"title":729,"type":605},[578],"2025-04-18 16:43:12","My girlfriend moved from Beijing to Shanghai for work, and I helped arrange broadband too. Shanghai Telecom’s 500M connection costs more than a 1000M line in Hangzhou, frustratingly. I set out to connect the two cities’ networks: transparent proxying in Shanghai, selected traffic exiting through Hangzhou, and access between both LANs. Hangzhou already had a simple setup with a software router and an AP, configured to route my everyday traffic home and ready for the next networking adventure.","https:\u002F\u002Fblog-img.774352199.xyz\u002FO6cAGh.webp","\u002Ffiddling\u002Fcross-city-network-setup",{"text":721,"minutes":722,"time":723,"words":724},"5 min read",4.865,291900,973,[726,601,727,603,728],"Tailscale","Site-to-site networking","Home networking","Connecting My Shanghai and Hangzhou Networks",{"categories":731,"date":732,"description":733,"image":734,"path":735,"readingTime":736,"recommend":588,"tags":741,"title":747,"type":605},[578],"2021-12-27 00:09:00","The labs are an essential part of learning CSAPP, but setting up Linux can be discouraging. Virtual machines bring installation errors, compatibility problems, and broken networking. WSL (Windows Subsystem for Linux), especially on Windows 10 version 2004 and later, provides a simpler, more direct Linux environment without the complexity and performance bottlenecks of a traditional VM.","https:\u002F\u002Fblog-img.774352199.xyz\u002FvqOC7N.webp","\u002Ffiddling\u002Fcsapplab0",{"text":737,"minutes":738,"time":739,"words":740},"6 min read",5.295,317700,1059,[742,743,744,745,746],"CSAPP","WSL2","Linux","GDB","Lab environment","Setting Up the CSAPP Lab Environment",{"categories":749,"date":750,"description":751,"image":752,"path":753,"readingTime":754,"recommend":588,"tags":759,"title":763,"type":605},[578],"2024-07-13 17:49:00","Using Debian as a side router offers a more stable and flexible alternative without depending on OpenWrt and LuCI. Configuring Debian directly gives you greater control over the system and avoids the limitations and instability of a GUI. Compared with common side-router setups, this approach makes transparent proxying more reliable and offers another option for those who value performance and efficiency.","https:\u002F\u002Fblog-img.774352199.xyz\u002FpPRU5x.webp","\u002Ffiddling\u002Fdebian-as-bypass-router",{"text":755,"minutes":756,"time":757,"words":758},"12 min read",11.79,707400,2358,[760,761,601,762,603],"Debian","Side router","AdGuard Home","Using Debian as a Side Router",{"categories":765,"date":766,"description":767,"image":768,"path":769,"readingTime":770,"recommend":588,"tags":774,"title":778,"type":605},[578],"2024-08-16 23:53:00","This FakeIP-based transparent proxy design addresses the single point of failure, poor performance, and awkward port forwarding of a traditional side router. Switching to the sing-box proxy core improves forwarding performance and simplifies configuration, with broader protocol support and better optimization than the previous Clash setup. Clash can implement the same design, but sing-box provides a flexible alternative.","https:\u002F\u002Fblog-img.774352199.xyz\u002FS2HHD5.webp","\u002Ffiddling\u002Ffake-ip-based-transparent-proxy",{"text":668,"minutes":771,"time":772,"words":773},9.395,563700,1879,[775,776,603,777,604],"FakeIP","sing-box","Policy-based routing","Routing Transparent Proxy Traffic with FakeIP",{"categories":780,"date":781,"description":782,"image":783,"path":784,"readingTime":785,"recommend":588,"tags":789,"title":793,"type":605},[578],"2024-08-15 23:50:00","Port forwarding on the main router often stops working when a side router is introduced. Setting the side router as the gateway changes the forwarding path, breaking mappings that previously relied on the main router. A gateway translates addresses and forwards traffic from the internal network to the outside, and each internal device needs one to communicate externally. Understanding this mechanism helps explain how to fix the forwarding problem.","https:\u002F\u002Fblog-img.774352199.xyz\u002FmRqws9.webp","\u002Ffiddling\u002Ffix-port-forward-in-bypass-router",{"text":616,"minutes":786,"time":787,"words":788},3.03,181800,606,[761,790,791,792],"Port forwarding","NAT","Network troubleshooting","Fixing Port Forwarding with a Side Router",{"categories":795,"date":796,"description":797,"image":798,"path":799,"readingTime":800,"recommend":588,"tags":805,"title":809,"type":605},[578],"2023-02-02 23:24:55","Inspiration struck during Chinese New Year: an article read on the train about running Go on bare metal sparked an interest in low-level system interfaces. Its successful implementation suggested exciting possibilities for combining a high-level language with an OS. Further research revealed earlier work on the idea, and that growing enthusiasm became a project full of anticipation that ultimately did not work out.","https:\u002F\u002Fblog-img.774352199.xyz\u002FxB1Ni5.webp","\u002Ffiddling\u002Fgo-os",{"text":801,"minutes":802,"time":803,"words":804},"8 min read",7.29,437400,1458,[806,712,659,807,808],"Go","Bare-metal programming","Runtime","A Project That Failed: Seven Not-So-Happy Days over Chinese New Year",{"categories":811,"date":812,"description":813,"image":814,"path":815,"readingTime":816,"recommend":588,"tags":821,"title":825,"type":605},[578],"2022-08-15 01:05:01","While refactoring a system, converting entities between layers made deep copying surprisingly awkward. A product VO in the view layer, an entity in the domain layer, and a PO in the persistence layer can look nearly identical, yet small type differences complicate direct conversion. I used reflection to build a general conversion method, reducing repetitive assembler methods and making the code more maintainable and flexible.","https:\u002F\u002Fblog-img.774352199.xyz\u002FBlhm0I.webp","\u002Ffiddling\u002Fgolang-deepcopy-between-different-type",{"text":817,"minutes":818,"time":819,"words":820},"3 min read",2.855,171300,571,[806,822,823,824],"Reflection","Deep copy","Struct conversion","Deep Copying Between Different Struct Types in Go",{"categories":827,"date":828,"description":829,"image":830,"path":831,"readingTime":832,"recommend":588,"tags":836,"title":842,"type":605},[578],"2025-03-31 23:51:00","Periodically syncing heart-rate data from Apple Health to a server and visualizing it in Grafana provides an intuitive way to monitor it. Health Auto Export sends the data to an HTTP endpoint through its REST API automation, the server stores it in InfluxDB, and Grafana presents a clear dashboard for tracking and analyzing personal heart-rate changes.","https:\u002F\u002Fblog-img.774352199.xyz\u002FF4qD2T.webp","\u002Ffiddling\u002Fheart-rate-to-grafana",{"text":817,"minutes":833,"time":834,"words":835},2.45,147000,490,[837,838,839,840,841],"Apple Watch","Health Auto Export","InfluxDB","Grafana","Heart rate monitoring","My Heart Beats for U: Visualizing Heart Rate in Grafana",{"categories":844,"date":845,"description":846,"image":847,"path":848,"readingTime":849,"recommend":853,"tags":854,"title":860,"type":605},[578],"2025-06-10 20:18:00","macOS and iPadOS 26 introduce the Liquid Glass design language, refreshing icons and windows with a more modern look. Windowed apps on iPad are a step toward making it a productivity device. The transparent Control Center and the integration of Launchpad have been controversial, however, and the experience still needs work. Despite its shortcomings, this update lays groundwork for a promising future.","https:\u002F\u002Fblog-img.774352199.xyz\u002FTPSaLE.webp","\u002Ffiddling\u002Fmacos-26-trial",{"text":616,"minutes":850,"time":851,"words":852},3.305,198300,661,5,[855,856,857,858,859],"macOS 26","iPadOS 26","Liquid Glass","Apple Intelligence","Operating system impressions","Trying Out macOS 26 and iPadOS 26",{"categories":862,"date":863,"description":864,"image":865,"path":866,"readingTime":867,"recommend":588,"tags":871,"title":875,"type":605},[578],"2026-06-11 10:00:00","Another WWDC brings new Apple Intelligence features in macOS 27. The workarounds from macOS 26 no longer get through, so the battle of wits with Apple continues.","https:\u002F\u002Fblog-img.774352199.xyz\u002FfOFucm.webp","\u002Ffiddling\u002Fmacos-27-apple-intelligence-chatgpt",{"text":737,"minutes":868,"time":869,"words":870},5.1,306000,1020,[872,858,873,874],"macOS 27","ChatGPT","Mainland China Mac","Getting Apple Intelligence Working on a China-Market Mac: Region Changes and Extracting ChatGPT",{"categories":877,"date":878,"description":879,"image":880,"path":881,"readingTime":882,"recommend":588,"tags":886,"title":892,"type":605},[578],"2025-07-20 23:32:00","MoonTV is a new video aggregation platform built with Next.js and React to make following shows convenient. It began as an attempt to improve LibreTV and has attracted substantial attention and usage over several months of development. Cursor made development efficient, although multi-platform support and complex data dependencies posed challenges. As its user base grows, MoonTV continues improving in response to feedback.","https:\u002F\u002Fblog-img.774352199.xyz\u002FnIeONi.webp","\u002Ffiddling\u002Fmoontv-vibe-coding",{"text":737,"minutes":883,"time":884,"words":885},5.76,345600,1152,[887,888,889,890,891],"MoonTV","LunaTV","Vibe Coding","Cursor","AI-assisted development","MoonTV: An Experiment in Vibe Coding",{"categories":894,"date":895,"description":896,"image":897,"path":898,"readingTime":899,"recommend":588,"tags":903,"title":906,"type":605},[578],"2024-10-07 16:51:00","BGP-based routing for Chinese and overseas IPs makes transparent proxying more efficient and accurate. Marking overseas destinations with FakeIP lets the main router route traffic more intelligently for smoother connectivity. The sing-box DNS configuration is also refined to handle queries more flexibly and efficiently, improving the overall network experience.","https:\u002F\u002Fblog-img.774352199.xyz\u002FMOmM1s.webp","\u002Ffiddling\u002Fmore-accurate-chnroute",{"text":616,"minutes":900,"time":901,"words":902},3.805,228300,761,[904,905,777,604],"BGP","IP address database","More Accurate Routing for Chinese and Overseas IPs with BGP",{"categories":908,"date":909,"description":910,"image":911,"path":912,"readingTime":913,"recommend":588,"tags":917,"title":922,"type":605},[578],"2025-12-14 14:31:00","A friend’s comment got me interested in simulation games. After buying Microsoft Flight Simulator, I discovered streamed maps and models, another account login, oddly hidden tutorials, and awkward keyboard controls. A flight stick and some tinkering with Pico VR finally rounded out the experience.","https:\u002F\u002Fblog-img.774352199.xyz\u002FlfhEuE.webp","\u002Ffiddling\u002Fmsfs2024-joystick-and-pico",{"text":721,"minutes":914,"time":915,"words":916},4.68,280800,936,[918,919,920,921],"Microsoft Flight Simulator 2024","Thrustmaster TCA","PICO VR","Flight simulation","Adventures with Microsoft Flight Simulator 2024",{"categories":924,"date":925,"description":926,"image":927,"path":928,"readingTime":929,"recommend":588,"tags":933,"title":939,"type":605},[578],"2025-06-05 23:26:00","My frequent phone changes took me from OnePlus to iPhone, and from enjoying tinkering to relying on an ecosystem. At my girlfriend’s suggestion, I recently bought an OPPO Find X8 Ultra to improve my photos. Migrating apps out of Apple’s ecosystem reminded me how uneven Android’s app selection remains and how difficult finding replacements can be. A month of migration has been an exercise in friction and adaptation between platforms.","https:\u002F\u002Fblog-img.774352199.xyz\u002F19NIhZ.webp","\u002Ffiddling\u002Fone-month-using-android",{"text":801,"minutes":930,"time":931,"words":932},7.61,456600,1522,[934,935,936,937,938],"Android","OPPO","iOS","Smartphone impressions","Ecosystem migration","One Month After Switching to Android",{"categories":941,"date":942,"description":943,"image":944,"path":945,"readingTime":946,"recommend":588,"tags":950,"title":956,"type":605},[578],"2026-05-11 22:43:00","I found a trip-planning tool on GitHub Trending and wanted to host my own instance. Since I was getting a new VPS anyway, I might as well install a little extra. Well, quite a lot extra.","https:\u002F\u002Fblog-img.774352199.xyz\u002FBBdDWW.webp","\u002Ffiddling\u002Fone-trek-twenty-stacks",{"text":737,"minutes":947,"time":948,"words":949},5.735,344100,1147,[951,952,953,954,955],"TREK","Docker Compose","Dockge","VPS","Self-hosting","One TREK, Twenty Stacks",{"categories":958,"date":579,"description":580,"image":583,"path":587,"readingTime":959,"recommend":588,"tags":960,"title":5,"type":605},[578],{"text":590,"minutes":591,"time":592,"words":593},[599,600,601,602,603,604],{"categories":962,"date":963,"description":964,"image":965,"path":966,"readingTime":967,"recommend":563,"tags":971,"title":977,"type":605},[578],"2025-03-15 20:35:00","Distinguishing user-defined type names from ordinary variables is a challenge during parsing. A statement such as `a*b;` can be either an arithmetic expression or a declaration. Grammar rules, especially those involving type specifiers, can misidentify variables as types, affecting correctness and readability. The prevalence of declarations without initializers makes this ambiguity especially common.","https:\u002F\u002Fblog-img.774352199.xyz\u002F2VKHK9.webp","\u002Ffiddling\u002Fparser-type-variable-ambiguity",{"text":801,"minutes":968,"time":969,"words":970},7.475,448500,1495,[972,973,974,975,976],"Parsing","GLR","Symbol table","Scope","Disambiguation","Resolving Type Name and Variable Name Ambiguity in Parsing",{"categories":979,"date":980,"description":981,"image":982,"path":983,"readingTime":984,"recommend":588,"tags":988,"title":993,"type":605},[578],"2023-05-24 17:51:09","To install the RISC-V toolchain, first obtain the riscv-gnu-toolchain source. Using `--depth=1` when cloning reduces the download size. Check the README’s Prerequisites section and install the required dependencies. On Debian, a simple package installation command prepares the environment for building the toolchain.","https:\u002F\u002Fblog-img.774352199.xyz\u002FrWNOKx.webp","\u002Ffiddling\u002Fspike-install",{"text":685,"minutes":985,"time":986,"words":987},1.92,115200,384,[712,989,990,991,992],"Spike","riscv-pk","Cross-compilation","Toolchain","Installing the RISC-V Toolchain and Emulator",{"categories":995,"date":996,"description":997,"image":998,"path":999,"readingTime":1000,"recommend":1004,"tags":1005,"title":1012,"type":605},[578],"2026-09-09 23:59:00","This counts as NTR too, surely.","https:\u002F\u002Fblog-img.774352199.xyz\u002FNDPUwc.webp","\u002Ffiddling\u002Fsteamdeck-switch-60fps",{"text":801,"minutes":1001,"time":1002,"words":1003},7.345,440700,1469,7,[1006,1007,1008,1009,1010,1011],"Steam Deck","Nintendo Switch","EmuDeck","Eden","Lossless Scaling","Frame generation","A Second Life for the Steam Deck: Switch Emulation and Frame Generation at 60 FPS",{"categories":1014,"date":1015,"description":1016,"image":1017,"path":1018,"readingTime":1019,"recommend":588,"tags":1023,"title":1028,"type":605},[578],"2024-06-23 15:31:32","The GFW does more than monitor an exit gateway: it inspects international traffic through passive taps, copying inbound and outbound IP packets to a cluster for analysis and filtering. Understanding where and how this happens matters when studying censorship circumvention. Examining the GFW’s network topology helps explain its blocking mechanisms and how to work around them.","https:\u002F\u002Fblog-img.774352199.xyz\u002FsOpJuL.webp","\u002Ffiddling\u002Ftech-about-gfw",{"text":755,"minutes":1020,"time":1021,"words":1022},11.885,713100,2377,[1024,1025,1026,1027],"GFW","DNS poisoning","TCP","Internet censorship","How the Great Firewall Works",{"categories":1030,"date":1031,"description":1032,"image":1033,"path":1034,"readingTime":1035,"recommend":588,"tags":1039,"title":1045,"type":605},[578],"2022-04-16 00:01:28","In Java, using `this` can prevent the compiler from optimizing constants. Although `ab1` and `ab2` in this example appear to refer to the same static final variable `s`, comparing them produces different results. `ab1` concatenates a direct reference to the static variable, while `ab2` accesses it through `this`, preventing the same constant propagation optimization. This illustrates how a small syntactic difference can change compilation behavior.","https:\u002F\u002Fblog-img.774352199.xyz\u002FgKtkYe.webp","\u002Ffiddling\u002Fthis-in-javac-string-concat",{"text":817,"minutes":1036,"time":1037,"words":1038},2.305,138300,461,[1040,1041,1042,1043,1044],"Java","javac","String concatenation","Constant propagation","Bytecode","How Java’s this Keyword Can Prevent Compile-Time Constant Propagation",{"categories":1047,"date":1048,"description":1049,"image":1050,"path":1051,"readingTime":1052,"recommend":588,"tags":1057,"title":1063,"type":605},[578],"2025-01-29 21:58:00","A microblog lets you share short thoughts whenever they occur, without the overhead of publishing a full static-blog post. This implementation uses Cloudflare Workers for the backend and KV for storage and management. A Vue component embedded in VitePress displays the updates, adding a lively, interactive element to the blog.","https:\u002F\u002Fblog-img.774352199.xyz\u002FhZX6hr.webp","\u002Ffiddling\u002Fvitepress-memos-component",{"text":1053,"minutes":1054,"time":1055,"words":1056},"19 min read",18.96,1137600,3792,[1058,1059,1060,1061,1062],"VitePress","Vue","Cloudflare Workers","Cloudflare KV","Microblogging","Adding a Microblog to VitePress",{"categories":1065,"date":1066,"description":1067,"image":1068,"path":1069,"readingTime":1070,"recommend":588,"tags":1074,"title":1077,"type":605},[578],"2025-03-15 16:24:00","A Telegram notification on an ordinary afternoon introduced a tempting VPS deal: a direct China Telecom CN2 route and 2.5G bandwidth. The plan includes IPv6, useful for unlocking streaming services, but not every connection needs to go through WARP. My previous script was convenient, yet its effects on speed and traffic routing called for a more flexible solution.","https:\u002F\u002Fblog-img.774352199.xyz\u002FMcjrrF.webp","\u002Ffiddling\u002Fvps-warp-ipv6",{"text":817,"minutes":1071,"time":1072,"words":1073},2.34,140400,468,[1075,1076,954,777,604],"Cloudflare WARP","IPv6","Routing Selected VPS Traffic Through WARP over IPv6",{"categories":1079,"date":1080,"description":1081,"image":1082,"path":1083,"readingTime":1084,"recommend":1088,"tags":1089,"title":1095,"type":605},[578],"2026-08-17 23:59:21","The more I tinker, the more I want to tinker. Still going strong in my old age, still going strong!","https:\u002F\u002Fblog-img.774352199.xyz\u002FqfxB0h.webp","\u002Ffiddling\u002Fxiaomi17-root-and-hide-root",{"text":721,"minutes":1085,"time":1086,"words":1087},4.545,272700,909,6,[1090,1091,1092,1093,1094],"Xiaomi 17","Bootloader","Android Root","Firmware flashing","Root hiding","Making the Xiaomi 17 My Daily Driver: Unlocking, Flashing, Rooting, and Hiding Root",{"categories":1097,"date":1099,"description":1100,"image":1101,"path":1102,"readingTime":1103,"recommend":588,"tags":1107,"title":1112,"type":605},[1098],"notes","2022-01-20 22:29:00","Lab 1 asks us to implement a MapReduce system with two core components: a master and workers. This requires a good command of Go RPC and concurrent programming, along with a thorough understanding of the MapReduce workflow. I built two versions, starting with mutex locks and then moving to a more elegant channel-based implementation without explicit locks, whose design is simpler and clearer. The key to understanding the lab is to read the relevant documentation carefully, especially the flowcharts and explanations.","https:\u002F\u002Fblog-img.774352199.xyz\u002FibVwPJ.webp","\u002Fnotes\u002F65840\u002Fmapreducelab",{"text":801,"minutes":1104,"time":1105,"words":1106},7.21,432600,1442,[1108,1109,806,1110,1111],"MIT 6.5840","MapReduce","RPC","Concurrent programming","6.5840 Lab 1: MapReduce",{"categories":1114,"date":1115,"description":1116,"image":1117,"path":1118,"readingTime":1119,"recommend":588,"tags":1123,"title":1128,"type":605},[1098],"2022-01-16 17:32:00","MapReduce is an efficient parallel computing model designed to simplify processing large datasets. By defining the two key functions, Map and Reduce, users can break complex tasks into simple operations. The framework automatically handles data distribution and task scheduling, allowing developers to focus on the algorithm rather than low-level details. Its widespread use in distributed systems demonstrates its flexibility and practical value.","https:\u002F\u002Fblog-img.774352199.xyz\u002FApIDdC.webp","\u002Fnotes\u002F65840\u002Fmapreducepaper",{"text":721,"minutes":1120,"time":1121,"words":1122},4.12,247200,824,[1109,1124,1125,1126,1127],"Distributed systems","Paper notes","Parallel computing","Fault tolerance","Reading the MapReduce Paper",{"categories":1130,"date":1131,"description":1132,"image":1133,"path":1134,"readingTime":1135,"recommend":588,"tags":1140,"title":1144,"type":605},[1098],"2022-12-16 02:06:10","Lab 2A focuses on implementing Raft leader election and heartbeats so that elections and term changes work correctly even under extreme conditions. The lab has four stages and lays the foundation for the distributed key-value store that follows. A design without explicit locks simplifies the Raft struct. The lab instructions provide the necessary background, but compared with the previous lab, this one relies on almost no reference material and places greater emphasis on implementing the system independently.","https:\u002F\u002Fblog-img.774352199.xyz\u002Fc11Uk4.webp","\u002Fnotes\u002F65840\u002Fraftlab2a",{"text":1136,"minutes":1137,"time":1138,"words":1139},"13 min read",12.235,734100,2447,[1108,1141,806,1142,1143],"Raft","Leader election","Distributed consensus","6.5840 Lab 2A: Leader Election",{"categories":1146,"date":1147,"description":1148,"image":1149,"path":1150,"readingTime":1151,"recommend":588,"tags":1156,"title":1158,"type":605},[1098],"2022-12-03 21:40:09","Raft is a consensus algorithm designed to improve the efficiency of log replication. It is particularly suited to clusters of machines, allowing them to keep providing service even when some machines fail. It uses the replicated state machine model: logs record the order of commands so that every machine in the cluster can reach the same state. In Search of an Understandable Consensus Algorithm explores Raft’s design and compares it with Paxos, highlighting its understandability and providing a foundation for building reliable large-scale software systems. These reading notes aim to help explain the paper’s core concepts and their applications.","https:\u002F\u002Fblog-img.774352199.xyz\u002F7mmvIZ.webp","\u002Fnotes\u002F65840\u002Freftextendedpaper",{"text":1152,"minutes":1153,"time":1154,"words":1155},"16 min read",15.98,958800,3196,[1141,1143,1157,1125],"Log replication","Reading the Raft Paper",{"categories":1160,"date":1162,"description":1163,"image":1164,"path":1165,"readingTime":1166,"recommend":1170,"tags":1171,"title":1175,"type":605},[1161],"projects","2021-11-27 14:43:00","MYDB is a personal project exploring and implementing the fundamentals of databases, built in my spare time over a little more than half a month. I picked up some basic knowledge in my university database systems course, though during my internship I mostly used the classes as an excuse to slack off. My candid answers in an interview did not cause too much trouble, but they did make me reconsider what I knew about databases and decide to learn through hands-on practice. That was how this project began.","https:\u002F\u002Fblog-img.774352199.xyz\u002Fxfci2J.webp","\u002Fprojects\u002Fmydb\u002Fmydb0",{"text":721,"minutes":1167,"time":1168,"words":1169},4.15,249000,830,1,[1172,1040,1173,1174],"MYDB","Database implementation","Database architecture","MYDB 0. Project Structure and a Few Things I Had to Say",{"categories":1177,"date":1178,"description":1179,"image":1180,"path":1181,"readingTime":1182,"recommend":588,"tags":1186,"title":1189,"type":605},[1161],"2021-11-28 16:10:00","MYDB manages transactions through an XID file. Each transaction has a unique XID, incrementing from 1; XID 0 denotes a super transaction whose state is always committed. TransactionManager maintains this file and records three states: active, committed, and aborted. This mechanism supports accurate transaction state queries and management, providing a foundation for system stability and reliability.","https:\u002F\u002Fblog-img.774352199.xyz\u002FH4zZAK.webp","\u002Fprojects\u002Fmydb\u002Fmydb1",{"text":721,"minutes":1183,"time":1184,"words":1185},4.755,285300,951,[1172,1040,1187,1188],"Transaction management","XID","MYDB 1. Starting with the Transaction Manager",{"categories":1191,"date":1192,"description":1193,"image":1194,"path":1195,"readingTime":1196,"recommend":588,"tags":1200,"title":1204,"type":605},[1161],"2021-12-25 18:26:00","MYDB uses a client\u002Fserver architecture similar to MySQL, allowing multiple clients to connect to a server over sockets, execute SQL queries, and receive results. Communication uses a special binary format, though plain text would also be an option for a simpler implementation. The basic transport structure supports effective communication and processing between client and server.","https:\u002F\u002Fblog-img.774352199.xyz\u002FPAHrUZ.webp","\u002Fprojects\u002Fmydb\u002Fmydb10",{"text":721,"minutes":1197,"time":1198,"words":1199},4.305,258300,861,[1172,1040,1201,1202,1203],"Socket","Client-server architecture","Communication protocol","MYDB 10. Implementing the Server, Client, and Wire Protocol",{"categories":1206,"date":1207,"description":1208,"image":1209,"path":1210,"readingTime":1211,"recommend":588,"tags":1215,"title":1219,"type":605},[1161],"2021-11-30 23:18:00","The Data Manager (DM) bridges higher-level modules and the filesystem, handling paging and caching while ensuring data safety and recovery. Its cache uses reference counting rather than traditional LRU, aiming for a reusable, efficient foundation for subsequent data operations.","https:\u002F\u002Fblog-img.774352199.xyz\u002FWdIGoG.webp","\u002Fprojects\u002Fmydb\u002Fmydb2",{"text":737,"minutes":1212,"time":1213,"words":1214},5.725,343500,1145,[1172,1040,1216,1217,1218],"Reference counting","Cache design","Shared memory","MYDB 2. A Reference-Counted Cache Framework and Shared Byte Arrays",{"categories":1221,"date":1222,"description":1223,"image":1224,"path":1225,"readingTime":1226,"recommend":588,"tags":1230,"title":1233,"type":605},[1161],"2021-12-05 15:28:00","DM abstracts the filesystem into pages and uses them as the unit of reading, writing, and caching. The default page size is 8K, with larger pages available to improve write performance under heavy loads. With the general-purpose cache framework already in place, we now define the page structure and implement efficient page caching.","https:\u002F\u002Fblog-img.774352199.xyz\u002FjlFC4E.webp","\u002Fprojects\u002Fmydb\u002Fmydb3",{"text":721,"minutes":1227,"time":1228,"words":1229},4.7,282000,940,[1172,1040,1231,1232],"Data pages","Cache management","MYDB 3. Caching and Managing Data Pages",{"categories":1235,"date":1236,"description":1237,"image":1238,"path":1239,"readingTime":1240,"recommend":588,"tags":1244,"title":1247,"type":605},[1161],"2021-12-08 22:55:00","Log files are essential to MYDB’s design, allowing data to be recovered after a crash. DM logs every operation on underlying data, forming a continuous sequence of records. Stored in a specific binary format with checksums and individual operation records, these logs let the database accurately reconstruct its data on restart and maintain consistency and integrity.","https:\u002F\u002Fblog-img.774352199.xyz\u002FTRcbsj.webp","\u002Fprojects\u002Fmydb\u002Fmydb4",{"text":801,"minutes":1241,"time":1242,"words":1243},7.885,473100,1577,[1172,1040,1245,1246],"Database logging","Crash recovery","MYDB 4. Log Files and Recovery Strategies",{"categories":1249,"date":1250,"description":1251,"image":1252,"path":1253,"readingTime":1254,"recommend":588,"tags":1258,"title":1261,"type":605},[1161],"2021-12-11 15:16:00","The page index is an important part of DM, optimizing insertions by caching the free space available on each page. It lets higher-level modules quickly locate a suitable page without a lengthy search, making data operations more efficient. Its implementation works closely with the DataItem abstraction to support efficient database operation.","https:\u002F\u002Fblog-img.774352199.xyz\u002F22PSG1.webp","\u002Fprojects\u002Fmydb\u002Fmydb5",{"text":590,"minutes":1255,"time":1256,"words":1257},6.37,382200,1274,[1172,1040,1259,1260],"Page index","Data management","MYDB 5. The Page Index and the Data Manager",{"categories":1263,"date":1264,"description":1265,"image":1266,"path":1267,"readingTime":1268,"recommend":588,"tags":1272,"title":1276,"type":605},[1161],"2021-12-18 14:58:00","VM uses two-phase locking to ensure serializable schedules and introduces multiversion concurrency control (MVCC) to eliminate blocking between reads and writes. This chapter also defines conflicts between database operations, focusing on the interaction between updates and reads as a foundation for understanding transaction isolation levels.","https:\u002F\u002Fblog-img.774352199.xyz\u002F8YzotA.webp","\u002Fprojects\u002Fmydb\u002Fmydb6",{"text":633,"minutes":1269,"time":1270,"words":1271},8.64,518400,1728,[1172,1040,1273,1274,1275],"MVCC","Transaction isolation","Two-phase locking","MYDB 6. Record Versions and Transaction Isolation",{"categories":1278,"date":1279,"description":1280,"image":1281,"path":1282,"readingTime":1283,"recommend":588,"tags":1287,"title":1290,"type":605},[1161],"2021-12-23 21:20:00","VM must handle version skipping introduced by MVCC as well as deadlocks. By simply marking a transaction, MYDB can cancel or roll it back and keep data from aborted transactions from affecting others. This design makes concurrent transaction handling more efficient and reliable, avoids the deadlock risks common with traditional 2PL, and improves overall stability and performance.","https:\u002F\u002Fblog-img.774352199.xyz\u002FBF3yDW.webp","\u002Fprojects\u002Fmydb\u002Fmydb7",{"text":801,"minutes":1284,"time":1285,"words":1286},7.265,435900,1453,[1172,1040,1288,1289],"Deadlock detection","Version management","MYDB 7. Deadlock Detection and the Version Manager",{"categories":1292,"date":1293,"description":1294,"image":1295,"path":1296,"readingTime":1297,"recommend":588,"tags":1298,"title":1301,"type":605},[1161],"2021-12-24 21:01:00","MYDB implements a clustered index using a B+ tree. IM interacts directly with the Data Manager (DM), bypassing the Version Manager (VM), so index data is written directly to the database file. This chapter details the binary-tree index structure and its basic node fields, including the leaf flag, key count, and sibling identifier, establishing the framework for indexed lookups.","https:\u002F\u002Fblog-img.774352199.xyz\u002Ff92X4o.webp","\u002Fprojects\u002Fmydb\u002Fmydb8",{"text":721,"minutes":1167,"time":1168,"words":1169},[1172,1040,1299,1300],"B+ tree","Database indexing","MYDB 8. Index Management",{"categories":1303,"date":1304,"description":1305,"image":1306,"path":1307,"readingTime":1308,"recommend":588,"tags":1312,"title":1315,"type":605},[1161],"2021-12-25 15:44:00","The Table Manager (TBM) manages field and table structures. Parser turns SQL-like statements into structured representations, wrapping their information in the corresponding classes to simplify subsequent operations. This chapter also covers MYDB’s SQL syntax as a foundation for understanding the management process.","https:\u002F\u002Fblog-img.774352199.xyz\u002FzOMyv5.webp","\u002Fprojects\u002Fmydb\u002Fmydb9",{"text":737,"minutes":1309,"time":1310,"words":1311},5.035,302100,1007,[1172,1040,1313,1314],"SQL parsing","Table management","MYDB 9. Field and Table Management",1789914051701]