[{"data":1,"prerenderedAt":313},["ShallowReactive",2],{"post:\u002Fagent-never-grades-itself":3,"posts":293},{"id":4,"title":5,"authors":6,"body":9,"category":276,"cover":277,"coverAlt":278,"date":279,"description":280,"draft":281,"extension":282,"featured":283,"meta":284,"minutes":285,"navigation":283,"path":286,"searchDescription":287,"searchTitle":288,"seo":289,"stem":290,"updated":291,"__hash__":292},"posts\u002Fagent-never-grades-itself.md","The agent never grades itself",[7],{"name":8},"Tomás Barak",{"type":10,"value":11,"toc":267},"minimark",[12,23,26,31,41,44,47,50,54,57,61,64,68,82,85,102,193,200,204,207,226,234],[13,14,15,22],"p",{},[16,17,21],"a",{"href":18,"rel":19},"https:\u002F\u002Fprovenboard.com",[20],"nofollow","Provenboard"," is a desktop app that designs printed circuit boards. You describe a board in plain words. An agent picks the parts, writes the circuit, places and routes the board, and produces what a factory needs to build it: gerbers, a parts list and a placement file for JLCPCB.",[13,24,25],{},"Getting a language model to produce a board is the easier half. The harder half is knowing whether the board is any good, and a model asked to check its own work tends to find that it did well. So the agent does not get to decide whether a board passes.",[27,28,30],"h2",{"id":29},"who-writes-what","Who writes what",[32,33,38],"drawing",{"alt":34,"height":35,"source":21,"src":36,"width":37},"Three columns. The agent writes the board. The checks write the verdicts, one of them failed. The person signs the checks only a person can answer, one still unsigned. A dashed line carries every verdict and sign-off back to the agent, which reads them and writes none.",600,"\u002Ffigures\u002Fwho-writes-what.svg",1200,[13,39,40],{},"Who writes what.",[13,42,43],{},"The verdicts come from separate checks. Each one answers a single question about the design, such as whether everything is connected or whether the board meets the factory's rules, and records its answer as pass or fail. The agent can read every verdict. It cannot write or change one.",[13,45,46],{},"Some questions only a person can answer. Has someone compared the pin map with the datasheet? Did the board work when it came back from the factory? Those checks wait for the person; the agent cannot pass them.",[13,48,49],{},"The agent fixes a failing check with layout work. When it thinks a finding is a false alarm, it can only ask. The person decides, and an accepted finding stays on the record with its reason.",[27,51,53],{"id":52},"every-decision-says-why","Every decision says why",[13,55,56],{},"The agent records its decisions as it goes: what it chose, why, and where the reason came from. That can be a datasheet, the parts catalogue, the brief, the person's own words, or, often, the agent's own judgement, marked as such. The record only grows; nothing in it is rewritten.",[27,58,60],{"id":59},"the-person-decides","The person decides",[13,62,63],{},"At each decision that matters, the agent lays out two or three whole options, recommends one, and you set how far it goes without asking. The order is always yours: it never places one.",[27,65,67],{"id":66},"passing-is-not-the-same-as-good","Passing is not the same as good",[13,69,70,71],{},"On 2026-10-08 we showed a board Provenboard had built on a KiCad Discord. Every automatic check had passed; the ones only a person can do were still open. The reviewers called the routing messy, and one wrote \"I don't see schematics\".",[72,73,74],"sup",{},[16,75,81],{"href":76,"ariaDescribedBy":77,"dataFootnoteRef":79,"id":80},"#user-content-fn-1",[78],"footnote-label","","user-content-fnref-1","1",[13,83,84],{},"They were right, and no automatic check of ours could have said so. A check says a board is valid. It does not say that an experienced person would have laid it out that way, and experienced people judge a board by exactly that.",[13,86,87,88,92,93,101],{},"So we measure that too. We route boards that people already routed by hand and compare our copper with theirs, board by board (",[16,89,91],{"href":90},"\u002Frouting-bench","how we measure the router","). The boards come from PCBWorld-Bench's D3 set.",[72,94,95],{},[16,96,100],{"href":97,"ariaDescribedBy":98,"dataFootnoteRef":79,"id":99},"#user-content-fn-2",[78],"user-content-fnref-2","2"," On our latest run, with one attempt per board, 85 % of the 96 D3-A boards came out fully routed and clean against their own design rules. Next to what the benchmark's paper reports:",[103,104,105,125],"table",{},[106,107,108],"thead",{},[109,110,111,115,119,122],"tr",{},[112,113,114],"th",{},"Routed and clean",[112,116,118],{"align":117},"right","Attempts",[112,120,121],{"align":117},"D3-A, 96 boards",[112,123,124],{"align":117},"D3-B, 9 boards",[126,127,128,141,154,167,180],"tbody",{},[109,129,130,133,135,138],{},[131,132,21],"td",{},[131,134,81],{"align":117},[131,136,137],{"align":117},"85 %",[131,139,140],{"align":117},"89 %",[109,142,143,146,148,151],{},[131,144,145],{},"Freerouting 2.1",[131,147,81],{"align":117},[131,149,150],{"align":117},"75 %",[131,152,153],{"align":117},"64 %",[109,155,156,158,161,164],{},[131,157,145],{},[131,159,160],{"align":117},"best of 5",[131,162,163],{"align":117},"80 %",[131,165,166],{"align":117},"78 %",[109,168,169,172,174,177],{},[131,170,171],{},"Reinforcement learning (PPO)",[131,173,160],{"align":117},[131,175,176],{"align":117},"94 %",[131,178,179],{"align":117},"45 %",[109,181,182,185,187,190],{},[131,183,184],{},"GPT-5.4 as an agent",[131,186,160],{"align":117},[131,188,189],{"align":117},"65 %",[131,191,192],{"align":117},"0 %",[194,195,197],"note",{"label":196},"read with care",[13,198,199],{},"Our row comes from our own evaluator, the others from the paper's. Ours passes 94 % of the D3-A boards and 89 % of the D3-B boards as the person routed them, where the paper's passes all of them, so the columns are close but not the same measure. Our router is built on a newer Freerouting (2.5), which ships inside the app. The 9 hardest boards are not run yet.",[27,201,203],{"id":202},"not-there-yet","Not there yet",[13,205,206],{},"A few things are still ahead:",[208,209,210,214,217,220,223],"ul",{},[211,212,213],"li",{},"The densest boards, where many signals leave one fine-pitch chip, can still need a hand to finish routing.",[211,215,216],{},"Pairs that must match in length are measured, not yet tuned automatically.",[211,218,219],{},"The schematic connects parts by name; drawn wires come next.",[211,221,222],{},"Signed installers are on the way.",[211,224,225],{},"Pricing is not set yet.",[13,227,228,229,233],{},"Provenboard is in early access at ",[16,230,232],{"href":18,"rel":231},[20],"provenboard.com",". If you design boards, we most want to hear where its layout and your judgement differ.",[235,236,239,244],"section",{"className":237,"dataFootnotes":79},[238],"footnotes",[27,240,243],{"className":241,"id":78},[242],"sr-only","Footnotes",[245,246,247,258],"ol",{},[211,248,250,251],{"id":249},"user-content-fn-1","A KiCad Discord, 2026-10-08. ",[16,252,257],{"href":253,"ariaLabel":254,"className":255,"dataFootnoteBackref":79},"#user-content-fnref-1","Back to reference 1",[256],"data-footnote-backref","↩",[211,259,261,262],{"id":260},"user-content-fn-2","PCBWorld-Bench, arXiv 2607.05915. Our run of 2026-10-09; the other rows as the paper reports them. ",[16,263,257],{"href":264,"ariaLabel":265,"className":266,"dataFootnoteBackref":79},"#user-content-fnref-2","Back to reference 2",[256],{"title":79,"searchDepth":268,"depth":268,"links":269},2,[270,271,272,273,274,275],{"id":29,"depth":268,"text":30},{"id":52,"depth":268,"text":53},{"id":59,"depth":268,"text":60},{"id":66,"depth":268,"text":67},{"id":202,"depth":268,"text":203},{"id":78,"depth":268,"text":243},"technical","\u002Fcovers\u002Fverdicts.svg","A circuit board on one side of a dashed line. On the other, a column of ticked boxes, one crossed, one signed and one still empty.","2026-10-12","How Provenboard designs a board, and what decides it passes.",false,"md",true,{},4,"\u002Fagent-never-grades-itself","Provenboard takes a circuit board from a plain-language idea to JLCPCB factory files. Checks the agent cannot write to decide whether the board passes.","An AI agent that designs PCBs and never grades its own work",{"title":5,"description":280},"agent-never-grades-itself",null,"agXNmG3YtjHE_MH60ZSr8xdcLvmJJ67xXcDTTWnziJs",[294,297,306],{"path":286,"title":5,"description":280,"date":279,"category":276,"authors":295,"cover":277,"coverAlt":278,"featured":283,"draft":281,"minutes":285},[296],{"name":8},{"path":298,"title":299,"description":300,"date":279,"category":276,"authors":301,"cover":303,"coverAlt":304,"featured":281,"draft":281,"minutes":305},"\u002Fhardware-that-improves-itself","What if hardware could enhance itself?","Every board you make could teach the next one.",[302],{"name":8},"\u002Fcovers\u002Fnext-board.svg","Three versions of the same small board in a row, the copper tidier on each, a dashed arrow running from each to the next.",7,{"path":90,"title":307,"description":308,"date":279,"category":276,"authors":309,"cover":311,"coverAlt":312,"featured":281,"draft":281,"minutes":268},"How we measure the router","On 105 boards a person already routed by hand.",[310],{"name":8},"\u002Fcovers\u002Fbench.svg","A grid of 105 small board outlines, nine of them dashed.",1791814350296]