Burak Köken

A veteran developer's honest ledger.

Yes we work together, AI and I

Published

This one is for those of you using AI to code your favorite hobby app or any app for your boss. Here are a few tips and tricks, learned by reading and watching and unfortunately spending actual stealing-from-life hours.

Discovery

First a very niche one, if you are doing some discovery stuff. Prepare a DISCOVERY_PROCESS_GUIDELINES.md, keep it somewhere it can see it. I am leaving this part open: It depends on which app/harness/CLI you are working with. Unfortunately, it will not read it at every step. When you sense slippage, say “Read DISCOVERY_PROCESS_GUIDELINES.md” Mine never ever responded with other than a variant of “I read them, and my walk-51 report broke rules 1, 2, 3 and the near-miss wire-in”. Hah. As if accepting its error brings your life-minutes back. The contents, well a little bit up to you. Mine is 103 lines long, so I cannot copy paste here but do not be alarmed because AI itself wrote it, under my guidance. Yes, sick. Guidelines written under guidance. Well, no one handed me ten stone tablets, I started somewhere, it formalized the finished document. One sample item for you:

3. REPORT IN ORDER OF STRUCTURE STRENGTH, not in order of hypothesis match. The operator reads findings; the hypothesis is the assistant’s own artifact. If the strongest structure in the output is not the registered one, the strongest structure is reported first. (Read, always read)

Today we are writing code

Put down rules.

The first part of writing a function (sub process) is writing the test for it first. Writing the function itself must come later.

It will not obey.

Test first, and the red must name the defect. Failing test written, run, its red output in the transcript. Every fix names the test that should have caught it.

It will find out there is none. None exists → write it; one existed and was green → that test is the defect. Behavior-preserving changes are judged by [dear reader, please put your “father of all scripts” here] (old vs new on real data). A wrong comment or stale figure has no test → say “no test, this is prose”.

Some design rules D1–D4.

This one is about code reviews, see below.

Two more rules I pulled from the pile of code writing guidelines, in the agent’s own words verbatim, which I asked it to put down for me:

Runs: no fleet without (e.g.) cost_gate.py measuring one real unit first; no run without an explicit “go”.

Every displayed value cites its producing script; timestamps are read from the host clock, never estimated.

Surprising, this last one is the most broken one, 1 out of 3, roughly. After every response, the agent is supposed to provide me with an Operator Summary. Capped as a short paragraph, saving me from scrolling pages upon pages of output to understand what has been done and what is expected of me to decide. And every Operator Summary must end with a timestamp in my local timezone. Sometimes it is missing altogether. Sometimes it advances only 10 minutes in two hours. Sometimes it is a summary from the near future. Completely bonkers. The agent “invented” a hook, the timestamp went through it, and the hook poked the agent to correct the hallucinated timestamp. I can see it work, sometimes the Operator Summary is repeated. The agent apologizes, tells me the hook worked (prideful) and the timestamp is corrected and then the whole summary, once more, verbatim, with the correct timestamp. But the hook does not fire IF there is nothing to trigger it. Clever but not so clever. And believe me when I say, that will be the story of your life when you work together, AI and you.

Bug Hunting and Squashing

I used Claude, Codex, DeepSeek Harness with GLM 5.3 Flash. With my token-limited budget, this is what I do.

The idea is to use a coding-capable model, usually one tier below the fancy, new and better one. After you feel there is enough code produced to sink the whole project with its weight, I say something like that to (Claude example) Opus, “Perform a Fable cold review on the pipeline you just built. Tight scoped. Attack execution engine.” (Fable is Claude’s flagship model, as of this writing, one level above Opus.) or if you are rich and Fable is writing the code, anyway, then “Perform an adversarial self review on pipeline code. Attack execution engine. Try to break it.” On a good day this will get you 4–12 defects, some blocking, some critical, some edge cases, and finally some documentation issues.

DO NOT let it fix the defects just after the review comes in. You will get into trouble: It will create another 5 defects while fixing the first round. You will need to ask for another fixing round. More errors (hopefully fewer than you fixed). But I have not one instance where I reached 0. Even after 8 rounds. Ever.

This is not the full solution, but it helps.

“Method: Do not batch-fix. Go one by one. Fix, attack adversarially, fix, attack, loop. When green, move on to the next”

Documents, everywhere

Anything you want done in an orderly fashion, put it in a document. One teaser example for you.

I have CONCEPT_INTAKE_PIPELINE.md — 24 numbered research-code conventions. The opening statement pasted here.

Purpose. The machine is a refusal engine: a solo operator cannot hand-feed ideas one at a time, so this is the standing process for pulling candidates from the wild and letting the honest scorer refuse or promote each. The goal is a book of proven mechanisms, not one edge. Re-runnable with any source/query; runs unattended.

I have a CHARTER.md. It is like the constitution. The opening statement pasted here.

CHARTER.md : Operator’s standing decisions Purpose: This file records decisions the operator has already made, so no session re-litigates them. Load this at session start. If anything here conflicts with a whim of the day — the charter wins until the operator explicitly amends this file.

As you can see, this is not a democracy, maybe a constitutional dictatorship, if there is ever such a thing. Now, this file is loooong. 4028 lines as of this writing. One (very careful) job: removing duplicates and trimming it a little bit. Another day.

I have a PROVENANCE.md, where every load-bearing number came from. The opening statement pasted here.

Purpose. REVIEWER_FAILURE_MODES.md B-1 requires that no advisor-proposed threshold be adopted without its derivation recorded, and B-4 requires the cold reviewer be told which numbers the advisory channel authored. Both were unexecutable because that provenance lived only in recollection — which is operator vigilance wearing a different hat. This file is the artifact those checks read from.

Rule. One line per numeric threshold, written when the number enters a document, never reconstructed later. Fields: value · source · derivation · date. Source is one of: operator · advisor-proposed · derived-from-<artifact>. A number whose derivation is absent is tagged UNDERIVED, and every verdict resting on it inherits that tag until a written derivation or a cold review clears it.

This was another example of a document born out of necessity, which is the mother of all inventions.

Addendum, 8th of Sept, 2026

I wrote this piece a few weeks ago. I did a little bit of editing, honestly I did not keep track. It is set to be published tomorrow and the following happened. I asked for a review on a repo I am working on. Here is what the agent actually did. Too good an opportunity to pass. Some of the above, in action: I am not touching it, this is a direct copy paste, all editing I did was remove whitespace and extra empty lines.

Two fixes, one at a time. W-001 first — it's my incomplete fix from this morning. Red test, using the review's witness:

Created test_embedding_failure_does_not_poison_retry.py, ran a command +159 -0

5  red, reproducing the review's witness plus the partial-note and ingest cases. Reading the current ingest boundary before changing both:

Ran 2 commands, created fix_w001.py +110 -0

398 pass. Attacking the fix before moving on:

Created attack_w001.py, ran a command +64 -0

W-001 committed, both paths perturbed independently. Now W-003 — and its root cause is a rule I have written down: if correctness depends on a caller remembering, the design is wrong. status remembered; the three commands that actually send data did not. So the disclosure goes where it can't be forgotten. Red test:

Created test_egress_is_announced_before_sending.py, ran a command +155 -0

All six red. Fixing at the choke point every processing path must pass:

Created fix_w003.py, ran 2 commands +69 -0

Both perturbations killed. Live, with a remote host configured, the disclosure now precedes the connection attempt:

Committed W-003

Both fixed, one at a time, each perturbed before moving on.

And so the story goes… I hope this helps someone, somewhere, somehow, somewhen.

Addendum to the addendum, 11th of Sept, 2026

The repository went public. The two failures recorded above, W-001 and W-003, are fixed commits in it — github.com/bkoken20/winnow — cloneable, runnable, breakable. The receipt that was a promise is now an artifact. How it got there is its own announcement: Generalized, Sanitized.