I spent an afternoon earlier this week trying to get two of the most capable models on the planet to draw a map of the United States. Not a good map. Any map. I failed. A lot.
The setup was innocent enough. I was building an internal training module at Google for people who haven’t done much agentic development yet. The goal was to show them different ways to drive these tools, including the parts that don’t involve an IDE. One section covered calling the Antigravity CLI (agy) in headless mode with -p, so you can wire an agent into a shell script like any other command. I’ve written about how Antigravity’s async model cuts context switching before, and headless is the same idea taken all the way: the agent becomes a unix tool.
I needed an example. Something real enough to be useful and common enough that people would remember it.
The idea was good
Here’s what I landed on: IP address exhaustion. Every cloud architect I’ve worked with has been burned by a subnet that ran out of room at the worst possible moment. It’s a real operational concern and the data is easy to get.
gcloud compute networks subnets describe my-subnet \
--region=us-central1 \
--view=WITH_UTILIZATIONThat --view=WITH_UTILIZATION flag gives you allocated and free addresses per range. Loop it across your regional subnets, and you’ve got a picture of where you’re about to have a bad day.
So far, so good. Now the fun part. What if the output wasn’t a table? What if the agent drew me a color-coded ASCII map of the US, with each region sitting roughly where it belongs geographically, shaded green to red by how close it was to running dry?
That would be extremely cool. Training modules live and die on whether people remember them, and nobody remembers a table.
It also happened to be the day Gemini 3.6 Flash shipped. Fresh model, fresh capabilities, low latency. New and shiny. Perfect.
The idea did not survive contact
Test one: a potato. Not a metaphor. A lumpy asymmetrical blob with us-central1 written somewhere inside it.
OK. My prompt was probably too loose. Let me be specific. I described the rough proportions, told it the West Coast is on the left, gave it a target width in characters.
Test two: squares. Just squares. A grid of them, region names crammed inside, no relationship to anything on Earth.
I refined again. I gave it an example of the output shape I wanted. I described Florida.
Test three: something I can only describe as an ASCII rendering of a Jackson Pollock painting. Characters everywhere. Zero structure. Confident labels attached to nothing.
I kept going. Six or seven more rounds. Confusion turned to genuine irritation, which is a weird feeling to have toward a matrix multiplication. At some point a shape appeared that was unmistakably a Mon Calamari cruiser. Home One, drifting across my terminal, where Texas was supposed to be.
WHAT HAVE WE RELEASED.
Then I did what any reasonable person does when they suspect a tool is broken. I opened Claude Code with Opus 4.8 and gave it the exact same task.
Same disaster. Different flavor of disaster, but same category. Two different labs, two different frontier models, one shared inability to draw a shape most people can approximate on a napkin.
The reality is more nuanced
My first instinct was that this was a quality problem. Something regressed. Somebody shipped too early.
That instinct was wrong, and there’s research to prove it.
A paper out of UC Santa Cruz in April, Learning to Draw ASCII Improves Spatial Reasoning in Language Models, names the exact thing I ran into. The authors call it a read-write asymmetry: “LLMs interpret ASCII representations effectively but struggle to produce them from text.” Models can look at ASCII art and tell you what it is. Ask them to make it, and they fall apart.
The numbers are consistent enough to be a little funny:
| Model | Reading | Writing | Gap |
|---|---|---|---|
| Qwen2.5-7B | 38.4% | 16.1% | 22.3 |
| gpt-oss-20B | 58.1% | 44.0% | 14.1 |
| Llama-3-70B | 57.4% | 43.9% | 13.5 |
| GPT-4.1 | 72.5% | 53.6% | 18.9 |
Every model tested showed the gap, somewhere between 13 and 27 percentage points. The authors identify construction, not comprehension, as “the primary constraint.” Scale doesn’t fix it. GPT-4.1 has the same shape of problem as a 7B model, just at a higher floor.
Once you see why, it stops being surprising. A model generates tokens in a line. ASCII art is a two-dimensional object encoded in one dimension. To put a character in the right spot on row 14, you have to have tracked what’s on rows 1 through 13 and counted columns the whole way. There’s no canvas. There’s no going back to nudge a character left. It’s a plotter drawing blind, one row at a time, from memory.
Every refinement I gave it made this worse, not better. More constraints meant more spatial bookkeeping across a longer generation. I was politely asking it to fail harder.
This is what the jagged frontier feels like
We talk about the jagged frontier a lot. Models are superhuman at some things and startlingly bad at others, and the boundary doesn’t follow human intuitions about difficulty.
That’s easy to nod along to in the abstract. It’s a different experience to find the edge with your face.
The thing that stuck with me is the mismatch. In that same session, the agent wrote a clean shell loop across regions, parsed the utilization output, and got the thresholds right. That’s the part I would have called hard. It nailed it on the first pass.
The part a bored middle schooler could do with graph paper? Ten attempts, zero successes, one Rebel Alliance capital ship.
There’s no rule I can give you for predicting which side of that line a task lands on. I’ve been doing this work daily for a couple of years now and I still get it wrong. Precise character positioning turns out to be a cliff. Counting things in a long list is often a cliff. Anything requiring an internal canvas the model doesn’t have is usually a cliff. But the map of the cliffs isn’t published anywhere, and it moves with every release.
What actually worked
I didn’t solve it. I gave up on it, which turned out to be the right call.
I threw out the map. The example became a bar chart of GitHub star counts for a few well-known projects:
microsoft/vscode ██████████████████████ 187.9k
ollama/ollama █████████████████████ 176.9k
anthropics/claude-code ████████████████ 139.1k
google-gemini/gemini-cli ████████████ 106.2kFirst attempt. Both models. Every time.
Once you know about the read-write asymmetry, it’s obvious why. A horizontal bar chart has no second dimension to track. Each row is independent: a label, some blocks, a number. Get row three wrong and row four doesn’t care. There’s no canvas to hold in working memory, just a repeated pattern of “count to N and stop.” I hadn’t made the model any smarter. I’d stopped asking it for the one thing it can’t do.
The example got better in ways I wasn’t aiming for, too. The GitHub API needs no auth, so nobody has to have a cloud project set up to follow along. The data is one request instead of a loop across regions. And star counts are something people recognize on sight, so the visualization does its job without a setup explanation.
That was a found lesson, not an intended one. I set out to teach headless agents and accidentally learned something about choosing an output format. The constraint made the example simpler, and the simpler example was the better teaching tool. I’d love to claim I planned that.
So what
I’m not writing this to dunk on Gemini 3.6 Flash, which is a genuinely great model, or on Opus 4.8, which I also use daily. I’m writing it because the failure mode was so much more interesting than the success would have been.
Here’s what I actually took away.
The failure was silent. Not once did either model say “I’m not good at this.” Every single attempt came back confident and well-formatted, with correct region names, plausible-looking color codes, and a shape that was nonsense. If I hadn’t been able to look at the output and know what North America is shaped like, I would have shipped a training module with a Mon Calamari cruiser in it. That’s the whole job right there. Not writing the code. Knowing when the answer is wrong. It’s the same lesson I got the hard way when an agent broke a feature it had just built, except this time the failure was funny instead of expensive.
Refinement isn’t free. The instinct to iterate is a good one and it’s usually right. It was exactly wrong here. Ten rounds of increasingly specific prompting bought me nothing, because the constraint wasn’t understanding. Recognizing an architectural limit and stopping is a skill, and it saves more time than better prompting does.
Route around it, don’t push through it. The fix wasn’t a better prompt. It was a different output format. When the precise part is fighting you, the first question isn’t how to force it. It’s whether you need it at all. Model for the fuzzy part, deterministic code for the precise part, and be willing to change the goal when the goal is the problem. That’s not a workaround. That’s just system design.
I’ve written before that these tools were never magic, and that line count tells you nothing about whether work is good. This is the same idea from a sillier angle. The models are extraordinary. They’re also weirdly, specifically bad at things nobody thought to warn you about, and they won’t tell you when they’ve wandered into one.
Somebody has to be in the loop who can tell a map of Texas from a Star Wars spaceship. For now, that’s still us.
The tools change. The craft endures.