> It can take a whole day to find 10 good lines to write.
So we've come full circle to code writing speed being a factor again? :)
In all seriousness, this just feels like a never-ending list of attempts to try to resist any notion that LLMs might accelerate software development, however small the increment. The original article was arguing that organizational and collaboration was the bottleneck and that taking a whole day to think about the code was not.
And sometimes an LLM can find those 10 lines in 10 minutes. Or it can find a 100 and you cut them down to 10 in two hours total. Yes I've seen this in practice. The amount of code an LLM can tirelessly ingest is super human.
The best code is no code. The second-best code is the code I delete.
My favorite JIRAs are the ones I prevent from being worked on in the first place because they were unnecessary.
The ideal prompt is the one I don't fire because it would be a waste.
In an application with an LLM component, the ideal amount of inference is zero.
Ultimately this seems to lead to "the ideal amount of computers in the world is none" but for the sake of my continued employment let's let that one go by. :)
You are speaking out of my soul. Thank you. Great example. I have grinded AI extensively 14 hours a day on my own project for months. I’ve been using AI since GPT-2.
I maxxed out Claude Max $200 subscription and before I justified spending $100/day.
And it was worth it, but not because it wrote me so good code, but because I learnt the lessons of software engineering fast. I had the exact ride you are describing. My software was incredible broken.
Now I see all the cracks, lies and "barking the wrong tree" issues clearly.
NOW i treat it as an untrustworyth search engine for domains I’m behind at. I also use predict next edit and auto-complete, but I don’t let AI do any edit on my codebase anymore.
I deleted 75000 lines of code of my codebase in the last 2 months and that was tremendously more useful to by business than the 75000 AI has written the 2 months before...
> I know full well that if you ask Claude Code to build a JSON API endpoint that runs a SQL query and outputs the results as JSON, it’s just going to do it right. It’s not going to mess that up. You have it add automated tests, you have it add documentation, you know it’s going to be good.
I feel like this is just not true. An JSON API endpoint also needs several decisions made.
- How should the endpoint be named
- What options do I offer
- How are the properties named
- How do I verify the response
- How do I handle errors
- What parts are common in the codebase and should be re-used.
- How will it potentially be changed in the future.
- How is the query running, is the query optimized.
…
If I know the answer to all these questions, wiring it together takes me LESS time than passing it to Claude Code.
If I don’t know the answer the fastest way to find the answer is to start writing the code.
Additionally, whilst writing it I usually realize additional edge cases, optimizations, better logging, observability and what else.
The author clearly stated the context for this quote is production code.
I don’t see any benefits in passing it to Claude Code. It’s not that I need 1000s of JSON API endpoints.
> If I know the answer to all these questions, wiring it together takes me LESS time than passing it to Claude Code.
That's just not true, and if it is in your case, then you're not great at writing prompts yet.
> Take the todo_items table in Postgres and build a Micronaut API based around it. The base URL should be /v1/todo_items. You can connect to Postgres with pguser:pgpass@1.2.3.4
That's about all it takes these days. Less lines of code than your average controller.
Every day I do something where the llm writes it ten times faster than I would with twice the test coverage.
And every day I do something else where the LLM output is off enough that I end up spending the same amount of time on it as if I'd done it by hand. It wrote a nice race condition bug in a race I was trying to fix today, but it was pretty easy for me to spot at least.
And once a week or so I ask for something really ambitious that would save days or even weeks, but 90% of the time it's half-baked or goes in weird directions early and would leave the codebase a mess in a way that would make future changes trickier. These generally suggest that I don't understand the problem well enough yet.
But the interesting things are:
1) many of the things it saves 90% of the time on are saving 5+ hours
2) many of the things I have to rework only cost me 2+ hours
3) even the things that I throw away make it way faster to discover that 'oh, we don't understand this problem well enough yet to make the right decisions here yet' conclusion that it would be just starting out on that project without assistance
This. There is definitely a ratio. A year ago, it was 50/50. It felt better because the hard things it did fast while I sipped coffee outweighed in my mind the negatives.
Now that ratio is swinging way over towards the LLMs favor.
How do you reconcile that with your example prompt, which demonstrates no skill requirement whatsoever. It’s the first thing any developer would think of.
It’s simple but contains all the necessary info. You can say “build an endpoint to get user data” and it will absolutely do something, but it might be stupid, and when you compound 1000 stupid prompts like that you get spaghetti.
It doesn’t contain any information at all about the structure of the JSON output. Is this a greenfield endpoint and anything will work is does it need to conform to an existing API? What about response codes for different failure modes? What about logging?
Your comment exemplifies what a lot of people complain about vibe coding: it works great for greenfielding CRUD apps, but it’s a bitch to use in a real code base.
I've drank the AI koolaid so I'm not a hater, but to say "you're just not prompting right" is such a cop-out. Prompting right takes a metric fuck ton of effort. I'm actually kinda agreeing with you, if you make it to where you're dev environment is sufficiently harnessed, then you can give it one-liner magic prompts. But getting there, learning to get there, paying that cost, hot mother of god it's a lot of effort.
Communicating, in words, is extremely hard. I don't think this should be as controversial as it's seems in the prompt era.
VS: someone has mastered one of the myriad openAPI generators, and it's shipped.
it does take a little while to get good at this new skill, yes. Just like, say, learning a new programming language and the ecosystem around it takes some effort. After you get over the hump it's really very straightforward and mostly a matter of knowing the kinds of mistakes the LLM is likely to make ahead of time, and then kindly asking it to do something smarter. If you've successfully mentored junior engineers you already have this skill.
that's well put. But i'd stress mentoring junior engineers is really a high effort, high leverage, high demand skill. A good teacher is gold. and not common.
I'll go in the other direction and say that if you're spending a lot of your time learning to prompt better then you're wasting it because LLMs are only going to get better at understanding your intent regardless of "prompt engineering". The JSON API example to wire up a database can be one-shot pretty easily by the latest models without much context and without setting up any harness. The more time you spend perfecting your harness, the more time you would have wasted when the next model comes out to make it obsolete.
The hardest thing about software engineering has always been that your intent often has to be decided on the fly once you get into complicated edge cases, weird-or-legacy-business requirements, or things that the spec literally has no answers for.
Letting the tool figure out your assumed intent on those things is a double-edged sword. Better than you never even thinking of them. But potentially either subtle broken contracts that test coverage missed (since nobody has full combinatoric coverage, or the patience to run it) or just further steps into a messy codebase that will cost ever-more tokens to change safely.
I was thinking of this interpretation as I read that:
"I'll go in the other direction and say that if you're spending a lot of your time learning to [program] better then you're wasting it because [computer]s are only going to get better at [computing] regardless of "[software] engineering". The JSON API example to wire up a database can be [run] pretty easily by the latest [computer]s without much [design] and without setting up any [optimizations]. The more time you spend perfecting your [program], the more time you would have wasted when the next [computer] comes out to make it obsolete."
I don't think it does. If I had to guess, the top comment was using an older version of AI or a local model which wouldn't be able to solve the JSON API task. A lot of AI skepticism comes from people who used it once a while back and decided not to keep up with the latest developments. If I only had experience with gpt-3.5 then I'd also assume what the original commenter said.
An experiment I'd love to do, but which isn't actually possible anymore, is run GPT 3.5 or the original 4 API release through a modern "agentic" harness for a task like this.
I think 3.5 would probably need more frequent intervention than a lot of harnesses give. But I bet 4 could do a simple JSON API one-shot with the right harness. Just back then I had to manually be the harness.
I disagree it's a cop-out, but I agree it's hard to get good at writing prompts and takes a lot of effort. But so is programming. We're trading one skill set for another and getting a bigger return on it.
I started as a skeptic and have similarly drank the kool-aid. The reality is AI can read code faster than I can, including following code paths. It can build and keep more context than I can, and do it faster as well. And it can write code faster than I can type. So the effort to learn how to tell it what to do is worthwhile.
yep fully agree. i'm taking issue with the flippant "not prompting right" as if they're holding it upside down vs it's actually a meaningful skill to have to invest in so it's fully believable that someone trained in normal code gen is much more proficient up front.
this seems disingenuous. even if your premise is true (which i don't think it is), it only really holds for the first few endpoints. most systems have many, and the models are very good at copying established patterns to the point that you wouldn't normally have to re-explain every detail for every endpoint. so you might be right for the first (you're not), but you're definitely wrong for the next 50.
Like writing code to me is not slower than writing text?
When I write code every character I type in my computer has less ambiguity than when I write it in human language? I also have the help of LSPs, Linters and Auto-completes.
I use AI to look things up and I try to learn. That part is speed up, but once I know how X works I’m faster doing it myself. My assumption is that most people seeing things differently, compare their performance of not knowing how X works with Claude, but not with someone who’s really good at X. Which makes a lot of sense given LLMS are prediction generators. My take is that the best use of AI is to get you to the point where you are really good with X and then naturally your AI usage will go down.
I have a similar sentiment. Subject that makes the claim that AI writing code is fast is going to matter a lot because some programmers heavily use "LSPs, Linters and Auto-completes", key bindings, snippets, CLI commands, etc to speed up writing code
It's not much to go on by, but I kinda feel ya. I think one exception I'd perhaps make is doing a large mechanic refactor. I find them incredibly daunting. So, I'll just ask AI for that. I mean it probably takes me a similar time to do, but it feels less daunting.
I've been trying to get into agentic coding and there are non-refactoring instances where I might reac for it (like any time I need to work on something using tailwind; I'm dyslexic and I'd get actual headaches, not exaggerating, trying to decipher Tailwind gibberish while juggling their docs before AIs came around)
I use Jetbrains features for that usually, it has great tools for that.
Lets say on that JSON API I want to extract part of the logic in a repositiory file i CTRL + W the function then I have almost all of my shortcuts with left alt + two character shortcuts. So once marked i do LAlt + E + M for Extract Method then it puts me in a step in between to rename the function
and then LAlt + M+V for MoVe and then it puts me in an interface to name the function.
Once you used to it its like a gamer doing APMS and its deterministic and fast. I also have R+N (rename), G+V (generate vitest) Q+C(query console), Q+H(Query history) and many more. Really useful. Probably also doable with other editors.
I highly recommend looking into codemods for larger mechanical refactorings. I did things like converting large test suites from one testing library to another by having codex write a codemod to convert it as a first pass.
I use voice to text and for me coding is way faster now. You don't need to sit down and type up a perfect spec lol. I give it terrible prompts with poor grammar and typos from incorrect transcriptions and it does an amazing job. Definitely not perfect I iterate with it a ton but it's still faster than typing it out by hand
You're still typing? I don't know how fast you can type, but I can speak way faster than I can type. Somewhere in the neighborhood of 300 wpm. Speech-to-text is pretty good now, and prompting an AI means I'm not trying to speak curly brace semicolon new line.
Average speaking speed for english speakers is 100-120 wpm for complex topic. I type 130wpm peak and I have the most common coding characters on my home row using neo layout.
This may have been a problem a year or two ago but any premium model will be exploring the codebase to check similar routes to answer all these questions, if you don't specify them.
Exactly.
As long as the codebase is consistently following some given patterns, LLMs nowadays stick to it.
Understanding that limiting number of “design patterns”
in a codebase made it better (easier to code and understand) was a good proxy for seniority before LLMs.
Now it’s even better: if all of a sudden “unusual code” is in a PR, either the person opening the PR or the one reviewing it has lost touch with the codebase.
Very important signal, since you don’t want that to happen with code you care about.
Yeah I can and I’ve done it and for fun project it’s fun and cool. But its like using templates to build your website. You’ll be annoyed and at one point your project goes in the endless graveyary of abandoned projects
I think most people are finding the opposite. Claude Code is not only reducing how many projects get abandoned, it's also resurrecting projects from the graveyard.
The number of Show HNs recently that have a days worth of commits and are never touched again disagrees. It's creating a lot of projects that are immediately abandoned
I think its a direct reflection of the fact that most people really prefer to go-go-go and not spend the time up-front thinking about what their project even is, why it matters and is it worth dedicated resources toward it. The abandonment usually reflects the answer - no it was not worth it.
There is a difference between a project that is eventually abandoned out of annoyance because you couldn't accomplish what you wanted and a project that gets a day or two of attention and then gets aborted because you figured out it wasn't worth it or got interested in something else. I think the parent comment is talking about the former and I'm responding to that, while you're talking about the latter.
I don’t want every verb implemented, I also dont want an IETF standard. I want as little as possible, so I have to worry about as little as possible in the future.
Use-cases differ, you described a complete REST API, which can be as much of a problem as a too little.
Till it has explored the codebase, asked me follow up questions, suggested the code change, incorporating my fixes after losing time on context switch + the extra time I need when somebody requests a change in 3 months to learn the mental model. I’m way faster to just write it myself (mental model included)
If it's genuinely the case that you can write code faster than you can prompt it into existence then you're not being ambitious enough with your coding agent. Ask it to do more. Tackle bigger problems.
1. It's unclear why creating more code faster is a good thing. Software engineering wisdom for decades has been that code is a cost, not a product. There are great reasons for that, which haven't changed with the appearance of LLMs.
2. There absolutely are cases where modifying code "manually" is unquestionably faster than prompting an LLM. There are trivial examples for this - eg only an insane person would ask an LLM to rename a variable rather than using an LSP for that. It would provably and consistently take more keystrokes. There are less trivial examples as well, like, you know, having an understanding of your codebase and using good abstractions/libraries within it that let you make large changes to the program's behavior with little boilerplate code.
One can argue that producing a lot of complex changes through an LLM is faster, which I would agree with, but then see point #1. Sustainable software development has up to this point relied on iterative discovery of the right small components that together form a complete, functional, stable system (see "Programming as Theory Building").
There's zero indication so far that LLMs are capable of speeding up the process of creating complete, functional, stable systems. What every org within my career and friend circle is seeing (and research into productivity impacts of LLMs on software development is showing) is the same story - fast prototypes that either turn into abandonware, personal tools, or maintenance nightmares.
I think the motion that Claude Code and Anthropic has is trying to force-hide stuff from you. Some hopefully remember the shitstorm that happened, when they changed. Reading xxx.yy to reading 1 file or reading 2 files.
More changes like this came and they were not or very hard to configure. I understand the business idea behind it. Make them to use AI as much as possible, get the human out of the loop. More training data. More Token Usage JUHUU.
However I think that made Claude Code so much worse and so much more untrusthworthy. It’s a sneaky attempt to take away the driving wheel from you. And if you follow that logic, way more and way more things seem reasonable.
But mainly for now it just generated a lot of distrust for me
That's how you end up with those cooking recipes that only "take 5 minutes". Sure, if you don't count buying all the ingredients, cleaning and preparing them, cleaning up the pots and pans (and probably the worktop, stove, etc), a lot of things can take 5 minutes. Even trivial stuff like scrambled eggs don't actually take 5 minutes when you take everything into account.
Reminds me when I automated a manual service deployment that only "took 5 minutes". Sure, copying the binaries only took 5 minutes, but coordinating between various departments to deactivate the relevant monitoring bits, turning off the services, invalidate the caches, etc, etc actually took half a day with humans involved. Once automated and parallelized the thing took about 10 minutes for a data center.
But have you thought about “fun factor”? It’s where you sit like an addict in a casino for weeks and burn tokens in a hope of winning a software that you could’ve written?
Who doesn’t consider “fun” thinking about work crap all the time, writing to your agent, verifying walls of slop?
It's not, but the author did say they have used this test against models when they come out. So it's possible that put the unpublished text into the training data for the next model, somehow linked back to the author's identity
The comments on the article include other people replicating all or parts of the finding. I'm also pretty confident Kelsey Piper wouldn't fail to disable memory while simultaneously talking about how Claude incognito mode is insufficient to prevent the app from handing it your name.
"I did not have memory enabled, nor did I have information about me associated with my account; I did these tests in Incognito Mode. To make sure it wasn’t somehow feeding my account information to Claude even in Incognito Mode, I asked a friend to run these tests on his computer, and he received the same result; I also got the same result when I tested it through the API."
Given those precautions if it is just memory or some form of deanonymization that's also cause for concern.
It can take a whole day to find 10 good lines to write.
reply