Where I let decent be enough
My job title says AI and ML engineer, and I started out as a data scientist. These days, for the AI workspace my colleagues use, I build the frontend, the backend and the system design under both. I’m decent at most of that, nowhere near great, and so far decent has been enough to ship.
I don’t think that’s unusual any more. The first React app I shipped was mostly written by a model, and I didn’t know enough to question any of it. The models have got a lot better since then (GPT-6.1 Sol, Fable 5.2), and each release takes me a bit further into a field I never trained in. So the lines between roles are getting blurry, and “be a generalist” is the advice everyone gives.
I don’t find that advice very useful, tbh. It doesn’t tell me where to spend my effort, and I can’t go deep on everything. What I keep running into is a smaller question. Which parts of the job can stay decent, and which ones do I need to know properly?
The parts where decent is fine
Frontend is the main one, and really anything user-facing. That first React app was a mess when I went back through it a few weeks later. It had packages I didn’t need and state being passed around in ways I couldn’t follow (I wrote about that in What I lose first when the AI goes down). Cleaning it up took ages. But look at how that mistake behaved. The app ran the whole time. I could see every screen, and when I fixed a component I could see the fix on the next refresh. A UI mistake is in front of me the moment I make it, and the cost of redoing it is my own time.
Unit tests are the other one, and this is where I’d expect people to argue with me. I never learned to write them properly and I’m not planning to. I know the few things that have to be checked for a feature, I say what they are, and the model writes the tests. Is that risky? A bit, yeah. Tests are how I check the model’s work in the areas where I’m weak, and a model can write a test that passes without proving much. So I’ve kept the part where I decide what gets tested and handed over the typing. Whether that holds up, I don’t know yet.
A video is the same kind of thing. Say I need a short video to show what an app does. The tools I have won’t make what a specialist would make. So I have to ask who’s going to watch it and whether it does the job for them. There’s a line somewhere. Below it the thing looks like slop, and people can tell. Above it, more polish is nice, and I’m not sure anyone needs it.
Where it’s cost me
System design is the part I want to know properly, and I can point at why.
That workspace went from about 200 people a week in June to about 900 by the end of September. About 600 people use it on a weekday now. Ordinary traffic has never come close to hurting it. From 17 to 30 September the servers sat at 24% CPU or less for 95% of the time. What hurt was one person doing something I hadn’t planned for, and it was nearly always files.
On 29 September one project added 953 files in about half an hour. Nothing limited how many files a project could add at once, and the worker that indexes files shared two servers with chat. The worker took about half the CPU, the slowest 5% of API requests went from about 0.1 seconds to 0.9, and some of those files waited 49 minutes before indexing started. The queue was strictly oldest first, so a file from a completely different project sat behind that upload for 19 minutes. That’s called head-of-line blocking, which I learned afterwards.
A week later it didn’t even need a big upload. One person retried 48 failed PDFs. Six indexing batches overlapped, and 43 PDFs were being read at the same moment at roughly 450 MB each. Worker memory went from about 1.2 GB to about 11 GB on each server. Both servers hit 97 to 98% memory, the chat process was killed and restarted, and the API stalled for about four minutes. I’d put a memory cap on each read. I hadn’t capped how many reads could run together. The memory alert stayed silent too, because it watched a five-minute average and the spike was shorter than that.
Then from 3 to 5 November one project uploaded 7,558 files, and indexing stalled for every project for about two days. The concurrency cap was in by then. This time it was the query that hands the next file to a worker. It recounted the files in progress once for every row in the queue, while holding a lock that everything else was waiting on. Some files waited 53 hours.
None of the fixes are exotic, and each one has a textbook name.
Indexing now takes one file per task, with at most four in flight across the whole worker. That’s a concurrency limit. The next free slot goes to the project with the fewest files in progress, so a small upload doesn’t wait behind a huge one, which is fair queueing. The worker moved to its own servers, so a memory spike there can only slow background work. People call that workload isolation.
Chat uploads got their own worker processes, because someone watching a spinner shouldn’t queue behind a bulk job. During one big upload a chat attachment had waited a median 70 seconds to be processed, against 0.3 seconds the week before. A worker that dies now gives up its file within three minutes (a lease), where it used to hold the slot for about half an hour. And there are checks at the door. An oversized upload is refused before the server reads it, and the file type comes from the file’s bytes, since the name can say anything.
Every one of those is a limit I didn’t think I needed. It isn’t only queues, either. When I added teams to an app built for one user, two booleans that should’ve been one choice ended in a migration. And this year our prompt cache broke four different ways without throwing a single error. I found each one on a bill.
A model wrote most of that code with me, and honestly it did a decent job. It also helped me fix every one of those problems once I could see them. I think that’s how it goes. The model is good at reacting to a problem after it has shown up. And I had asked the obvious question. Every file had a size limit and every read had a memory cap. Someone who has run a queue under load asks the next one, which is what happens when forty of those run at once, and I didn’t know to ask. I hate that. So for now the model does the maintenance, which is the reacting it’s good at, and I spend that time learning this side of software. I’m guessing a person with real architecture experience would have hit half as many of these.
None of it stopped the app, and I want to be fair about that. No chats failed during the 953-file burst. I hit more issues than a specialist would have and I still got there. It cost me weeks, though. Next time I’ll know what to look out for and what to ask.
How I choose
When I put those two lists next to each other, what separates them is when the mistake shows up and what it costs to undo.
Redone before anything sat on top of it.
Found in production. Two pieces changed while people were using it.
Frontend mistakes show up straight away and are cheap to redo, so I let the model carry me. Architecture mistakes sit quietly until there are real users or a real bill, and by then other things are built on top of them. That’s where the gap between me and a specialist costs the most, so that’s where my learning time goes.
The other thing that sets the bar is how many people I’m serving. A tool for my colleagues can live with a UI that a public product with paying customers couldn’t. If the audience got ten times bigger, some of what I’ve left at decent would have to move.
The same thing outside software
I work at a construction consultancy. We have project managers, cost managers and so on, and each has their own lane. I think those lines are going to blur the way mine did, and I don’t think most people are ready for it. Part of that is not having seen what the current models can do. If the last thing you tried was a chatbot from two years ago, why would you think anything had changed? I’m guessing a lot of people have only tried the free versions of ChatGPT or Claude, which aren’t the models I’m talking about and do a mediocre job in a business setting.
I want to be careful here, though. I crossed lanes in software, where a mistake usually shows up fast because the thing breaks in front of you. I’m not a cost manager, so I’m guessing, but a cost plan that’s wrong probably doesn’t break in front of anyone. It shows up months later on site, when it’s expensive to put right. By my own rule that’s the kind of work where depth matters most. So people in those roles have a real reason to be slower about this than I’ve been.
I still think it’s coming. A project manager with a good model will get to decent at parts of a cost manager’s job, and the other way round, the same way I got to decent at frontend. What stays with the specialist is knowing where the line sits for a given job, and everything above it. That’s a smaller patch than a whole lane, and I can see why nobody’s in a hurry to accept it.
I don’t have this worked out. I know which of my own skills I’m happy to leave at decent, and I know the one I’m not. Whether I’ve put the line in the right place, I’ll probably find out the usual way, when something I left at decent turns up as a problem.