Working with LLMs
Written without LLM assistance.
I recently started a new role and have been working with some lovely people. However, I’ve noticed that many people in this team heavily use LLMs.
At first, I thought nothing of it and said nothing of it because I use LLMs myself. Wer im Glashaus sitzt, sollte nicht mit Steinen werfen. Yet, I’ve noticed that my annoyance at LLM users is steadily increasing since I joined this team. I’m afraid that I’m becoming a Luddite.
Another interesting aside is that most LLM users are more senior engineers. Many are vibe coded features that were not reviewed by what should have been watchful eyes. The correlation of seniority to LLM usage is positive, although I’m also skeptical that there may be underreporting or unobserved usage by our tracking tools. This also poses other problems on team psychological safety. If an engineer is wont to hide their LLM usage, how can one trust that they’re allowed to take risks and to be honest?
I also found out a few hours ago, that some people’s LLM spend was equivalent to the median monthly wage here in Japan. This post is frankly an expression of my bewilderment. While some of us are careful not to spend over $30 per month, a few are spending money on AI tokens that could feed a family. At this point I’m asking myself: do I even work with humans? Or am I working with LLMs with human agents?
I want to process all of this carefully. Maybe even go through the stages of grief, but I don’t have the time. The next logical question then is: how do I work with LLMs? Alright, maybe that was a bit harsh. How do I work with LLM superusers?
My reflex was to throw an LLM right back at their work. Which I did, tonight, because I’m tired and at this point at 1:00AM need sleep. This kind of response is not sustainable and would only produce more AI slop. I do not want to perpetuate this race to the bottom of mediocrity. Could extreme agentic engineering be actually addressed at the root? Could we enact a cultural change within the team so that 1) LLM use is less haphazard and more thoughtful and 2) people don’t have to hide their LLM usage?