Everybody has a plan until they get punched in the mouth
It is rare for me to quote Mike Tyson, be he was onto something with this. Here is the thing with coding agents: you will not tell a coding agent to do something in one sentence and then something usable falls out. I mean… there is a chance - depending on your prompt. If you tell it to copy something that exists millions of times on GitHub. But anything even remotely more involved than that, things will be a bit more tricky, especially if you are dealing with an existing system and not one shot-ing a Tetris clone.
Building out a small fleet of coding and research agents for LazerBunny means I regularly have requests to LLMs running, which are hosted on various hardware in my local network. This is not a big deal most of the time, but things get a little bit more interesting when multiple agents compete for the limited local resources. I spent some time this week to look into potential solutions.
I have worked on software for Apple devices on and off during my career. It even goes as far back as working on a desktop application, before things were called apps or the little glass slabs became our daily companion. Some of my highlights include having an app featured in the AppStore, working on an Apple Watch integration and having day one support (mostly, thanks review process) and some other fun stuff.
This week was highly unstructured in regards to working on Lazerbunny, to the point that you could have rolled a dice to figure out which task from the todo list I will pick up at any given day. The two most important ones were progress on the avatar and adding context compression to the coding agent.
I am a very big fan of self-hosting services I use on a daily basis or that hold my data. Some services are mostly a backup, some are in day to day production use for my company, some are to toy around. Whenever one of the big providers messes up, has a data breach or is bought and shuts down people advocate for self-hosting similar services, and you might think I’d be one of the,. Sadly it is not that easy.