Remember the name

I think this is the first week in which I do not have spent any time on feature development on the coding part of LazerBunny. I was using my coding agent, router and controller every day and things just worked. A pleasant surprise to be honest. Instead of rushing into working on the rest of the stack I only implement a single feature for the brain.

I went a bit beyond what Fort Minor suggested in their song and implemented a personalisation layer. There have been a few different approaches I have seen how to do this. I decided to keep it as lightweight as possible to not burn GPU cycles while an actual request is waiting to be processed.

Once a day at 2am a goroutine runs and checks all sessions that happened during the last 24 hours, to see if there are any facts the LLM believes are worth remembering about me, past conversations or what I am working on. Once the facts are identified, the embeddings are generated and stored in the database. When a request is made the brain runs a quick check about the topic, does a query (via reciprocal rank fusion) and injects the three most relevant facts.

This is not deployed yet, so I might be in for a bad surprise, but benchmarks suggest this takes around 200 to 300ms additional time per request. Not great, but also not bad. Especially because this only happens on the first request of a chat session.

The blind spot is obviously the current day. If this turns into a problem I might run the embedding process more aggressively. But without a few more GPUs in my rack I have to be a bit smart how and when "maintenance" requests are running.

Really?

I really have to start spending some more time with Xcode to see how Apple implemented their coding assistant. With the exact same model it produces a lot more garbage that is not working and does not seem to have any tools to check if the generated code is actually passing a linter or is compiling.

I noticed this when testing a feature of my controller. I told both my agent and Xcode to generate an iOS application that can communicate with the controller and provides the same functionality as the web interface.

My agent produced an okay foundation. A bit convoluted as most LLM output is, when all it gets is a spec and no handholding. So far I did not implement any LSP, shell session or way for the agent to interact with the codebase, so all it can do is iterate over and over until it assumes it is correct.

Meanwhile Xcode could not get dispatching, variable initialisation or the difference between let and var right. I am not sure if it is the system prompt that causes the LLM to become a lot worse, but it might be a possibility. What is ridiculous is Xcode not even checking if the code is sound, nor running another iteration.

Hair?

If you ever decide to learn 3D modelling, just plan for the character to wear a hat. The most miserable time I had in the process (so far) is working on the hair. I never wanted to go realistic but highly stylised. Think Overwatch or Fortnite. So just a few large paths, how hard can it be. The short answer: very hard.

I never had to think so much about how hair flows, how it overlaps, how to add volume and where it is fine to clip into the head, and let me tell you, there’s a lot of possibilities for things to go awry.

Luckily there are some really good tutorials on YouTube. Funnily enough all of them make it a lot easier than whatever the tutorial series I am following does. The worst offender are eyelashes. Instead of a texture or modelling them the instructor literally plays with a gaussian noise slider for 40 minutes until the noise kind of looks like eyelashes.

I might not know anything, but this cannot be how you do this!?

Progress

I am mostly in the planning phase for the next part. Getting the brain right, adding a function as a service component to run random tasks and a signal or data to the brain to forward it to the text to speech component… and a little physical component I hope I will soon talk about, because this one is unbelievable stupid and I am so hyped about it.

I might spend a few minutes on the agent to add language server protocol support, which would mean my agent will by definition outperform Xcode and could be a bit smarter when generating code to at least check if it compiles. What an idea Apple, am I right? Also if you add this to Xcode in the next release I will now claim you stole my idea.