Lazerbunny

I am starting a new series of articles for a project I have been working on for a bit, but only recently have been getting more serious about. I am perpetually a little bit annoyed by the state of software - projects constantly changing, being abandoned or adding features that make no sense for my use case - so I started writing small tools for myself which I use on a daily basis. And it has not only been fun, but also useful. For the rest of the year I will focus on a project I have been thinking about for a few years: Building a useful, personal AI assistant.

My primary objective is having fun while working on this. But as this is the Internet consider this a disclaimer that this post as well will be partly for my amusement. While the project and its goals are serious business, I will enjoy the ride and do stupid things on the way. Do not take them too seriously or judge them as if this was a professional project, because I surely will not.

I know there are already projects trying to fit into this niche, but I am not too excited about them and I do not plan changing workflows that work exceptionally well for me - like taking notes via email a separate email account. But this also means many off the shelf tools will not work for me.

I also find most tools lacking when it comes to doing research while I am walking Triss. If I ask "how to do X" I do not just want a summary of available software. I want a comparison. I want pros and cons listed out. I want follow up questions I would ask answered automatically. And most importantly I want to see the sources for all these information.

Here is a very high level view of what I am planning to do:

1: Build several yet unbuilt services for day to day use. Even the more simple things like setting a timer. Sure, I could do this on my phone. But if I am in the living room and the phone is ringing in my office I have no way to turn the timer off. (Except for walking up the stairs. Why would a small company like Apple let me manage timers across all my devices?)

2: Setup a local LLM to be able to interact with all those services. So likely an orchestration agent, some subagents to delegate to, some small, specialised models and lots of RAG.

3: Add interfaces: Delta Chat, a small web interface and - I cannot stress enough that this is the one I am excited about - a fully 3D character modelled in Blender, rigged and running in a Unity scene with their own, little world.

3D?!

Now some of you might raise an eyebrow at the 3D character. Let me try to explain my thought process here. I never did a significant amount of graphic programming and never did any creative work. It is a nice way to stretch my legs a bit and learn something new. Also Cortana is so much cooler than HAL. there I said it, come at me This will likely take the most time and will be done in very small chunks as I learn new things. Meanwhile there are asset stores (thankfully) and I can just buy a fully rigged model for a few Euros.

If you still think this is a stupid idea I am fairly certain you are just jealous that in 12 to 24 months of weekends "very well spent" my AI assistant will be cooler than yours.

Beside a completely new and foreign field there will an opportunity for some other things I did not have a chance to play with yet. Generating voices for text to speech, having wake words for speech to text, building out a form of memory that can go back to previous conversations and use it as base context without manually selecting the correct one from a list and so on.

At the same time I will over engineer the apps and infrastructure a bit, including (most likely) building out a small event bus and other components no one should ever build themselves. What I learned over the years mentoring junior engineers is that nothing beats observing and understanding the thought process behind decisions. But finding resources for "how do I build out a distributed system of services" including tradeoffs, design considerations and so on are hard to find and rare. So I hope to build out some nice resources over time to make my life in future a bit simpler, by being able to point at existing reading resources and code.

Lazerbunny?!

Naming things is hard. We can all agree on that, right? Beside, it is finally a chance to answer the age old question: What if the bunny in Monty Pythons classic movie would have been a cyborg sent from the future and could shoot lasers from its eyes? If I am not the one working on asserting this heavily debated topic who else is?

Also it was a name I could get a domain for, my wife had fun creating a small logo and I am fairly certain it will be easy to find if someone, for some reason, would ever Google it.

Now what?

I just spent a week learning Unity and thinking about the basic UI. But this will have to wait for a few weeks as I will start with a proper plan what will be built for V1, how the components will interact and the focus will likely be on building out two or three services, the LLM piece and the Delta Chat integration.

While the most exciting part will be the 3D work and having Endirillia - that will be the name of my assistant - running on a screen or TV in my office when I walk in, there is not much point building the best frontend possible while there is no functionality... even though it's the more fun part.