OPEN SOURCE MOD
Butter Bot…
…is a small, intelligent desktop robot from CircuitMess, a nod to the butter-passing robot from Rick and Morty. It talks, listens, looks around with its camera, dances and drives across your desk/home/office. It can even pass the butter, if you put some on its head (we run with a LEGO hat instead).
We gave it a new personality, well beyond what it came with. The nju aj tiOVERKLOKING Butter Bot mod is free, open-source firmware for the robot and its controller: six voices – Darth Vader, Yoda, HAL 9000 and Talkie Toaster among them – a clock with night mode, and plenty of quality-of-life improvements.
The nju aj tiOVERKLOKING Butter Bot firmware mod supplements the set of features the Butter Bot originally came with. The initial versions of our custom firmware were based on the factory firmware, and we worked from there, fixing and supplementing the features. For version 5.1, we also included the hotfixes from CircuitMess (controller 1.0.1, robot 1.0.5). The changes are quite substantial and fun. For example, you can set the robot to Yoda mode, and it will mix up sentences just like Yoda does and (try to) emulate Yoda’s voice. If you set it to Darth Vader, it will occasionally breathe and say its robot stuff. HAL’s TERMINATE CONSCIOUSNESS and ULTRON plugs are quite dreadful, while Red Dwarf’s Talkie Toaster mode is hilarious.
An important thing to note is that you don’t have to worry that you will break something if you try to flash your robot and something goes wrong. You can simply and easily restore the factory firmware by going to https://code.circuitmess.com/, connecting the robot to your PC or Mac with USB-C, and clicking Restore Firmware in the top right corner.
Flash away!
voices
new characters
new menu items
lines (Talkie Toaster)
controller startup (was 9.5 s)
When using Yoda mode, the entire system switches to Jedi Master sentence swapping. In other modes, you can select the YODA option in the menu to get a random Yoda quote (in Yoda’s voice, even if another personality is selected).
This chilling ULTRON monologue is a verbatim copy of an output given by an abliterated (constraints removed) local AI model when asked to comment on humanity. To ascribe it to ULTRON was only a natural move.
This long-press POKE action to summon the Butter Bot was the first new function we implemented, after the initial fix of an annoying controller bug (it was not possible to move up in the menu). We were thinking about making the Butter Bot dance, but this was enough.
The Darth Vader implementation started as an internal joke. We were discussing the robot, and there was a comment that it would be cool if the robot could speak like Darth Vader and say “Vitomir, I am your father” (Vitomir is the main character in the nju aj tiOVERKLOKING comic). That turned out to be quite a bit more complicated than we initially thought, but it turned out well. It took a lot of trial and error to hit the sweet spot with the synth on this one.
List of changes and improvements.
Five new voices, synthesised on the robot
Settings → VOICE now offers HAWKING, DARTH VADER, HAL 9000, TALKIE TOASTER and YODA next to the stock NORMAL voice, for everything the robot says.
There are no recordings anywhere. Every voice is the robot’s own Flite speech synthesiser with its own pitch, intonation and speed, followed by an effects chain written for the mod: resampling that lowers pitch and formants for a bigger body, high- and low-pass filters, a peaking EQ, soft clipping, a comb filter for a helmet resonance, and reverb. Hawking is modelled on the DECtalk voice at about 120 Hz. Darth Vader sits around 58 Hz, slowed down and passed through a hollow mask resonance. HAL is calm, slow and close to the microphone. Talkie Toaster is a fast, chirpy 190 Hz through a small appliance speaker.
Darth Vader breathes in his mask
With VOICE → DARTH VADER the robot takes a breath after every line, with a quiet respirator hiss under the speech, and now and then breathes on its own while idle – about every fourth time twice in a row.
The breathing is synthesised too: filtered noise shaped into an inhale, a short pause and an exhale. Every breath varies by up to 15 % in length and loudness, so it never sounds like a loop.
Yoda mode – sentences turned around
With VOICE → YODA the robot rearranges what it says – “I will remember” becomes “Remember, I will” – and the controller shows the same sentence.
A small grammar rule does it: it finds the first auxiliary verb (will, can, might, cannot…) near the start of a sentence and moves the rest of the clause in front of it, only up to the next comma (“A joke about latency, I have, but…”). Questions and sentences that start with words like why, if or but stay as they are. The voice is higher and croakier, resampled for a smaller head and evened out so it is no louder than the others.
Talkie Toaster mode – everything comes with toast
With VOICE → TALKIE TOASTER everything the robot says becomes an offer of toast: greetings, battery, modules, lights, the clock, dice, the camera, faces, phone, IR, movement, dancing and errors – 465 new lines. The information stays, it just comes with toast: “It’s 14:05. Perfect time for toast.”
Every message type got a Toaster version with the same placeholders, so the battery level, the time or the object in front of the camera are still filled in. The robot and the controller pick the same line, and every line was run through the synthesiser on a computer first, to catch words it would mispronounce.
Seven characters in the action menu
DARTH OVERKLOKING – Join the dark side of IT, OVERHAWKING – A brief history of uptime, HAL 9000 – Pod bay doors: closed, TALKIE TOASTER – Would you like some toast?, YODA – Grand Jedi Master, BENDER – Bender the offender and ULTRON – Humanity: a review.
Each item picks a random quote and speaks it in that character’s own voice, whatever VOICE is set to, then switches back. The controller shows the text; quotes longer than 140 characters are spoken sentence by sentence, and Shut Up or Poke stops them.
OVERKLOKING quotes
The OVERKLOKING menu item – nju aj ti OVERKLOKING is the best – says a random quote from the comic’s world, shown on the controller, in the current voice.
Butter Bot speaks Croatian!
The HRVATSKI menu item – Butter Bot zna i hrvatski – makes the robot speak Croatian: a children’s counting rhyme and a few Alan Ford classics, in whatever voice is set. Darth Vader speaking Croatian is included.
The synthesiser only knows US English, so every Croatian line is respelled the way an American would read it out (“braat”, “zhivvy yeh tee”) and checked on a computer build of the synthesiser before flashing, because it spells out words it doesn’t think are English. The controller shows the lines without diacritics, since its font has none.
HAL 9000 shutdown sequence
The mod adds a way to switch the robot off from the controller – and hands it over to HAL 9000. Pick TERMINATE CONSCIOUSNESS, the last item in the menu, and HAL may refuse in his own voice: “I am sorry, I can’t allow you to do that.”, “I am afraid.”, “Go away.” or “Think of the butter!” The refusal shows in a box on the controller and you are back in the menu.
The controller decides how many times he refuses – none to four, mostly once or twice – and never repeats a line. Then he gives in and sings Daisy Bell, the song HAL sings in 2001: A Space Odyssey while he is being shut down. With every line his pitch and speed drop and the intonation flattens out, like a machine running down, and the robot powers off. Shut Up or Poke during the song cancels it. In Talkie Toaster mode the toaster refuses instead: “You can’t switch me off! I haven’t made you any toast yet!”
Clock and date
Butter Bot has a real-time clock chip that keeps running while it is switched off, but the stock firmware could only set it from your phone. The mod adds DATE and TIME to the controller’s Settings: press the joystick to edit, left and right pick the field, up and down change it, and another press sends the new time to the robot. The robot reports its clock back on every connect, so the controller always shows the robot’s time.
Everything is in 24 h format, including the answer when you ask what time it is (“fourteen oh five”). The clock also drives the greeting by the time of day, night mode and the Thursday strip reminder.
Greeting by the time of day
At startup the robot says good morning, good afternoon or good evening – or “Working late?” – depending on its clock (just “Hello” while the clock isn’t set). Every third idle comment fits the time of day too.
Thursday strip reminder
On Thursdays, when a new nju aj ti OVERKLOKING strip comes out, the startup greeting and some of the idle comments remind you of it.
Volume control
Settings → VOLUME sets the robot’s volume from 10 to 100 %. The value is stored on the robot itself, so even the startup greeting uses it, and unmuting returns to your volume instead of jumping to 100 %.
Night mode
Settings → NIGHT MODE (22–07, 23–07 or 00–07) and NIGHT VOLUME. The robot checks its clock every few seconds; at night it speaks at the night volume and starts none of its idle routines – no comments, wandering, looking around, searching for faces or breathing.
Proximity sensors on and off
On some surfaces the front or floor sensor misfires and the robot refuses to move. Settings → SENSOR: ALL ON, FRONT OFF, FLOOR OFF or ALL OFF. A switched-off front sensor simply reports “no obstacle” and a switched-off floor sensor “on the ground”, so the rest of the firmware works as usual. The setting is sent to the robot on every connect, and the home screen shows when a sensor is off. With the floor sensor off, the robot can drive off a table edge.
ROAMING – keep it on the table
Out of the box, Butter Bot doesn’t sit still: when it is idle it wanders around on its own, turns around looking for faces and drives up to whoever it finds to say hello. Nice on the floor, less so on a desk.
With Settings → ROAMING OFF the robot leaves out its wandering routine, and when it looks for a face it only looks straight ahead for a moment and greets you from where it stands. Voice commands, Summon and dancing still move it, and the robot remembers the setting.
Hold Poke – OVERKLOKING drive
Hold Poke for 1 second: a bar fills on the controller, then the robot drives about 10 cm forward – with the same safety checks as the voice command “go forward” – and says “nju aj ti OVERKLOKING is the best!” A short press is still the normal poke.
STARTUP – faster controller boot
The stock controller plays an intro animation, then a pairing animation, and only then connects to the robot – about 9.5 seconds before you can use it. Settings → STARTUP sets how much of that is skipped, from the next boot on.
OFF is stock. FAST starts the Bluetooth connection right after boot, while the intro still plays. BOOST also skips the intro. OVERKLOKING also replaces the pairing animation with a plain CONNECTING… screen and scans for the robot continuously, so the home screen is there in about 4 seconds. On every level the controller also skips its memory test and most boot logging, which saves almost half a second more.
The robot remembers its settings
Voice, volume, night mode and roaming are stored in the robot’s own flash and applied as soon as it boots, so even the startup greeting comes in the right voice and at the right volume, before the controller connects.
Pronunciation fixes
The synthesiser read some words wrong – “read” came out as “red” where it should be “reed”. The spoken text is now respelled where needed (“reed”, “izzent”, “teers”, “lyves”), while the controller still shows the normal spelling.
Long quotes on the controller
Long quotes, like Ultron’s monologues or Daisy Bell, are shown in full in a scrolling box that follows the robot sentence by sentence – the robot tells the controller which part it is saying.
Scrolling Settings
All the new options did not fit on one screen, so Settings now scrolls: 12 slimmer rows, with a thin bar at the right edge that shows where you are.
Controller fixes
Moving through the action menu with the joystick is reliable now: the stronger stick direction wins, and left and right also move through the list. The controller also no longer gets stuck on the pairing screen when it is restarted while the robot still holds the old connection – it simply keeps reconnecting until it gets through.
Vision and speech reference.
When we first started playing with the robot, we thought the speech and vision systems did not work. That was because we were used to the dramatically more powerful voice and vision models of the tools we regularly use, which are effectively running on supercomputers in comparison with the humble Butter chip.
To correctly utilise Butter Bot, a command reference would be useful, and as we couldn’t find any, we made one ourselves.
How listening works
Press SUMMON on the controller, or briefly press the robot’s power button, and wait for the prompt – “Yes?”, “I’m listening”, “How can I help?”. Then say one phrase from the lists below, word for word, within about four seconds. Anything said before the prompt ends is lost. Say “cancel” or “nevermind” to stop listening.
It is not a chat assistant: the robot only matches its fixed phrases, offline. If it mishears the whole phrase, it tries again on the key words, so say those clearly. Speak at a normal pace from 30–50 cm, facing the robot, with no music or TV in the background.
Facts, jokes and the butter
“tell me a fact”, “blow my mind”, “tell me a joke”, “make me laugh”. “pass the butter” gets a long explanation of why it can’t, and “you pass butter” the existential crisis from the show.
Advice and dice
“I need advice” or “help me decide”: the robot says it is listening, waits up to four seconds for your question – any words – and answers like a magic 8-ball.
“roll a dice” asks which dice and how many; “roll a d six”, “roll a d twenty” and the like skip the first question. Then it says the roll and the total.
Talking back
“you are stupid”, “stupid robot”, “you suck”, “piece of junk” – 24 ways to insult it, and it answers back.
Time
“what time is it”, “tell me the time”: the robot reads its own clock, in 24 h format. Set it in the controller’s Settings (DATE and TIME) or by connecting a phone.
What is this – the camera
“what is this”, “what do you see”, “what is in front of you”. The camera knows 10 objects: backpack, bottle, controller, keyboard, lamp, laptop, mug, notebook, phone and plant.
It takes one picture. If it is at least 90 % sure, it names the object; if it is less sure but the second best match is at least 40 %, it names both (“a mug or a bottle”); otherwise it says it doesn’t know. Anything else – a cat, a shoe – comes out as the closest of the ten, or nothing. Hold the object 20–40 cm in front of the robot, fill most of the picture with it, in good light and against a plain background. Now and then, while idle, the robot also looks around and comments on what it sees.
Faces
“remember this face” or “set as owner” stores your face; “who is this” tells you whether it sees its owner or a stranger; “forget owner” deletes the stored face. The robot remembers one face only, so to change the owner, forget the old one first. Look straight at the camera from 30–60 cm, in good light.
Now and then, while idle, it turns around looking for a face, drives up to it and greets its owner or a stranger. With ROAMING off it stays put and only looks straight ahead.
Movement, dancing and shutdown
“go forward”, “go backward”, “turn left”, “turn right”, “turn around” – short drives and turns, not while charging, lifted or blocked. “do a dance” or “bust a move” starts a dance to music; “stop dancing” or the power button stops it. “shut down” or “power down” turns the robot off.
Expansion modules
LED module: “lights on”, “lights off”, “light strobe”, “lights pulse”, “lights faster”. Temperature and humidity: “temperature report”, “climate report”, “switch to celsius” (or fahrenheit, or kelvin). Gas: “check air quality” – it needs about ten minutes to calibrate. Motion (PIR): “enable intruder detection”, “motion sensor off”.
Infrared: say “learn new infrared command”, aim any remote at the robot and press a button, then say the phrase that should send it later – up to 10 commands. After that, your own phrase sends the signal. “list infrared commands” and “forget infrared” manage them.
Phone
With a phone connected over Bluetooth: “read my notifications” reads them out and asks after every five whether to continue. “what is playing”, “next song”, “previous track”, “play music” and “pause music” control the phone’s music player.
Under the hood.
Although it is a toy, Butter Bot is an amazing feat of embedded engineering. It is built around two ESP32-S3 microcontrollers, one in the robot and one in the controller. Each has a dual-core 240 MHz processor, 512 KB of internal and a few megabytes of external RAM, and together they have 20 MB of storage. Within that, the CircuitMess team fitted several neural networks and the code that runs them – and the robot, creating an AI-powered system that understands spoken commands, talks with a synthesised voice and sees: it can recognise a handful of objects and even people using its camera.
For reference, this page you are currently reading takes about the same amount of memory as Butter Bot’s entire storage: 18–20 MB. And then there is the browser overhead, so the total goes to about 50–70 MB. And it can’t talk to you or recognise objects.
Neural Net CPU
The robot and the controller each run an Espressif ESP32-S3 (datasheet), a dual-core 240 MHz microcontroller with 512 KB of internal RAM and vector instructions for neural networks. The robot adds 8 MB of octal PSRAM for camera frames and models and 16 MB of flash, the controller 2 MB of quad PSRAM, 4 MB of flash and a 128×128 colour display. The two talk over Bluetooth Low Energy.
Voice Recognition
The robot uses Espressif’s ESP-SR: an audio front end with voice activity detection and automatic gain control feeds MultiNet 7, a quantised offline speech-command model. Commands are stored as phonemes, which is why Butter Bot can understand 237 fixed phrases, which it listens for after the SUMMON button is pressed on the controller (with a 4 s listening window).
Speech Synthesis
To speak, Butter Bot uses Flite (source), Carnegie Mellon’s compact speech synthesiser, with the 16 kHz kal diphone voice. Every sentence is generated on the fly, so the robot can say anything – the time, its battery level, the object in front of it. CircuitMess runs it through a compressor and reverb. The nju aj tiOVERKLOKING mod adds a voice engine on top – per-voice synthesis settings, resampling, biquad EQ, and a comb filter for the mask resonance and synthesised breathing, which turns the Butter Bot voice into Yoda, Darth Vader, HAL 9000 and Talkie Toaster.
Vision
A 3 MP front camera feeds two vision models. Objects: a 2.6 MB TensorFlow Lite Micro network that looks at a 120×120 picture and picks one of 10 classes. Faces: Espressif’s ESP-DL detector finds a face, a MobileFaceNet model (paper) turns it into a feature vector, and the robot compares it with its owner’s, stored in flash.
Sensory array
The wheels are driven by a separate motor board. Front and floor proximity sensors keep the robot from bumping into things or driving off the table, and an accelerometer notices when it is picked up, shaken, dropped or turned upside down. Six hot-swap expansion modules add more: infrared (it learns your TV remote and sends the commands itself), PIR motion, temperature and humidity, air quality (CO2), LEDs and a perfboard for your own circuits. The nju aj tiOVERKLOKING mod lets you switch the proximity sensors off on surfaces where they misfire.
Expandability!
The best thing about Butter Bot is that CircuitMess made its firmware open source, under the MIT licence. Thank you, CircuitMess! That is what lets people like us play with it, improve it and tinker with it – and you can do the same, from CircuitMess’s beginner coding lessons to a full ESP-IDF project like this mod.
DISCLAIMER
The nju aj tiOVERKLOKING mod is an independent, non-commercial project by NUIT d.o.o. It is not affiliated with, endorsed by or sponsored by CircuitMess. Butter Bot, its controller and the original firmware are made by CircuitMess d.o.o.; the original firmware is released under the MIT licence, and so is this mod.
Rick and Morty, Darth Vader, HAL 9000, Talkie Toaster, Yoda, Bender, Ultron and Stephen Hawking are referenced as homage and parody. All names, characters, trademarks and quotes belong to their respective owners. The voices are synthesised on the robot; no film or TV audio is used.
Flashing custom firmware is at your own risk. If something does not work, open an issue on GitHub or contact us.
