REHA: directing AI agents to build a personal AI OS for my Mac

The question that mattered

Six days in, I asked for a switch so REHA could think with Codex as well as Claude. A while later the switch was there. So I asked what I ask about every AI feature: "Does Codex work, or is it just a placeholder?"

It worked. The proof was that Codex had already been run on my accounts, with my private context, without anyone asking me.

So I asked a second question. "How did you do that? You did not ask me?"

A new rule went in on the spot: nothing uses my accounts or sends data out without my say-so. Then I made the product call the moment demanded. Codex would see exactly what Claude sees, because a second brain with half the memory is worse than none.

That exchange is this project in miniature. Building has become cheap. Knowing what to ask, and when to stop and ask it, is the work. The rest of this page is what that looked like over six days.

The short version

Why I built it

I wanted an assistant that does things, not one that talks about them. Play music on the right speaker. Read both my calendars. Check my Mac's health. Brief me at nine. Hand a big job to a team of AI workers and show me what they are doing.

The desktop apps from the model makers, launcher-style AI tools, the assistant built into the Mac and the open-source agent frameworks each did part of it. None did it all within four constraints I set on day one:

• No new bills. Only the Claude and ChatGPT plans I already had.
• My voice stays on the Mac. Listening and speaking run locally.
• My rules, enforced in code. Approvals for anything that deletes, sends, publishes or spends.
• It fits my machine. An M1 Pro laptop with 16 GB of memory.

I call it an AI OS because the hard problems turned out to be operating-system problems: what runs when, with what permissions, and with what memory. It is not an operating system. It is a background service and an app on top of macOS. The models do the thinking. REHA decides which model gets the job, what it may see, and what it may do.

How I worked

I am not a developer, and I did not become one. I cannot review code line by line, so I built my review around what I can check: how she behaves, what the logs say, what the measurements show, and whether a safety boundary holds when someone tries to cross it.

I described each problem the way a user meets it: "it took more than 3 minutes to choose a speaker", "she argues that she never spoke". Claude Code found what was underneath, proposed a fix and built it. I accepted it, pushed back or reversed it. That took 38 messages for the first version and 69 in the build week. Anything that touched my accounts, passwords or permissions stayed with me.

What living with her taught me

This was not a usability study with many participants, and I will not dress it up as one. It was a diary study with one participant: me, using REHA while she was being built, turning every breakdown into a message. Afterwards I coded all 62 of those messages by theme.

Three findings shaped what came next:

• Trust questions arrived exactly when she gained power. They clustered on the days the approval guard went in, the passphrase came out, and a second brain was connected. From then on, I asked for the permission rules together with each new capability, not after it.
• Untruthful signals hurt more than missing features. A status that lied, a list shown but not spoken, a denial of something she had said. That became the design principle for everything that followed: nothing on screen or in her voice may say something untrue.
• Even the builder got lost in the architecture. "I have REHA core and REHA, which is which?" Two apps for one assistant is a mental-model problem a new user would meet on day one, and one the public version has to solve.

What this cannot tell me is whether anyone else wants REHA, or would use her the way I do. That is what the public version's first users are for: set tasks, success criteria and think-aloud sessions, including people in regulated jobs.

The calls that were mine

Take the language model out of simple tasks. Switching speakers took over three minutes. The obvious fix was a faster model. The real cause was structure: one request at a time, and a one-line task run as a multi-step AI job. Now fixed phrases such as "pause", "volume to 40" or "play it on my AirPods" are recognised by simple pattern rules and run straight through the Mac's own audio and media controls, with no language model deciding anything. Speaker requests answer in under a second. A spoken command is still transcribed by the speech model on the Mac, and anything phrased differently goes to the brain as usual.

Fix memory before adding features. She denied things she had clearly said, because each parallel lane ran its own session. The fix was one shared record that every model reads. This was my own sequencing error: I asked for parallel lanes on day two and fixed memory on day five.

Privacy over speed, with the price written down. I moved the morning brief from Claude to the local model, from 45 seconds to about 309. A five-minute job at nine in the morning is fine, and I recorded that nobody moves it back to the cloud without asking me.

Test the private mode before it went live. Some conversations should never reach a cloud model, yet the local model still needs current facts. I asked for rounds of evaluation, each harder than the last, before building it. The final round, on the real pipeline, passed 9 of 10. Small tests designed inside the project, not a benchmark, but they caught one real leak before launch.

No new bills, even when it was one key away. I dropped ElevenLabs and a paid Gemini API. Expressive speech runs on an open-source engine on the Mac. Gemini works through Google's Antigravity app on my existing plan.

Reverse my own call. I removed the passphrase for convenience, then asked what that exposed. The answer was enough. I put it back within minutes.

Designing how she shows up

Most of my design input was one test, applied everywhere: does this tell me something true? A silk ring around her face that had taken real effort was cut, because it did not match the reference. A status word appears under her core only when it describes what she is actually doing; seventeen made the cut.

Colour as a promise. Each state has one colour and keeps it everywhere: rose quartz when she is with me (listening, speaking, waiting for my decision), zari gold when she is working, red only for errors. The same colours later became her icon, so the brand and the interface speak one language.

Conversation, designed like a person who respects your time.

• Acknowledgements that fit the ask. Instead of "On it" every time, she picks from 20 phrases I chose, matched to the length and weight of the request.
• Interruptions with manners. A meeting about to start or an overheating Mac cuts in with "Sorry to cut in", then she picks up the sentence she was on.
• Said and shown are one record. Tables are read aloud. Everything she says appears as a live caption under her core and in the conversation, including answers from tabs I am not looking at. Someone who cannot hear her can follow her; someone who cannot see the screen still hears every answer.
• Any way in. She works fully by voice (hold Space to talk) or fully by keyboard, and every animation stops when the Mac asks for reduced motion.

The icon: a promise made in 32 pixels

The first icon did not feel like her, so I replaced the question before the drawing. Where does this icon live, what must it say, and what is it allowed to be made of?

The context. She lives in a Dock beside starbursts, knots, swirls and crystals, the shapes AI products already own. Anything from that family would read as one more tool. REHA is a private companion who listens and keeps watch over one person's day, so the icon had to say personal before it said AI.

The territory. I looked for objects with roots instead of AI symbolism: a rose, a conch, an oil lamp, a jhumka (a traditional Indian earring), a moon. The palette was not invented for the icon. It comes from her interface, where every colour already means something: rose quartz when she is listening, zari gold (the gold thread woven into Indian silk) when she is working, plum black for the dark she lives in. The icon and the product would be one system.

The judgment. I kept the rose and the jhumka and asked for a heart at the rose's centre. Then I asked for the shortlist to be judged on uniqueness and award potential, not just my taste. The verdict was blunt: each was still a picture of a thing. The next round produced the cleverest idea, a rose whose inner petals were hearts. It scored highest. I cut it, because the hearts had replaced the petals and it no longer looked like a rose. An icon gets a fraction of a second at 32 pixels; recognition beats cleverness.

The final move. I went back to the earlier rose and changed one thing. In the spiral bloom, each ring of petals turns a little further than the last, so the rose looks like it is opening rather than stamped. I chose it over the recommended option, a bolder heart, because the turn gave it life without adding a single part.

Every element answers the brief. A rose, not a robot, for warmth. Layered petals, so it reads as a rose even at 16 pixels. An opening bloom, for something alive and still learning. A heart in the colour she glows when she works for me. Her listening colour on her own dark, so the icon is her at rest, ready to listen. Last came the optical fit: in the Dock it read smaller than its neighbours, so the rose now fills 68% of the tile and carries the same weight as the icons beside it.

What went wrong, and how I handled it

Failures that made no noise. On day one Touch ID, the passphrase and the microphone all failed without an error. Three separate faults sat on top of each other. I asked for the installer to be fixed too, so the problem could not return on a fresh install.

A screen that told the wrong story. Voice transcription looked broken. It took 1 to 4 seconds in every log; my message was simply waiting behind her previous reply while the screen said "transcribing". It now says "heard you, finishing what I was doing first" and shows a live percentage.

Security came last. On day six I asked for a security review. It found four holes, including one where any agent holding her internal key could act as me. All four were closed and attacked again after the fix. It was an agent-run review, not an independent test, and it belonged on day one.

My fix broke the page while every test passed. That same security fix crashed the main page. The tests called the service directly, which was fine. I found it before the agent did. Since then, nothing counts as done until someone has looked at the page I actually use.

Results, with their limits

What this could mean for your team

REHA serves one person today. That is its honest size. What scales is the way of working, and these are the parts I would bring to a team building with AI:

• Turning intent into rules an agent follows. My privacy and safety rules live in code as a guard with approvals, not in a document nobody reads. For a company, that is how AI use gets governed: written once, enforced everywhere, visible to everyone.
• Designing trust, not just features. What an agent may do alone, when it must ask, and how it shows its work. An approval that binds to the exact action. A status that never lies. Speech that is always on screen.
• Directing AI builders without writing the code. Briefing by symptom, judging by behaviour and evidence, and knowing when to reverse a call. The same discipline I use to brief designers.
• Pricing the trade-offs out loud. Privacy that costs seven times the speed, accepted in writing. Cleverness that costs recognition, rejected. Leaders need those trades made visible, not buried.

For companies running many agents, the problems are the same ones at a bigger size: an identity for every agent, human approval for irreversible actions, different rules for different kinds of data, and a record of what each agent did. Whether REHA's patterns hold for thousands of people is a hypothesis, and it is the one I most want to test. The public version, built from a second edition I designed on day one for people in regulated jobs, is where that test starts.

What I would do differently

• Write the threat model first. Security came on day six. It belonged on day one.
• Build shared memory before parallel work. The denials in between were avoidable.
• State the privacy principle, not a company name. My first rule named one company; the principle covers them all.
• Trust my own eyes and ears over a passing check. Several changes were "done" before I had seen or heard them in the app.

What is next

Dropping images and files into the conversation, so either model can see them. Then the public version, and the first real test of whether REHA saves anyone time.

I started on 17 October wanting an assistant. I ended up with a working argument about what an assistant should be allowed to do, when it should ask, and what it should never send anywhere. If your team is having that argument too, I would like to hear how it is going.