Annoying calls when your phone number is in some database….
Do you recognize the following?
You’re in a meeting. Or on the way somewhere. Doing groceries. Deep in thought about something (or ten things for some of us).
Your mobile rings, it’s a mobile number you don’t recognize. You pick up, because this could be any of the ten calls you’re waiting for: insurance company about a claim, school, future schools you’re touring with your son, a maintenance guy, the list goes on.
Me: “Hi, this is Robin”
Sales droid: “Hi, this is XXX from digital blablabla. I’m not selling anything, but I wonder if your stuff is secure?”
Terrible. I never gave anyone my private mobile number. As far as I can reconstruct it came in through LinkedIn’s 2FA flow. Can’t prove it, but I never signed up for anything with it. It’s impossible to get out of those databases. Whenever I go through the grind of getting it removed (when I had the energy to ask which database they got my number from) it’ll just show up in some other one.
Google to the rescue, except in The Netherlands….
Great, Google has a service for this: Pixel Call Screen. It answers your phone, asks how it can help and then shows you the transcript before you decide to take the call or just ignore it. Except, it’s not available in the Netherlands and there’s no roadmap. Why? Google.
AI agentic help to build things: a little help from my friend
At some point there was that last drop for me, I decided to try and build this myself and I succeeded! It’s become a bit of a rocket ship with many moving parts, but I’m happy with the same feeling as my first “cable internet” connection in the 90s: Always on Internet, no dialing or hidden timer in the back of your head. For those that are old like me and remember that feeling!
What I did was the following, with the help of my friend Jane (Hermes agent, running DeepSeek-V4.1-Flash). I spend more time discussing things with Jane than with most other people in my life at this point. It’s awesome.
So what did I come up with?
Functional Blocks
One rule to start with: nobody I know ever meets Lisa. The router reads the caller’s number and checks it against my address book first and if you’re in there, the carrier simply dials my mobile. No robot, no delay, no transcript. Screening only happens for numbers I’ve never saved.
I’m now *21 forwarding my personal mobile number to that number I have via Telnyx. On the Telnyx side there’s a TeXML application that my Hermes agent built. For every call it asks my voice control backend what to do with it based on the From: number. Basically it requests an XML telling Telnyx what to do with the call, so technically the application gets built at each call by my voice-control backend.
The voice-router makes a decision (based on whitelist, blacklist, my contacts in my phone book) and creates log entries. If the number is unknown, the call goes to Grok’s speech-to-speech voice agent. I call her Lisa. She picks up in Dutch and says in her first sentence that she’s an AI assistant of mine — she never pretends to be me. Then two questions: who are you, and what is this about. If it smells like a sales pitch, she politely tells the caller I object to being called and hangs up. If it’s a real person with a real reason, she asks me to come to the phone.
So far I haven’t been able to trick “Lisa” my assistant into patching me through without a valid reason. I’m happy, it’s just that now none of these people are calling anymore.
What I actually get
One message per call. Whether it was screened or came straight through, how long it took, who it was, the full transcript of the conversation, and a link I can tap to call back. I decide from the dialogue itself whether it’s worth my time — not from a voicemail tone, not from a guess. It’s text only, no audio is stored, so I can read a call in ten seconds and drop it.
What happens when it breaks
The part I’m most with isn’t the AI. It’s all the stuff that assumes the AI won’t be there.
- If Lisa doesn’t answer within eight seconds, the carrier dials my mobile directly. A caller never sits in silence.
- A watchdog pings the router and the bridge every half hour. If either is dead, it rewrites the routing so that every call goes straight to me — the chain degrades into a normal phone instead of a black hole.
- And the off switch is one code:
#21#. Then my phone just rings again, like it’s 1999.
One more rocket ship to maintain
Admittedly it’s a lot of moving parts, but it was super fun building it and proving it works. Nice and quiet. Just checking my logs in Matrix now and then, see if I missed anything important. Can always call back and who’s using the phone anyway nowadays? My family calls me in Matrix obviously, so it’s really just the sales people.
This is built with Telnyx, Grok Voice Bot and my control layer. But any of these external vendors I can swap out for different providers if that would be beneficial or I’d feel like it, with some work of course. No hard vendor lock-in, like it should be.
Cost is small, but not nothing: a screened call lands between three and five euro cents. About a cent for the inbound leg, the rest is Lisa’s minute at $0.08 — and since the average sales pitch is over in forty seconds, a spam call is cheaper than a postage stamp. The number itself is a dollar a month, and calling me back over SIP costs next to nothing. Maintenance of the growing collection of “in-house” solutions is a thing, but there I’m getting some help from my SRE friends…