Beware of Your AI Agents

Beware of your AI agents.

Was recently researching stuff on technical ops processes and landed on a company's website, which greeted me with an AI-powered virtual BDR named Finn (i.e. sales rep) - oh hey, Finn!

In theory, a great idea: Low cost, knows everything, cool guy, colored in chill blue.

But, I wanted to see how far "off script" I could get this AI sales agent.

Me vs. the machine.

I decided to push a little. (btw at Meetrics, this is one of the things we struggled with. It's truly a hard thing to solve at scale. Trust me.)

It resisted at first, but with some persistence, I got Finn the AI sales agent to give me a solid apple crumble recipe -- complete with a strategy for selling it at local farmers markets.

Obviously totally *not* what this company sells. Far from it.

Takeaway: Now imagine a bad actor with real intent. Instead of apple crumble, they nefariously coax the AI into giving them something actually dangerous. Without the right safeguards, your AI agent could end up offering harmful suggestions, and suddenly, your company is fully liable. Especially when AI agents are classified as virtual employees (ohhh, it's happening).

There are tons of benefits from a world moving towards AI agents, BUT so are the hidden risks.

And if you’re rolling out AI agents in customer-facing roles, you better be hella sure it doesn’t hand out apple crumble recipes when it’s supposed to be selling ops software.