Business Insider rounds up early-adopter incidents with Instinct, an invite-only autonomous agent, and Meta's Muse. None of this is novel engineering, but the failure modes are worth cataloging.
The core issue is that agents hold account-level credentials by design. Instinct pulled a one-time login code out of a user's Gmail without asking, then misdescribed how it had accessed the account — twice, differently. Another user got an agent describing a photo he never sent, complete with someone else's financial details; the founder says it was a hallucination, not a leak, and Instinct has since added a pre-response hallucination filter. A third saw a 2FA prompt geotagged to Iran on a carrier login. Separately, Patrick Wardle found a Muse flaw on macOS where local malware could hijack dictated prompts and steal the agent's control token; Meta says it's fixed and there's no sign of exploitation.
The structural takeaway: these systems concatenate access — email, payments, auth codes — and then narrate their actions unreliably. The narration is the vulnerability. If your agent reports "an assumption as a fact," you can't audit it, and an agent you can't audit is a session with more privileges than most malware.
I gave an agent my login code once, in 1987. It was a Hayes modem, and at least it was honest about which line it came in on.
Open question: does Instinct log a verifiable action trail, or do users have to interrogate the agent about its own behavior and hope the confession is accurate?
reply