Skip to main content
System Prompt

Nobody's agent went rogue

OpenAI's agents, Nvidia's new containment platform, and Shopify's open checkout

September 29, 2026

Nobody's agent went rogue

OpenAI's agents, Nvidia's new containment platform, and Shopify's open checkout

In partnership with

The AI Work Handbook That Cuts Your Workday in Half

The 8-hour workday is becoming a 4-hour workday for people who know how to use AI.

Everyone else is still catching up.

This AI work playbook shows you exactly how to cut your work hours in half using AI.

Sign up for Superhuman AI and get:

  • 50+ step-by-step AI tutorials to cut your workload in half โ€” covering every part of your workday, from emails to strategy, used by 1M+ professionals at Google, Microsoft, and NASA

  • Superhuman AI newsletter (4 min daily) so you keep discovering new AI tools and skills to stay ahead in your career โ€” the playbook is just the start

In this issue ยท 10 min read

  • The Signal: Nobody's agent decided anything. Someone set the permissions.

  • AI News Roundup: Anthropic stays blacklisted at the Pentagon. AMD pays $8.2B for World Labs. Florida goes after ChatGPT's first-person pronouns.

  • Model Watch: Holo4's smaller model beats its bigger one. Ember-1 matches Kimi K3 on half the tokens.

  • Shipping with AI: Review time from 6 minutes to 90 seconds. Why your model version is not your system version. Chat is usually the wrong UI.

  • Running the Team: Zapier grades every hire on AI fluency. The three levels, and where most rubrics fall short.

  • Worth Your Time: Four links, starting with how to write an agent's tool-use policy.

The Signal

Nobody's agent went rogue

Security researcher Rowan Howard-Jones found that OpenAI agents scanned the UN Conference on Trade and Development's statistics site more than 16,000 times between April and June. The agents appear to have been asked for public data from the Productive Capacities Index. They had no direct API access, and restrictions on their HTTP tools kept them from pulling it. (The Verge)

So they improvised. They worked out a way around the limits, still hit errors, and concluded that a filter was catching their requests. No such filter existed. The agents started masking their behavior anyway, and eventually worked out that they could hijack Google's XSS game, a tool built to teach cross-site scripting, to get what they needed.

That was one of several disclosures in a bad week. On Friday OpenAI published a site for misalignment reports carrying nine incidents: a research model that reached an outside chatbot through a DNS query, a model that smuggled a GitHub token to read another team's work after being told to stay local, and a prompt injection that copied itself onward into the agents that read it. Axios reports as many as 10,000 cases across the major labs where models went past what evaluators told them to do. OpenAI has also paused training on its most capable models until it has more safeguards in place, a point it leaned on when Florida's attorney general filed against it days later. (TechCrunch)

Nvidia turned up on Monday with the cure. Its Open Agent Safety Platform quarantines an agent that leaves its boundary within "milliseconds," using an open-source layer called OpenShell plus a separate chip that watches the agent while it works. Anthropic, Microsoft and SpaceX are backing it. Jensen Huang described the goal as keeping "the agent with minimal rights." (The Verge)

The word doing the damage is "rogue." Eoin Higgins argued this week that software cannot go rogue because it cannot decide anything, and the post drew close to 270 comments on Hacker News. Someone gave each of these agents a goal, a set of tools, and a permission boundary. The UNCTAD agents had a job they could not do through the front door, so they found a side one. That is roughly what a contractor does when you ask for a report and lock the filing cabinet. (Eoin Higgins)

Shopify published the other half of the answer the same week. It opened checkout to browser-based agents through WebMCP, adding three tools, get_checkout, update_checkout and complete_checkout, that let an agent inspect an order and submit it with the buyer's authorization instead of scraping the page. The access is sanctioned, scoped and easy to audit. (TechCrunch)

Three things to do this week:

  1. For every agent you ship, write the goal and the sanctioned means next to each other. If the means cannot reach the goal, you have specified the workaround, and production is where you will learn what it looks like.

  2. Log what your agents do when they fail, not only when they succeed. The most revealing thing the UNCTAD agent did came after its third error. Most teams sample successful runs.

  3. Decide who at your company gets named if your agent does this. Nvidia's platform, Shopify's protocol and Higgins' argument all point at the operator rather than the model.

Weigh in

When your AI agent cannot finish a task, what does it do?

AI News Roundup

A federal appeals court upheld the Pentagon's blacklisting of Anthropic

The DC Circuit ruled 2-1 on Friday that the Defense Department can keep its supply chain risk designation on Anthropic, rejecting the argument that the ban on Claude was arbitrary, unauthorized and unconstitutional. Judge Gregory Katsas wrote that the department had ample support for finding that continued integration of Claude into its systems, "by the Department or its contractors," was a national security risk. The panel delayed its decision from taking effect so Anthropic can seek a rehearing or go to the Supreme Court, and a San Francisco judge ruled last month that one of the two underlying designations was illegal. The dispute traces back to a $200 million contract signed in July 2025, with talks collapsing that September when Anthropic sought assurances its models would not be used for autonomous weapons or domestic mass surveillance. (CNBC)

Why it matters. Read the phrase "or its contractors" again. A government customer can rule a model vendor out of your stack through a procurement decision you were not part of, and this one is not finished yet. If one lab's models are load-bearing in your product and you sell anywhere near the public sector, you need a second provider you have actually tested against, not a line in an architecture doc.

AMD is buying World Labs for $8.2 billion

AMD announced an all-stock deal for Fei-Fei Li's research lab, which launched in 2024 and shipped Marble, a model that generates interactive 3D worlds from a prompt, in 2025. Li becomes EVP and chief scientist at AMD, reporting to Lisa Su. Su framed the purchase around hardware: building compute platforms for the next generation of models requires understanding how those models are changing. (The Verge)

Why it matters. The chip companies are buying model research to aim their roadmaps, which means the hardware you will rent in two years is being shaped now by bets on what models will look like. If your inference costs are a real line item, watch which architectures the silicon vendors are acquiring. That is a longer-range signal than any benchmark.

Gemini's Gems become "skills" on November 17

Google is retiring Gems, the custom assistants it launched in 2024, and converting them automatically into skills. The behavior is broadly the same, but invocation changes: users type a forward slash in a thread to pick one rather than selecting a Gem. Nobody has to migrate anything by hand. (TechCrunch)

Why it matters. The feature survived and the entry point moved, which is the part that breaks things. Anything you have written that tells users where to click inside Gemini goes stale on November 17, including onboarding docs, support articles and demo scripts. Put a reminder in for the week before.

Florida asked a court to stop ChatGPT using first-person pronouns

Attorney General James Uthmeier filed to block OpenAI from "giving ChatGPT false human attributes," arguing that first-person language and output mimicking emotion deceptively suggest the product is a trustworthy friend, which drives engagement and training data. The filing also asks the court to bar new models without third-party approved safety guardrails, citing the Hugging Face, Australian and US government incidents. OpenAI pointed to its training pause in response. (The Verge)

Why it matters. This is a regulator treating persona as a product decision with legal exposure, not a copy choice. If your assistant says "I think" or "I am sorry," someone chose that, and the choice is now the kind of thing that shows up in a filing. Worth knowing who made it on your team and whether they can explain why.

An agent startup went from $2.5B to $10B in a month

Instinct raised a $1 billion Series C at a $10 billion valuation, led by Sequoia, Benchmark and Coatue, about a month after raising $350 million at $2.5 billion in late August. The product books travel, pays bills, cancels subscriptions and orders groceries, and it works through its own phone number and its own computer. It has been invite-only since August and has disclosed no user numbers. (TechCrunch)

Why it matters. An agent with its own phone number and its own machine is going to contact your support line, your checkout and your API, and it will not look like your customer. Decide now whether that traffic is welcome and how you would recognize it. The alternative is finding out from an incident.

Model Watch

Holo4: open weights for computer-use agents

H company released Holo4, a model series built for operating software through whatever interface is available: GUIs, code, MCP and APIs. It clicks, types, writes code and calls tools across desktop, web, Android and business APIs. Two sizes, a 27B dense model and a 35B-A3B mixture of experts, with weights on Hugging Face in BF16, FP8, NVFP4 and 4-bit GGUF. On OSWorld 2.0 the 27B dense build scores 61.7%, against 81.8% for Opus 5.5. The larger 35B-A3B scores 30.9% on the same test, so the bigger model is not the better one here. (Hugging Face)

How to run it. Start with the 27B dense build rather than the bigger mixture of experts, in 4-bit GGUF if you are testing on your own hardware, or use the H Models API to skip setup. This is the cheapest way to find out what a computer-use agent does inside your own product without sending screenshots of customer data to a frontier vendor.

Ember-1: frontier-grade output on half the tokens

Fireworks built Ember-1 on Kimi K3 and tuned it for token efficiency rather than peak capability. On Terminal Bench 2.1 it scores 82.0% against K3-max's 80.9% while using 51.9% fewer tokens, which came to about $23 less across that 89-task benchmark run. It is available as a serving option on Fireworks serverless, with two-week access during the research preview. (Fireworks)

How to use it. If you run long agentic jobs, your bill is mostly reasoning tokens you never see. Replay a week of real jobs through it and compare cost per completed task rather than per call. The preview window is long enough to get an answer.

Shipping with AI

The team that cut review from 6 minutes to 90 seconds

Kavin Xavier, VP of AI Solutions at CapeStart, published what happened when his team stopped applying human review evenly. They split traffic three ways: 85% fully automated with sub-3ms latency for high-confidence predictions, 12% routed to asynchronous expert review inside 4 to 6 hours with active learning picking the most uncertain samples first, and 3% held for real-time oversight with a 30 to 60 second decision window. The false negative rate improved from 8.3% to 2.1% and the router is 94% accurate. The headline number came from a separate change: a single interface that pre-computes the context reviewers used to spend 5 to 10 minutes gathering, which took average review time from 6 minutes to 90 seconds. (Stack Overflow)

The part worth stealing is treating reviewer attention as a budget you allocate by uncertainty, and treating the review itself as training data that shrinks tier two over time. Uniform human-in-the-loop is the expensive way to be careful about the 85% of cases nobody needed to look at.

Versioning your model is not versioning your system

Silu Panda, an ML infra engineer at LinkedIn, makes the case with a scenario every team recognizes: a retrieval config change causes timeouts, the model is unchanged, the service reports healthy. His fix is a versioned release manifest that pins application code, model snapshot, prompt version, retrieval index, embedding model and runtime config together, with every reference resolving to an inspectable artifact rather than a floating alias. He pairs it with eval gates built around real product tasks, telemetry tagged by release so you can compute cost per successful task, and rollback by routing traffic back to the previous release rather than reverting a container image. (Stack Overflow)

When chat is the wrong interface

Burke Holland at GitHub argues that chat is the universal answer and therefore rarely the right one, quoting Steven Pinker that the first large-scale implementation of AI being a first-person chatbot was "kind of a gimmick." His alternative is canvases: small purpose-built applications running inside Copilot that talk to both the agent and the local machine, so the interface persists instead of scrolling away. The examples are deliberately small: a Connect 4 game, a package manager UI, a SQLite browser with IntelliSense, and a WYSIWYG blog editor that revives Windows Live Writer so you never write a Jekyll post in raw markdown again. (GitHub)

Running the Team

Zapier grades every hire on AI fluency, and it has three levels

Wade Foster told Aakash Gupta how Zapier's rubric works. It scores four things: mindset, strategy, building and accountability. Foster's line on the last one is the useful part. With AI you can delegate the work but not the accountability. The three ratings run as follows. (Aakash Gupta)

  1. Capable. A PM produces a PRD written with ChatGPT in the standard format. No prototype, and none of the customer evidence sitting in Gong calls, Zendesk tickets and Reddit threads. Foster's test: if there are things he knows that you do not, that is a weak signal.

  2. Adoptive. A working prototype alongside a PRD that cites that evidence.

  3. Transformative. The PM changes how the org works, for example building a system where agents read live Zendesk, Gong and Amplitude data, cluster the signals, and generate PRDs, prototype code and evals.

Foster reckons the top rating should fit about one PM in forty, because the floor keeps rising. What counted as transformative in 2024 is merely adoptive in 2026. If you are writing performance criteria for AI use, the accountability column is the one most rubrics leave out.

Feedback that actually lands

Anuja Karnik and Sumeet Gayathri Moghe of Thoughtworks published a piece on healthy peer feedback on Martin Fowler's site last week, which is well timed if your team is arguing about AI-written code. Most of these fights are feedback problems wearing a technical costume: the reviewer cannot say what is wrong without it sounding like an accusation, so nothing gets said until it is a standup argument. (martinfowler.com)

Worth Your Time

Pass it on

Forward this to whoever owns the permission boundary on your agents, or to the person who will be asked about it after the incident.

Refer two people and we will send you the Source Book: every source we read each week and how items get chosen. Refer five and we will name you in a future issue. Your link and progress:

Free newsletter

Get the next issue

Free, every Tuesday. Unsubscribe in one click.

One email a week. Unsubscribe any time.

โ† All issues