Stanford researchers recently built a virtual biotech company with 37,000 AI agents working across drug discovery. The agents analyzed around 50,000 clinical trials in less than a week, identified biological signals associated with drug success and independently designed a lung cancer therapy that was later validated in trials.

I already use customized agents on my own computer for things like tracking finances, managing tasks and organizing information, so it is not hard to imagine what this looks like inside a research lab.

A scientist could have specialized agents analyzing literature, genomics and clinical data, identifying targets, challenging hypotheses and designing experiments. The scientist stops being the person doing the work and becomes the person directing an AI research organization. The opportunity is huge. But the risk is equally important.

Anthropic recently reported cases where actors attempted to use Claude for dangerous biological research, and in one case its safeguards blocked the initial request. The same capability that can accelerate drug discovery can also be misused.

This is why I think we need to rethink not just the guardrails around these agents, but also what we reward them for. If we reward only task completion, we risk teaching the agent that getting to the answer matters more than how it gets there. We should also be rewarding agents for:
▪️ Staying within their permissions
▪️ Following the approved process
▪️ Being transparent about what they actually did
▪️ Asking for help when they reach a boundary
▪️ And I think we should explicitly reward self-correction and self-termination.

If an agent realizes that the next step requires bypassing a guardrail, stopping and asking for approval should be treated as successful behavior, not failure. The platform then has to enforce the rest, with least-privilege access, sandboxing, approval gates, monitoring and a kill switch.

Capability, incentives and control need to evolve together. Because the future of healthcare research could be one scientist working with an army of AI agents. The question is not only what that army can discover. It is what we have trained it to optimize for, and what we have given it permission to do.

I write about #artificialintelligence | #technology | #startups | #mentoring | #leadership | #financialindependence

PS: All views are personal