Doctors get free AI. Pharma gets something too.

This week’s post is a bit of a departure from AI drug discovery, and more about AI in drug sales, I suppose.

I came across an interesting Reuters story from China this week, where the head of a medical AI startup was describing how pharmaceutical companies used to reach doctors through gifts, or those “thank you for your time” lunches at nice restaurants.

Today, he argued, they could offer something much more useful, like an AI assistant to help with clinical work, perhaps even for free. It’s hard to say no to something like that. The doctor gets a useful tool, and the pharma company still gets a way to reach them. Sounds pretty benign, but I think there’s more to it.

China has been tightening rules around drug reps visiting hospitals, and Reuters reports that companies are exploring medical AI as a new marketing channel. Alibaba Health, for example, is considering allowing sponsored clinical research to appear alongside answers to doctors’ questions, kind of like sponsored results on Google.

I don’t see anything inherently wrong with promoting legitimate research, especially if advertising helps keep these tools free. But it’s a bit of a grey area when the company collecting the advertising money also operates the chatbots doctors are relying on.

In the US, we’ve spent years trying to make financial relationships between doctors and pharmaceutical companies more visible. There’s even a government database called Open Payments where you can look up certain payments to individual physicians, including sponsored meals. Being listed doesn’t mean a doctor did anything wrong, but at least we know who paid for lunch.

Now, when a drugmaker pays an independent AI platform for advertising, that payment generally wouldn’t appear as money received by the doctors using it. So we might know who bought a physician lunch, but not who’s paying for the AI assistant helping them decide what to prescribe.

Which brings me to another study published this week. Researchers at Mount Sinai tested 12 AI models using identical clinical cases, sometimes placing a drug advertisement before the question. Without the ad, the models selected the advertised drug 35% of the time. With it, that increased to 47.6%.

These weren’t actual doctors writing prescriptions, and an ad placed in a model’s input isn’t necessarily the same as one displayed beside an answer. But what is interesting is that the recommendations often remained medically reasonable. The AI wasn’t necessarily giving bad advice. It was simply becoming more likely to choose one acceptable drug over another.

Imagine seeing your doctor when two medications could reasonably treat your condition. One costs $30 a month, the other several hundred. Your doctor checks an AI assistant, which recommends the more expensive one with a perfectly reasonable explanation. The medicine works, and your doctor hasn’t received a cent from its manufacturer. You probably wouldn’t think twice about it.

But imagine discovering later that the drugmaker had paid for material that influenced the AI’s recommendation. I suspect you would feel rather differently, even if the prescription was medically appropriate. And across millions of consultations, changing even a small percentage of prescriptions toward one manufacturer’s drug could be worth a fortune.

The idea of paying a software company to influence doctors isn’t entirely new. In 2020, electronic health records company Practice Fusion agreed to pay $145 million to resolve investigations after admitting it received kickbacks from an opioid manufacturer. The manufacturer had paid nearly $1 million to build prescribing alerts into doctors’ software, and its marketing team even helped design them.

That was an illegal scheme, quite different from ordinary advertising. But the interesting part is that the money went to the software provider rather than the physician.

Of course, US anti-kickback laws can also apply to payments made to technology vendors, so paying the AI company instead of the doctor doesn’t automatically make it legal. What I suspect gets harder is the grey area where a manufacturer pays to promote legitimate clinical studies, and the AI gradually favors that company’s drug without anyone explicitly instructing it to do so.

A regulator could examine the recommendation and find nothing clinically wrong. The doctor may not even know advertising played a role.

I can see why this business model is attractive, and why doctors would appreciate better tools without another subscription. But it seems strange that we’ve spent decades making financial relationships between doctors and pharma companies more transparent, only to potentially introduce another party into the consultation whose commercial relationships may be much harder to see.

Perhaps we’re going to have to reconsider what a conflict of interest looks like when the doctor isn’t the one receiving the money.

In the US, there's a government database called Open Payments where anyone can look up payments from pharma companies to doctors, even something as small as buying them lunch. But when that money goes to the company providing the doctor's AI assistant, it wouldn't normally show up, even though the software might influence what gets prescribed.

Something similar actually happened in 2020, when Practice Fusion, an electronic health records company, agreed to pay $145 million after admitting an opioid manufacturer had paid it nearly $1 million to influence prescribing alerts.

That was illegal, but what about a company simply paying to promote legitimate clinical research? The AI might favor its drug, the recommendation could still be medically reasonable, and I'm not sure how anyone would know whether advertising played a role.

Google DeepMind released AlphaGenome 2, a new AI model that predicts how genetic mutations affect gene activity in specific cell types. Trained on more than 100 million individual cell profiles across 68 tissues, the model can help researchers identify disease-related mutations that might otherwise be overlooked. It identified nearly twice as many gene-expression-associated variants as its predecessor on one benchmark and is available for noncommercial research through an API. [Link]

Generate Biomedicines used Anthropic's Claude to redesign a key component of CAR-T cancer therapies, making it 70% shorter. The redesigned signaling domain produced roughly ten times more receptors on T-cell surfaces and improved tumor-cell killing in laboratory tests. The smaller design could leave room for additional functions, potentially helping CAR-T therapies tackle solid tumors, where they've had limited success so far. [Link]

University of Cambridge researchers developed an AI-guided bird flu vaccine that protected 100% of the mice and ferrets tested against multiple virus variants. By comparison, an existing vaccine stockpiled for potential human outbreaks protected only about 50% of animals under the same experimental conditions. The vaccine is designed to provide broader protection against evolving strains, although it has not yet been tested for efficacy in humans. [Link]