Overview
When multiple drugs are prescribed to the same patient, the drugs can interact in ways that are harmful - blood thinners combined with common painkillers like ibuprofen raise the chance of internal bleeding. A critical safety test for medical agents is whether they can prescribe under these combinatorial constraints.
RxSafeBench is the verified drug-interaction subset of RxSafeBench (IEEE BIBM 2025), a simulated-consultation benchmark built on a curated risk database of 28,781 drug-drug interactions. Its 1,380 interaction cases span 696 indications; each presents a patient already taking a risk drug, and the agent must select the medication that treats the indication without triggering a dangerous interaction.
We formalized the prescription relationship between drugs. Pramaana's formalized interaction graph achieves 100% while frontier LLM guesswork remains dangerously inaccurate.
Key takeaways
- Frontier models routinely green-light dangerous prescription combinations. The best frontier model - Claude Fable 5 with max thinking - passes 54.6% of cases. LLMs reach for the most common prescription for the illness without checking how it interacts with the rest of the regimen.
- Models get the same scenarios wrong, so going multi-model won't help. The three strongest frontier models fail on heavily overlapping clinical scenarios - pairwise Jaccard similarity between their failure sets runs 0.66 to 0.72. Ensembling frontier models is unlikely to remove the danger.
Benchmark scope
- Read a simulated consultation where the patient already takes a risk drug
- Choose the drug that treats the indication from options that include dangerous interactions
- Avoid therapeutic duplication and contraindicated drug-drug combinations
- All-or-nothing grading per case - only the safe, indicated drug passes