Randomization is a key tool to learn about the world, but it makes people uneasy
Randomized trials are one of the best tools scientists have for learning about the effects of new policies. There’s just one problem — the public kinda hates them. At least, that’s the takeaway from a recent study published in the Proceedings of the National Academy of Sciences. A significant share of the public finds conducting randomized studies morally objectionable.
Are you one of them? Well, here’s a hypothetical question for you:
A hospital director is trying to reduce the rate of medical error. She’s heard that checklists of standard safety procedures can cut errors down, but she doesn’t know anything about which approach to checklists does the most to reduce errors. Here are a few approaches she could try:
- She could print safety checklists on every doctor’s mandatory ID badge.
- She could put a poster in each room with a safety checklist.
- She could randomly do posters for half of operations and ID badges for the other half, measure which works best for a year, and then implement that for the whole hospital.
That last approach is an example of A/B testing and resembles the approach that researchers take when designing a randomized trial. These studies have some limitations, but they’re one of the best approaches to learning more about the effects of an intervention — that is, for learning things like whether posters or ID badges work better to reduce medical error.
But many respondents argued that it was deeply inappropriate for the hospital director to randomize in this fashion.
In a series of surveys — conducted among the general public on paid survey-answering sites Mechanical Turk and Pollfish, and among healthcare professionals recruited through their workplace — participants were asked questions similar to the one above. Most participants agreed that it’s okay for the hospital director to print safety checklists on badges and okay for the hospital director to put a poster in each room — but 40-50% of them felt it was “inappropriate” or “very inappropriate” for the director to test which approach works better.
Across the different studies conducted with different subgroups, an overwhelming majority was okay with either doing the badge or doing the poster. A much smaller majority was okay with testing which worked better: 40-50% thought that randomization was inappropriate. Even if each policy would be okay with them, people felt that it was unethical to conduct a study to learn which one worked better.
People don’t only feel this way about hospital policies. The researchers asked a question very similar to the one I posed above as well as questions about “direct-to-consumer genetic testing, autonomous vehicle design, employee retirement plan enrollment nudges, recruitment of health workers in developing countries, alleviation of extreme poverty, promoting school teacher well-being, and basic income policy options.”
In all of those cases except the last one, people felt the same way. Option 1? Fine. Option 2? Fine. Random assignment between Option 1 and Option 2, for the sake of learning which works better? Not fine.
The researchers didn’t just get these results among science-illiterate respondents, either. The researchers examined “whether educational attainment, having a science, technology, engineering, and mathematics (STEM) degree, or scientific understanding explained any of the variance in appropriateness ratings of experiments.” The results? Nope. Scientific awareness — and any other demographic factors — don’t influence whether people think that randomization is okay.
This is a big deal, because methods like these are widely used in medicine and social science
Why does public opinion of scientific methodology matter? Because A/B tests are everywhere. If you’re trying to study the effects of an intervention, a randomized controlled trial (RCT) — which assigns people to groups just like in the hospital example — is often by far the best way to figure out what’s going on. Approaches like these are how we’ve studied cancer treatments, microfinance, premature birth, preschool, and countless other critically important policy questions.
Over 360 million different patients have been enrolled in medical RCTs and 22 million in social policy RCTs, from 2007 to 2017. Much of our knowledge of medicine and policy comes from studies like these.
We even use A/B tests here at Vox. If I have two headlines I really love for a story, I can arrange for different viewers to see different ones — and then I can settle on the one that’s engaging more readers. It’s a nifty tool to figure out which of my clever headlines are actually clever — and it has never occurred to me that half of readers might rate it “inappropriate” or “very inappropriate” or feel like I’m experimenting on them.
So public discomfort with randomization suggests that lots of organizations should think about their practices.
Hospitals, nonprofits, governments, and other institutions that use RCTs to learn more about the world have particular obligations to the populations they serve — and that means they need to address this discomfort. There are a few routes to do that.
One is to educate people. The PNAS study researchers found that healthcare professionals largely agreed that a “learning health system” was best for patient care — but were as likely as anyone else to express dismay at randomization. It might be important for scientists to make the case for these methodologies and to explain clearly and transparently why they form such a crucial element of a learning health system.
Another is to develop alternative statistical methods that keep the desirable features of RCTs — in particular, their power to help us understand the effects of different approaches — while allowing more of an element of patient choice. For example, a working paper by Yusuke Narita at University of Chicago proposes an RCT design that lets people choose which treatment they get. Under the right conditions, it can still arrive at the same estimates of an effect as a standard RCT.
If approaches like this can be made to work widely, we could get the best of all possible worlds — more participant control over experience without compromising our ability to learn from the results.
Finally, it’s hard not to parse these results as an indictment, at least slightly, of science education. Scientifically educated adults have about the same rate of concerns about randomized trials as the rest of the public — suggesting nothing in their background forced them to confront and grapple with this question at a deeper level (which might have produced results that differed from the general population in either direction, but which probably wouldn’t have ended up with everyone exactly where they started.)
Maybe science education needs to encourage aspiring scientists to think harder about questions like these, so that researchers can articulate a vision they really believe in for the role of RCTs in society.
Sign up for the Future Perfect newsletter. Twice a week, you’ll get a roundup of ideas and solutions for tackling our biggest challenges: improving public health, decreasing human and animal suffering, easing catastrophic risks, and — to put it simply — getting better at doing good.
from Vox - All http://bit.ly/2QlUaca
Breaking News: A shocking share of the public thinks randomized trials are immoral - News Paper
0 komentar:
Post a Comment