Here are the factual allegations, as set forth in last week’s long opinion by Justice Mark Hanasono, joined by Justice Anne Egerton, in Dinho v. Netflix, Inc.:

Plaintiffs’ home is on a ridgeline in the Hollywood Hills. The property is guarded by a private gate and the home is not visible from any nearby street. The closest publicly accessible vantage point from which the home can be seen is 1,034 feet away….

Netflix began using a photo of plaintiffs’ home in an advertisement for Buying Beverly Hills, one of its reality shows which depicts the operations of a real estate firm. The photo was taken by nonparty Ashwin Rao. Rao published the photo on Shutterstock.com (Shutterstock), “a website that allows any person with [I]nternet access to license photographs to the public for specific uses.” Netflix hired Williams Creative Agency (WC+A) to produce the advertisement, and WC+A licensed the photo from Shutterstock for use in the advertisement.

Rao allegedly took the photo without [plaintiff] Dihno’s knowledge or consent using a drone or other specialized photography equipment. The photo depicted interior and exterior details of the home not visible from any public location, including the “room layout” and the entrances and exits….

Netflix published the advertisement on its own website and on several other websites. Netflix did so without plaintiffs’ knowledge or permission. Both Netflix and WC+A knew that the home was not associated with or depicted in Buying Beverly Hills….

[P]eople began to visit plaintiffs’ home “on a daily basis” asking to see it and claiming they learned it was for sale through the Buying Beverly Hills advertisement. One woman rang the doorbell, demanded to enter the home, and refused to leave. Dihno called the police and the woman was arrested. Other people attempted to open plaintiffs’ front gate and climb over their fence. [Plaintiffs] would only answer the doorbell for friends or relatives who provided advanced notice of their visit. In addition, [Plaintiffs] received calls “more than once daily” from real estate agents who sought to represent the family in selling the home.

Note that the ad apparently didn’t include the address of the home, or the names of the owners. More from the allegations:

Plaintiffs’ mental health and their reputations suffered. They had negative interactions with their neighbors, including one who “alienate[d]” them and another who “angrily confronted” them because the advertisement increased the number of unwanted visitors to the formerly quiet street. Plaintiffs did not feel safe in their home, could not sleep, and received medical treatment for insomnia and stress disorders. Out of fear for their own safety, plaintiffs spent approximately $20,000 on security measures. …

Plaintiffs sued, but the Court of Appeal ruled against them. In particular, it rejected plaintiffs’ claim that defendants were responsible for invading plaintiffs privacy on an “intrusion upon seclusion” theory. The court concluded that, whether or not Rao had committed the tort in photographing the house, Netflix wasn’t responsible for that, partly because plaintiffs didn’t allege “that Netflix had full knowledge of Rao’s conduct” and therefore couldn’t be said to have “Rao’s conduct.” Nor could the ad be faulted for being “an ‘extension’ of Rao’s intrusion”: “the tort of intrusion,” unlike the disclosure, false light, and appropriation torts, “does not create liability for publications.”

And Netflix also wasn’t liable for third parties’ intrusions:

In support of this argument, plaintiffs rely on Vescovo v. New Way Enterprises, Ltd. (Cal. App. 1976). There, the defendant published a woman’s address in a classified advertisement, accompanied by lewd language suggesting that the woman wished to engage in sex acts. More than 100 people trespassed at the property thereafter. The appellate court concluded that the woman’s minor child could state a claim for “the physical intrusion by various unsavory characters on her own solitude in her own home” based on allegations that the “defendants published the advertisement ‘with intent and design to injure, disgrace and aggrieve'” the child….. [But] Vescovo was published nearly 50 years ago, and some 22 years before Shulman v. Group W Productions, Inc. (1998), our Supreme Court’s leading case on common law intrusion. As Shulman and its progeny now instruct, to establish an intrusion claim, plaintiffs must allege that “defendants ‘intentionally intrude[d]'” upon their seclusion….

[And whether or not Vescovo survives Shulman,] Vescovo is distinguishable…. Unlike in Vescovo, the advertisement in this case depicted only the exterior of plaintiffs’ house, not their address or any other personal information; and the advertisement did not encourage third parties to visit plaintiffs’ home. And while the complaint alleges that Netflix intentionally published the advertisement with knowledge that it might harm plaintiffs, it does not allege that Netflix intended for third parties to harass plaintiffs….

Justice Egerton also added her views rejecting “appellants’ proposal to expand—in my view, in a sweeping and unwarranted way—the intentional tort of intrusion”:

[A]ppellants contend they “should have some right under the law to limit Netflix’s exploitation of their home, life, and privacy.” Appellants’ claim that Netflix “exploit[ed]” “their home” sounds suspiciously like a proposed right of publicity for houses. For good reason, there is no such tort. As for the alleged “exploitation” of appellants’ “life,” the photograph at issue did not depict any of the appellants. It is simply a picture of a house. The advertisement did not mention appellants nor did it say anything negative about them….

Appellants’ proposed expansion of liability for an intentional tort—they pray for compensatory damages of five million dollars in their privacy count, and for punitive as well as compensatory damages in three other causes of action—is breathtaking in its scope. Let’s say the Los Angeles Times decides to do a piece on “five houses in Los Angeles that look like they came out of a fairy tale.” You know—with those cute, curving brown roofs. People read the piece and think, “Wow, I’d like to see that.” They drive by, or walk by, the houses. Maybe some even knock and ask to come inside. Let’s say lots of people do that. Let’s say the “publisher” of the piece is not the Los Angeles Times but an influencer on Instagram who’s interested in architecture.

Can the owners or residents of those homes sue for intrusion? One can imagine myriad other examples….

Justice Lee Smalley Edmon dissented on these matters:

[A]t least a few cases have recognized claims for intrusion where defendants published information about the plaintiffs that caused third parties to intrude into their private spaces. The first such case was Kerby v. Hal Roach Studios, Inc. (Cal. App. 1942). The defendant in that case was a movie producer, and the plaintiff was an actress. To create interest in one of its movies, the defendant sent copies of a letter, which appeared to have been handwritten and signed by the plaintiff, to 1,000 men in Los Angeles. The letter said the plaintiff was ” ‘in the mood for fun'” and invited the recipients to meet her “‘in front of Warners Downtown Theatre at 7th and Hill on Thursday'” for ” ‘an evening you won’t forget.'” Although the letters did not include the plaintiff’s address or phone number, the plaintiff alleged that both were listed in a public phone directory, and thus the letter resulted in a large number of phone calls and a visit to her home, including one call that led the plaintiff to fear being shot….

[The] Court of Appeal [allowed the case to go forward]: “‘The right of privacy has been defined as the right to live one’s life in seclusion, without being subjected to unwarranted and undesired publicity. In short it is the right to be let alone.’ … Here the plaintiff was, without her consent, plucked from her regular routine of life and thrust before the world, or at least 1,000 of its persons, as the author of a letter not written by her and of a nature to at least cast doubt on her moral character, and this was done in a manner to call down on her a train of highly undesirable consequences. This constituted as strong an invasion of the right of privacy as any of those described in the cases.” The Court of Appeal reached a similar conclusion in Vescovo ….

Significantly, neither Kerby nor Vescovo concerned allegations that the defendants themselves physically intruded into the plaintiffs’ homes or ratified intrusions by third parties. Instead, the essence of the claimed intrusions in those cases was the defendants’ publication of information that created interest in the plaintiffs and led to foreseeable physical intrusions by third parties that significantly disturbed the plaintiffs’ solitude. Under those circumstances, the courts found the plaintiffs had adequately alleged claims for intrusion. The present case is analogous….

I do not agree [that Vescovo is inconsistent with Shulman]. While it is true that Shulman requires an intentional intrusion by the defendant, it does not limit “intrusion” to a physical intrusion by a defendant. To the contrary, Shulman says that to state a claim for intrusion, a plaintiff must demonstrate that the defendant “‘intentionally intrude[d], physically or otherwise, upon the solitude or seclusion of another,’ that is, into a place or conversation private to” the plaintiff….

The majority also suggests that the present case is distinguishable from Vescovo because there the defendant published the plaintiffs’ address and was alleged to have intended to “injure, disgrace and aggrieve” the plaintiffs, while here Netflix did not publish plaintiffs’ address and is not alleged to have intended harassment by third parties. A specific intent to cause harm—as opposed to the intent to intrude—is not an element of a cause of action for intrusion, and thus I do not find this distinction material. Nor do I consider it relevant that Netflix did not publish plaintiffs’ address. The plaintiff’s address was not published in Kerby, but third parties nonetheless were able determine where the plaintiff lived because her name and address were listed in the telephone directory….

The dissent also added, perhaps in response to the concurrence, “Importantly, the photograph of plaintiffs’ house was published in connection with an advertisement, not a news story.  The present case thus does not raise the constitutional issues present in many privacy cases.” The concurrence, on the other hand, reasoned, “Finally, that the photograph appeared in an advertisement does not strip it of constitutional protection.  New York Times Co. v. Sullivan (1964)—the seminal First Amendment case—involved an advertisement.” (Note that the Sullivan ad was a political ad, not a commercial ad for a TV show, as was the case here.)

The majority and dissent also disagreed on whether plaintiffs had adequately stated a claim for private nuisance, but I omit that for space reasons; I also omit the discussion of the false light claim, the negligent and intentional infliction of emotional distress claims, and a couple of statutory claims. You can read more about them in the full opinion.

Mark R. Yohalem and Madelyn Y. Chen (Wilson Sonsini Goodrich & Rosati) and Jonathan Segal, Rachel R. Goldberg, and Samantha Lachman (Davis Wright Tremaine) represent Netflix.

The post Showing Plaintiffs' House in an Ad for Netflix Real Estate Reality Show Isn't Actionable Invasion of Privacy, appeared first on Reason.com.

from Latest – Reason.com https://ift.tt/9MHzNRX
via IFTTT

As I write, European Union (E.U.) officials are debating the details of a proposal to either require or pressure tech companies to scan all private messages for child sexual abuse material. Dubbed “chat control,” the scheme inevitably entails mass surveillance of private communications—targeting one sort of content for the moment, though it’s difficult to see how that would long remain limited in any way. It’s an illustration of the continuing decline in online liberty documented in a new report from Freedom House.

No Right to Private Messages?

As the E.U.’s discussion over chat control heated up over the summer, Danish Minister of Justice Peter Hummelgaard, a proponent of surveillance, commented, “We must break with the totally erroneous perception that it is everyone’s civil liberty to communicate on encrypted messaging services.”

Since Hummelgaard’s Orwellian comments, which would have necessitated government backdoors into all encryption used by the public, the E.U. has backed off a bit from mandating such access. Current proposals would make private companies liable for the content of their customers’ communications, leaving them to choose how to mitigate their legal liability—with scanning messages a likely choice.

“Chat Control is not dead,” privacy activist and former member of the European Parliament Patrick Breyer commented, “it is just being privatized.”

More accurately, private companies are likely going to be jawboned—strong-armed—into doing the government’s dirty work. And there’s a lot of dirty government work to go around on today’s internet.

“Suppression of mass protests, deepening censorship, and threats to free speech fueled the 15th consecutive year of decline in global internet freedom,” the U.S.-based Freedom House notes in its report, Freedom on the Net 2025, released November 13.

It’s unsurprising that countries already recognized as authoritarian are continuing repressive practices. Nobody expects China or Iran to suddenly develop a taste for protecting online dissent and respecting privacy of communications. More disturbingly though, as seen in the European debate over chat control, nominally free countries are becoming increasingly intrusive when it comes to the digital world.

Online Authoritarianism in the ‘Free’ World

“In a concerning sign, half of the 18 countries with an internet freedom status of Free suffered score declines during the coverage period, while only two received improvements,” according to the report. “People in Georgia experienced the most significant decline among these countries, followed by Germany and the United States.”

Georgia made the list by forcing private organizations and media operations that receive foreign funding to register with the government. The government also imposed “criminal penalties of up to 45 days in prison for insulting public officials.”

As the report notes, Germany’s government has infamously “pursued criminal prosecutions against people who made memes about politicians, invoking laws against insult and hate speech.” Such censorship continues even after the replacement of the previous thin-skinned traffic light coalition of the Social Democratic Party (SDP), the Free Democratic Party (FDP), and the Greens by a grand coalition of the Christian Democratic Union (CDU), its ally the Christian Social Union (CSU), and the SDP.  That suggests there’s little appetite among the country’s political class for leaving people alone.

In the U.S., “the administration of President Donald Trump,” the report says, “detained several foreign nationals for one to two months after revoking their visas over nonviolent online expression” and that the government “threatened or carried out politicized investigations into civil society organizations and media and technology companies, often focusing on their content moderation, editorial decision-making, or forms of speech that are protected by the US Constitution’s First Amendment.” It should have also included the pressure brought by the previous Biden administration on social media companies to suppress criticism of administration policies and stories inconvenient to the powers that be.

Some countries did register improvements in online freedom. But “of the 72 countries assessed in Freedom on the Net 2025, conditions deteriorated in 28, while 17 countries registered overall gains.” And, as mentioned above, half of the countries ranked as “free” lost ground over the assessed period, while only two improved. Consequences for those targeted for suppression could be severe.

“People in at least 57 of the 72 countries covered by Freedom on the Net 2025 were arrested or imprisoned for online expression on social, political, or religious topics during the coverage period—a record high.”

Politicians Want to Regain Lost Control

It’s interesting to speculate on why government officials around the world seem so intent on suppressing online speech and monitoring digital communications, but the most likely explanation, to my mind, is the democratization of communications brought about by the internet. People can share ideas—good, bad, or flat-out nuts—with one another without the permission or assistance of governments or established media companies which long dominated mass communication.

“Whereas establishment institutions once exercised an informational monopoly, managing media and mainstream discourse to protect elite interests and perspectives, social media makes such narrative control impossible,” Dan Williams, an academic philosopher from the United Kingdom, wrote this week.

Williams builds on arguments advanced by former CIA analyst Martin Gurri, the author of The Revolt of the Public and the Crisis of Authority in the New Millennium (2014). Gurri believes “technology has categorically reversed the balance of power between the public and the elites who manage the great hierarchical institutions of the industrial age—government, political parties, and the media.”

Surveillance and censorship are ways to try to reassert that lost control. By monitoring people’s private communications and punishing them for their online statements, government officials, even in countries that once prided themselves on open debate, try to regain power over ideas expressed by the public and the esteem (or lack thereof) in which people regard officialdom.

That’s not going to happen. Politicians can’t regain status and respect by pushing for mass surveillance and muzzling their critics. They can erode norms around privacy and free speech to punish dissenters. But they undermine their own standing even as they lash out and further alienate the public.

The post The 'Free' World Is Coming for Your Private Messages appeared first on Reason.com.

from Latest – Reason.com https://ift.tt/YR6ICfw
via IFTTT

12/1/1897: Justice Stephen Field resigns.

The post Today in Supreme Court History: December 1, 1897 appeared first on Reason.com.

from Latest – Reason.com https://ift.tt/IQeVRN4
via IFTTT

A century ago, two oddly domestic puzzles helped set the rules for what modern science treats as “real”: a Guinness brewer charged with quality control and a British lady insisting she can taste whether milk or tea was poured first.

Those stories sound quaint, but the machinery they inspired now decides which findings get published, promoted, and believed—and which get waved away as “not significant.” Instead of recognizing the limitations of statistical significance, fields including economics and medicine ossified around it, with dire consequences for science. In the 21st century, an obsession with statistical significance led to overprescription of both antidepressant drugs and a headache remedy with lethal side effects. There was another path we could have taken.

Sir Ronald Fisher succeeded 100 years ago in making statistical significance central to scientific investigation. Some scientists have argued for decades that blindly following his approach has led the scientific method down the wrong path. Today, statistical significance has brought many branches of science to a crisis of false-positive findings and bias.

At the beginning of the 20th century, the young science of statistics was blooming. One of the key innovations at this time was small-sample statistics—a toolkit for working with data that contain only a small number of observations. That method was championed by the great data scientist William S. Gosset. His ideas were largely ignored in favor of Fisher’s, and our ability to reach accurate and useful conclusions from data was harmed. It’s time to revive Gosset’s approach to experimentation and estimation.

Fisher’s approach, “statistical significance,” is a simple method for drawing conclusions from data. Researchers gather data to test a hypothesis. They compute the p-value under the null hypothesis—that’s the probability of observing their data if the effect they are testing is absent. They compare that p-value to a cutoff, usually 0.05. If the p-value is below the cutoff—in other words, the data we observe are unlikely under the null hypothesis—then the effect is present.

Fisher pioneered many statistical tools still in use today. But writing in the early 20th century, when science was carried out with fountain pens and slide rules, he could not have anticipated how those tools would be misused in an era of big data and limitless computing power.

Fisher was able to attend Cambridge only by virtue of winning a scholarship in mathematics. In 1919 he was offered a job as a statistician at the Rothamsted Agricultural Experiment Station, the oldest scientific research farm in England. At Rothamsted, then at University College London and Cambridge, Fisher grew into an awesomely productive polymath. He invented p-values, significance testing, maximum likelihood estimation, analysis of variance, and even linkage analysis in genetics. The Simply Statistics blog estimates that if every paper that used a Fisherian tool cited him by 2012 he would have amassed over 6 million citations, making him the most influential scientist ever.

In 1925 Fisher published his first textbook, Statistical Methods for Research Workers, which defined the field of statistics for much of the 20th century. Before its publication, a researcher who wished to draw conclusions from data would make use of a large-sample formula such as the normal distribution. Discovered a hundred years earlier by Carl Friedrich Gauss, the normal distribution is the standard “bell curve” formula for an entire population of observations around a central mean. Fisher’s textbook provided tools to analyze more limited samples of data.

How Beer Led to a Breakthrough

The most important small-sample formula in Statistical Methods was discovered by a correspondent of Fisher’s, William Sealy Gosset, who studied mathematics and chemistry at Oxford. Instead of staying in academia, Gosset moved to Dublin to work for Guinness.

Guinness was expanding rapidly and shipping its product worldwide. By 1900 it was the largest beer maker in the world, producing over a million gallons of stout porter each year. At such scale, it no longer made sense to test each batch by taste and feel. Guinness set up an experimental lab within its giant brewery in St. James’s Gate, Dublin, to systematically improve quality and yield, and staffed it with talented young university graduates. Gosset and his colleagues were among the first industrial data scientists.

Gosset’s interest in small-sample statistics flowed from his everyday work. Beer takes three ingredients: yeast, hops, and grain. The grain in Guinness is malted barley. To prepare the barley, you steep it in water, allow it to germinate, and then dry and roast it, which malts the starch into sugar that the yeast can digest. The amount of sugar in a batch of malt affects the taste of the beer, its shelf life, and its alcohol content, and was measured at that time in “degrees saccharine.”

Guinness had established that 133 degrees saccharine per barrel was the ideal level, and was willing to tolerate a margin of error of 0.5 degrees on either side. The brewer could take spoonfuls from a barrel of malt, test each spoonful, then take the average. But how accurate would that average be—should he take five spoonfuls or 10? Gosset verified that small-sample estimates are more spread out than a normal distribution, because you might draw a spoonful that is unusually high or unusually low in sugar, and in a small sample such outliers will have outsize influence.

Gosset was unable to mathematically solve the distribution of a small sample. Impressively, he wrote out a formula based solely on his intuition. To check his work, Gosset made use of a set of physical measurements taken by the British police from 3,000 prisoners. He wrote each prisoner’s height and finger length on a card, then shuffled the cards and divided them into groups of four. The averages of the four-card samples had a distribution that matched closely with his formula.

That formula was immediately useful in his job of industrial quality control. Guinness now had an exact formula for the number of samples that would yield a desired level of accuracy. Guinness scientists were allowed to publish their work in academic journals on two conditions: The paper must not mention Guinness or any beer-related topics, and it must use a pseudonym. Under the name “Student”—he kept his laboratory notes in a notebook whose cover read “The Student’s Science Notebook”—Gosset published his small-sample formula in 1908.

The height of plants and prisoners, the degrees saccharine of a batch of malt, the test scores of college applicants, and many other real-world examples are approximately normally distributed. Thus, knowing the distribution of a small sample drawn from such a population would be useful in many areas of science. Ronald Fisher, then a Cambridge undergraduate, recognized the potential of Student’s distribution, and in 1912 he mathematically proved the formula that Gosset had guessed.

Gosset’s formula was a product of his work at Guinness. But the formal proof was all Fisher, as was the organization of Statistical Methods for Research Workers. One of the reasons Fisher’s textbook was so influential was an innovative feature: It included a set of statistical tables. The researcher could pick the distribution that corresponded to their data and look up the p-value in the corresponding table.

Gosset was unimpressed with this emphasis on p-values. In a 1937 letter, talking about a comparison of crop yields at different farm stations, he wrote that “the important thing in such is to have a low real error, not to have a ‘significant’ result at a particular station. The latter seems to me to be nearly valueless in itself.” Gosset was not interested in statistical significance, and he had no null hypothesis to test. His interest was in accurately measuring the yield. The economists Aaron Edlin and Michael Love call this philosophy “estimation culture.”

The Flaws in Fisher

In his 1936 textbook, The Design of Experiments, Fisher formally laid the foundation of significance testing. On page 6, Fisher describes an acquaintance, a British lady who claims that she can distinguish by taste whether the milk or the tea was poured first. Fisher is skeptical. He proposes having the lady drink eight cups of tea, four milk-first and four tea-first, in random order, and try to guess which was which. How, he asks, should we evaluate the data from this experiment?

Fisher advised calculating the probability of observing your data under the null hypothesis that the lady has no tea-tasting ability. If she is picking at random, she might get all eight cups of tea correct by chance, but the probability of that happening is one in 72—a p-value of 0.014. Fisher then proposes that a p-value under 5 percent is “statistically significant,” while a p-value greater than 5 percent is not.

The Fisherian approach flows from the specifics of the tea-tasting example. The binary, yes-or-no approach arises because the fundamental question is “Does the lady have tea-tasting ability, or does she not?” We are not trying to estimate how much tea-tasting ability she has. This feature of the tea-tasting experiment—the binary yes-or-no question—is critical to the Fisherian approach. Without it, statistical significance is unhelpful at best, actively misleading at worst.

The original sin of statistical significance is that it sets up an artificial yes-or-no dichotomy in how we learn from data. Fisher himself would agree that his 0.05 threshold for significance is arbitrary. But this arbitrary cutoff leads us to discard useful information. Imagine you run an expensive randomized trial of a new drug and find that it miraculously cures patients of pancreatic cancer—but only in a few rare cases, so that p = 0.06 and the result is statistically insignificant. Hopefully, we would not toss the project. Put differently, experiments do not exist merely to test the null hypothesis. Their purpose is to advance human knowledge, such as finding cancer cures.

Many important scientific experiments are completely outside of the Fisherian statistical significance model. For one sterling example, 16 years before the publication of The Design of Experiments, Robert Millikan and his graduate student, Harvey Fletcher, performed their oil-drop experiment at the University of Chicago. Millikan and Fletcher set up a mechanism that sprayed droplets of oil past an X-ray tube and then between two metal plates wired with an electric current.

As the droplets passed the X-ray tube, they became ionized—that is, they absorbed one or more extra electrons and gained a negative charge. By gently varying the voltage between the metal plates, Millikan and Fletcher could see when the oil drops floated in midair, precisely balancing the electrical force with the pull of gravity. That level of electrical current, plus the weight of the average oil drop, let them estimate the charge of the electron.

The oil-drop experiment was such a tour de force that Millikan received the Nobel Prize in physics just 14 years later, in 1923—two years before the publication of The Design of Experiments. But what null hypothesis were they testing? Nobody thought that the charge of an electron might be zero. Instead, the oil-drop experiment is a towering example of estimation culture: an experiment to measure the world we live in.

How Statistical Significance Causes Publication Bias

Significance culture versus estimation culture might seem like a minor technical debate. If you’re not a tenure-track academic, you may find it hard to believe how much one’s career depends on getting “significant” publishable results. In my own field of economics, Isaiah Andrews and Maximilian Kasy estimated that a statistically significant result is 30 times more likely to be published than a nonsignificant result. In the prestigious American Economic Review in 1983, the economist Ed Leamer wrote, “This is a sad and decidedly unscientific state of affairs…hardly anyone takes anyone else’s data analysis seriously. Like elaborately plumed birds who have long since lost the ability to procreate but not the desire, we preen and strut and display our [p]-values.”

We might imagine that medical studies are held to a higher standard. But in a 2008 study in the New England Journal of Medicine, researchers gathered data from 74 preregistered studies of antidepressant medications. Of the 38 studies that found a statistically and medically positive result, 37 were ultimately published. Of the 36 that found negative or insignificant results, 33 were either not published or published in a way the researchers considered misleading.

If a doctor relied on the published literature, then, they would see 94 percent of studies yielding a positive outcome. In reality, only 51 percent of studies found a positive outcome—no better than a coin flip.

Individual findings are also distorted by significance culture. Vioxx is a nonsteroidal anti-inflammatory drug (or NSAID) similar to aspirin. In the year 2000, a clinical trial funded by its producer—the pharmaceutical firm Merck—reported that the difference between the treated and control groups in the rate of cardiac complications “did not reach statistical significance.”

But heart issues were quite rare in their patient population. One patient out of 2,772 in the control group had cardiac complications. In the treated group, five patients out of 2,785 had cardiac complications. The risk of the deadly side effect was five times higher among treated patients. But because one and five out of 2,800 are both very low numbers, the p-value of the difference is above 0.05. As a result, it was reported and marketed as having no significant difference.

Less than a year later, Merck pulled Vioxx from the market; follow-up studies had found the NSAID led to an increased risk of heart attack and stroke. In the meantime, millions of prescriptions were handed out.

Statistical significance had a doubly malign effect on the Vioxx study. First, it caused the researchers, editors, and reviewers to ignore the higher rate of cardiac complications because the difference was not statistically significant. Second, and more insidiously, statistical significance meant that they did not think about the relative costs and benefits. We already have aspirin, Tylenol, Advil, and other NSAID drugs, so the contribution to human well-being of Vioxx’s ability to alleviate aches and pains was always going to be modest. By contrast, an increase in the risk of cardiac complications is deadly serious. But statistical significance does not consider the importance of the outcome; all that matters is the p-value.

The false-positive problem is getting worse, not better, in the world of big data. As we run more and more studies, we sieve down to smaller and less important effects. At the same time, we increase our sample sizes and run more and more significance tests. As a result, the flood of nonsense findings is rising over time—as predicted in a 2005 PLoS Medicine article by John Ioannidis titled “Why Most Published Research Findings Are False.”

Can “Estimation Culture” Improve Practical Science?

Gosset disliked statistical significance. He was interested in estimating useful quantities in the real world, quantifying his confidence around that estimate, and adding up the costs and benefits involved. His 1908 Biometrika paper on the small-sample formula states that “any series of experiments is only of value in so far as it enables us to form a judgment as to the statistical constants of the population.” That’s estimation culture.

Could we shift the norms and practices of entire fields (social science and medicine, for starters) toward estimation culture? More than 100 years later, we can still look to industry for inspiration. In recent years, there’s been a flood of quantitative researchers into software and marketing jobs analyzing the oceans of data collected by companies such as Google and Microsoft.

A 2025 Information Research Studies paper examined over 16,000 statistical tests run by e-commerce data scientists. The researchers found no evidence of p-hacking or selective publication. Unlike academics, data scientists in industry don’t need to publish or perish, so they have no incentive to distort their findings; they are paid for accuracy. Just like their forebear at the Guinness brewery, they naturally cleave to estimation culture.

Academics and journalists should do likewise. Start with the estimate itself and our confidence in that estimate. Put the number in context, evaluate its relevance and plausibility, and try to add up the potential costs and benefits on each side. Finally, recognize that the scientific record in many fields presents a severely distorted picture. When reading the academic literature—to say nothing of the media coverage—keep in mind that we see all the positive and significant findings, and far fewer of the negative or insignificant findings.

There is a better way, and it was there in the Guinness brewery with William Gosset 100 years ago. It starts with dethroning statistical significance, moving away from all-or-none frequentist thinking and toward estimation culture.

The post Our Obsession With Statistical Significance Is Ruining Science appeared first on Reason.com.

from Latest – Reason.com https://ift.tt/4XifhMD
via IFTTT