Categories → Process , Causal Inference , Validation , Method
Statistical refutation is a method of using statistical evidence to test and refute hypotheses, including those related to causal inference.
NOTE: Please read this article for a detailed guide to refutation and statistical significance testing in DoWhy, a Python library used in Causal Wizard.
Refutation is a key concept in causal inference, which refers to the process of testing a hypothesis by attempting to prove it false.
Falsifying or refuting an outcome should not been seen as a disappointment:
Strict refutation helps to ensure - but does not guarantee - that results are sound and trustworthy.
Statistical refutation is a process of testing a hypothesis or a claim using statistical methods to determine whether the evidence contradicts or refutes it. In other words, it involves assessing the likelihood of the observed data under the null hypothesis, and if the probability of obtaining such data is very low, the null hypothesis is rejected in favor of an alternative hypothesis (i.e. the experimental hypothesis).
In the context of causal inference, statistical refutation is often used to test whether a hypothesized causal relationship between two variables is supported by the available data, or not. For example, suppose we hypothesize that smoking causes lung cancer. We can test this hypothesis using statistical methods by collecting data on a large group of people, some of whom smoke and some of whom do not, and analyzing the relationship between smoking and lung cancer incidence.
If our analysis finds a significant association between smoking and lung cancer incidence, we can infer that smoking is a causal factor in lung cancer. On the other hand, if we fail to find a significant association, we can conclude that there is no evidence to support the hypothesis that smoking causes lung cancer, and the hypothesis is refuted.
In summary, statistical refutation is a powerful tool for testing hypotheses and claims using statistical evidence. In the context of causal inference, it is used to evaluate the strength of evidence supporting or contradicting a hypothesized causal relationship between two variables.
Causal Wizard automatically applies a set of refutation tests after each Estimation request, and includes the refutation outcomes in your Results. We do this because we believe it's important to apply these good practices to every analysis and every answer.
Causal Wizard uses the DoWhy library, which implements several refutation mechanisms. Some of these methods make very few assumptions about your data and the system being modelled, which means we can safely use them in a wide range of problems.
Specifically, Causal Wizard uses: