What is an Experiment?

Its purpose is to provide evidence that supports or refutes a hypothesis, and provides insight into the causal mechanisms underlying a phenomenon.

We can broadly identify three types of experimental methodology:

• Experiment / Field Study / Randomized-Controlled Trial (RCT): Actively divide subjects into control and treatment group randomly to evaluate the causal link between treatment and outcome of interest. Randomness is key to ensure trustworthy results. "Randomized controlled trials (RCT) are prospective studies that measure the effectiveness of a new intervention or treatment. Although no study is likely on its own to prove causality, randomization reduces bias and provides a rigorous tool to examine cause-effect relationships between an intervention and outcome."
• Observational Experiment: Requires Causal Inference, based on observational data. Since there are intentional and/or unintentional randomness in the data, researchers need to choose methodologies carefully and consciously adopt relevant assumptions. There is no direct intervention in the system.
• Natural Experiment: Empirical studies that expose subjects to treatment and control group by taking advantage of naturally-occurring events, such as a lottery draw, birth of the month, immigration law, etc. Since the randomness mainly comes from the natural events that have already happened, researchers can analyze the causal link with observational data and Causal Inference methods.

Randomized, Controlled Trials

A randomized controlled trial (RCT) is a type of scientific experiment designed to test the effectiveness of an intervention or treatment. It involves randomly assigning participants to a treatment group or control group and then comparing the outcomes between the two groups. The goal is to determine whether the intervention or treatment has a significant effect on the outcome of interest, while minimizing bias and other confounding factors.

The strength of RCTs in proving causation lies in their ability to control for confounding variables and establish a cause-and-effect relationship between the intervention and outcome. By randomly assigning participants to groups, researchers can ensure that any observed differences between the groups are due to the intervention, rather than other factors such as age, gender, or socioeconomic status.

Observational and Natural Experiments

Causal inference is the process of drawing conclusions about causal relationships between variables based on statistical analysis of data from observational studies (or other types of experiment). In causal inference experiments, researchers use sophisticated statistical methods to estimate the causal effect of an intervention or treatment on an outcome while accounting for other factors that may influence the outcome.

Causal inference in combination with natural or observational experiments can be used to prove causation in situations where RCTs are not feasible or ethical. For example, in cases where it is not possible to randomly assign participants to a treatment group or control group, such as when studying the effects of smoking on lung cancer, observational studies can be used to estimate the causal effect of smoking on the risk of lung cancer.

The value of Causal Inference and observational experiment 

Scientists have argued that avoiding the explicit study of causation outside the interventional experimental setting "impairs study design and data analysis, holds back cumulative research, leads to a disconnect between original findings and how they are interpreted in subsequent work, and limits the relevance of nonexperimental psychology for policymaking." and "the taboo does not prevent researchers from interpreting findings as causal effects—the inference is simply made implicitly, and assumptions remain unarticulated." This is the worst of both worlds - potentially faulty conclusions are not made clear and tested.

Graphical causal models, as used in Causal Wizard, help to make the assumptions underlying observational or natural experiments explicit and clear.

Overall, the key to proving causation is the ability to isolate the effects of one variable from other potential confounding factors. RCTs and causal inference experiments are two powerful tools for doing so and provide valuable evidence for making informed decisions about interventions and treatments.