Check (Verification and Identification)

Categories → Causal Wizard Concept , Graph , Data , Process , Causal Inference , Variables

When editing a Study, clicking Check will check your config and determine if the Causal Effect can be identified.

Check Button

The Check button is available when editing a Study. You can click it at any time.

The Check process will broadly do two things:

1. It will review and check the setup and configuration of your Study, ensuring you have:
   • entered all necessary data,
   • selected Treatment and Outcome variables,
   • identified the Control and Treated Cohorts, and 
   • completed a Causal Diagram.
2. The Wizard will attempt to Identify the Causal Estimand (the statistical quantity which can be used to Estimate the desired Causal Effect).

Check result

If the Check fails, you can't get a Result unless you change the way the problem is modelled, or provide more information.

• If there is a problem, you will be given feedback about specific issues that you can fix. These usually involve the setup and configuration options, the data, or the structure of the Causal Diagram.
• You may receive a warning that your data is highly skewed. You can proceed, but bear in mind this may affect your results.
• In some cases, the result of Identification will confirm that the Causal Effect is zero, which is a valid result! Congratulations - you used the Causal Diagram editor to confirm that there cannot be any causal relationship from Treatment to Outcome.

Identification result

Assuming your Study passes the Check, Causal Wizard will attempt to identify an Estimand that would provide an Estimate of the Causal Effect you want. Think of an Estimand as a strategy to perform the estimation.

If an estimand can be identified:

• You'll see a green panel which invites you to select the Estimation method and model type, and the Calculate button will become available.
• Select the desired model and then press the Calculate button. Causal Wizard will then go and estimate your Result.

If an estimand cannot be identified:

• The causal effect you desire cannot be directly estimated given the domain knowledge you have provided.
• You will have to change your Study design to find a quantity which can be estimated.
• Often, this can be achieved by simplifying the Causal Diagram, in particular by removing Unobserved Variables. 
  
  • But remember, that this implicitly creates new assumptions which you need to consider when interpreting your results - the edges or variables you removed may have a material effect, which has not been reflected in the estimate. 
  • You will need to use your judgement to decide which edges (relationships) and variables to keep in your Causal Diagram. 
  • You can compare Results given different Causal Diagrams, removing different relationships or variables each time and checking to see if this has a significant effect on the result.
  • Ensure to document variables or relationships not present in the Causal Diagram, but suspected to exist in the real world.
• Sometimes you can change the way you define the problem - perhaps by measuring a proxy of the desired treatment or outcome, or replacing an unobserved confounding variable with a proxy which is present in your data (i.e. observable).
  • A proxy just means a variable which can be used instead of the real variable, perhaps because they are highly correlated.
• Occasionally, adding new Variables to the graph (such as Instrumental Variables) may create a way to Identify the desired Causal Effect. Check whether you have data for these variables already.