What is Doubly Robust Estimation? [Part 2] — What Does Doubly Robustness Protect and What Does It Not? —

In the first part, we calculated AIPW as "prediction by the outcome model + residual correction using the treatment model."

However, using two models does not mean that the premises of causal inference are doubly guaranteed.

In this second part, we will compare cases where only the outcome model is appropriate, only the treatment model is appropriate, and both are inappropriate, to clarify the scope of doubly robustness.

First, the conclusion: what it protects is "a certain robustness against model misspecification."

AIPW is a method that predicts the A-time and B-time outcomes for everyone using an outcome model, and corrects the observed value - predicted value for the treatment actually received by the reciprocal of the treatment probability.

Under the necessary causal assumptions and regularity conditions, if at least one of the outcome model or the treatment model is correctly specified and estimated to converge to its true value, the AIPW will converge to the target quantity in large samples.

This is doubly robustness. What it protects is a certain robustness against the misspecification of one of the two auxiliary models. It is not a shield that eliminates unmeasured confounding or violations of positivity.

A brief review of the same 200 people and AIPW

This is a 200-person example examining health deterioration within 30 days. There were 100 people each in treatment group A and group B.

Mild: A is 1/20 = 5%, B is 8/80 = 10%

Severe: A is 24/80 = 30%, B is 8/20 = 40%

Of the total 200 people, 50% are mild and 50% are severe. The main target quantity is the risk difference of the ATE, which compares the average outcomes of everyone at time A and everyone at time B across these 200 people.

A-time 17.5% - B-time 25% = -7.5 percentage points

The marginal risk ratio, which expresses the same two marginal risks as a ratio, is 0.70. This is a quantity that expresses the same target population on a different scale and is not the same expression as the ATE defined as the difference in average outcomes.

The propensity score, or the probability of receiving A viewed from pre-treatment background, is 0.20 for mild cases and 0.80 for severe cases.

Let X be the treatment, Y the outcome, C the background, e(C) the propensity score, and m_A(C) and m_B(C) the outcome predictions for time A and time B. The contribution of one person to time A and time B is,

H_Ai = m_A(C_i) + X_i/e(C_i) × {Y_i - m_A(C_i)}

H_Bi = m_B(C_i) + (1 - X_i)/{1 - e(C_i)} × {Y_i - m_B(C_i)}

The correction for the side of the treatment not actually received is 0. We average each over the 200 people and compare time A and time B.

What happens if only the outcome model is appropriate?

Suppose the outcome model appropriately represents the conditional risks as: Mild A 5%, Mild B 10%, Severe A 30%, and Severe B 40%.

On the other hand, the treatment model ignores severity and incorrectly sets the propensity score for everyone to 0.50.

In this aggregation example, since the predicted values and observed averages match within each treatment and severity group, the total prediction error is 0. Even after adjustment, the results remain A 17.5% and B 25%.

In general finite samples, the adjustment term does not necessarily become 0. Nevertheless, if the outcome model is correctly specified and estimated to converge to its true value, AIPW will approach the target quantity in large samples even if the treatment model is misspecified.

What happens if only the treatment model is appropriate?

Now, suppose the propensity scores are appropriately estimated as 0.20 for mild cases and 0.80 for severe cases.

The outcome model is a crude model that does not distinguish between treatment or severity, predicting 20% for everyone.

The total adjustment for Mild A and Severe A is equivalent to -5 people, so for A, it becomes 20% - 5/200 = 17.5%. The total adjustment for Mild B and Severe B is equivalent to +10 people, so for B, it is 20% + 10/200 = 25%.

In this well-organized teaching table, it accurately returns to the reference value. In general, this is the property that if the treatment model is correctly specified and estimated to converge to its true value, consistency can be maintained in large samples even if the outcome model is misspecified.

If both models are inappropriate, there is no guarantee

Consider a case where both models are crude: the outcome model predicts 20% for everyone, and the treatment model sets everyone to 0.50.

In this case, after adjustment, it returns to the values observed in each treatment group, resulting in A 25% and B 16%.

The risk difference is +9 percentage points, which is even in the opposite direction of the standardized -7.5 percentage points.

Even if both are inappropriate, it is possible to accidentally obtain a result close to the target quantity. However, there is no guarantee provided by doubly robustness. This is because it is not a method where 'two errors cancel each other out'.

Organizing with four combinations

Out of the four combinations, if at least one is appropriate, it can converge to the target quantity in large samples. Only the bottom-right case, where both are inappropriate, is not guaranteed.

However, this does not mean that 'it is sufficient if two models are half correct.' Also, AIPW itself does not automatically determine which one is appropriate.

'Consistent estimation' does not mean getting the correct answer every time in a finite sample

Doubly robustness is a matter of consistent estimation, where the estimator approaches the target value as the sample size increases.

In small samples, estimation error remains even if one model is appropriate. There is no guarantee that it is always unbiased, always highly accurate, or that confidence intervals are automatically correct. Especially when propensity scores are close to 0 or 1, a small number of large adjustments can influence the results.

The "consistent estimation" in statistics referred to here is different from consistency, which is an assumption in causal inference. The latter means that the observed outcome corresponds to the counterfactual outcome under the treatment actually received.

"The model is appropriate" does not just mean that variables have been included.

An appropriate outcome model means that it sufficiently represents the conditional mean according to the treatment and background. If necessary interactions or non-linear relationships are omitted, it can be misspecified even if the variable names are included in the equation.

An appropriate treatment model means that it sufficiently represents the treatment probability according to the background. If important pre-treatment confounders are omitted or the form of the relationship is incorrect, the propensity score will not be estimated appropriately.

It is not a simple rule that it is always safe to include a single variable in only one of the models. At least one of the models as a whole must appropriately represent the necessary conditional mean or treatment probability.

Even with doubly robustness, the premises of causal inference cannot be protected.

To interpret it as a causal effect, at least three assumptions are required.

Conditional exchangeability: Group A and Group B can be compared if the measured pre-treatment background is taken into account

Positivity: In each target background, the probability of receiving either A or B is not zero

Consistency: The observed outcome corresponds to the outcome under the treatment actually received

If important confounding factors are not measured, they cannot be handled appropriately in either the outcome model or the treatment model. In backgrounds where the treatment probability is zero, both treatments cannot be compared from observational data.

Ambiguous definitions of treatment or outcome, measurement error, missing data, and bias due to subject selection are also separate issues. Using two models does not mean that causal assumptions are doubly guaranteed.

Diagnosing two models and extreme corrections

In actual research, not only point estimates but also two models are checked.

In the treatment model, we look at the overlap of propensity scores, values close to 0 or 1, extreme weights, and background balance after weighting.

In the outcome model, we check the discrepancy between prediction and observation, calibration, residuals, extrapolation, and the handling of interactions or non-linear relationships.

In AIPW, we also check whether the contribution of one person is not extreme and whether the results do not change significantly when the model specification is changed. Even if the diagnosis is good, it does not prove that there is no unmeasured confounding.

In a paper, check in this order

The points to check can be summarized into the following four groups.

Target: For whom and which effect measure was sought

Models: What was input into the two models, and how was the form of the relationship specified?

Diagnostics: Were overlap, covariate balance, prediction error, and extreme weights/contributions checked?

Inference: Is the 95% confidence interval consistent with the entire estimation procedure?

The 95% confidence interval must also reflect the uncertainty of estimating both models from the same data. Double robustness of point estimation does not automatically guarantee the validity of the confidence interval. In the paper, we check whether a calculation method corresponding to the entire AIPW procedure was used.

Summary

Double robustness is the property that, under the necessary causal assumptions and regularity conditions,

if at least one of the outcome model or the treatment model is appropriate, the target quantity can be consistently estimated in large samples

— this is the nature of it.

On the other hand, there is no guarantee if both models are inappropriate. It does not mean that you will get the correct answer every time in finite samples, nor does it solve unmeasured confounding, positivity violations, measurement error, missing data, or selection bias.

Rather than feeling secure because you used two models, it is important toseparately check the target quantity, the two models, causal assumptions, diagnostics, and the 95% confidence interval.

Next time,

What is TMLE? — What changes when AIPW correction is performed as an 'update of the prediction model'?

We will organize this by linking double robustness with the range of estimated values.

References

Bang H, Robins JM. Doubly Robust Estimation in Missing Data and Causal Inference Models. Biometrics. 2005;61(4):962–973.

https://doi.org/10.1111/j.1541-0420.2005.00377.x

Funk MJ, Westreich D, Wiesen C, Stürmer T, Brookhart MA, Davidian M. Doubly Robust Estimation of Causal Effects. Am J Epidemiol. 2011;173(7):761–767.

https://doi.org/10.1093/aje/kwq439

Kang JDY, Schafer JL. Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data. Stat Sci. 2007;22(4):523–539.

https://doi.org/10.1214/07-STS227

Austin PC, Stuart EA. Moving towards Best Practice when Using Inverse Probability of Treatment Weighting (IPTW) Using the Propensity Score to Estimate Causal Treatment Effects in Observational Studies. Stat Med. 2015;34(28):3661–3679.

https://doi.org/10.1002/sim.6607

Hernán MA, Robins JM. Causal Inference: What If. Boca Raton: Chapman & Hall/CRC. 2020.

https://miguelhernan.org/whatifbook

Tags:

#Statistics

#CausalInference

#DoublyRobustEstimation

#AIPW

#MedicalStatistics