Skip to contents

Permutation Inference via MSPE Ratio for SCM

Source: R/scm.R
mspe_ratio_pval.Rd

Computes the Abadie et al. (2010) / Abadie (2021) permutation p-value. For each control unit, a leave-one-out synthetic control is fitted.

Usage

mspe_ratio_pval(
  fit,
  mspe_threshold = 0,
  max_iter = 100L,
  tol = 1e-04,
  use_covariates = NULL,
  alternative = c("two.sided", "greater", "less"),
  mspe_prune = Inf,
  statistic = c("ratio", "post_mspe")
)

Arguments

fit

A coresynth object from scm_fit() with method = "scm".

mspe_threshold

Minimum pre-treatment MSPE for including a control unit in the two-sided test. Ignored for one-sided tests. Default: 0 (no filtering).

max_iter

Passed to scm_placebo_cpp() (outcomes-only fits) or scm_placebo_x_cpp() (covariate fits). Default 100L.

tol

Passed to scm_placebo_cpp() or scm_placebo_x_cpp(). Default 1e-4.

use_covariates

Controls which predictor specification the placebo refits use. Default NULL (recommended) mirrors the treated fit: if the fit was estimated with a predictors specification, each placebo unit is refit with that same specification (the Abadie et al. 2010 / Synth convention – treated and placebo statistics are computed under one common spec); outcomes-only fits use the fast C++ outcomes-only placebo. Set TRUE/FALSE to force either path; note that FALSE on a covariate fit compares a covariate-based treated statistic against outcomes-only placebo statistics, which breaks the exchangeability logic of the permutation test.

alternative

Direction of the alternative hypothesis: "two.sided" (default) uses the MSPE ratio statistic; "greater" tests whether the treatment increased the outcome; "less" tests whether the treatment decreased the outcome. One-sided tests use the signed ATT as the test statistic.

mspe_prune

Discard placebo runs whose pre-treatment MSPE exceeds this multiple of the treated unit's, following Abadie, Diamond & Hainmueller (2010, Figures 5-7), who use 20, 5 and 2. A run the synthetic control could not fit before treatment carries no information about how rare a large post-treatment gap is, so leaving it in the permutation distribution only adds noise. Inf (default) keeps every run. This is a different control from mspe_threshold, which is a floor guarding the ratio's denominator against near-zero pre-treatment MSPE.

statistic

Which quantity to rank, for two-sided tests. "ratio" (default) is the post/pre MSPE ratio of Abadie, Diamond & Hainmueller (2010, Figure 8), whose denominator adjusts for how well each unit was fitted and so needs no cutoff. "post_mspe" is the alternative of Abadie (2021, S.3.5): rank the post-treatment MSPE itself, having discarded the ill-fitting runs through mspe_prune. Pair it with a finite mspe_prune; without one, placebo runs that were never fitted dominate the distribution.

Value

An object of class scm_placebo (a list) with:

  • p_value: Permutation p-value between 0 and 1

  • mspe_ratio_treated: MSPE_post / MSPE_pre for the treated unit (two.sided only)

  • mspe_ratios_all: Named numeric vector (treated first, then controls); two.sided only

  • placebo_effects: Named N_co-vector of placebo ATT estimates

  • treated_effect: ATT estimate for the treated unit

  • n_placebo_used: Number of control units used

  • n_placebo_pruned: Number discarded by mspe_prune

  • mspe_prune, statistic: the settings the p-value was computed under

  • gaps: T x N_co matrix of placebo gap paths (unit minus its synthetic control over all periods), for the Abadie et al. (2010) Figure 4-7 plot

  • treated_gap: T-vector of the treated unit's gap path

  • mspe_pre_treated, mspe_pre_placebo: Pre-treatment MSPEs used for the relative pruning rule in plot.scm_placebo()

  • times, T_pre: Time axis metadata for plotting

Details

When alternative = "two.sided" (default), the test statistic is the post/pre MSPE ratio, following Abadie et al. (2010). When alternative = "greater" or "less", the test statistic is the signed average post-treatment gap (ATT), giving a one-sided permutation test as recommended by Abadie (2021) S.3.5 for improved power when the direction of the treatment effect is known.

The placebo refits mirror the treated fit's outer optimiser and evaluation window: a fit estimated with the multi-start outer search (v_optim = "multistart", or the "auto" default with predictors) or with a v_window runs every placebo unit through the same configuration, keeping the permutation statistic exchangeable across units.

See also

plot.scm_placebo() for the placebo gap and MSPE ratio plots.