Executive Summary
What 13 studies of 'log risk ratio' say once pooled — and how much they disagree.
The short answer
Across 13 studies, BCG vaccination is associated with reduced tuberculosis risk: the random-effects pooled estimate is -0.7141 (95% CI -1.064 to -0.364), excluding zero. But the 13 studies do not agree on the size of that effect—92.12% of the variation between them exceeds sampling error—so the pooled number is an average across genuinely different study effects, not a single true effect.
The detail
The random-effects estimate of -0.7141 is the one to quote because the studies are highly heterogeneous (I-squared = 92.12%, Cochran's Q p < 0.001, tau-squared = 0.3088). The fixed-effect model gives -0.4303 with a narrower interval, but that interval reports a precision the evidence does not support. The 95% prediction interval (-1.999 to 0.571) is the honest range for what a new comparable study would find. Egger's test detects no significant funnel asymmetry (p = 0.189). No subgroup analysis was performed because no grouping column with two or more poolable levels was mapped.
What this can't tell you
The prediction interval is much wider than the confidence interval and includes zero, so a new study could plausibly find no protective effect or even harm. The small study count (13) limits the power of the funnel asymmetry test to detect publication bias. Pooling cannot repair selective reporting or make observational associations causal.
Analysis Overview
Fixed-effect and random-effects pooling of 13 studies on 'log risk ratio'.
The short answer
Two models are fitted because they make different assumptions about whether all 13 studies estimate the same effect or different effects. The heterogeneity statistics decide which is defensible: with I-squared at 92.12%, the studies do not share one effect, so the random-effects model is the right one to use.
The detail
The fixed-effect model assumes all studies estimate one shared quantity and weights each by 1 divided by its variance, yielding -0.4303. The random-effects model allows effects to differ by adding a between-study variance component (tau-squared = 0.3088) and yields -0.7141. Standard errors came from the reported 'std error' column. Cochran's Q of 152.233 (p < 0.001) and I-squared of 92.12% mean that more than 92% of the variation across the 13 studies exceeds what sampling error alone can explain, so the random-effects estimate is an average across a distribution of genuinely different study effects, not a better measurement of one shared effect.
What this can't tell you
Pooling inherits whatever biases the 13 studies carry. A pooled association remains an association and cannot establish causation.
Data Quality
Which rows became studies, and which could not.
The short answer
All 13 studies were usable. Data quality checks confirmed that every row carried both a numeric effect and a strictly positive standard error—the prerequisites for inverse-variance pooling. Blank study labels were replaced with positional names and duplicates were suffixed to ensure each study appears separately.
The detail
13 rows loaded; 13 rows became studies; 0 rows were removed. A row can only enter pooling if it carries a numeric effect AND a strictly positive standard error, because a zero or negative standard error would assign that study infinite weight and silently decide the entire result. Standard errors came from the 'std error' column. All 13 met this criterion. Blank study labels were replaced with positional names and duplicate labels were suffixed, so every study appears as a separate row on the forest plot.
What this can't tell you
This check confirms that the data structure supports pooling; it does not verify that the 13 studies measured tuberculosis risk using identical methods or definitions.
Forest Plot
Every study's effect and 95% interval, with both pooled estimates.
The short answer
The 13 studies scatter widely around the pooled estimate, with effects ranging from +0.4459 (study 12) to -1.6209 (study 7). Because I-squared is 92.12%, this scatter reflects real differences between studies, not just sampling noise. The narrowest intervals (study 8 at 0.0405 standard error, study 6 at 0.0831) dominate the random-effects weighting, carrying 10.22% and 10.12% respectively.
The detail
Each study is plotted at its log risk ratio estimate with 95% confidence interval whiskers. Studies are ordered by effect size. Study 12 has the widest interval (ci_low -0.9843, ci_high 1.8762, weight 3.8%) and study 8 the narrowest (ci_low -0.1114, ci_high 0.1353, weight 10.22%). The fixed-effect pooled estimate sits at -0.4303 (95% CI -0.5097 to -0.3509); the random-effects estimate at -0.7141 (95% CI -1.0644 to -0.3638). With I-squared at 92.12%, the studies are not scattered around one value; the pooled row summarizes a distribution rather than a better measurement of a single number.
What this can't tell you
The wide scatter and high I-squared mean the pooled estimate is an average across different study contexts, not a universal effect size. Individual study intervals crossing zero (studies 12, 1, 3) indicate those studies alone could not rule out no effect.
Pooled Estimates
Fixed-effect and random-effects results side by side.
| Model | Estimate | Std Error | CI Low | CI High | P Value | Interpretation |
|---|---|---|---|---|---|---|
| Fixed-effect (inverse-variance) | -0.4303 | 0.0405 | -0.5097 | -0.3509 | < 0.001 | Assumes every study estimates the SAME effect and all variation is sampling error. Weights are 1 divided by the study variance, so precise studies dominate. That assumption is contradicted here by the heterogeneity, so this number understates the real uncertainty. |
| Random-effects (DerSimonian-Laird) | -0.7141 | 0.1787 | -1.064 | -0.3638 | < 0.001 | Allows the true effect to differ across studies by tau-squared = 0.3088, which flattens the weights and widens the interval. This is the estimate to quote, but as an average across differing effects, not as one true effect. |
The short answer
The random-effects estimate of -0.7141 (95% CI -1.0644 to -0.3638) is the one to report because tau-squared = 0.3088 indicates the studies estimate different effects. The fixed-effect estimate of -0.4303 has a narrower interval but reports false precision.
The detail
Fixed-effect: estimate -0.4303, standard error 0.0405, 95% CI -0.5097 to -0.3509, interval width 0.159, p < 0.001. Random-effects: estimate -0.7141, standard error 0.1787, 95% CI -1.0644 to -0.3638, interval width 0.701, p < 0.001. The estimates differ by 0.284. Both exclude zero. The random-effects interval is 4.4 times wider because it admits between-study variance; the fixed-effect model assumes all variation is sampling error, contradicted by the heterogeneity. Neither model corrects bias shared by the studies.
What this can't tell you
If the 13 studies share a systematic flaw—measurement error, selection bias, or confounding—both pooled estimates inherit it. The confidence interval around a biased average is still narrow.
Heterogeneity
How much the studies disagree, and what that does to the pooled estimate.
| Statistic | Value | Meaning |
|---|---|---|
| Cochran's Q | 152.233 | Total weighted dispersion of the study estimates around the fixed-effect pooled value. |
| Degrees of freedom | 12 | Number of studies minus one (13 studies pooled). |
| Q p-value | < 0.001 | Probability of dispersion this large if every study shared one true effect. Q is known to have low power at small study counts, so a non-significant Q with 13 studies does not establish homogeneity. |
| I-squared (%) | 92.12 | Share of the total variation that exceeds sampling error: 92.12% here, which is considerable by the conventional bands. |
| H-squared | 12.686 | Ratio of total variation to sampling variation. One means no excess heterogeneity. |
| Tau-squared | 0.3088 | Estimated variance of the true effects across studies (DerSimonian-Laird). Zero means no detectable between-study variance. |
| Tau | 0.556 | The standard deviation of the true effects, on the same scale as the effect itself. |
| 95% prediction interval | -1.999 to 0.571 | The range a NEW comparable study's true effect would fall in about 95% of the time. This is the honest width of the finding, and it is always wider than the confidence interval around the average. |
The short answer
The 13 studies are highly heterogeneous: 92.12% of the variation between them exceeds sampling error, so the random-effects estimate of -0.7141 is an average across genuinely different study effects, not an estimate of one shared effect. The 95% prediction interval (-1.999 to 0.571) is much wider and the honest range for what a new study would find.
The detail
Cochran's Q = 152.233 on 12 degrees of freedom (p < 0.001), I-squared = 92.12% (considerable by convention), tau-squared = 0.3088, tau = 0.556, H-squared = 12.686. The prediction interval (-1.999 to 0.571) is the range a new comparable study's true effect would fall in about 95% of the time. It is always wider than the confidence interval around the average (-1.0644 to -0.3638) and here includes zero, meaning a future study could plausibly find no protective effect or even harm.
What this can't tell you
Q has low power with 13 studies, so a non-significant Q would be weak evidence of homogeneity rather than proof. The I-squared estimate itself carries uncertainty not shown here. The high heterogeneity means the pooled estimate describes an average context, not a universal effect.
Subgroup Pooling
Does the effect hold across the groups, or only in some of them?
| Subgroup | Studies | Estimate | CI Low | CI High | I Squared | Interpretation |
|---|---|---|---|---|---|---|
| All studies | 13 | -0.7141 | -1.064 | -0.3638 | 92.1 | 13 study/studies pooled to -0.714 (95% CI -1.064 to -0.364); within-group I-squared 92.10%. |
The short answer
No subgroup comparison was performed because no grouping column with two or more levels was mapped to the data. The analysis reports only the single overall pool of 13 studies.
The detail
The table shows all 13 studies pooled together: random-effects estimate −0.714 (95% CI −1.064 to −0.364), within-group I-squared 92.10%. To test whether the vaccination effect differs across trial design, population, region, or other characteristics, a grouping column holding at least two levels with two or more studies each must be mapped. That comparison is often the first response to high heterogeneity (I-squared = 92.12% here), and it would reveal whether the protective effect is consistent across populations or concentrated in some subgroups.
What this can't tell you
Without stratification, the analysis cannot determine whether the large heterogeneity reflects real differences in the effect of vaccination across populations or settings. A subgroup comparison would locate where the effect differs but not explain why.
Funnel Plot
Effect against precision — the shape that publication bias distorts.
The short answer
No significant funnel asymmetry is detected (Egger's p = 0.189). The 13 studies scatter across the plot with no obvious gap in one bottom corner—the classic small-study signature. Reading a funnel by eye is unreliable, which is why the regression test is decisive here.
The detail
Each point is one study: its log risk ratio effect on the horizontal axis against its precision (1 divided by standard error) on the vertical. Precise studies sit at the top and should cluster tightly; imprecise studies sit at the bottom and should scatter symmetrically. Egger's regression intercept is -2.112 (p = 0.189). Study 8 (effect 0.012, precision 15.8879) and study 6 (effect -0.7861, precision 12.0337) are the most precise. Study 12 (effect 0.4459, precision 1.3704) and study 3 (effect -1.3481, precision 1.5516) are the least precise. The vertical reference line marks zero.
What this can't tell you
At 13 studies, Egger's test detects only one specific pattern—effects varying systematically with their own standard error—and has limited power. Funnel asymmetry has causes other than publication bias, including real differences between small and large studies. The test cannot rule out selective reporting of outcomes within studies.
Small-Study Effects
Egger's regression test for funnel asymmetry, with its power stated honestly.
| Quantity | Value | Meaning |
|---|---|---|
| Egger intercept (bias) | -2.112 | Regression of each study's standardised effect on its precision. An intercept away from zero means small (imprecise) studies report systematically different effects from large ones. |
| Standard error of the intercept | 1.507 | How well determined that intercept is; small study counts make it large. |
| t statistic | -1.401 | The intercept divided by its standard error. |
| p-value | 0.189 | Two-sided test that the intercept is zero. The test uses all 13 studies. Even at this size it detects only one specific pattern — estimates that vary systematically with their own standard error — and asymmetry has causes other than publication bias, including real differences between small and large studies. |
| Studies in the test | 13 | Every one of the 13 pooled studies enters the regression. |
| Bias-adjusted slope | -0.191 | The slope estimates what the pooled effect would be for a study of infinite precision. It is a crude correction, not a bias-free estimate. |
The short answer
Egger's regression finds no significant funnel asymmetry (intercept −2.112, p = 0.189), so no systematic difference between small and large studies is detected. However, at 13 studies this test has low power and cannot rule out small-study effects.
The detail
Egger's intercept (bias) is −2.112 with standard error 1.507, t-statistic −1.401, p-value 0.189 (two-sided). The test uses all 13 studies. The bias-adjusted slope is −0.191, a crude correction estimate. The test detects only one specific pattern: estimates that vary systematically with their own standard error. Asymmetry has causes other than publication bias, including real differences between small and large studies (for example, different trial designs, population characteristics, or follow-up duration).
What this can't tell you
Even at this study count, the test detects only one pattern and cannot see studies that were never written down. A symmetric funnel is consistent with a literature where an entire class of null results—say, all trials in low-incidence settings—is missing uniformly. The p-value of 0.189 is not evidence that small-study effects are absent; it reflects low power at 13 studies.
Methods & Disclosure
Every formula, and the specific things pooling cannot fix.
| Item | Detail |
|---|---|
| Input shape | One row per study: the effect estimate in 'log risk ratio', the study label in 'study', and its precision. 13 of 13 rows were usable. |
| Precision source | Standard errors came from the reported standard error in 'std error'. |
| Fixed-effect model | Inverse-variance weights (1 divided by the squared standard error): pooled estimate -0.430, standard error 0.040, 95% CI -0.510 to -0.351. |
| Random-effects model | DerSimonian-Laird moment estimator for tau-squared (0.3088), weights 1 divided by (variance plus tau-squared): pooled estimate -0.714, standard error 0.179, 95% CI -1.064 to -0.364. |
| Heterogeneity | Cochran's Q = 152.233 on 12 degrees of freedom (p < 0.001); I-squared = 92.12%; H-squared = 12.686. |
| Prediction interval | Random-effects estimate plus or minus the 97.5th percentile of a t distribution on 11 degrees of freedom times the square root of (tau-squared plus the squared standard error of the pooled estimate). |
| Subgroup test | No between-subgroup test was run: a grouping column with at least two levels of two or more studies each was not mapped. |
| Small-study effects | Egger's regression of the standardised effect on precision; intercept -2.112 (p = 0.189). The test uses all 13 studies. Even at this size it detects only one specific pattern — estimates that vary systematically with their own standard error — and asymmetry has causes other than publication bias, including real differences between small and large studies. |
| Null value | The pooled effect is tested against a null value of zero. That is correct for mean differences, standardised mean differences and log-transformed ratios. Ratio measures such as odds ratios or risk ratios must be log-transformed BEFORE they are pooled, otherwise both the weighting and this test are wrong. |
| What pooling cannot fix | Pooling averages the studies it is given. It cannot repair selective reporting, cannot detect that the studies measured subtly different things, and cannot turn observational comparisons into causal ones — a pooled association is still an association. |
The short answer
All quantities are closed-form: inverse-variance weights for the fixed-effect model, DerSimonian-Laird moment estimation for tau-squared (0.3088), Cochran's Q against chi-square on 12 degrees of freedom, and Egger's ordinary regression. Standard errors came from the reported 'std error' column; 13 of 13 rows were usable. The analysis describes what these 13 studies reported when combined; it cannot verify they measured the same construct or make an association causal.
The detail
Fixed-effect pooled estimate -0.430 (95% CI -0.510 to -0.351, standard error 0.040). Random-effects pooled estimate -0.714 (95% CI -1.064 to -0.364, standard error 0.179). Heterogeneity: Cochran's Q = 152.233 on 12 degrees of freedom (p < 0.001), I-squared = 92.12%. Prediction interval calculated as random-effects estimate ± (97.5th percentile of t on 11 degrees of freedom) × √(tau-squared + squared standard error of pooled estimate). Egger's intercept -2.112 (p = 0.189). No subgroup test was run: no grouping column with two or more levels of two or more studies each was mapped.
What this can't tell you
Pooling raw ratio measures instead of log-transformed ones breaks both the weighting and the test. Pooling cannot repair selective reporting, detect that studies measured different constructs, or establish causation from observational comparisons.
Meta-Analysis & Forest Plot
Pools per-study effect sizes into one estimate. Each row is one study carrying an effect estimate and its standard error (or a 95% confidence interval the analysis inverts into a standard error). The analysis fits BOTH the fixed-effect (inverse-variance) and the random-effects (DerSimonian-Laird) model, reports the heterogeneity diagnostics that decide which of the two means anything (Cochran's Q, I-squared, tau-squared), draws the forest plot, and checks the funnel for small-study effects with Egger's regression.
Why This Method?
A single study is one draw from a noisy process. Pooling weights each study by its precision, so the combined estimate is tighter than any one study — but only if the studies are estimating the same thing. The heterogeneity diagnostics are what tell you whether that premise holds, which is why they are reported next to the pooled number rather than buried underneath it.
What This Analysis Covers
- Forest plot: every study's effect and confidence interval, plus both pooled estimates
- Fixed-effect (inverse-variance) and random-effects (DerSimonian-Laird) models
- Heterogeneity: Cochran's Q, I-squared, tau-squared, and a prediction interval
- Subgroup pooling with a between-subgroup Q test when a grouping column is mapped
- Funnel plot and Egger's regression for small-study effects, with an honest
statement of how little power that test has at the study count in hand
Standard Library
Platform standard-library module (LAT-1441): runs on ANY dataset via the semantic mapping {study, effect, std_error | ci_low + ci_high, subgroup}. All narrative is derived from the user's own column names and computed values.
suppressPackageStartupMessages(library(DT))
suppressPackageStartupMessages(library(htmlwidgets))
suppressPackageStartupMessages(library(arrow))
suppressPackageStartupMessages(library(knitr))
suppressPackageStartupMessages(library(rmarkdown))
suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(tidyr))
suppressPackageStartupMessages(library(ggplot2))
suppressPackageStartupMessages(library(stringr))
suppressPackageStartupMessages(library(lubridate))
suppressPackageStartupMessages(library(broom))
suppressPackageStartupMessages(library(Matrix))
suppressPackageStartupMessages(library(cluster))
suppressPackageStartupMessages(library(data.table))Core Analysis Pipeline
compute_shared <- function(df, params, col_map = list()) {
# === SHARED EXPORTS ===
# initial_rows/final_rows/rows_removed $ row accounting
# k $ studies pooled
# study_h/effect_h/se_h/sub_h $ humanized user column names
# se_source $ where each study's standard error came from
# drop_df $ data.frame(reason, studies) — excluded rows
# fit $ pool_iv() result for ALL studies
# pi_lo/pi_hi $ 95% prediction interval for a new study
# het_word/het_band $ computed heterogeneity language (Cochrane bands)
# re_is_average $ TRUE when I-squared >= 50 — the RE estimate then
# describes a DISTRIBUTION, not one true effect
# egger_* $ Egger regression intercept/se/t/p + power verdict
# forest_df $ study + effect + ci_low + ci_high + std_error + weight_pct
# funnel_df $ effect + precision (+ study label)
# models_df $ the two pooled models side by side
# het_df $ heterogeneity statistics table
# subgroup_df $ per-subgroup pooling (or one "All studies" row)
# egger_df $ small-study-effect table
# methods_df $ full disclosure table
# metrics / json_output
# === /SHARED EXPORTS ===Step 1: Resolve the mapped columns (humanized for every sentence)
initial_rows <- nrow(df)
study_h <- humanize_semantic("study", col_map)
effect_h <- humanize_semantic("effect", col_map)
se_h <- humanize_semantic("std_error", col_map)
lo_h <- humanize_semantic("ci_low", col_map)
hi_h <- humanize_semantic("ci_high", col_map)
sub_h <- humanize_semantic("subgroup", col_map)
has <- function(nm) nm %in% names(df)
if (!has("effect")) {
stop(sprintf("Meta-analysis needs a column of per-study effect estimates mapped as the effect('%s' was not found in the data).", effect_h))
}Step 2: Coerce the effect column (95% rule — refuse, never guess)
coerce_strict <- function(v, label_h, what) {
if (is.numeric(v)) return(as.numeric(v))
ch <- as.character(v)
non_blank <- !is.na(ch) & trimws(ch) != ""
conv <- suppressWarnings(as.numeric(ch))
if (sum(non_blank) == 0 ||
sum(!is.na(conv[non_blank])) < 0.95 * sum(non_blank)) {
stop(sprintf("The column '%s' mapped as %s does not look numeric — fewer than 95%% of its values parse as numbers. Map a numeric column.",
label_h, what))
}
conv
}
y_all <- coerce_strict(df$effect, effect_h, "the effect estimate")Step 3: Standard errors — reported, or inverted from the interval
z95 <- stats::qnorm(0.975)
has_se <- has("std_error")
has_ci <- has("ci_low") && has("ci_high")
se_from_ci <- NULL
if (has_ci) {
lo_v <- coerce_strict(df$ci_low, lo_h, "the lower confidence limit")
hi_v <- coerce_strict(df$ci_high, hi_h, "the upper confidence limit")
se_from_ci <- (hi_v - lo_v) / (2 * z95)
se_from_ci[!is.na(se_from_ci) & se_from_ci <= 0] <- NA_real_
}
n_filled <- 0L
if (has_se) {
se_all <- coerce_strict(df$std_error, se_h, "the standard error")
if (!is.null(se_from_ci)) {
fill <- is.na(se_all) & !is.na(se_from_ci)
n_filled <- sum(fill)
se_all[fill] <- se_from_ci[fill]
}
se_source <- if (n_filled > 0)
sprintf("the reported standard error in '%s', with %d study standard error(s) recovered from the '%s' to '%s' interval",
se_h, n_filled, lo_h, hi_h)
else
sprintf("the reported standard error in '%s'", se_h)
} else if (!is.null(se_from_ci)) {
se_all <- se_from_ci
se_source <- sprintf("the reported 95%% confidence interval('%s' to '%s'), inverted as half-width divided by %s",
lo_h, hi_h, r2(z95))
} else {
stop(sprintf("Meta-analysis needs each study's precision: map either a standard-error column, or both confidence-limit columns, alongside '%s'.",
effect_h))
}Step 4: Study labels — never dropped, blanks filled, duplicates suffixed
labels_all <- if (has("study")) trimws(as.character(df$study)) else rep(NA_character_, initial_rows)
auto_lab <- paste0("Study ", seq_len(initial_rows))
blank_lab <- is.na(labels_all) | labels_all == ""
labels_all[blank_lab] <- auto_lab[blank_lab]
labels_all <- make.unique(labels_all, sep = " #")Step 5: Usable rows — an effect AND a strictly positive standard error
bad_effect <- !is.finite(y_all)
bad_se_missing <- !bad_effect & !is.finite(se_all)
bad_se_nonpos <- !bad_effect & is.finite(se_all) & se_all <= 0
keep <- !bad_effect & !bad_se_missing & !bad_se_nonpos
drop_rows <- list()
if (sum(bad_effect) > 0)
drop_rows[[length(drop_rows) + 1]] <- data.frame(
reason = sprintf("No usable %s value", effect_h),
studies = sum(bad_effect), stringsAsFactors = FALSE)
if (sum(bad_se_missing) > 0)
drop_rows[[length(drop_rows) + 1]] <- data.frame(
reason = "No usable standard error(and none recoverable from a confidence interval)",
studies = sum(bad_se_missing), stringsAsFactors = FALSE)
if (sum(bad_se_nonpos) > 0)
drop_rows[[length(drop_rows) + 1]] <- data.frame(
reason = "Standard error is zero or negative, so the study would carry infinite weight",
studies = sum(bad_se_nonpos), stringsAsFactors = FALSE)
drop_df <- if (length(drop_rows) > 0) do.call(rbind, drop_rows) else
data.frame(reason = "None — every row was usable",
studies = 0L, stringsAsFactors = FALSE)
rownames(drop_df) <- NULL
y <- y_all[keep]; se <- se_all[keep]; labs <- labels_all[keep]
k <- length(y)
final_rows <- k
rows_removed <- initial_rows - final_rows
n_dropped <- rows_removed
if (k < 3) {
stop(sprintf("Meta-analysis needs at least 3 studies with both a usable '%s' value and a usable standard error; only %d of the %d rows qualified (studies identified by '%s'). Pooling two studies produces a number, but Cochran's Q, I-squared and any funnel check are meaningless at that size.",
effect_h, k, initial_rows, study_h))
}Step 6: Pool — fixed-effect and DerSimonian-Laird random-effects
fit <- pool_iv(y, se)Step 7: Prediction interval for the effect in a NEW study
pi_lo <- NA_real_; pi_hi <- NA_real_
if (k >= 3) {
tq <- stats::qt(0.975, df = k - 2)
spread <- sqrt(fit$tau2 + fit$se_re^2)
pi_lo <- fit$theta_re - tq * spread
pi_hi <- fit$theta_re + tq * spread
}Step 8: Heterogeneity language — Cochrane bands, computed
het_band <- if (fit$i2 < 25) "low" else if (fit$i2 < 50) "moderate" else
if (fit$i2 < 75) "substantial" else "considerable"
re_is_average <- fit$i2 >= 50
het_word <- sprintf("I-squared of %s%% (%s heterogeneity)", r2(fit$i2), het_band)
models_agree <- abs(fit$theta_fe - fit$theta_re) < 0.05 * max(1e-9, abs(fit$theta_re))The single most important honest sentence this module can produce.
het_caveat <- if (re_is_average) {
sprintf("Because %s of the variation across studies is more than sampling error can explain, the random-effects estimate of %s is the AVERAGE of a distribution of genuinely different study effects — it is not an estimate of one true effect that every study shares. The 95%% prediction interval(%s to %s) is the honest range for what a new comparable study would find, and it is much wider than the confidence interval around the average.",
paste0(r2(fit$i2), "%"), r3(fit$theta_re), r3(pi_lo), r3(pi_hi))
} else {
sprintf("Heterogeneity is %s(I-squared %s%%, %s), so the studies are consistent with estimating a common effect and the pooled number can be read as an estimate of it.",
het_band, r2(fit$i2), fmt_pp(fit$p_q))
}Step 9: Egger's regression for small-study effects (funnel asymmetry)
precision <- 1 / se
egger_b0 <- NA_real_; egger_b0_se <- NA_real_; egger_t <- NA_real_
egger_p <- NA_real_; egger_slope <- NA_real_
egger_ok <- k >= 3 && isTRUE(stats::var(precision) > 0)
if (egger_ok) {
snd <- y / se
efit <- tryCatch(stats::lm(snd ~ precision), error = function(e) NULL)
if (!is.null(efit)) {
ecf <- tryCatch(summary(efit)$coefficients, error = function(e) NULL)
if (!is.null(ecf) && nrow(ecf) == 2) {
egger_b0 <- ecf[1, 1]; egger_b0_se <- ecf[1, 2]
egger_t <- ecf[1, 3]; egger_p <- ecf[1, 4]
egger_slope <- ecf[2, 1]
}
}
}
egger_sig <- !is.na(egger_p) && egger_p < 0.05
egger_underpowered <- k < 10
egger_verdict <- if (is.na(egger_p)) "not testable" else
if (egger_sig) "detected" else "not detected"The power statement always names the actual study count — a null Egger result on a handful of studies is not evidence of no small-study effect.
egger_power_note <- if (is.na(egger_p)) {
sprintf("Egger's test could not be computed: all %d studies report the same precision, so the funnel has no spread to regress against.", k)
} else if (egger_underpowered) {
sprintf("With only %d studies, Egger's test has almost no power — the standard guidance is not to interpret funnel asymmetry below 10 studies. A p-value of %s here is close to uninformative, and the absence of a significant result is NOT evidence that small-study effects are absent.",
k, fmt_p(egger_p))
} else {
sprintf("The test uses all %d studies. Even at this size it detects only one specific pattern — estimates that vary systematically with their own standard error — and asymmetry has causes other than publication bias, including real differences between small and large studies.",
k)
}Step 10: Subgroup pooling (only when a grouping column is mapped)
sub_ok <- has("subgroup")
sub_levels_note <- ""
q_bet <- NA_real_; q_bet_df <- NA_integer_; q_bet_p <- NA_real_
n_groups <- 0L
if (sub_ok) {
g_all <- trimws(as.character(df$subgroup))[keep]
g_all[is.na(g_all) | g_all == ""] <- "Missing"
tab <- sort(table(g_all), decreasing = TRUE)Lump past 8 levels, then lump anything with fewer than 2 studies.
keep_lv <- names(tab)[seq_len(min(8, length(tab)))]
g <- ifelse(g_all %in% keep_lv, g_all, "Other")
tab2 <- table(g)
sparse <- names(tab2)[tab2 < 2]
if (length(sparse) > 0) g <- ifelse(g %in% sparse, "Other(fewer than 2 studies)", g)
lumped <- sum(!(g_all %in% keep_lv)) + sum(g == "Other(fewer than 2 studies)")
if (lumped > 0) {
sub_levels_note <- sprintf(" %d study/studies fell into levels of '%s' that were too small or too numerous to pool separately and were grouped together.",
lumped, sub_h)
}
gl <- names(sort(table(g), decreasing = TRUE))
gl <- gl[sapply(gl, function(x) sum(g == x) >= 2)]
n_groups <- length(gl)
if (n_groups >= 1) {
rows <- lapply(gl, function(lv) {
idx <- which(g == lv)
f <- pool_iv(y[idx], se[idx])
data.frame(
subgroup = lv, studies = length(idx),
estimate = round(f$theta_re, 4),
ci_low = round(f$re_lo, 4), ci_high = round(f$re_hi, 4),
i_squared = round(f$i2, 1),
se_g = f$se_re,
stringsAsFactors = FALSE)
})
subgroup_df <- do.call(rbind, rows)
} else {
subgroup_df <- NULL
}
if (n_groups >= 2) {
wg <- 1 / (subgroup_df$se_g^2)
theta_bar <- sum(wg * subgroup_df$estimate) / sum(wg)
q_bet <- sum(wg * (subgroup_df$estimate - theta_bar)^2)
q_bet_df <- n_groups - 1L
q_bet_p <- stats::pchisq(q_bet, df = q_bet_df, lower.tail = FALSE)
}
} else {
subgroup_df <- NULL
}
if (is.null(subgroup_df)) {
subgroup_df <- data.frame(
subgroup = "All studies", studies = k,
estimate = round(fit$theta_re, 4),
ci_low = round(fit$re_lo, 4), ci_high = round(fit$re_hi, 4),
i_squared = round(fit$i2, 1), se_g = fit$se_re,
stringsAsFactors = FALSE)
n_groups <- 1L
}
subgroup_df$interpretation <- vapply(seq_len(nrow(subgroup_df)), function(i) {
r <- subgroup_df[i, ]
sprintf("%d study/studies pooled to %s(95%% CI %s to %s); within-group I-squared %s%%.",
r$studies, r3(r$estimate), r3(r$ci_low), r3(r$ci_high), r2(r$i_squared))
}, character(1))
subgroup_df$se_g <- NULL
rownames(subgroup_df) <- NULL
sub_differs <- !is.na(q_bet_p) && q_bet_p < 0.05Step 11: Forest dataset — studies (capped) plus both pooled rows
max_plot <- 50L
ord <- order(-y)
plot_idx <- ord
forest_note <- ""
if (k > max_plot) {Keep the most precise studies for a readable plot; the pooled estimates below still use every study.
keep_pl <- order(se)[seq_len(max_plot)]
plot_idx <- keep_pl[order(-y[keep_pl])]
forest_note <- sprintf(" The plot shows the %d most precise of the %d studies for readability; both pooled estimates use all %d.",
max_plot, k, k)
}
w_pct <- 100 * fit$w_re / sum(fit$w_re)
forest_df <- data.frame(
study = labs[plot_idx],
effect = round(y[plot_idx], 4),
ci_low = round(y[plot_idx] - z95 * se[plot_idx], 4),
ci_high = round(y[plot_idx] + z95 * se[plot_idx], 4),
std_error = round(se[plot_idx], 4),
weight_pct = round(w_pct[plot_idx], 2),
stringsAsFactors = FALSE)
pooled_rows <- data.frame(
study = c(sprintf("Pooled — fixed-effect(k = %d)", k),
sprintf("Pooled — random-effects(k = %d)", k)),
effect = round(c(fit$theta_fe, fit$theta_re), 4),
ci_low = round(c(fit$fe_lo, fit$re_lo), 4),
ci_high = round(c(fit$fe_hi, fit$re_hi), 4),
std_error = round(c(fit$se_fe, fit$se_re), 4),
weight_pct = c(NA_real_, NA_real_),
stringsAsFactors = FALSE)
forest_df <- rbind(forest_df, pooled_rows)
rownames(forest_df) <- NULLStep 12: Funnel dataset — effect against precision (1 / standard error)
set.seed(42)
fidx <- if (k > 1000) sample(k, 1000) else seq_len(k)
funnel_df <- data.frame(
effect = round(y[fidx], 4),
precision = round(precision[fidx], 4),
study = labs[fidx],
stringsAsFactors = FALSE)
funnel_df <- funnel_df[order(funnel_df$effect), , drop = FALSE]
rownames(funnel_df) <- NULLStep 13: Model comparison table
models_df <- data.frame(
model = c("Fixed-effect(inverse-variance)", "Random-effects(DerSimonian-Laird)"),
estimate = round(c(fit$theta_fe, fit$theta_re), 4),
std_error = round(c(fit$se_fe, fit$se_re), 4),
ci_low = round(c(fit$fe_lo, fit$re_lo), 4),
ci_high = round(c(fit$fe_hi, fit$re_hi), 4),
p_value = c(fmt_p(fit$fe_p), fmt_p(fit$re_p)),
interpretation = c(
sprintf("Assumes every study estimates the SAME effect and all variation is sampling error. Weights are 1 divided by the study variance, so precise studies dominate. %s",
if (re_is_average)
"That assumption is contradicted here by the heterogeneity, so this number understates the real uncertainty."
else
"The heterogeneity statistics are consistent with that assumption here."),
sprintf("Allows the true effect to differ across studies by tau-squared = %s, which flattens the weights and widens the interval. %s",
r4(fit$tau2),
if (fit$tau2 <= 0)
"Here tau-squared came out at zero, so this model collapses onto the fixed-effect one and the two rows agree by construction."
else if (re_is_average)
"This is the estimate to quote, but as an average across differing effects, not as one true effect."
else
"The two models agree closely, which is what low heterogeneity implies.")
),
stringsAsFactors = FALSE)Step 14: Heterogeneity table
het_df <- data.frame(
statistic = c("Cochran's Q", "Degrees of freedom", "Q p-value",
"I-squared(%)", "H-squared", "Tau-squared", "Tau",
"95% prediction interval"),
value = c(r3(fit$q), as.character(fit$df), fmt_p(fit$p_q),
r2(fit$i2), r3(fit$h2), r4(fit$tau2), r3(fit$tau),
if (is.na(pi_lo)) "not available" else paste0(r3(pi_lo), " to ", r3(pi_hi))),
meaning = c(
"Total weighted dispersion of the study estimates around the fixed-effect pooled value.",
sprintf("Number of studies minus one(%d studies pooled).", k),
sprintf("Probability of dispersion this large if every study shared one true effect. Q is known to have low power at small study counts, so a non-significant Q with %d studies does not establish homogeneity.", k),
sprintf("Share of the total variation that exceeds sampling error: %s%% here, which is %s by the conventional bands.", r2(fit$i2), het_band),
"Ratio of total variation to sampling variation. One means no excess heterogeneity.",
"Estimated variance of the true effects across studies(DerSimonian-Laird). Zero means no detectable between-study variance.",
"The standard deviation of the true effects, on the same scale as the effect itself.",
"The range a NEW comparable study's true effect would fall in about 95% of the time. This is the honest width of the finding, and it is always wider than the confidence interval around the average."
),
stringsAsFactors = FALSE)Step 15: Small-study-effect table
egger_df <- data.frame(
quantity = c("Egger intercept(bias)", "Standard error of the intercept",
"t statistic", "p-value", "Studies in the test",
"Bias-adjusted slope"),
value = c(r3(egger_b0), r3(egger_b0_se), r3(egger_t), fmt_p(egger_p),
as.character(k), r3(egger_slope)),
meaning = c(
"Regression of each study's standardised effect on its precision. An intercept away from zero means small (imprecise) studies report systematically different effects from large ones.",
"How well determined that intercept is; small study counts make it large.",
"The intercept divided by its standard error.",
sprintf("Two-sided test that the intercept is zero. %s", egger_power_note),
sprintf("Every one of the %d pooled studies enters the regression.", k),
"The slope estimates what the pooled effect would be for a study of infinite precision. It is a crude correction, not a bias-free estimate."
),
stringsAsFactors = FALSE)Step 16: Methods disclosure
methods_df <- data.frame(
item = c("Input shape", "Precision source", "Fixed-effect model",
"Random-effects model", "Heterogeneity", "Prediction interval",
"Subgroup test", "Small-study effects", "Null value",
"What pooling cannot fix"),
detail = c(
sprintf("One row per study: the effect estimate in '%s', the study label in '%s', and its precision. %d of %d rows were usable.",
effect_h, study_h, k, initial_rows),
sprintf("Standard errors came from %s.", se_source),
sprintf("Inverse-variance weights(1 divided by the squared standard error): pooled estimate %s, standard error %s, 95%% CI %s to %s.",
r3(fit$theta_fe), r3(fit$se_fe), r3(fit$fe_lo), r3(fit$fe_hi)),
sprintf("DerSimonian-Laird moment estimator for tau-squared(%s), weights 1 divided by(variance plus tau-squared): pooled estimate %s, standard error %s, 95%% CI %s to %s.",
r4(fit$tau2), r3(fit$theta_re), r3(fit$se_re), r3(fit$re_lo), r3(fit$re_hi)),
sprintf("Cochran's Q = %s on %d degrees of freedom (%s); I-squared = %s%%; H-squared = %s.",
r3(fit$q), fit$df, fmt_pp(fit$p_q), r2(fit$i2), r3(fit$h2)),
sprintf("Random-effects estimate plus or minus the 97.5th percentile of a t distribution on %d degrees of freedom times the square root of(tau-squared plus the squared standard error of the pooled estimate).",
max(1, k - 2)),
if (!is.na(q_bet))
sprintf("Between-subgroup Q on the '%s' column = %s on %d degrees of freedom (%s), comparing the random-effects estimate of each subgroup.",
sub_h, r3(q_bet), q_bet_df, fmt_pp(q_bet_p))
else
"No between-subgroup test was run: a grouping column with at least two levels of two or more studies each was not mapped.",
sprintf("Egger's regression of the standardised effect on precision; intercept %s (%s). %s",
r3(egger_b0), fmt_pp(egger_p), egger_power_note),
"The pooled effect is tested against a null value of zero. That is correct for mean differences, standardised mean differences and log-transformed ratios. Ratio measures such as odds ratios or risk ratios must be log-transformed BEFORE they are pooled, otherwise both the weighting and this test are wrong.",
"Pooling averages the studies it is given. It cannot repair selective reporting, cannot detect that the studies measured subtly different things, and cannot turn observational comparisons into causal ones — a pooled association is still an association."
),
stringsAsFactors = FALSE)Step 17: Metrics + one-paragraph computed answer
dir_word <- if (fit$theta_re > 0) "positive" else if (fit$theta_re < 0) "negative" else "exactly zero"
re_sig <- (fit$re_lo > 0) || (fit$re_hi < 0)
sig_clause <- if (re_sig)
"the interval excludes zero, so the pooled effect is distinguishable from no effect"
else
"the interval includes zero, so the pooled effect is not distinguishable from no effect"
metrics <- list(
`Studies Pooled` = k,
`Fixed-Effect Estimate` = round(fit$theta_fe, 4),
`Random-Effects Estimate` = round(fit$theta_re, 4),
`Random-Effects 95% CI` = paste0(r3(fit$re_lo), " to ", r3(fit$re_hi)),
`I-squared(%)` = round(fit$i2, 1),
`Tau-squared` = round(fit$tau2, 4),
`Cochran Q p-value` = fmt_p(fit$p_q),
`Prediction Interval` = if (is.na(pi_lo)) "not available" else
paste0(r3(pi_lo), " to ", r3(pi_hi)),
`Small-Study Effect` = egger_verdict,
`Egger p-value` = fmt_p(egger_p)
)
sub_clause <- if (sub_differs) {
sprintf(" Pooling by '%s' shows the subgroups do NOT share one effect (between-group Q = %s on %d degrees of freedom, %s), so the single pooled number averages across real differences.",
sub_h, r3(q_bet), q_bet_df, fmt_pp(q_bet_p))
} else if (!is.na(q_bet_p)) {
sprintf(" Pooling by '%s' gives no evidence that the subgroups differ (between-group Q = %s, %s).",
sub_h, r3(q_bet), fmt_pp(q_bet_p))
} else ""
egger_clause <- if (egger_sig) {
sprintf(" Egger's regression finds funnel asymmetry (intercept %s, %s): smaller studies report systematically %s effects, which is what publication bias looks like — though real differences between small and large studies produce the same pattern.",
r3(egger_b0), fmt_pp(egger_p),
if (!is.na(egger_b0) && egger_b0 > 0) "larger" else "smaller")
} else if (!is.na(egger_p)) {
sprintf(" Egger's regression finds no significant funnel asymmetry (intercept %s, %s), but %s",
r3(egger_b0), fmt_pp(egger_p),
if (egger_underpowered)
sprintf("with only %d studies that test is close to powerless and cannot rule small-study effects out.", k)
else
"asymmetry tests only detect one specific pattern and cannot rule publication bias out.")
} else ""
json_output <- list(
answer = paste0(
"Pooling ", k, " studies on '", effect_h, "': the random-effects (DerSimonian-Laird) estimate is ",
r3(fit$theta_re), " (95% CI ", r3(fit$re_lo), " to ", r3(fit$re_hi), "), and ", sig_clause,
". The fixed-effect estimate is ", r3(fit$theta_fe), " (95% CI ", r3(fit$fe_lo), " to ", r3(fit$fe_hi),
"). Heterogeneity: Cochran's Q = ", r3(fit$q), " on ", fit$df, " degrees of freedom (", fmt_pp(fit$p_q),
"), I-squared = ", r2(fit$i2), "% (", het_band, "), tau-squared = ", r4(fit$tau2), ". ",
het_caveat, sub_clause, egger_clause
),
cards = lapply(
c("tldr", "overview", "preprocessing", "forest_plot", "pooled_models",
"heterogeneity", "subgroup_pooling", "funnel_plot",
"small_study_effects", "methods"),
function(cid) list(id = cid, metrics = metrics)
)
)
list(
initial_rows = initial_rows, final_rows = final_rows,
rows_removed = rows_removed, n_dropped = n_dropped, k = k,
study_h = study_h, effect_h = effect_h, se_h = se_h, sub_h = sub_h,
se_source = se_source, n_filled = n_filled,
has_subgroup = sub_ok, sub_levels_note = sub_levels_note,
drop_df = drop_df, fit = fit, pi_lo = pi_lo, pi_hi = pi_hi,
het_band = het_band, het_word = het_word, het_caveat = het_caveat,
re_is_average = re_is_average, models_agree = models_agree,
dir_word = dir_word, re_sig = re_sig, sig_clause = sig_clause,
egger_b0 = egger_b0, egger_b0_se = egger_b0_se, egger_t = egger_t,
egger_p = egger_p, egger_slope = egger_slope, egger_sig = egger_sig,
egger_underpowered = egger_underpowered, egger_verdict = egger_verdict,
egger_power_note = egger_power_note,
q_bet = q_bet, q_bet_df = q_bet_df, q_bet_p = q_bet_p,
n_groups = n_groups, sub_differs = sub_differs,
forest_df = forest_df, forest_note = forest_note,
funnel_df = funnel_df, models_df = models_df, het_df = het_df,
subgroup_df = subgroup_df, egger_df = egger_df, methods_df = methods_df,
metrics = metrics, json_output = json_output
)
}