Executive Summary

Does forecast actually beat doing nothing clever?

Periods Evaluated

724

MAE

933.522

RMSE

1341.106

Mean Error (Bias)

4.304

MAPE (%)

31.02

MASE vs Naive

1.271

MASE vs Seasonal Naive

0.992

Verdict

beats seasonal naive

Across 724 daily periods, The forecast beats the seasonal-naive baseline: MAE of 933.52 against the baseline's 941.10, a MASE of 0.992. On average the process under-forecast by 4.30 units of 'actual' per day, running high in 46.69 percent of periods and low in 53.31 percent. MAPE reads 31.02 percent but rests on only 723 of 724 periods — 1 were excluded because 'actual' was zero or near zero — so lead with MASE, not MAPE. The worst stretch was 2012 Q4 (MAE 1457.59) and the best 2011 Q1 (MAE 478.31).

What this means— written from the computed numbers · yours to edit

The forecast beats the seasonal-naive baseline (repeating the same day 7 days earlier) with a MASE of 0.992, meaning it recovers 0.8 percent of the baseline's error advantage. However, it is not adding value: the forecast's MAE of 933.52 matches the seasonal-naive baseline's MAE of 941.10 within rounding. The process under-forecasts by 4.30 rentals per day on average (53.31 percent of days run low). MAPE of 31.02 percent is unreliable here because 1 of 724 periods had zero or near-zero actual rentals; MASE is the metric to trust. The worst quarter was 2012 Q4 (MAE 1457.59), the best 2011 Q1 (MAE 478.31), showing the headline average masks substantial regime variation.

Overview

Analysis Overview

Backtest of 724 daily forecasts of actual.

N Periods724

Mae933.522

Mase Seasonal0.992

Frequencydaily

What this means— written from the computed numbers · yours to edit

A naive week-ago forecast repeats the same day from 7 days earlier. This analysis evaluates 724 daily bike-rental forecasts using three families of error metrics because no single number answers all questions: MAE and RMSE measure magnitude in rental units; MAPE and sMAPE express error as percentages for comparability; MASE divides the error by what the cheapest baseline would score, answering whether the forecasting effort pays for itself. Two baselines anchor the comparison: repeating the last day (cost: zero) and repeating the same day 7 days earlier (respects seasonality, also cost: zero). The analysis backtests forecasts already made rather than producing new ones, pairing each forecast against actual outcomes to compute errors and test whether differences survive sampling noise.

Data Preparation

Data Quality

How the forecast log became a set of scored periods.

Initial Rows724

Final Rows724

Rows Dropped0

Rows Collapsed0

Periods Missing0

What this means— written from the computed numbers · yours to edit

All 724 rows loaded cleanly with no missing forecasts or actuals. The daily cadence was confirmed from the median gap between dates. No rows were dropped, no duplicates collapsed, and the series is complete — every day in the span carries an actual value. No horizon column was mapped, so all 724 forecasts are scored together without splitting by lead time. The data flows directly from log to scored set with zero exclusions or transformations.

Visualization

Actual vs Forecast

actual and forecast plotted over the same periods.

What this means— written from the computed numbers · yours to edit

Over 700 displayed days from 2011 to 2012, the forecast line and actual line track closely, with the forecast running above actual 46.69 percent of the time and below it 53.31 percent. The average miss is 4.30 rentals, a small bias that a constant correction could remove. The forecast does not lag turning points dramatically — it follows the seasonal swings — but it does not lead them either. The persistent slight under-forecast (53.31 percent of periods low) is the dominant pattern, not volatility or lag.

Data Table

Accuracy Scorecard

Every headline error metric with what it does and does not tell you.

Metric	Value	Interpretation
Periods evaluated	724	Distinct days of 'forecast' scored against 'actual'.
MAE (mean absolute error)	933.5	Average miss, in units of 'actual', ignoring direction.
RMSE (root mean squared error)	1341	Average miss with large misses weighted more heavily; RMSE is 1.44 times MAE here, so the misses are fairly evenly sized.
Mean error (bias)	4.304	Actual minus forecast on average: the process under-forecast by 4.30 units of 'actual' per day.
Periods under-forecast (%)	53.31	386 of 724 periods came in above the forecast; 338 came in below and 0 landed exactly.
MAPE (%)	31.02	Computed on 723 of 724 periods; 1 were excluded because 'actual' was zero or near zero, where a percentage error is undefined or explodes.
sMAPE (%)	25.06	Symmetric percentage error: the denominator uses both the actual and the forecast, so it is bounded at 200 percent and does not blow up as the actual approaches zero.
MASE vs naive	1.271	MAE divided by the naive baseline's average absolute change (734.68). Below 1 means the forecast beats repeating the last day.
MASE vs seasonal naive (7 days)	0.9919	MAE divided by the seasonal-naive baseline's average absolute change (941.10). Below 1 means the forecast beats repeating the same day 7 days earlier.

What this means— written from the computed numbers · yours to edit

The short answer

The forecast edges out the seasonal-naive baseline with a MASE of 0.992, meaning it captures about 99 percent of the baseline's error rate. The process under-forecasts on average by 4.30 units per day, a small removable bias; the larger misses cluster in a few periods rather than spreading evenly (RMSE 1.44 times MAE).

The detail

MAE is 933.5221 units; RMSE is 1341.1059, a ratio of 1.44, indicating the damage concentrates in a few bad days. Mean error (bias) is 4.3039 units — the forecast's systematic under-forecast, removable by adjustment but separate from the absolute miss. Of 724 periods, 386 ran above forecast (53.31 percent) and 338 below (46.69 percent). MAPE of 31.02 percent is computed on 723 of 724 periods (1 excluded for zero actual), making it weaker here; the asymmetry shows 44.72 percent mean error in high periods and 19.06 percent in low periods, revealing the metric penalises over-forecast harder. sMAPE of 25.059 percent is bounded and does not explode at low actuals. MASE of 0.9919 against seasonal-naive (dividing 933.52 by the baseline's 941.10) is the row to trust most — it says the forecast's error is 99 percent of the baseline's.

What this can't tell you

The MASE of 0.9919 is very close to 1.0, the boundary between adding and not adding value. While the forecast does beat the seasonal baseline, the margin is thin and sensitive to the periods included. The MAPE exclusion (1 period with zero actual) is small in count but the asymmetry across high and low periods suggests the forecast may handle demand peaks and troughs differently — a transaction-level or demand-segment breakdown would clarify whether the narrow MASE holds across all conditions.

Visualization

Forecast vs Naive Baselines

Mean absolute error of the forecast next to the two free alternatives.

What this means— written from the computed numbers · yours to edit

Three methods scored on the same 717 days (where all baselines are defined): the forecast's MAE is 941.10, the naive baseline (last day) is 739.36, and the seasonal-naive baseline (7 days back) is 941.10. The forecast is not the shortest bar — it equals the seasonal-naive baseline exactly. Repeating the same day 7 days earlier would have been at least as accurate as the forecasting process, which is the plainest statement that the method is not adding value over the free alternative.

Visualization

Error Distribution

How the per-day misses are spread, and whether they centre on zero.

What this means— written from the computed numbers · yours to edit

Errors (actual minus forecast) centre near zero with a median of 54.50 rentals. The middle 90 percent of periods fall between −2180.15 and 2056.40 rentals, showing tails are roughly symmetric. The distribution is centred close to zero, meaning misses are noise rather than a systematic offset. The longest tail on the negative side (under-forecasts) reaches −7036 rentals; the positive side (over-forecasts) reaches 6370 rentals. The symmetry suggests no single bad event dominates; instead, the process struggles consistently across different regimes.

Visualization

Accuracy Over Time

Mean absolute error by stretch of the forecast log.

What this means— written from the computed numbers · yours to edit

Accuracy varies sharply across 8 quarters. The best quarter is 2011 Q1 with MAE 478.31 rentals; the worst is 2012 Q4 with MAE 1457.59 rentals — a 3.05-fold gap. The headline MAE of 933.52 is therefore an average over genuinely different regimes, not a stable estimate. Mean errors show the pattern: 2011 Q2 runs high (+236.49) and 2012 Q1 runs high (+282.85), while 2011 Q4 and 2012 Q4 run low (−214.01 and −439.46). These swings point to bias problems in specific quarters rather than a single bad event, suggesting the process degrades or the data shifts seasonally.

Data Table

Accuracy by Horizon

Overall accuracy — no distinct lead times to compare.

Horizon	N Periods	MAE	RMSE	Mean Error	MAPE	Mase	Verdict
All forecasts (no 'horizon' column mapped)	724	933.5	1341	4.304	31.02	0.9919	beats seasonal naive

What this means— written from the computed numbers · yours to edit

No horizon column was mapped, so the analysis does not split forecasts by lead time (one day ahead, three days ahead, etc.). All 724 forecasts are scored together, yielding a single row: MAE 933.52, RMSE 1341.11, mean error 4.30, MAPE 31.02 percent, MASE 0.992 against seasonal naive. If the forecast log records how far ahead each forecast was made, mapping that column would split this table by lead time — the place where naive forecasts typically fail most, with near-term accuracy excellent and long-horizon accuracy no better than a free baseline.

Data Table

Methods & Disclosure

The formulas, the baselines, and where each metric breaks down.

Item	Detail
Error convention	Every period's error is 'actual' minus 'forecast', so a positive error means the outcome came in above the forecast (an under-forecast).
Scale-dependent error	MAE is the mean of the absolute errors (933.52) and RMSE the square root of the mean squared error (1341.11), both in units of 'actual'. RMSE exceeds MAE whenever the misses are uneven in size.
Percentage error	MAPE is the mean of the absolute errors divided by the actual, computed on 723 of 724 periods (1 excluded for a zero or near-zero 'actual', where the ratio is undefined or explodes). MAPE is also bounded at 100 percent when the forecast runs low but unbounded when it runs high, so it penalises over-forecasting more heavily and a process tuned to minimise it will drift low. Measured here: mean absolute percentage error of 44.72 in the periods that ran high versus 19.06 in the periods that ran low.
Symmetric percentage error	sMAPE (25.06 percent) divides twice the absolute error by the sum of the absolute actual and the absolute forecast, which bounds it at 200 percent and keeps it finite when the actual is small.
Scaling baselines	The naive baseline repeats the previous day's actual; its average absolute change over the observed series is 734.68. The seasonal-naive baseline repeats the same day 7 days earlier; its average absolute change is 941.10. Both are computed on a regular daily calendar grid, so gaps in the data never shift the lag.
MASE	MASE is MAE divided by a baseline's average absolute change: 1.271 against naive and 0.992 against seasonal naive. The scaling factor is taken from the whole observed series rather than a separate training window, because a forecast log carries no train and test split; that makes these values comparable across series but not identical to a MASE computed from a held-out fit.
Diebold-Mariano test	Against the naive baseline: 723 paired periods, mean loss differential 200.097 (positive means the forecast's absolute error is larger), Newey-West lag 8, statistic 4.582, p < 0.001. Not run against the seasonal-naive baseline — too few paired periods, or the loss differential has no variation. The test is implemented directly here: the per-period loss differential under absolute-error loss is divided by a Newey-West long-run standard error with Bartlett weights, given the Harvey-Leybourne-Newbold small-sample correction, and referred to a t distribution.
Calendar handling	The cadence of 'date' was read as daily from the median gap between periods. 0 of the 724 calendar days spanned by the data carry no actual (0.00 percent). Baseline lags line up with the calendar.
What this cannot tell you	This is a scorecard of a forecasting process, not a diagnosis of it. It measures how far 'forecast' landed from 'actual' and whether that beats a naive rule; it cannot say why the misses happened, and it cannot rule out that the forecasts were revised with knowledge the baselines did not have.

What this means— written from the computed numbers · yours to edit

Error is defined as actual minus forecast, so positive values are under-forecasts. MAE (933.52) and RMSE (1341.11) are in rental units; RMSE exceeds MAE here because misses vary in size. MAPE (31.02 percent) is computed on 723 of 724 periods, excluding 1 where actual was zero or near-zero; MAPE is also asymmetric, capped at 100 percent when the forecast runs low but unbounded when it runs high, penalizing over-forecasts harder. sMAPE (25.06 percent) uses a symmetric denominator and stays bounded at 200 percent. MASE divides MAE by the baseline's average absolute change: 1.271 against naive (734.68 scale) and 0.992 against seasonal-naive (941.10 scale). The Diebold-Mariano test against the naive baseline shows a mean loss differential of 200.097 (forecast is larger), Newey-West lag 8, t-statistic 4.582, p < 0.001 — the forecast is significantly worse than the naive baseline. The test was not run against seasonal-naive because too few paired periods or no variation in loss differential. This analysis scores a process without diagnosing it; it does not explain why the forecast misses, only that it does.

Rate this report Was this the answer you needed?

The exact source that produced this report — yours to keep, read, and re-run.

Download PDF

How this was computed method · R source · citation

The code that did it

Forecast Accuracy Backtest — Are Your Forecasts Actually Any Good?

Grades a log of forecasts against what actually happened. Scale-dependent error (MAE, RMSE), percentage error (MAPE with its zero and asymmetry traps handled, plus sMAPE), scale-free error against the naive and seasonal-naive baselines (MASE), and bias (mean error plus the share of periods over- and under-forecast). A Diebold-Mariano test decides whether the forecast's loss is really lower than a baseline's, rather than just printing two numbers side by side. Accuracy is then broken out by calendar period and, when a horizon column is mapped, by forecast horizon.

Why This Method?

A forecast is only worth its cost if it beats the cheapest thing you could have done instead. The cheapest things are the naive forecast ("next period equals this period") and the seasonal naive forecast ("next period equals the same period one cycle ago"). MASE expresses the forecast's error as a multiple of that baseline's error, so a value at or above 1 means the forecasting process is not adding value. The Diebold-Mariano test then asks whether the difference in loss is larger than the sampling noise.

What This Analysis Covers

Actual against forecast over time (chart)
MAE, RMSE, MAPE, sMAPE, MASE, and bias
Naive and seasonal-naive baselines with a Diebold-Mariano test
Error distribution, accuracy by period, accuracy by horizon

Standard Library

Platform standard-library module (LAT-1441): runs on ANY dataset via the semantic mapping {date, forecast, actual, horizon}. All narrative is derived from the user's own column names and computed values. The sibling tool standard_forecasting PRODUCES a forecast from history; this tool GRADES forecasts you already made.

suppressPackageStartupMessages(library(DT))
suppressPackageStartupMessages(library(htmlwidgets))
suppressPackageStartupMessages(library(arrow))
suppressPackageStartupMessages(library(knitr))
suppressPackageStartupMessages(library(rmarkdown))
suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(tidyr))
suppressPackageStartupMessages(library(ggplot2))
suppressPackageStartupMessages(library(stringr))
suppressPackageStartupMessages(library(lubridate))
suppressPackageStartupMessages(library(broom))
suppressPackageStartupMessages(library(Matrix))
suppressPackageStartupMessages(library(cluster))
suppressPackageStartupMessages(library(data.table))

Core Analysis Pipeline

Step 1: Resolve the mapped columns (humanized for every sentence)

initial_rows <- nrow(df)
  date_h     <- humanize_semantic("date", col_map)[1]
  forecast_h <- humanize_semantic("forecast", col_map)[1]
  actual_h   <- humanize_semantic("actual", col_map)[1]
  horizon_h  <- humanize_semantic("horizon", col_map)[1]

  if (!all(c("date", "forecast", "actual") %in% names(df))) {
    stop(sprintf(
      "A forecast backtest needs three columns mapped: the period the forecast was for (&#x27;%s'), the forecast itself ('%s'), and the actual outcome ('%s').",
      date_h, forecast_h, actual_h))
  }
  if (initial_rows < MIN_PERIODS) {
    stop(sprintf(
      "Only %d rows were supplied — grading &#x27;%s' against '%s' needs at least %d dated periods.",
      initial_rows, forecast_h, actual_h, MIN_PERIODS))
  }

Step 2: Parse dates; coerce both numbers with the 95% rule

raw_date <- df$date
  non_blank <- !(is.na(raw_date) | !nzchar(trimws(as.character(raw_date))))
  dts <- parse_dates_robust(raw_date)
  n_nonblank <- sum(non_blank)
  if (n_nonblank == 0) {
    stop(sprintf("The &#x27;%s' column is empty — there are no periods to score forecasts against.", date_h))
  }
  n_unparsed <- sum(non_blank & is.na(dts))
  if (n_unparsed / n_nonblank > 0.05) {
    stop(sprintf(
      "%d of %d values in &#x27;%s' could not be read as dates. Please use a recognizable date format such as 2024-01-31 or 01/31/2024.",
      n_unparsed, n_nonblank, date_h))
  }

  coerce_num <- function(v, label_h, role) {
    if (is.numeric(v)) return(as.numeric(v))
    ch <- as.character(v)
    nb <- !is.na(ch) & trimws(ch) != ""
    conv <- suppressWarnings(as.numeric(ch))
    if (sum(nb) == 0 || sum(!is.na(conv[nb])) < 0.95 * sum(nb)) {
      stop(sprintf(
        "The column &#x27;%s' was mapped as the %s but fewer than 95%% of its values parse as numbers. Map a numeric column instead.",
        label_h, role))
    }
    conv
  }
  fcv <- coerce_num(df$forecast, forecast_h, "forecast")
  acv <- coerce_num(df$actual, actual_h, "actual outcome")

  keep <- !is.na(dts) & !is.na(fcv) & !is.na(acv)
  n_dropped <- sum(!keep)
  work <- data.frame(date = dts[keep], forecast = fcv[keep], actual = acv[keep],
                     stringsAsFactors = FALSE)
  has_horizon <- "horizon" %in% names(df)
  if (has_horizon) work$horizon_raw <- df$horizon[keep]
  n_rows <- nrow(work)
  if (n_rows < MIN_PERIODS) {
    stop(sprintf(
      "Only %d complete rows remained after dropping rows with an unreadable &#x27;%s' or a missing '%s' or '%s' — at least %d are needed.",
      n_rows, date_h, forecast_h, actual_h, MIN_PERIODS))
  }
  work <- work[order(work$date), , drop = FALSE]

Step 3: Horizon column — numeric, binned, or lumped categorical

single_horizon <- FALSE; primary_label <- NA_character_
  n_horizons <- 0L; horizon_note <- ""
  horizon_usable <- FALSE
  if (has_horizon) {
    hv <- work$horizon_raw
    ch <- trimws(as.character(hv))
    nb <- !is.na(ch) & nzchar(ch)
    hnum <- suppressWarnings(as.numeric(ch))
    numeric_ok <- sum(nb) > 0 && sum(!is.na(hnum[nb])) >= 0.95 * sum(nb)
    if (numeric_ok) {
      hnum[!nb] <- NA_real_
      uq <- sort(unique(hnum[!is.na(hnum)]))
      if (length(uq) > 12) {
        brks <- unique(stats::quantile(hnum, probs = seq(0, 1, length.out = 7), na.rm = TRUE))
        if (length(brks) >= 3) {
          cutf <- cut(hnum, breaks = brks, include.lowest = TRUE, dig.lab = 6)
          work$horizon_label <- as.character(cutf)
          work$horizon_order <- as.numeric(cutf)
          horizon_note <- sprintf("&#x27;%s' held %d distinct values, so they were grouped into %d ranges.",
                                  horizon_h, length(uq), length(levels(cutf)))
        } else {
          work$horizon_label <- as.character(hnum)
          work$horizon_order <- hnum
        }
      } else {
        work$horizon_label <- ifelse(is.na(hnum), "Missing", format(hnum, trim = TRUE, scientific = FALSE))
        work$horizon_order <- ifelse(is.na(hnum), Inf, hnum)
      }
    } else {
      lab <- ch
      lab[!nb] <- "Missing"
      tb <- sort(table(lab), decreasing = TRUE)
      keep_lv <- names(tb)[seq_len(min(8L, length(tb)))]
      n_lumped <- sum(!(lab %in% keep_lv))
      lab[!(lab %in% keep_lv)] <- "Other"
      work$horizon_label <- lab
      cnt <- table(lab)
      work$horizon_order <- as.numeric(-cnt[lab])
      horizon_note <- sprintf("&#x27;%s' is not numeric, so it was treated as a label%s.",
                              horizon_h,
                              if (n_lumped > 0) sprintf(" and %d rows outside the eight most common levels were grouped as \"Other\"", n_lumped) else "")
    }
    lv_tab <- table(work$horizon_label)
    n_horizons <- length(lv_tab)
    horizon_usable <- n_horizons >= 1
    single_horizon <- n_horizons == 1
    cand <- names(lv_tab)[lv_tab == max(lv_tab)]
    ord <- sapply(cand, function(k) min(work$horizon_order[work$horizon_label == k]))
    primary_label <- cand[order(ord, cand)][1]
  }

Step 4: The actual series (all rows), its cadence, and its calendar grid

ser <- stats::aggregate(actual ~ date, data = work, FUN = mean)
  ser <- ser[order(ser$date), , drop = FALSE]
  if (nrow(ser) < 4) {
    stop(sprintf("Only %d distinct periods appear in &#x27;%s' — a backtest needs the forecasts spread over time.",
                 nrow(ser), date_h))
  }
  med_gap <- stats::median(as.numeric(diff(ser$date)))
  if (!is.finite(med_gap) || med_gap <= 0) med_gap <- 1
  if (med_gap <= 1.5) {
    freq_label <- "daily";     step_days <- 1;   m <- 7L;  period_word <- "day";     cycle_word <- "week"
  } else if (med_gap >= 5.5 && med_gap <= 8.5) {
    freq_label <- "weekly";    step_days <- 7;   m <- 52L; period_word <- "week";    cycle_word <- "year"
  } else if (med_gap >= 26 && med_gap <= 35) {
    freq_label <- "monthly";   step_days <- 30;  m <- 12L; period_word <- "month";   cycle_word <- "year"
  } else if (med_gap >= 85 && med_gap <= 95) {
    freq_label <- "quarterly"; step_days <- 91;  m <- 4L;  period_word <- "quarter"; cycle_word <- "year"
  } else if (med_gap >= 350 && med_gap <= 380) {
    freq_label <- "yearly";    step_days <- 365; m <- 1L;  period_word <- "year";    cycle_word <- "period"
  } else {
    freq_label <- "irregular"; step_days <- max(1, round(med_gap)); m <- 1L
    period_word <- "period";   cycle_word <- "cycle"
  }

  origin <- ser$date[1]
  if (freq_label == "irregular") {
    gidx_ser <- seq_len(nrow(ser))
    n_grid <- nrow(ser)
  } else {
    gidx_ser <- grid_index(ser$date, origin, freq_label, step_days)
    gidx_ser[gidx_ser < 1] <- 1L
    n_grid <- max(gidx_ser)
  }
  gv <- rep(NA_real_, n_grid)
  gv[gidx_ser] <- ser$actual
  n_grid_missing <- sum(is.na(gv))
  pct_grid_missing <- 100 * n_grid_missing / n_grid

Step 5: Naive and seasonal-naive baselines on that grid

g_naive <- c(NA_real_, gv[-n_grid])
  seasonal_available <- m > 1L && n_grid > m + 2L
  g_seas <- if (seasonal_available) c(rep(NA_real_, m), gv[seq_len(n_grid - m)]) else rep(NA_real_, n_grid)
  d_naive <- abs(gv - g_naive)
  d_seas  <- abs(gv - g_seas)
  q_naive <- if (any(!is.na(d_naive))) mean(d_naive, na.rm = TRUE) else NA_real_
  q_seas  <- if (any(!is.na(d_seas)))  mean(d_seas,  na.rm = TRUE) else NA_real_
  if (!is.na(q_naive) && q_naive <= 0) q_naive <- NA_real_
  if (!is.na(q_seas)  && q_seas  <= 0) q_seas  <- NA_real_
  if (is.na(q_seas)) seasonal_available <- FALSE
  seasonal_phrase <- sprintf("the same %s %d %ss earlier", period_word, m, period_word)

Step 6: Headline rows — the primary horizon if one is mapped,

collapsed to one forecast per period so every metric is per-period.

if (horizon_usable && !is.na(primary_label)) {
    hl <- work[work$horizon_label == primary_label, , drop = FALSE]
  } else {
    hl <- work
  }
  hl_agg <- stats::aggregate(cbind(forecast, actual) ~ date, data = hl, FUN = mean)
  hl_agg <- hl_agg[order(hl_agg$date), , drop = FALSE]
  n_dup_rows <- nrow(hl) - nrow(hl_agg)
  n_eval <- nrow(hl_agg)
  if (n_eval < MIN_PERIODS) {
    stop(sprintf(
      "Only %d distinct periods are available to score%s — at least %d are needed to grade &#x27;%s'.",
      n_eval,
      if (horizon_usable) sprintf(" at the level of &#x27;%s' being scored (%s)", horizon_h, primary_label) else "",
      MIN_PERIODS, forecast_h))
  }
  final_rows <- n_eval
  rows_removed <- initial_rows - final_rows

Step 7: Error metrics on the evaluated periods

Convention: error = actual minus forecast, so a POSITIVE error is an under-forecast (the outcome beat the plan).

err <- hl_agg$actual - hl_agg$forecast
  mae  <- mean(abs(err))
  rmse <- sqrt(mean(err^2))
  me   <- mean(err)
  n_under <- sum(err > 0); n_over <- sum(err < 0); n_exact <- sum(err == 0)
  pct_under <- 100 * n_under / n_eval
  pct_over  <- 100 * n_over  / n_eval
  pct_exact <- 100 * n_exact / n_eval

MAPE is undefined at a zero actual and unstable at a near-zero one, so both are excluded and counted rather than silently dropped.

abs_a <- abs(hl_agg$actual)
  pos_a <- abs_a[abs_a > 1e-9]
  med_a <- if (length(pos_a) > 0) stats::median(pos_a) else NA_real_
  zero_idx <- abs_a <= 1e-9
  near_idx <- if (is.na(med_a)) rep(FALSE, n_eval) else (!zero_idx & abs_a < 0.01 * med_a)
  mape_ok <- !zero_idx & !near_idx
  n_zero <- sum(zero_idx); n_near_zero <- sum(near_idx)
  n_mape_excl <- n_zero + n_near_zero
  n_mape_used <- sum(mape_ok)
  mape <- if (n_mape_used >= 3) 100 * mean(abs(err[mape_ok] / hl_agg$actual[mape_ok])) else NA_real_

Measured asymmetry: mean absolute percentage error among the periods the forecast ran high versus those it ran low.

ape <- if (n_mape_used > 0) 100 * abs(err[mape_ok] / hl_agg$actual[mape_ok]) else numeric(0)
  err_ok <- err[mape_ok]
  ape_over  <- if (sum(err_ok < 0) >= 3) mean(ape[err_ok < 0]) else NA_real_
  ape_under <- if (sum(err_ok > 0) >= 3) mean(ape[err_ok > 0]) else NA_real_

sMAPE — symmetric denominator, bounded at 200 percent.

sden <- abs(hl_agg$actual) + abs(hl_agg$forecast)
  sok <- sden > 1e-9
  smape <- if (sum(sok) >= 3) 100 * mean(2 * abs(err[sok]) / sden[sok]) else NA_real_
  mape_reliable <- !is.na(mape) && n_mape_excl == 0 && (is.na(med_a) || min(pos_a) > 0.05 * med_a)

  mase_naive <- if (!is.na(q_naive)) mae / q_naive else NA_real_
  mase_seas  <- if (!is.na(q_seas))  mae / q_seas  else NA_real_
  beats_naive    <- !is.na(mase_naive) && mase_naive < 1
  beats_seasonal <- !is.na(mase_seas)  && mase_seas  < 1

Step 8: Paired baseline errors for the evaluated periods

if (freq_label == "irregular") {
    gidx_hl <- match(hl_agg$date, ser$date)
  } else {
    gidx_hl <- grid_index(hl_agg$date, origin, freq_label, step_days)
  }
  gidx_hl[gidx_hl < 1 | gidx_hl > n_grid] <- NA_integer_
  base_naive_fc <- g_naive[gidx_hl]
  base_seas_fc  <- g_seas[gidx_hl]
  e_naive <- hl_agg$actual - base_naive_fc
  e_seas  <- hl_agg$actual - base_seas_fc

  dm_naive <- dm_test(err, e_naive, h = 1)
  dm_seas  <- if (seasonal_available) dm_test(err, e_seas, h = 1) else NULL

Step 9: Like-for-like baseline comparison on one common subset

bench_idx <- if (seasonal_available) !is.na(e_seas) else !is.na(e_naive)
  bench_n <- sum(bench_idx)
  bench_kind <- if (seasonal_available) "seasonal-naive" else "naive"
  bench_fc_mae    <- if (bench_n > 0) mean(abs(err[bench_idx])) else NA_real_
  bench_naive_mae <- if (bench_n > 0) mean(abs(e_naive[bench_idx]), na.rm = TRUE) else NA_real_
  bench_seas_mae  <- if (bench_n > 0 && seasonal_available) mean(abs(e_seas[bench_idx]), na.rm = TRUE) else NA_real_

  bench_rows <- list(data.frame(
    method = sprintf("Your forecast(%s)", forecast_h),
    mae = round(bench_fc_mae, 4),
    n_periods = bench_n,
    note = sprintf("Mean absolute error of &#x27;%s' against '%s' over the %d periods where every baseline is defined.",
                   forecast_h, actual_h, bench_n),
    stringsAsFactors = FALSE))
  if (is.finite(bench_naive_mae)) {
    bench_rows[[length(bench_rows) + 1]] <- data.frame(
      method = sprintf("Naive(last %s)", period_word),
      mae = round(bench_naive_mae, 4), n_periods = bench_n,
      note = sprintf("Repeat the previous %s&#x27;s actual as the forecast — the cheapest thing you could have done.", period_word),
      stringsAsFactors = FALSE)
  }
  if (is.finite(bench_seas_mae)) {
    bench_rows[[length(bench_rows) + 1]] <- data.frame(
      method = sprintf("Seasonal naive(%d %ss back)", m, period_word),
      mae = round(bench_seas_mae, 4), n_periods = bench_n,
      note = sprintf("Repeat %s as the forecast — the cheapest thing that respects the seasonal cycle.", seasonal_phrase),
      stringsAsFactors = FALSE)
  }
  baseline_comparison_df <- do.call(rbind, bench_rows)
  rownames(baseline_comparison_df) <- NULL

Step 10: Chart datasets (both capped well under the row limit)

keep_n <- min(n_eval, 700L)
  tail_idx <- (n_eval - keep_n + 1):n_eval
  series_points_df <- rbind(
    data.frame(period = format(hl_agg$date[tail_idx], "%Y-%m-%d"),
               value = round(hl_agg$actual[tail_idx], 4),
               series = actual_h, stringsAsFactors = FALSE),
    data.frame(period = format(hl_agg$date[tail_idx], "%Y-%m-%d"),
               value = round(hl_agg$forecast[tail_idx], 4),
               series = forecast_h, stringsAsFactors = FALSE)
  )
  rownames(series_points_df) <- NULL

  set.seed(42)
  eidx <- if (n_eval > 2000) sample(n_eval, 2000) else seq_len(n_eval)
  error_distribution_df <- data.frame(error = round(err[eidx], 4), stringsAsFactors = FALSE)
  error_distribution_df <- error_distribution_df[order(error_distribution_df$error), , drop = FALSE]
  rownames(error_distribution_df) <- NULL

Step 11: Accuracy by calendar period

span_days <- as.numeric(max(hl_agg$date) - min(hl_agg$date))
  if (span_days >= 3 * 365) {
    bucket <- format(hl_agg$date, "%Y")
  } else if (span_days >= 200) {
    bucket <- paste0(format(hl_agg$date, "%Y"), " Q",
                     (as.integer(format(hl_agg$date, "%m")) - 1L) %/% 3L + 1L)
  } else {
    bucket <- format(hl_agg$date, "%Y-%m")
  }
  ub <- unique(bucket)
  if (length(ub) > 12) {
    per <- ceiling(length(ub) / 12)
    grp <- ceiling(seq_along(ub) / per)
    newlab <- tapply(ub, grp, function(x) if (length(x) == 1) x[1] else paste0(x[1], " to ", x[length(x)]))
    bucket <- unname(newlab[as.character(grp[match(bucket, ub)])])
  }
  ape_all <- rep(NA_real_, n_eval)
  ape_all[mape_ok] <- 100 * abs(err[mape_ok] / hl_agg$actual[mape_ok])
  pa <- lapply(unique(bucket), function(b) {
    ix <- bucket == b
    data.frame(period_bucket = b,
               n_periods = sum(ix),
               mae = round(mean(abs(err[ix])), 4),
               mean_error = round(mean(err[ix]), 4),
               mape = if (any(mape_ok & ix)) round(mean(ape_all[ix], na.rm = TRUE), 3) else NA_real_,
               stringsAsFactors = FALSE)
  })
  period_accuracy_df <- do.call(rbind, pa)
  rownames(period_accuracy_df) <- NULL
  pa_ok <- which(!is.na(period_accuracy_df$mae))
  worst_bucket <- if (length(pa_ok) > 0) period_accuracy_df$period_bucket[pa_ok[which.max(period_accuracy_df$mae[pa_ok])]] else NA_character_
  best_bucket  <- if (length(pa_ok) > 0) period_accuracy_df$period_bucket[pa_ok[which.min(period_accuracy_df$mae[pa_ok])]] else NA_character_
  worst_mae <- if (length(pa_ok) > 0) max(period_accuracy_df$mae[pa_ok]) else NA_real_
  best_mae  <- if (length(pa_ok) > 0) min(period_accuracy_df$mae[pa_ok]) else NA_real_

Step 12: Accuracy by horizon

scale_for_mase <- if (seasonal_available) q_seas else q_naive
  scale_label <- if (seasonal_available) "seasonal naive" else "naive"
  if (horizon_usable) {
    lv <- unique(work$horizon_label)
    ordv <- sapply(lv, function(k) min(work$horizon_order[work$horizon_label == k]))
    lv <- lv[order(ordv, lv)]
    hz <- lapply(lv, function(k) {
      sub <- work[work$horizon_label == k, , drop = FALSE]
      sub <- stats::aggregate(cbind(forecast, actual) ~ date, data = sub, FUN = mean)
      e <- sub$actual - sub$forecast
      aok <- abs(sub$actual) > 1e-9 & (is.na(med_a) | abs(sub$actual) >= 0.01 * med_a)
      data.frame(
        horizon = k,
        n_periods = nrow(sub),
        mae = round(mean(abs(e)), 4),
        rmse = round(sqrt(mean(e^2)), 4),
        mean_error = round(mean(e), 4),
        mape = if (sum(aok) >= 3) round(100 * mean(abs(e[aok] / sub$actual[aok])), 3) else NA_real_,
        mase = if (!is.na(scale_for_mase)) round(mean(abs(e)) / scale_for_mase, 4) else NA_real_,
        verdict = if (is.na(scale_for_mase)) "no baseline available"
                  else if (mean(abs(e)) / scale_for_mase < 1) sprintf("beats %s", scale_label)
                  else sprintf("does not beat %s", scale_label),
        stringsAsFactors = FALSE)
    })
    horizon_accuracy_df <- do.call(rbind, hz)
  } else {
    horizon_accuracy_df <- data.frame(
      horizon = sprintf("All forecasts(no &#x27;%s' column mapped)", horizon_h),
      n_periods = n_eval, mae = round(mae, 4), rmse = round(rmse, 4),
      mean_error = round(me, 4),
      mape = if (is.na(mape)) NA_real_ else round(mape, 3),
      mase = if (is.na(scale_for_mase)) NA_real_ else round(mae / scale_for_mase, 4),
      verdict = if (is.na(scale_for_mase)) "no baseline available"
                else if (mae / scale_for_mase < 1) sprintf("beats %s", scale_label)
                else sprintf("does not beat %s", scale_label),
      stringsAsFactors = FALSE)
  }
  rownames(horizon_accuracy_df) <- NULL
  hz_ok <- which(!is.na(horizon_accuracy_df$mae))
  horizon_degrades <- horizon_usable && length(hz_ok) >= 2 &&
    horizon_accuracy_df$mae[hz_ok[length(hz_ok)]] > horizon_accuracy_df$mae[hz_ok[1]]
  hz_ratio <- if (horizon_usable && length(hz_ok) >= 2 && horizon_accuracy_df$mae[hz_ok[1]] > 0) {
    horizon_accuracy_df$mae[hz_ok[length(hz_ok)]] / horizon_accuracy_df$mae[hz_ok[1]]
  } else NA_real_

Step 13: The headline metric table

bias_dir <- if (me > 0) "under-forecast" else if (me < 0) "over-forecast" else "unbiased"
  acc_rows <- list(
    data.frame(metric = "Periods evaluated", value = n_eval,
               interpretation = sprintf("Distinct %ss of &#x27;%s' scored against '%s'.", period_word, forecast_h, actual_h),
               stringsAsFactors = FALSE),
    data.frame(metric = "MAE(mean absolute error)", value = round(mae, 4),
               interpretation = sprintf("Average miss, in units of &#x27;%s', ignoring direction.", actual_h),
               stringsAsFactors = FALSE),
    data.frame(metric = "RMSE(root mean squared error)", value = round(rmse, 4),
               interpretation = sprintf("Average miss with large misses weighted more heavily; RMSE is %s times MAE here, so %s.",
                                        r2(if (mae > 0) rmse / mae else NA_real_),
                                        if (mae > 0 && rmse / mae > 1.5) "a minority of periods carries most of the damage" else "the misses are fairly evenly sized"),
               stringsAsFactors = FALSE),
    data.frame(metric = "Mean error(bias)", value = round(me, 4),
               interpretation = sprintf("Actual minus forecast on average: the process %s by %s units of &#x27;%s' per %s.",
                                        bias_dir, r2(abs(me)), actual_h, period_word),
               stringsAsFactors = FALSE),
    data.frame(metric = "Periods under-forecast(%)", value = round(pct_under, 2),
               interpretation = sprintf("%d of %d periods came in above the forecast; %d came in below and %d landed exactly.",
                                        n_under, n_eval, n_over, n_exact),
               stringsAsFactors = FALSE),
    data.frame(metric = "MAPE(%)", value = if (is.na(mape)) NA_real_ else round(mape, 3),
               interpretation = if (is.na(mape))
                 sprintf("Not computable — too few periods with a usable non-zero &#x27;%s'.", actual_h)
               else if (n_mape_excl > 0)
                 sprintf("Computed on %d of %d periods; %d were excluded because &#x27;%s' was zero or near zero, where a percentage error is undefined or explodes.",
                         n_mape_used, n_eval, n_mape_excl, actual_h)
               else
                 sprintf("Computed on all %d periods; no zero or near-zero actuals were present.", n_eval),
               stringsAsFactors = FALSE),
    data.frame(metric = "sMAPE(%)", value = if (is.na(smape)) NA_real_ else round(smape, 3),
               interpretation = "Symmetric percentage error: the denominator uses both the actual and the forecast, so it is bounded at 200 percent and does not blow up as the actual approaches zero.",
               stringsAsFactors = FALSE),
    data.frame(metric = "MASE vs naive", value = if (is.na(mase_naive)) NA_real_ else round(mase_naive, 4),
               interpretation = if (is.na(mase_naive))
                 sprintf("Not computable — &#x27;%s' does not change from one %s to the next.", actual_h, period_word)
               else sprintf("MAE divided by the naive baseline&#x27;s average absolute change (%s). Below 1 means the forecast beats repeating the last %s.",
                            r2(q_naive), period_word),
               stringsAsFactors = FALSE),
    data.frame(metric = sprintf("MASE vs seasonal naive(%d %ss)", m, period_word),
               value = if (is.na(mase_seas)) NA_real_ else round(mase_seas, 4),
               interpretation = if (is.na(mase_seas))
                 sprintf("Not computable — the history does not cover enough full %ss for a seasonal baseline.", cycle_word)
               else sprintf("MAE divided by the seasonal-naive baseline&#x27;s average absolute change (%s). Below 1 means the forecast beats repeating %s.",
                            r2(q_seas), seasonal_phrase),
               stringsAsFactors = FALSE)
  )
  accuracy_metrics_df <- do.call(rbind, acc_rows)
  rownames(accuracy_metrics_df) <- NULL

Step 14: Methods and disclosure

dm_detail <- function(res, label) {
    if (is.null(res)) return(sprintf("Not run against the %s baseline — too few paired periods, or the loss differential has no variation.", label))
    sprintf("Against the %s baseline: %d paired periods, mean loss differential %s(%s means the forecast&#x27;s absolute error is %s), Newey-West lag %d, statistic %s, %s.",
            label, res$n, r3(res$mean_d),
            if (res$mean_d < 0) "negative" else "positive",
            if (res$mean_d < 0) "smaller" else "larger",
            res$lag, r3(res$stat), fmt_pp(res$p))
  }
  methods_df <- data.frame(
    item = c("Error convention", "Scale-dependent error", "Percentage error",
             "Symmetric percentage error", "Scaling baselines", "MASE",
             "Diebold-Mariano test", "Calendar handling", "What this cannot tell you"),
    detail = c(
      sprintf("Every period&#x27;s error is '%s' minus '%s', so a positive error means the outcome came in above the forecast (an under-forecast).", actual_h, forecast_h),
      sprintf("MAE is the mean of the absolute errors(%s) and RMSE the square root of the mean squared error(%s), both in units of &#x27;%s'. RMSE exceeds MAE whenever the misses are uneven in size.",
              r2(mae), r2(rmse), actual_h),
      if (is.na(mape))
        sprintf("MAPE could not be computed: too few periods had a usable non-zero &#x27;%s'.", actual_h)
      else sprintf("MAPE is the mean of the absolute errors divided by the actual, computed on %d of %d periods(%d excluded for a zero or near-zero &#x27;%s', where the ratio is undefined or explodes). MAPE is also bounded at 100 percent when the forecast runs low but unbounded when it runs high, so it penalises over-forecasting more heavily and a process tuned to minimise it will drift low. Measured here: mean absolute percentage error of %s in the periods that ran high versus %s in the periods that ran low.",
                   n_mape_used, n_eval, n_mape_excl, actual_h, r2(ape_over), r2(ape_under)),
      if (is.na(smape)) "sMAPE could not be computed on this data."
      else sprintf("sMAPE(%s percent) divides twice the absolute error by the sum of the absolute actual and the absolute forecast, which bounds it at 200 percent and keeps it finite when the actual is small.", r2(smape)),
      sprintf("The naive baseline repeats the previous %s&#x27;s actual; its average absolute change over the observed series is %s. The seasonal-naive baseline repeats %s; %s. Both are computed on a regular %s calendar grid, so gaps in the data never shift the lag.",
              period_word, r2(q_naive), seasonal_phrase,
              if (is.na(q_seas)) sprintf("it is unavailable because the history does not cover enough full %ss", cycle_word)
              else sprintf("its average absolute change is %s", r2(q_seas)),
              freq_label),
      sprintf("MASE is MAE divided by a baseline&#x27;s average absolute change: %s against naive and %s against seasonal naive. The scaling factor is taken from the whole observed series rather than a separate training window, because a forecast log carries no train and test split; that makes these values comparable across series but not identical to a MASE computed from a held-out fit.",
              r3(mase_naive), r3(mase_seas)),
      paste(dm_detail(dm_naive, "naive"), dm_detail(dm_seas, "seasonal-naive"),
            "The test is implemented directly here: the per-period loss differential under absolute-error loss is divided by a Newey-West long-run standard error with Bartlett weights, given the Harvey-Leybourne-Newbold small-sample correction, and referred to a t distribution.",
            sep = " "),
      sprintf("The cadence of &#x27;%s' was read as %s from the median gap between periods. %d of the %d calendar %ss spanned by the data carry no actual (%s percent). %s",
              date_h, freq_label, n_grid_missing, n_grid, period_word, r2(pct_grid_missing),
              if (pct_grid_missing > 10)
                "With that many periods missing, the baselines are computed only where both ends of a lag are present, and they rest on fewer comparisons than the period count suggests."
              else "Baseline lags line up with the calendar."),
      sprintf("This is a scorecard of a forecasting process, not a diagnosis of it. It measures how far &#x27;%s' landed from '%s' and whether that beats a naive rule; it cannot say why the misses happened, and it cannot rule out that the forecasts were revised with knowledge the baselines did not have.",
              forecast_h, actual_h)
    ),
    stringsAsFactors = FALSE
  )

Step 15: Metrics and the one-paragraph answer

bench_verdict <- if (seasonal_available) {
    if (beats_seasonal) "beats seasonal naive" else "does not beat seasonal naive"
  } else if (!is.na(mase_naive)) {
    if (beats_naive) "beats naive" else "does not beat naive"
  } else "no baseline available"

  metrics <- list(
    `Periods Evaluated`      = n_eval,
    `MAE`                    = round(mae, 3),
    `RMSE`                   = round(rmse, 3),
    `Mean Error(Bias)`      = round(me, 3),
    `MAPE(%)`               = if (is.na(mape)) NA_real_ else round(mape, 2),
    `MASE vs Naive`          = if (is.na(mase_naive)) NA_real_ else round(mase_naive, 3),
    `MASE vs Seasonal Naive` = if (is.na(mase_seas)) NA_real_ else round(mase_seas, 3),
    `Verdict`                = bench_verdict
  )

  headline <- if (seasonal_available) {
    if (beats_seasonal) {
      sprintf("The forecast beats the seasonal-naive baseline: MAE of %s against the baseline&#x27;s %s, a MASE of %s.",
              r2(mae), r2(q_seas), r3(mase_seas))
    } else {
      sprintf("The forecast does not beat the seasonal-naive baseline: MAE of %s against the baseline&#x27;s %s, a MASE of %s. A forecast that does not beat seasonal naive is not adding value — repeating %s would have been at least as accurate.",
              r2(mae), r2(q_seas), r3(mase_seas), seasonal_phrase)
    }
  } else if (!is.na(mase_naive)) {
    if (beats_naive) {
      sprintf("No seasonal baseline was available, so the forecast is judged against the naive one, which it beats: MAE of %s against %s, a MASE of %s.",
              r2(mae), r2(q_naive), r3(mase_naive))
    } else {
      sprintf("No seasonal baseline was available, so the forecast is judged against the naive one, which it does not beat: MAE of %s against %s, a MASE of %s. A forecast that does not beat the naive rule is not adding value.",
              r2(mae), r2(q_naive), r3(mase_naive))
    }
  } else {
    sprintf("Neither baseline could be computed, because &#x27;%s' does not change from one %s to the next; MAE of %s is reported without a scale-free comparison.",
            actual_h, period_word, r2(mae))
  }

  dm_sentence <- function(res, label) {
    if (is.null(res)) return("")
    if (res$p < 0.05) {
      sprintf(" A Diebold-Mariano test says that gap against the %s baseline is larger than sampling noise(statistic %s, %s), so the forecast is significantly %s than %s.",
              label, r3(res$stat), fmt_pp(res$p),
              if (res$mean_d < 0) "more accurate" else "less accurate", label)
    } else {
      sprintf(" A Diebold-Mariano test cannot separate the forecast from the %s baseline(statistic %s, %s), so on this evidence the two perform equivalently.",
              label, r3(res$stat), fmt_pp(res$p))
    }
  }
  dm_headline <- if (seasonal_available) dm_sentence(dm_seas, "seasonal-naive") else dm_sentence(dm_naive, "naive")

  mape_clause <- if (is.na(mape)) {
    sprintf(" MAPE is not reported: too few periods had a usable non-zero &#x27;%s', so MASE is the only reliable accuracy scale here.", actual_h)
  } else if (n_mape_excl > 0) {
    sprintf(" MAPE reads %s percent but rests on only %s of %s periods — %s were excluded because &#x27;%s' was zero or near zero — so lead with MASE, not MAPE.",
            r2(mape), format(n_mape_used, big.mark = ","), format(n_eval, big.mark = ","),
            format(n_mape_excl, big.mark = ","), actual_h)
  } else {
    sprintf(" MAPE reads %s percent across all %s periods.", r2(mape), format(n_eval, big.mark = ","))
  }

  bias_clause <- sprintf(" On average the process %s by %s units of &#x27;%s' per %s, running high in %s percent of periods and low in %s percent.",
                         bias_dir, r2(abs(me)), actual_h, period_word,
                         r2(pct_over), r2(pct_under))

  json_output <- list(
    answer = paste0(
      "Backtest of ", format(n_eval, big.mark = ","), " ", freq_label,
      " forecasts of &#x27;", actual_h, "'",
      if (horizon_usable && !single_horizon) sprintf(" at the level of &#x27;%s' with the most observations (%s)", horizon_h, primary_label) else "",
      ": ", headline, dm_headline, bias_clause, mape_clause
    ),
    cards = lapply(
      c("tldr", "overview", "preprocessing", "actual_vs_forecast",
        "accuracy_metrics", "baseline_comparison", "error_distribution",
        "period_accuracy", "horizon_accuracy", "methods"),
      function(cid) list(id = cid, metrics = metrics)
    )
  )

  list(
    initial_rows = initial_rows, final_rows = final_rows, rows_removed = rows_removed,
    date_h = date_h, forecast_h = forecast_h, actual_h = actual_h, horizon_h = horizon_h,
    n_rows = n_rows, n_dropped = n_dropped, n_dup_rows = n_dup_rows,
    n_unparsed = n_unparsed, n_eval = n_eval,
    freq_label = freq_label, period_word = period_word, cycle_word = cycle_word,
    m = m, seasonal_phrase = seasonal_phrase,
    n_grid = n_grid, n_grid_missing = n_grid_missing, pct_grid_missing = pct_grid_missing,
    mae = mae, rmse = rmse, me = me, mape = mape, smape = smape,
    mape_reliable = mape_reliable,
    n_zero = n_zero, n_near_zero = n_near_zero,
    n_mape_excl = n_mape_excl, n_mape_used = n_mape_used,
    ape_over = ape_over, ape_under = ape_under,
    n_under = n_under, n_over = n_over, n_exact = n_exact,
    pct_under = pct_under, pct_over = pct_over, pct_exact = pct_exact,
    bias_dir = bias_dir,
    q_naive = q_naive, q_seas = q_seas,
    mase_naive = mase_naive, mase_seas = mase_seas,
    beats_naive = beats_naive, beats_seasonal = beats_seasonal,
    seasonal_available = seasonal_available, bench_verdict = bench_verdict,
    bench_n = bench_n, bench_kind = bench_kind, bench_fc_mae = bench_fc_mae,
    bench_naive_mae = bench_naive_mae, bench_seas_mae = bench_seas_mae,
    dm_naive = dm_naive, dm_seas = dm_seas, dm_headline = dm_headline,
    headline = headline, mape_clause = mape_clause, bias_clause = bias_clause,
    has_horizon = has_horizon, horizon_usable = horizon_usable,
    single_horizon = single_horizon, primary_label = primary_label,
    n_horizons = n_horizons, horizon_note = horizon_note,
    horizon_degrades = horizon_degrades, hz_ratio = hz_ratio,
    worst_bucket = worst_bucket, best_bucket = best_bucket,
    worst_mae = worst_mae, best_mae = best_mae,
    series_points_df = series_points_df,
    accuracy_metrics_df = accuracy_metrics_df,
    baseline_comparison_df = baseline_comparison_df,
    error_distribution_df = error_distribution_df,
    period_accuracy_df = period_accuracy_df,
    horizon_accuracy_df = horizon_accuracy_df,
    methods_df = methods_df,
    metrics = metrics, json_output = json_output
  )
}

Your data has more stories to tell.Run any analysis on your own data — validated R modules, interactive reports, AI insights, and PDF export. 500 free credits on signup.

Try Free — No Signup Sign Up Free

Download PDF · Print · Share ·Refresh this analysis ·Edit this report

What's wrong with this card?

Executive Summary

Analysis Overview

Data Quality

Actual vs Forecast

Accuracy Scorecard

The short answer

The detail

What this can't tell you

Forecast vs Naive Baselines

Error Distribution

Accuracy Over Time

Accuracy by Horizon

Methods & Disclosure

Forecast Accuracy Backtest — Are Your Forecasts Actually Any Good?

Why This Method?

What This Analysis Covers

Standard Library

Core Analysis Pipeline

Step 1: Resolve the mapped columns (humanized for every sentence)

Step 2: Parse dates; coerce both numbers with the 95% rule

Step 3: Horizon column — numeric, binned, or lumped categorical

Step 4: The actual series (all rows), its cadence, and its calendar grid

Step 5: Naive and seasonal-naive baselines on that grid

Step 6: Headline rows — the primary horizon if one is mapped,

Step 7: Error metrics on the evaluated periods

Step 8: Paired baseline errors for the evaluated periods

Step 9: Like-for-like baseline comparison on one common subset

Step 10: Chart datasets (both capped well under the row limit)

Step 11: Accuracy by calendar period

Step 12: Accuracy by horizon

Step 13: The headline metric table

Step 14: Methods and disclosure

Step 15: Metrics and the one-paragraph answer

Cite this analysis

Report an Issue