Executive Summary
What contacting the top of the Glucose ranking actually captures
The short answer
Screening patients by glucose level captures 24.6% of all diabetes cases in the top 10%—a 2.46x improvement over random screening. This performance reaches 85.7% of the theoretical maximum for this base rate, and the signal remains strong through the top 30% (55.3% of cases).
The detail
Among 768 patients with 268 diabetes cases (34.90% base rate), ranking by glucose yields a top-decile lift of 2.46x, holding 24.6% of all responders. The top 20% captures 41.9% of cases (lift 2.10x); the top 30% captures 55.3% (lift 1.84x). A perfect score would reach 2.87x at 10% depth; glucose achieves 85.7% of that ceiling. No contact cost or value per response was supplied, so no profit-maximizing depth is computed. The break-even analysis shows that contacting the top 10% justifies the effort once a diabetes diagnosis is worth at least 1.17 times the cost of screening one patient. All gains are measured on the 768 rows for which glucose was supplied.
What this can't tell you
If the glucose ranking was fitted on these same 768 patients, the 2.46x gain is an in-sample estimate and will be optimistic for a fresh cohort. The data contains no marker indicating whether glucose came from a model trained on this population or from independent measurements. A transaction-level or temporal export would clarify whether the ranking reflects training-set inflation.
Analysis Overview
Gains and lift from ranking 768 rows by Glucose.
The short answer
Screening by glucose level finds diabetes cases far more efficiently than random selection. The top 10% of patients by glucose level contain 24.6% of all diabetes cases — roughly 2.5 times what you'd expect by chance. This pattern is strong and consistent across the first 20% of the population.
The detail
The population is 768 patients with 268 diabetes cases (base rate 34.90%). Sorting by glucose and contacting the best-scoring 10% delivers a cumulative lift of 2.455x — meaning you capture 24.6% of all responders in just 10% of the population. The theoretical maximum for this base rate is 2.87x, so glucose reaches 85.7% of the ceiling. Lift remains above 2.10x through the first 20% of patients, then declines toward 1.00 as you contact deeper into the population.
What this can't tell you
The data does not record whether glucose values were computed from a model already fitted to these same outcomes. If they were, the 2.455x lift is measured on in-sample rows and a fresh screening campaign should expect lower efficiency. If the scores are genuinely out-of-sample, this lift estimates future campaign performance, but carries sampling uncertainty tied to the 268 cases available.
Data Quality
Which rows were ranked, how the responder class was defined, and how ties were treated.
The short answer
All 768 patients loaded with complete glucose values and outcome labels. No rows were excluded. The responding class (diabetes cases) comprises 268 patients (34.90%); non-responders number 500. Glucose values are heavily tied — 97.5% of patients share their glucose reading with at least one other patient — so decile boundaries were handled by proportionally splitting responders within tied groups rather than arbitrarily assigning them.
The detail
768 rows retained; 0 removed. 268 responders ('1') and 500 non-responders ('0'). The 768 patients span 136 distinct glucose values, with the largest tied group holding 17 rows. Since 9 of the 10 decile boundaries fall inside a tied group, responders were apportioned in proportion to the fraction of the tied group contacted. This approach yields some non-integer responder counts but reflects the expected result of random tie-breaking rather than one arbitrary ordering.
What this can't tell you
The analysis measures gains on the same 768 rows for which glucose values were supplied. The data does not indicate whether those glucose values came from a model already exposed to the outcomes, versus a genuinely independent source. This distinction determines whether the reported lift is in-sample (and will overstate next campaign performance) or a forward-looking estimate. Consider documenting the source and timing of the glucose variable to clarify this.
Cumulative Gains Curve
Share of all responders captured against the share of the list contacted, ranked by Glucose.
The short answer
The cumulative gains curve rises sharply above the random-targeting diagonal, reaching 24.6% of diabetes cases at 10% population depth and continuing to climb steadily through 30%, where it captures 55.3% of cases. The curve flattens visibly beyond 50% depth, signalling diminishing returns from screening further down the glucose ranking.
The detail
The curve plots population share (horizontal) against the cumulative share of diabetes cases found (vertical). At 10% depth the curve sits at 24.6% gain; at 20%, 41.9%; at 30%, 55.3%; at 50%, 76.6%. The straight diagonal represents random targeting (x% of population yields x% of cases). The vertical gap between the curve and diagonal at any depth is the extra value the ranking delivers. The curve reaches 76.6% of all cases by screening half the population, then flattens considerably, indicating that glucose stratification loses discriminative power in the lower-scoring half.
What this can't tell you
The curve describes association between glucose and diabetes status in this cohort; it does not establish that screening high-glucose patients causes diabetes diagnosis or treatment response. Steep initial rise and later flattening are consistent with a working ranking but do not indicate why glucose separates cases from non-cases. A clinical context would be needed to interpret the biological meaning of the glucose threshold where the curve inflects.
Cumulative Lift Curve
How many times better than random targeting each contact depth performs.
The short answer
Lift starts strong at 2.455x in the top 10% and holds remarkably steady through the first 20% of patients (2.10x), then declines gradually toward random baseline. The curve shows no sharp drops or outliers — it is a smooth, expected decay pattern consistent with a well-ordered ranking.
The detail
Cumulative lift begins at 2.455x at 10% population depth and remains above 2.40x through the first 5.5% of patients. It falls to 2.10x by 20% and 1.84x by 30%, converging toward 1.00 at full depth (where contacting all patients captures all cases by definition). The tightest cluster occurs in the 5–10% range, where lift fluctuates between 2.38x and 2.46x. No single depth shows an anomalous spike or collapse; the downward trajectory is consistent throughout.
What this can't tell you
Lift always decays with depth — this is mechanical and not a weakness of the glucose ranking. The pattern here reflects how quickly the association between glucose and diabetes weakens as you move from highest to lower glucose patients. The curve does not reveal whether glucose association differs by patient subgroup (e.g., age or weight), nor does it estimate the effect of actually contacting patients (lift measures ranking association only, not intervention response).
Decile Table
Responders, response rate, lift and cumulative gain in each decile of the Glucose ranking.
| Bucket | Population PCT | Contacted | Responders | Response Rate PCT | Lift | Cumulative Gain PCT | Cumulative Lift |
|---|---|---|---|---|---|---|---|
| Decile 1 | 10 | 77 | 65.8 | 85.68 | 2.455 | 24.55 | 2.455 |
| Decile 2 | 20 | 154 | 46.6 | 60.68 | 1.739 | 41.94 | 2.097 |
| Decile 3 | 30 | 230 | 35.9 | 46.7 | 1.338 | 55.32 | 1.844 |
| Decile 4 | 40 | 307 | 32.8 | 42.75 | 1.225 | 67.57 | 1.689 |
| Decile 5 | 50 | 384 | 24.2 | 31.47 | 0.902 | 76.59 | 1.532 |
| Decile 6 | 60 | 461 | 21.6 | 28.07 | 0.805 | 84.64 | 1.411 |
| Decile 7 | 70 | 538 | 17.9 | 23.29 | 0.668 | 91.31 | 1.304 |
| Decile 8 | 80 | 614 | 10.8 | 14 | 0.401 | 95.33 | 1.192 |
| Decile 9 | 90 | 691 | 6.8 | 8.9 | 0.255 | 97.88 | 1.088 |
| Decile 10 | 100 | 768 | 5.7 | 7.4 | 0.212 | 100 | 1 |
The short answer
Decile 1 (top 10%) responds at 85.68%—a lift of 2.46x—while Decile 10 (bottom 10%) responds at only 7.40% (lift 0.21x). Lift decays monotonically down the ranking, with 4 of 10 deciles beating random targeting (lift ≥ 1.0). The top three deciles together capture 55.3% of all diabetes cases.
The detail
The top decile holds 65.8 responders at 85.68% response rate and 2.46x lift. Decile 2 drops to 60.68% (46.6 responders, lift 1.74x); Decile 3 to 46.7% (35.9 responders, lift 1.34x); Decile 4 to 42.75% (32.8 responders, lift 1.23x). By Decile 5 the response rate falls to 31.47% (lift 0.90x), below the base rate, and continues falling through Deciles 6–10. Cumulative gains show 24.6% of all cases in the top decile, 41.9% by the top two, and 55.3% by the top three. The monotone decay in lift (no reversals) confirms the ranking is working as expected; fractional responder counts reflect proportional tie-breaking at decile boundaries where glucose values are tied.
What this can't tell you
This table describes how well glucose stratifies the current cohort; it does not establish the threshold glucose level at which to screen or treat. The response rates shown are associations, not causal effects of screening. Deciles 5–10 show lift below 1.0, meaning they respond at rates below the population base rate, but the table alone cannot determine whether screening those patients has clinical value (e.g., for early intervention in borderline cases).
Targeting Economics
What each contact depth costs to justify, and the profit at it when the economics are supplied.
| Depth | Contacted | Responders | Response Rate | Breakeven Value To Cost | Net Profit |
|---|---|---|---|---|---|
| Top 10.0% | 77 | 65.8 | 85.68% | 1.17 | n/a |
| Top 20.0% | 154 | 112.4 | 73.18% | 1.37 | n/a |
| Top 30.0% | 230 | 148.3 | 64.35% | 1.55 | n/a |
| Top 40.0% | 307 | 181.1 | 58.95% | 1.70 | n/a |
| Top 50.0% | 384 | 205.3 | 53.46% | 1.87 | n/a |
| Top 60.0% | 461 | 226.8 | 49.23% | 2.03 | n/a |
| Top 70.0% | 538 | 244.7 | 45.52% | 2.20 | n/a |
| Top 80.0% | 614 | 255.5 | 41.58% | 2.40 | n/a |
| Top 90.0% | 691 | 262.3 | 37.95% | 2.64 | n/a |
| Top 100.0% | 768 | 268.0 | 34.90% | 2.87 | n/a |
The short answer
Screening the top 10% by glucose breaks even once a diabetes diagnosis is worth 1.17 times the cost of one screen. By the top 50%, the break-even ratio rises to 1.87x; by the full population, 2.87x. No contact cost or response value was supplied, so profit cannot be computed; use the break-even column to identify the screening depth that matches your economics.
The detail
The break-even value-to-cost ratio at each depth is the multiplier at which revenue from detected cases exactly covers screening costs. Top 10% requires 1.17x; top 20%, 1.37x; top 30%, 1.55x; top 40%, 1.70x; top 50%, 1.87x; top 60%, 2.03x; top 70%, 2.20x; top 80%, 2.40x; top 90%, 2.64x; top 100%, 2.87x. Read down the column and stop at the depth where the ratio matches or falls below the actual value of detecting one case relative to the cost of screening one patient. For example, if each detected case is worth 1.50 times the screening cost, screen to the top 30% depth.
What this can't tell you
The break-even ratios are computed on the same 768 rows used to score glucose, so they reflect in-sample performance. A fresh screening campaign should expect higher break-even ratios (lower response rates) than shown here, because the ranking was optimized on this cohort. The analysis does not account for the clinical value of early detection, treatment adherence, or the cost of false positives—only the statistical efficiency of the glucose ranking at capturing cases.
Method & Disclosure
How the ranking, the buckets, the ties and the economics were computed, and what the numbers do not establish.
| Item | Detail |
|---|---|
| Ranking | Rows are sorted by Glucose from highest to lowest; higher is assumed to mean more likely to be '1'. |
| Gains | Cumulative gain at depth d is the share of all 268 '1' rows found within the best-scoring d of the population. |
| Lift | Cumulative lift at depth d is that gain divided by d — how many times better than contacting the same share of the list at random. Random targeting sits at lift 1.00 and a perfect score would reach 2.87x at 10% depth given this base rate of 34.90%. |
| Decile split | 768 usable rows give ten buckets of 10% each. |
| Ties | 768 rows share 136 distinct Glucose values; 97.5% of rows sit on a value they share with at least one other row and the largest tied group holds 17 rows. 9 of the 10 decile boundaries fall inside a tied group, so the responders in that group were shared out in proportion to how much of it is contacted — the expected result of breaking those ties at random, rather than one arbitrary ordering. That is why some responder counts are not whole numbers. |
| Economics | No contact cost and value per response were supplied, so no profit figure is computed. The break-even column instead gives the value-to-cost ratio at which each depth would just wash its face. |
| Where these numbers come from | The gains are measured on the same 768 rows the Glucose values were supplied for. Nothing in the data records whether those scores were produced by a model that had already seen these outcomes, so this analysis cannot tell you which case you are in. If the model was fitted on these rows, the 2.46x top-decile lift is an in-sample figure and a fresh campaign should be expected to capture less; if the scores are genuinely out-of-sample, it is an estimate of what the next campaign captures, carrying the usual sampling uncertainty of 268 responders. |
The short answer
Gains and lift are computed by sorting 768 patients by glucose (highest first) and counting how many of the 268 diabetes cases accumulate at each depth. No model is fitted and no statistical test is run; the numbers are descriptive summaries of this dataset. Lift describes the association between glucose ranking and diabetes status, not the causal effect of screening.
The detail
Rows were sorted by glucose in descending order (higher glucose = higher diabetes likelihood). Cumulative gain at depth d is the share of all 268 cases found in the top d% of the list. Cumulative lift at depth d is that gain divided by d, expressing how many times better than random selection the ranking performs. The list was split into ten equal deciles of 77 rows each. Ties were handled by proportional apportionment: 768 rows carry 136 distinct glucose values, and 97.5% of rows share a value with at least one other. Nine of ten decile boundaries fall inside tied groups; responders within those groups were distributed proportionally to the share of the group contacted, mimicking random tie-breaking. Break-even ratios give the value-to-cost multiplier at which each depth covers itself, requiring no assumptions about economics. No profit is computed because no cost or value per case was supplied.
What this can't tell you
The analysis cannot determine whether glucose was measured independently or computed from this same cohort. If glucose came from a model fitted on these 768 patients, the 2.46x top-decile lift is optimistic and a fresh campaign should expect lower gains. If the scores are genuinely out-of-sample, the figures estimate real-world performance, subject to sampling variation from 268 cases. Gains describe statistical association; they do not establish that screening high-glucose patients causes better health outcomes or that the glucose threshold identified here is clinically optimal. A prospective validation study or hold-out test set would clarify generalization.
Lift & Gains — Targeting Value
Answers the campaign question "if I only contact the top N%, how much of the total value do I capture?". Takes a model score (or any ranking score) plus the actual binary outcome, sorts the population from best score to worst, and reports the cumulative gains curve, the lift curve, a decile/ventile table, and the profit-maximizing contact depth when the economics are supplied.
Why This Method?
A classifier's AUC tells you how well a score ranks; it does not tell you what a campaign captures. Gains and lift convert the same score into the operational number a targeting decision needs: contact the top N percent and you reach this share of all responders, this many times better than contacting N percent at random.
What This Analysis Covers
- The cumulative gains curve against the random-targeting diagonal
- The cumulative lift curve against the lift = 1 baseline
- A decile (or ventile / quintile) table: population share, responders,
response rate, within-bucket lift, cumulative gain, cumulative lift
- Targeting economics: the break-even value-to-cost ratio at every depth,
and — when a contact cost and a value per response are supplied — the profit-maximizing depth
- An explicit statement that the gains are measured on the rows the scores
were supplied for, and what that means if the model was fitted on them
Standard Library
Platform standard-library module (LAT-1441): runs on ANY dataset via the semantic mapping {actual, score} — the same mapping as the ROC analysis, so the two tools accept the same column choices. All narrative is derived from the user's own column names and computed values.
suppressPackageStartupMessages(library(DT))
suppressPackageStartupMessages(library(htmlwidgets))
suppressPackageStartupMessages(library(arrow))
suppressPackageStartupMessages(library(knitr))
suppressPackageStartupMessages(library(rmarkdown))
suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(tidyr))
suppressPackageStartupMessages(library(ggplot2))
suppressPackageStartupMessages(library(stringr))
suppressPackageStartupMessages(library(lubridate))
suppressPackageStartupMessages(library(broom))
suppressPackageStartupMessages(library(Matrix))
suppressPackageStartupMessages(library(cluster))
suppressPackageStartupMessages(library(data.table))Round before banding: an exact 3.0 arrives as 2.9999999999999996 out of floating-point division and must not fall into the band below.
lift <- round(lift, 3)
if (lift < 1.1) "no better than random targeting"
else if (lift < 1.5) "marginal"
else if (lift < 2) "modest"
else if (lift < 3) "strong"
else "very strong"
}Step 1: Row accounting + semantic column discovery
initial_rows <- nrow(df)
if (!"actual" %in% names(df)) {
stop("column_mapping must map an 'actual' column (the true yes/no outcome).")
}
if (!"score" %in% names(df)) {
stop("column_mapping must map a 'score' column (the ranking score, higher = better).")
}
actual_name <- humanize_semantic("actual", col_map)
score_name <- humanize_semantic("score", col_map)Step 2: Choose the positive class + binarize the outcome
Same rule as the ROC module so the two tools agree on which class is "positive" for an identical column mapping.
v_raw <- df$actual
if (is.logical(v_raw)) {
keep_a <- !is.na(v_raw)
df <- df[keep_a, , drop = FALSE]
y <- as.integer(df$actual)
positive_label <- "TRUE"
negative_label <- "FALSE"
positive_rule <- "the boolean TRUE value"
} else {
vc <- trimws(as.character(v_raw))
keep_a <- !is.na(v_raw) & !is.na(vc) & vc != ""
df <- df[keep_a, , drop = FALSE]
vc <- vc[keep_a]
lv <- sort(unique(vc))
if (length(lv) < 2) {
stop(sprintf(
paste0("The outcome column('%s') has only one value ('%s') — a lift and gains ",
"analysis needs both responders and non-responders present."),
actual_name, if (length(lv) == 1) lv[1] else "empty"))
}
pat <- paste0("^(1|yes|true|positive|pos|responder|responded|response|",
"convert|converted|conversion|purchase|purchased|buyer|bought|",
"subscribed|churn|churned|fraud|default|defaulted|disease)$")
hits <- lv[grepl(pat, tolower(lv))]
if (length(hits) >= 1) {
positive_label <- hits[length(hits)]
positive_rule <- "it matched a conventional positive label"
} else {
positive_label <- lv[length(lv)]
positive_rule <- "no conventional positive label was found, so the alphabetically-last class was used"
}
neg_levels <- setdiff(lv, positive_label)
negative_label <- if (length(neg_levels) == 1) neg_levels[1]
else paste0("not ", positive_label)
if (length(lv) > 2) {
positive_rule <- paste0(positive_rule,
"; the outcome had more than two values, so everything else was pooled as non-responding")
}
y <- as.integer(vc == positive_label)
}Step 3: Coerce the score to numeric under the 95% rule
sc_raw <- df$score
if (is.numeric(sc_raw)) {
sc <- as.numeric(sc_raw)
coercion_rate <- 1
} else {
chr <- trimws(as.character(sc_raw))
nonblank <- !is.na(chr) & chr != ""
conv <- suppressWarnings(as.numeric(chr))
n_nonblank <- sum(nonblank)
coercion_rate <- if (n_nonblank > 0) sum(!is.na(conv) & nonblank) / n_nonblank else 0
if (coercion_rate < 0.95) {
stop(sprintf(
paste0("The score column('%s') could not be read as a number — only %s of its ",
"non-blank values converted. A lift and gains analysis needs a numeric ",
"ranking score where a higher value means more likely to respond."),
score_name, fpct(coercion_rate)))
}
sc <- conv
}
keep_s <- !is.na(sc) & !is.na(y)
sc <- sc[keep_s]
y <- y[keep_s]
final_rows <- length(y)
rows_removed <- initial_rows - final_rowsStep 4: Guards — enough rows, both classes, a score that ranks
if (final_rows < 50) {
stop(sprintf(
paste0("Only %d rows have both a usable outcome('%s') and a numeric score ('%s') — ",
"at least 50 are needed before the ranked list can be split into buckets."),
final_rows, actual_name, score_name))
}
n_responders <- sum(y == 1L)
n_non <- sum(y == 0L)
if (n_responders < 10 || n_non < 10) {
stop(sprintf(
paste0("A lift and gains analysis needs at least 10 of each outcome, but '%s' has ",
"%d '%s' and %d '%s' rows."),
actual_name, n_responders, positive_label, n_non, negative_label))
}
if (length(unique(sc)) < 2) {
stop(sprintf(
paste0("The score column('%s') holds a single value for every row, so it cannot ",
"rank anyone above anyone else — there is no top N%% to target."),
score_name))
}
n_total <- final_rows
base_rate <- n_responders / n_totalStep 5: Aggregate to unique score LEVELS (descending)
Working at the level of tied score values rather than of rows makes every number below independent of the arbitrary order of tied rows.
ord <- order(sc, decreasing = TRUE)
s_o <- sc[ord]
y_o <- y[ord]
run_end <- which(c(diff(s_o) != 0, TRUE))
cum_n <- run_end # rows covered through each level
cum_r <- cumsum(y_o)[run_end] # responders covered through each level
lev_n <- diff(c(0, cum_n))
lev_r <- diff(c(0, cum_r))Expected responders captured when contacting exactly k best-scored rows. Inside a run of tied scores the responders are shared out in proportion to how much of the run is contacted — the expected value under a random tie-break, and the only order-independent answer available.
gain_at_k <- function(k) {
k <- pmin(pmax(k, 0), n_total)
stats::approx(x = c(0, cum_n), y = c(0, cum_r), xout = k, rule = 2)$y
}
gain_pct_at <- function(d) gain_at_k(d * n_total) / n_responders
lift_at <- function(d) {
g <- gain_pct_at(d)
out <- rep(NA_real_, length(d))
ok <- d > 0
out[ok] <- g[ok] / d[ok]
out
}Step 6: Tie diagnostics — how much the tie rule had to do
n_levels <- length(lev_n)
max_tie <- max(lev_n)
pct_rows_tied <- sum(lev_n[lev_n > 1]) / n_totalStep 7: Bucket resolution — deciles unless the data says otherwise
n_buckets <- if (n_total >= 2000) 20L else if (n_total >= 100) 10L else 5L
bucket_word <- switch(as.character(n_buckets),
"20" = "ventile", "10" = "decile", "5" = "quintile")
bucket_title <- switch(as.character(n_buckets),
"20" = "Ventile", "10" = "Decile", "5" = "Quintile")
bucket_reason <- if (n_buckets == 20L) {
paste0(format(n_total, big.mark = ","),
" usable rows give twenty buckets of 5% each")
} else if (n_buckets == 10L) {
paste0(format(n_total, big.mark = ","),
" usable rows give ten buckets of 10% each")
} else {
paste0("only ", format(n_total, big.mark = ","),
" usable rows, so the split is five buckets of 20% each")
}
bk <- (seq_len(n_buckets)) * n_total / n_bucketsWhich bucket boundaries land strictly inside a run of tied scores?
b_idx <- vapply(bk, function(b) which(cum_n >= b - 1e-9)[1], integer(1))
at_level_end <- abs(cum_n[b_idx] - bk) < 1e-9
boundaries_in_tie <- sum((!at_level_end) & (lev_n[b_idx] > 1))Step 8: The bucket table
cum_resp <- gain_at_k(bk)
bucket_resp <- diff(c(0, cum_resp))
bucket_rows <- diff(c(0, bk))
bucket_rate <- bucket_resp / bucket_rows
bucket_lift <- bucket_rate / base_rate
cum_gain <- cum_resp / n_responders
depth_seq <- seq_len(n_buckets) / n_buckets
cum_lift <- cum_gain / depth_seq
bucket_table_df <- data.frame(
bucket = paste(bucket_title, seq_len(n_buckets)),
population_pct = round(100 * depth_seq, 2),
contacted = round(bk),
responders = round(bucket_resp, 1),
response_rate_pct = round(100 * bucket_rate, 2),
lift = round(bucket_lift, 3),
cumulative_gain_pct = round(100 * cum_gain, 2),
cumulative_lift = round(cum_lift, 3),
stringsAsFactors = FALSE
)Step 9: Curve datasets on a fixed depth grid (<= 201 points each)
grid <- seq(0, 1, by = 0.005)
gains_curve_df <- data.frame(
population_pct = round(100 * grid, 2),
cumulative_gain_pct = round(100 * gain_pct_at(grid), 3),
stringsAsFactors = FALSE
)
grid2 <- grid[grid > 0]
lift_curve_df <- data.frame(
population_pct = round(100 * grid2, 2),
cumulative_lift = round(lift_at(grid2), 3),
stringsAsFactors = FALSE
)Step 10: Headline depths + the attainable ceiling
gain_10 <- gain_pct_at(0.10)
gain_20 <- gain_pct_at(0.20)
gain_30 <- gain_pct_at(0.30)
gain_50 <- gain_pct_at(0.50)
top_decile_lift <- gain_10 / 0.10
lift_20 <- gain_20 / 0.20
lift_30 <- gain_30 / 0.30A perfect score puts every responder first, so at depth d the most gain anyone could capture is min(1, d / base_rate) and the most lift is min(1/d, 1/base_rate).
max_top_decile_lift <- min(1 / 0.10, 1 / base_rate)
ceiling_ratio <- top_decile_lift / max_top_decile_lift
band <- lift_band(top_decile_lift)A score sitting on its own ceiling is the signature of a label that leaked into the score, or of a score read back off the same rows it was fitted on — flagged as a condition, never asserted as a fact.
near_ceiling <- is.finite(ceiling_ratio) && ceiling_ratio >= 0.95Step 11: Targeting economics
Always computable: the value-to-cost ratio at which contacting to a depth breaks even, which is one divided by the response rate achieved to it.
econ_contacted <- bk
econ_resp <- cum_resp
econ_rate <- econ_resp / econ_contacted
breakeven_ratio <- ifelse(econ_resp > 0, econ_contacted / econ_resp, NA_real_)
contact_cost <- param_positive(params, c("contact_cost", "cost_per_contact",
"cost", "contact_cost_per_person"))
value_per_response <- param_positive(params, c("value_per_response",
"revenue_per_response",
"value_per_conversion",
"value", "margin_per_response"))
economics_on <- !is.na(contact_cost) && !is.na(value_per_response)
opt_depth <- NA_real_; opt_profit <- NA_real_; opt_contacts <- NA_real_
opt_resp <- NA_real_; profit_all <- NA_real_; profit_positive <- NA
econ_profit <- rep(NA_real_, n_buckets)
if (economics_on) {
econ_profit <- value_per_response * econ_resp - contact_cost * econ_contacted
fg <- seq(0.005, 1, by = 0.005)
fg_profit <- value_per_response * gain_at_k(fg * n_total) -
contact_cost * (fg * n_total)
ok_idx <- which(!is.na(fg_profit) & is.finite(fg_profit))
if (length(ok_idx) > 0) {
best <- ok_idx[which.max(fg_profit[ok_idx])]
opt_depth <- fg[best]
opt_profit <- fg_profit[best]
opt_contacts <- opt_depth * n_total
opt_resp <- gain_at_k(opt_contacts)
}
profit_all <- value_per_response * n_responders - contact_cost * n_total
profit_positive <- is.finite(opt_profit) && opt_profit > 0
}
targeting_economics_df <- data.frame(
depth = paste0("Top ", fnum(100 * depth_seq, 1), "%"),
contacted = fint(econ_contacted),
responders = fnum(econ_resp, 1),
response_rate = fpct(econ_rate, 2),
breakeven_value_to_cost = fnum(breakeven_ratio, 2),
net_profit = if (economics_on) fnum(econ_profit, 2) else rep("n/a", n_buckets),
stringsAsFactors = FALSE
)Step 12: Methods and disclosure table
tie_detail <- if (max_tie <= 1) {
paste0("Every one of the ", format(n_total, big.mark = ","), " rows carries a distinct ",
score_name, " value, so no bucket boundary had to be split.")
} else {
paste0(format(n_total, big.mark = ","), " rows share ", format(n_levels, big.mark = ","),
" distinct ", score_name, " values; ", fpct(pct_rows_tied),
" of rows sit on a value they share with at least one other row and the ",
"largest tied group holds ", format(max_tie, big.mark = ","), " rows. ",
if (boundaries_in_tie > 0)
paste0(boundaries_in_tie, " of the ", n_buckets, " ", bucket_word,
" boundaries fall inside a tied group, so the responders in that group ",
"were shared out in proportion to how much of it is contacted — the ",
"expected result of breaking those ties at random, rather than one ",
"arbitrary ordering. That is why some responder counts are not whole numbers.")
else
paste0("No ", bucket_word, " boundary falls inside a tied group, so the counts ",
"are exact whole numbers and no tie-breaking was needed."))
}
provenance_detail <- paste0(
"The gains are measured on the same ", format(n_total, big.mark = ","),
" rows the ", score_name, " values were supplied for. Nothing in the data records ",
"whether those scores were produced by a model that had already seen these outcomes, ",
"so this analysis cannot tell you which case you are in. If the model was fitted on ",
"these rows, the ", fnum(top_decile_lift, 2), "x top-decile lift is an in-sample figure ",
"and a fresh campaign should be expected to capture less; if the scores are genuinely ",
"out-of-sample, it is an estimate of what the next campaign captures, carrying the ",
"usual sampling uncertainty of ", format(n_responders, big.mark = ","), " responders."
)
methods_details_df <- data.frame(
item = c("Ranking", "Gains", "Lift", paste0(bucket_title, " split"),
"Ties", "Economics", "Where these numbers come from"),
detail = c(
paste0("Rows are sorted by ", score_name,
" from highest to lowest; higher is assumed to mean more likely to be '",
positive_label, "'."),
paste0("Cumulative gain at depth d is the share of all ",
format(n_responders, big.mark = ","), " '", positive_label,
"' rows found within the best-scoring d of the population."),
paste0("Cumulative lift at depth d is that gain divided by d — how many times ",
"better than contacting the same share of the list at random. Random ",
"targeting sits at lift 1.00 and a perfect score would reach ",
fnum(max_top_decile_lift, 2), "x at 10% depth given this base rate of ",
fpct(base_rate, 2), "."),
paste0(bucket_reason, "."),
tie_detail,
if (economics_on)
paste0("Profit at depth d is ", fnum(value_per_response, 2),
" per response times the responders reached, minus ", fnum(contact_cost, 2),
" per contact times the contacts made. The depth reported as best was chosen ",
"on these same rows, so the profit at it is the most favourable reading of ",
"this data rather than a forecast.")
else
paste0("No contact cost and value per response were supplied, so no profit figure ",
"is computed. The break-even column instead gives the value-to-cost ratio at ",
"which each depth would just wash its face."),
provenance_detail
),
stringsAsFactors = FALSE
)Step 13: KPI metrics + machine channels
metrics <- list(
`Observations` = n_total,
`Responders` = n_responders,
`Base Rate` = round(base_rate, 4),
`Top Decile Lift` = round(top_decile_lift, 3),
`Gain at 10%` = round(gain_10, 4),
`Gain at 20%` = round(gain_20, 4)
)
lift_summary <- list(
n_total = n_total, n_responders = n_responders, base_rate = base_rate,
top_decile_lift = top_decile_lift,
gain_10 = gain_10, gain_20 = gain_20, gain_30 = gain_30, gain_50 = gain_50,
lift_20 = lift_20, lift_30 = lift_30,
max_top_decile_lift = max_top_decile_lift, ceiling_ratio = ceiling_ratio,
near_ceiling = near_ceiling, band = band,
n_buckets = n_buckets, bucket_word = bucket_word,
n_levels = n_levels, max_tie = max_tie, pct_rows_tied = pct_rows_tied,
boundaries_in_tie = boundaries_in_tie,
economics_on = economics_on, contact_cost = contact_cost,
value_per_response = value_per_response,
opt_depth = opt_depth, opt_profit = opt_profit, opt_contacts = opt_contacts,
opt_resp = opt_resp, profit_all = profit_all,
breakeven_10 = breakeven_ratio[1],
positive_label = positive_label, negative_label = negative_label,
bucket_resp = bucket_resp, cum_resp = cum_resp
)
json_output <- list(
answer = paste0(
"Ranking ", format(n_total, big.mark = ","), " rows by ", score_name,
" and counting '", positive_label, "' outcomes in ", actual_name,
": the best-scoring 10% of the list holds ", fpct(gain_10),
" of all ", format(n_responders, big.mark = ","),
" responders, a lift of ", fnum(top_decile_lift, 2),
"x over the ", fpct(base_rate, 2), " base rate. The top 20% holds ",
fpct(gain_20), " and the top 30% holds ", fpct(gain_30), ". ",
if (economics_on && isTRUE(profit_positive))
paste0("At the supplied economics, profit peaks at a contact depth of ",
fpct(opt_depth), ".")
else if (economics_on)
paste0("At the supplied economics no contact depth turns a profit on this data.")
else
paste0("Contacting the top 10% pays for itself once a response is worth at least ",
fnum(breakeven_ratio[1], 2), " times a contact."),
" These gains are measured on the rows the scores were supplied for, so if the model ",
"was fitted on them they overstate what a fresh campaign captures."
),
cards = lapply(
c("tldr", "overview", "preprocessing", "gains_curve", "lift_curve",
"bucket_table", "targeting_economics", "methods"),
function(cid) list(id = cid, metrics = metrics)
)
)
list(
initial_rows = initial_rows, final_rows = final_rows, rows_removed = rows_removed,
actual_name = actual_name, score_name = score_name,
positive_label = positive_label, negative_label = negative_label,
positive_rule = positive_rule,
n_total = n_total, n_responders = n_responders, n_non = n_non,
base_rate = base_rate,
n_buckets = n_buckets, bucket_word = bucket_word, bucket_title = bucket_title,
bucket_reason = bucket_reason,
n_levels = n_levels, max_tie = max_tie, pct_rows_tied = pct_rows_tied,
boundaries_in_tie = boundaries_in_tie,
top_decile_lift = top_decile_lift, band = band,
gain_10 = gain_10, gain_20 = gain_20, gain_30 = gain_30, gain_50 = gain_50,
lift_20 = lift_20, lift_30 = lift_30,
max_top_decile_lift = max_top_decile_lift, ceiling_ratio = ceiling_ratio,
near_ceiling = near_ceiling,
best_bucket_lift = bucket_lift[1], worst_bucket_lift = bucket_lift[n_buckets],
breakeven = breakeven_ratio,
economics_on = economics_on, contact_cost = contact_cost,
value_per_response = value_per_response,
opt_depth = opt_depth, opt_profit = opt_profit, opt_contacts = opt_contacts,
opt_resp = opt_resp, profit_all = profit_all, profit_positive = profit_positive,
provenance_detail = provenance_detail, tie_detail = tie_detail,
gains_curve_df = gains_curve_df, lift_curve_df = lift_curve_df,
bucket_table_df = bucket_table_df,
targeting_economics_df = targeting_economics_df,
methods_details_df = methods_details_df,
lift_summary = lift_summary, metrics = metrics, json_output = json_output
)
}