Group Lasso: When to Use It for Feature Selection

Hypothetical: a team has time, place, and product data for prices, and wants a model it can explain. The features fall into logical groups: time variables (hour, day, week, month), geography (city, state, region, country), and product attributes (category, subcategory, brand). Standard Lasso can keep "month" and drop "week," or keep "state" without "city." A model like that is hard to explain. Treating each group as a unit avoids the problem.

Group Lasso handles the case that standard regularization ignores: features do not always stand alone. They cluster into natural groups with shared meaning. When you encode "product_category" with 12 dummy variables, those are not 12 independent features. They are a single conceptual unit. Standard Lasso might keep 3 dummies and drop 9, leaving a model that claims only some categories matter. Group Lasso decides at the group level: either product category matters (the 12 dummies enter the model together), or it does not (all 12 are set to zero). Within a group that is kept, the penalty shrinks the coefficients together rather than zeroing individual ones; if you also want sparsity inside a kept group, that is the job of the Sparse Group Lasso variant.

Worked Example (Simulated): Which Groups Does Group Lasso Keep?

Question: Can group lasso find sets of inputs that help guess a result?

Inputs: Simulated data, seed 2718. The script makes 240 rows and 18 fields in 6 groups of 3 fields each. The result is built from two groups on purpose: usage and account health carry signal, and the other four groups are set to zero. Finding the two signal groups is therefore expected here. The script also adds noise. It uses 180 rows to train and 60 rows to test.

Method: Group lasso keeps linked fields as a set. The script sets each column's scale from the train rows, and the penalty is scaled by the square root of the group size. It splits the train rows into four parts and tries each lambda (the penalty value) on those parts. It picks the lambda with the lowest mean squared error, the mean of squared gaps.

import numpy as np

SEED = 2718
rng = np.random.default_rng(SEED)

n, groups, per_group = 240, 6, 3
X = rng.normal(size=(n, groups * per_group))
true_beta = np.zeros(groups * per_group)
true_beta[0:3] = [1.2, -0.8, 0.6]
true_beta[9:12] = [-1.0, 0.7, 0.9]
y = X @ true_beta + rng.normal(scale=1.4, size=n)

order = rng.permutation(n)
train_idx, test_idx = order[:180], order[180:]
group_names = ["usage", "engagement", "firmographic", "account_health",
               "temporal", "support"]
lambdas = [0.05, 0.1, 0.2, 0.4, 0.8, 1.2]


def standardize_fit(A):
    mean = A.mean(axis=0)
    scale = A.std(axis=0)
    scale[scale == 0] = 1.0
    return mean, scale


def group_lasso(A, b, lam, max_iter=5000, tol=1e-8):
    mean, scale = standardize_fit(A)
    Z = (A - mean) / scale
    y_mean = b.mean()
    target = b - y_mean
    spectral = np.linalg.norm(Z, ord=2) ** 2 / len(b)
    step = 1.0 / spectral
    beta = np.zeros(Z.shape[1])
    z = beta.copy()
    momentum = 1.0
    for _ in range(max_iter):
        grad = Z.T @ (Z @ z - target) / len(b)
        candidate = z - step * grad
        for g in range(groups):
            sl = slice(g * per_group, (g + 1) * per_group)
            norm = np.linalg.norm(candidate[sl])
            threshold = step * lam * np.sqrt(per_group)
            candidate[sl] *= max(0.0, 1.0 - threshold / norm) if norm else 0.0
        next_momentum = (1 + np.sqrt(1 + 4 * momentum**2)) / 2
        next_z = candidate + ((momentum - 1) / next_momentum) * (candidate - beta)
        if np.linalg.norm(candidate - beta) < tol * max(1.0, np.linalg.norm(beta)):
            beta = candidate
            break
        beta, z, momentum = candidate, next_z, next_momentum
    return beta, mean, scale, y_mean


def predict(A, fitted):
    beta, mean, scale, y_mean = fitted
    return ((A - mean) / scale) @ beta + y_mean


folds = np.array_split(rng.permutation(train_idx), 4)
cv_mse = []
for lam in lambdas:
    errs = []
    for i in range(4):
        val = folds[i]
        fit = np.concatenate([folds[j] for j in range(4) if j != i])
        fitted = group_lasso(X[fit], y[fit], lam)
        errs.append(np.mean((y[val] - predict(X[val], fitted)) ** 2))
    cv_mse.append(float(np.mean(errs)))

best_lam = lambdas[int(np.argmin(cv_mse))]
fitted = group_lasso(X[train_idx], y[train_idx], best_lam)
pred = predict(X[test_idx], fitted)
ss_res = np.sum((y[test_idx] - pred) ** 2)
ss_tot = np.sum((y[test_idx] - y[test_idx].mean()) ** 2)
r2 = 1 - ss_res / ss_tot
active = []
for g, name in enumerate(group_names):
    if np.linalg.norm(fitted[0][g * per_group:(g + 1) * per_group]) > 1e-6:
        active.append(name)

print(f"Simulated data, seed {SEED}")
print(f"Rows: {n}; features: {groups * per_group}; "
      f"groups: {groups} ({per_group} features each)")
print(f"Signal used to make y: {group_names[0]}, {group_names[3]}")
print(f"Split: {len(train_idx)} train rows, {len(test_idx)} held-out rows")
print("Method: four-fold cross-validation chose lambda by lowest mean "
      "squared error; group penalty used sqrt(group size)")
print("Cross-validation mean squared error by lambda:")
for lam, mse in zip(lambdas, cv_mse):
    print(f"  {lam:.2f}: {mse:.3f}")
print(f"Chosen lambda: {best_lam:.2f}")
print("Selected groups: " + (", ".join(active) if active else "none"))
print(f"Held-out R-squared: {r2:.3f}")

Printed output:

Simulated data, seed 2718
Rows: 240; features: 18; groups: 6 (3 features each)
Signal used to make y: usage, account_health
Split: 180 train rows, 60 held-out rows
Method: four-fold cross-validation chose lambda by lowest mean squared error; group penalty used sqrt(group size)
Cross-validation mean squared error by lambda:
  0.05: 2.180
  0.10: 2.131
  0.20: 2.292
  0.40: 3.127
  0.80: 5.957
  1.20: 6.799
Chosen lambda: 0.10
Selected groups: usage, engagement, account_health, support
Held-out R-squared: 0.773

Result: The chosen lambda was 0.10. The fit kept usage, engagement, account health, and support. Its held-out R-squared was 0.773 on 60 test rows. It also kept groups not marked as signal (engagement and support).

Limit: This is made-up data, not a test on real users. The result may change with other data or a new split.

Sources

• Tibshirani R (1996). Regression Shrinkage and Selection Via the Lasso. Journal of the Royal Statistical Society, Series B. doi 10.1111/j.2517-6161.1996.tb02080.x. Cited for the plain lasso only, not for the group penalty.

Why Feature Groups Matter

Group structure matters because business decisions are often made at the group level, not the coefficient level. Take a hypothetical customer lifetime value model for a SaaS company, with features organized into natural groups such as these:

To see how much your own drivers explain an outcome, and which matter most, try the free linear regression tool.

• Usage metrics: logins per week, features accessed, API calls, sessions, time on platform, clicks, uploads, downloads
• Engagement scores: NPS, support tickets, community posts, feature requests, beta participation
• Firmographic data: industry, company size, revenue band, growth stage, tech stack, employee count, funding status, office locations, publicly traded, years in business, number of subsidiaries, international presence
• Account health: payment history, contract length, renewal rate, upsell conversions, downgrade risk, seat utilization
• Temporal patterns: tenure, days since last login, contract days remaining, season of signup

With standard Lasso, the selection can be incoherent: it might keep "logins per week" and "time on platform" while dropping "sessions," or include "company size" but exclude "employee count." A customer success team would struggle to act on that. With group lasso, the whole usage group is either in or out, so the answer matches how the team thinks about the segments. Whether the grouped model predicts as well is something to test on held-out data, as in the worked example above.

The Mathematics of Group-Level Regularization

Standard Lasso solves this optimization problem:

minimize: (1/2n)||y - Xβ||₂² + λ∑|βⱼ|

The L1 penalty λ∑|βⱼ| encourages sparsity by shrinking individual coefficients to exactly zero. The absolute value creates a non-differentiable kink at zero, which is what allows the solution to land precisely on zero (unlike L2 ridge regression, which only shrinks coefficients close to zero).

Group Lasso extends this to grouped features. Partition your p features into G non-overlapping groups. Let β_g denote the coefficients for group g, with p_g features. The optimization becomes:

minimize: (1/2n)||y - Xβ||₂² + λ∑√(pₐ)||βₐ||₂

The group penalty λ∑√(pg)||βg||₂ has these parts. The L2 norm ||βg||₂ = √(β²₁ + β²₂ + ... + β²ₚₐ) treats all coefficients in a group together. It's the Euclidean distance from the origin. The √(pg) term adjusts for group size. The norm of a group's fit to pure noise grows roughly with √p_g, so without the weight a large group clears the threshold by chance more easily than a small one.

Why is this usually described as giving group-level sparsity? In the standard formulation, the L2 norm within each group combined with the L1-like sum across groups produces that structure. For group g to contribute to the model, at least one coefficient in β_g must be non-zero, which means paying the penalty λ√(pg)||βg||₂. But once you've paid that penalty, there's no additional L1 penalty for having multiple non-zero coefficients within the group. So either all coefficients in a group are zero, or the group is active, and the L2 penalty doesn't encourage sparsity within an active group, only across groups. An individual coefficient inside an active group can still land at zero by coincidence, but the penalty does not push it there.

How the Algorithm Decides Which Groups Matter

Group Lasso uses coordinate descent with block updates. Instead of updating individual coefficients β_j one at a time (as in standard Lasso), it updates entire groups β_g simultaneously. The algorithm cycles through groups, solving for each group's coefficients while holding all other groups fixed.

For group g, when the group's columns are orthonormalized (X_gᵀX_g/n = I, which standard implementations arrange by transforming each group before fitting), the update has a closed form with soft-thresholding at the group level. This holds only when X_gᵀX_g/n = I.

If ||β̃ₐ||₂ ≤ λ√(pₐ):  βₐ = 0 (entire group goes to zero)
Otherwise:           βₐ = (1 - λ√(pₐ)/||β̃ₐ||₂) · β̃ₐ

Here β̃_g = X_gᵀr_g/n is the least squares fit of group g to the partial residual r_g, the part of y not yet explained by the other groups at their current values. Without orthonormal groups there is no closed form, and the block update is solved iteratively (for example by a proximal gradient step). The condition ||β̃g||₂ ≤ λ√(pg) checks whether the group's unregularized contribution is strong enough to overcome the penalty. If not, the entire group is zeroed out. If yes, the group's coefficients are scaled by the factor (1 - λ√(pg)/||β̃g||₂), which shrinks them toward zero while maintaining their relative proportions.

This creates a natural decision boundary: groups with weak signal (small ||β̃_g||₂) are dropped entirely, while groups with strong signal are retained but shrunk. The penalty parameter λ controls how strong the signal must be, higher λ means fewer groups survive.

Structuring Your Features Into Groups

What Group Lasso gives you depends on how you define the groups. Poor grouping removes the benefit. These are common patterns:

Categorical Variables With Dummy Encoding

This is the most common use case. When you one-hot encode "industry" into 15 dummy variables, those 15 features form a natural group. Standard Lasso might keep 4 and drop 11, which is nonsensical. You can't say "industry matters, but only these 4 industries." Group Lasso makes the decision at the right level: either industry-as-a-concept predicts the outcome (the 15 dummies enter together), or it doesn't (all 15 are set to zero).

Polynomial and Interaction Terms

If you include x, x², and x³ for feature x, these should be grouped. Keeping x³ while dropping x creates an uninterpretable model. Similarly, if you create interaction terms between variables A and B (A×B, A×B², A²×B), those interactions form a logical group.

For example, with polynomial price terms and polynomial competitor_price terms, standard Lasso can keep price² and drop the linear and cubic terms, which is mathematically valid but hard to interpret. Grouping the terms keeps each polynomial whole.

Time-Lagged Features for the Same Variable

When forecasting demand, you might include sales_t-1, sales_t-2, ..., sales_t-7 (past 7 days). These 7 features represent the same conceptual variable (historical sales) at different lags. Group them together. Either recent history matters (the lags enter together), or it doesn't (all lags are set to zero).

Grouping the lags of each variable lets the model keep or drop a whole variable's history, rather than an arbitrary mix of lags.

Spatial or Hierarchical Features

Geographic data often has natural hierarchy: latitude, longitude, and altitude form a spatial group. Organizational hierarchy might group manager_id, department_id, division_id, and region_id together. Product taxonomy could group category, subcategory, brand, and SKU.

Grouping these keeps a hierarchy intact: the model keeps or drops the geography as a whole, not a few of its parts.

Choosing the Penalty Parameter: Balancing Sparsity and Fit

The penalty parameter λ controls the sparsity-accuracy tradeoff. Higher λ means stronger regularization, fewer active groups, and simpler models. Lower λ allows more groups, better fit, but less interpretability. How do you choose?

Cross-Validation Path Analysis

The standard approach: compute Group Lasso across a logarithmic grid of λ values (e.g., 0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10) and use 5-fold or 10-fold cross-validation to estimate test error for each λ. Plot held-out error versus number of active groups.

Plot the cross-validation error against the number of active groups. The lowest error may sit at a lambda that keeps several groups, while a larger lambda that keeps fewer groups has an error only a little higher. The one-standard-error rule suggests choosing the simplest model whose error is within one standard error of the minimum. The worked example above prints its own error by lambda.

The Bayesian Interpretation of Lambda

From a Bayesian perspective, λ encodes your prior belief about how many groups should be active. Higher λ means a stronger prior favoring sparsity. You believe most groups are irrelevant before seeing data. Lower λ is a weaker prior, allowing the data more influence.

How do you set this prior? Use domain knowledge. If you have 20 feature groups but deep expertise suggests only 3-5 genuinely drive the outcome, start with higher λ values that select 3-5 groups. If you genuinely have no idea which groups matter, use a flatter prior (lower λ) and let the data speak more loudly.

Cross-validation updates your prior with evidence. If CV suggests that 8 groups achieve minimum error, but your domain expertise said only 3-5 matter, you've learned something: either your prior was too sparse, or there's genuine complexity in the data you didn't anticipate. Update your beliefs accordingly.

Implementation Checklist: From Raw Data to Production Model

A step-by-step process, with decision points at each stage:

Step 1: Define Feature Groups

Action: Manually specify which features belong to which groups. Don't automate this, domain knowledge matters.

Decision point: For categorical variables, the grouping is obvious (all dummy codes for a variable form one group). For continuous features, group by conceptual similarity: all time lags of the same variable, all polynomial terms of the same base feature, all spatial coordinates, etc.

Example group specification:

groups = {
    'usage_metrics': ['logins', 'sessions', 'time_on_platform', 'features_used'],
    'account_health': ['payment_score', 'renewal_prob', 'support_tickets', 'nps'],
    'firmographic': ['industry_tech', 'industry_finance', ..., 'size_enterprise'],
    'temporal': ['tenure_days', 'days_since_login', 'contract_remaining']
}

Quality check: Do your groups reflect how business stakeholders think about feature importance? If a product manager says "Does usage matter?", can you answer by pointing to a single group rather than 15 scattered features?

Step 2: Standardize Features Within Groups

Action: Standardize all features to mean 0, standard deviation 1 before fitting. This ensures that the group penalty λ√(pg)||βg||₂ has consistent scale across groups.

Why it matters: If one group has features measured in dollars (ranging 0-1,000,000) and another has features measured in percentages (ranging 0-1), the unscaled coefficients will have vastly different magnitudes, making the L2 penalty ||βg||₂ incomparable across groups. Standardization fixes this.

Exception: If all features in a group are dummy variables from the same categorical variable, they're already on the same scale, standardization is optional but doesn't hurt.

Step 3: Run Cross-Validation Over λ Grid

Action: Fit Group Lasso on a grid of λ values using k-fold cross-validation (k=5 or k=10). For each λ, track held-out error and number of active groups.

Typical grid: Start with logarithmically spaced values from 0.001 to 10. Narrow the grid around the region where held-out error is lowest.

What to look for: Plot held-out error versus number of active groups. Look for an "elbow" where error stops improving much as you add more groups.

# Pseudocode for CV grid search
lambda_grid = [0.001, 0.003, 0.01, 0.03, 0.1, 0.3, 1, 3, 10]
cv_results = []

for lambda_val in lambda_grid:
    cv_errors = []
    for train_idx, val_idx in kfold_split(data, k=5):
        model = GroupLasso(lambda=lambda_val, groups=groups)
        model.fit(X[train_idx], y[train_idx])
        val_error = model.score(X[val_idx], y[val_idx])
        cv_errors.append(val_error)

    cv_results.append({
        'lambda': lambda_val,
        'mean_error': mean(cv_errors),
        'std_error': std(cv_errors),
        'active_groups': count_active_groups(model)
    })

Step 4: Apply One-Standard-Error Rule

Action: Find λ_min, the value that minimizes CV error. Compute the standard error SE of that minimum. Select λ_1SE, the largest λ (sparsest model) whose CV error is within CV_min + SE.

Why: Models with CV errors within one standard error of the minimum are not clearly different from the best one. Among such models, choose the simplest (fewest groups). This reduces overfitting risk and makes the model easier to explain.

Example: If the lowest CV error is 10.0 with a standard error of 0.5, any λ with a CV error up to 10.5 qualifies. Choose the largest of those λ, because it keeps the fewest groups. These numbers show the rule only.

Step 5: Assess Coefficient Stability With Bootstrap

Action: Refit the selected model on many bootstrap samples. For each group, track selection frequency and coefficient distributions.

Quality check:

• Groups selected in nearly all bootstraps: stable
• Groups selected in many but not all bootstraps: moderate confidence, quantify the uncertainty
• Groups selected in few bootstraps: unstable; avoid definitive business decisions based on these

Interpretation: The bootstrap selection frequency measures how stable a group's selection is under resampling. It is not the probability that the group matters: a group selected in most bootstraps is fairly stable, not "that likely to be relevant." Your confidence should reflect this instability.

Step 6: Check on Holdout Data

Action: Evaluate the final model on a true holdout set (data never used in CV or training). Compute the same metrics your business cares about: RMSE, MAE, R², AUC, etc.

Sanity check: Holdout performance should be close to CV performance. If holdout error is much worse, you may have overfit during the CV process: go back and increase λ or simplify group definitions.

Step 7: Interpret and Communicate Group Importance

Action: For each active group, compute interpretable importance metrics:

• Coefficient magnitudes: ||βg||₂ gives a rough sense of group contribution
• Permutation importance: Shuffle group g's features and measure error increase. This tells you how much the model relies on group g
• Bootstrap stability: Selection frequency across bootstraps

Communication template: Name the groups the model kept, say how often each was selected across bootstraps, say how much the error rises when its features are shuffled, and name the groups that did not improve predictions.

Four Settings Where Groups Are Natural

Group Lasso is not always the right choice. These are settings where the features come in groups. Compare it with standard Lasso on held-out data before you choose.

Scenario 1: High-Cardinality Categorical Variables

Setup: A pricing model with many product categories encoded as dummy variables.

Why groups: Standard Lasso may keep some category dummies and drop others, which is hard to explain. Group Lasso keeps or drops the category variable as a whole.

When to use: Categorical variables with several levels where partial selection makes no domain sense.

Scenario 2: Polynomial Feature Engineering

Setup: Demand forecasting with polynomial time trends (t, t², t³) and seasonal terms (sin(t), cos(t), sin(2t), cos(2t)).

Why groups: Standard Lasso can keep t² and t³ while dropping t. Grouping the terms keeps each trend or seasonal set whole.

When to use: Any model with polynomial or Fourier features where partial selection breaks interpretability.

Scenario 3: Time-Series Lag Selection

Setup: A prediction model with many lagged values of the same variable.

Why groups: Standard Lasso can pick an arbitrary mix of lags with no clear story. Grouping the lags of each variable keeps or drops that variable's history as a whole.

When to use: Any time-series model where you include multiple lags of the same variable.

Scenario 4: Multi-Level Hierarchical Features

Setup: Retail sales forecasting with a product hierarchy (category, subcategory, brand, SKU) and a geographic hierarchy (country, region, state, city, store).

Why groups: Standard Lasso can keep "state" and "store" but drop "region" and "city," which breaks the hierarchy. Grouping each hierarchy keeps its levels together.

When to use: Hierarchical data where levels should be selected or excluded together.

Common Mistakes

Mistake 1: Creating Artificial Groups Based on Correlation

The error: Running hierarchical clustering on feature correlations and defining groups based on clusters. "These 8 features are highly correlated, so I'll group them."

Why it fails: Group Lasso assumes groups reflect conceptual structure, not statistical correlation. Correlated features might have independent effects on the outcome. Grouping by correlation forces the model to select or drop correlated features together, even when some matter and others don't.

Example: "Revenue" and "profit margin" are correlated, so you group them. But in a customer LTV model, revenue predicts LTV strongly while profit margin doesn't (once you account for revenue). Group Lasso forces an all-or-nothing decision, likely dropping both or keeping both, when the right answer is "revenue yes, margin no."

Correct approach: Group features by conceptual similarity (same base variable at different lags, same categorical variable encoded as dummies, same polynomial expansion), not by correlation.

Mistake 2: Ignoring the √(p_g) Scaling Factor

The error: Implementing Group Lasso with penalty λ∑||βg||₂ instead of λ∑√(pg)||βg||₂.

Why it fails: Without the √(pg) weight, larger groups are favored, because the norm of a group's fit to noise grows with √p_g and so a large group is more likely to be selected by chance. A categorical variable with 20 dummies can displace a variable with 2 dummies even when it carries no more signal.

Example: "Industry" (15 dummies) and "plan_type" (3 dummies) both predict churn. Without √(pg), industry (15 dummies) has a built-in edge: its fit to noise has a norm about √15 ≈ 3.9 times that of a single coefficient, and about √(15/3) ≈ 2.2 times that of plan_type, so it can be selected even when its signal is no stronger. With the √(pg) weight, the threshold scales with group size and the two compete on more equal terms.

Correct approach: Include the √(pg) term. Check whether your Group Lasso library applies this weight by default, and add it yourself if it does not.

Mistake 3: Using Group Lasso When Features Don't Have Group Structure

The error: Forcing every feature into some group, even when natural groupings don't exist.

Why it fails: If your 50 features are genuinely independent (no categorical variables, no polynomial terms, no time lags) Group Lasso adds no value over standard Lasso. Worse, it can hurt performance by forcing unrelated features into artificial groups.

Example: A model with 30 diverse numerical features (age, income, credit score, transaction count, session duration, etc.) has no obvious group structure. Creating groups like "demographic features" (age, income) and "behavioral features" (transaction count, session duration) is arbitrary. Standard Lasso will likely outperform.

Correct approach: Use Group Lasso only when you have genuine structural groups: categorical variables, polynomial/interaction terms, time lags, or hierarchical features. If in doubt, compare Group Lasso and standard Lasso with cross-validation: let the data tell you which structure fits better.

Mistake 4: Not Checking Group Stability

The error: Fitting a single Group Lasso model, seeing that group G is selected, and concluding "group G definitely matters."

Why it fails: A single model gives you a point estimate, not a distribution. If group G is borderline (just barely strong enough to overcome the penalty), small changes in the data might drop it. You need to quantify uncertainty.

Example: A marketing mix model selects "radio advertising" as an active group. You recommend increasing radio spend. But bootstrap analysis reveals that radio is selected in only a small share of resamples. It is unstable. Your recommendation is based on weak evidence.

Correct approach: Run bootstrap resampling and track selection frequency. Make strong business recommendations only on groups that are selected in nearly all resamples. For groups selected in some of them, say so explicitly.

Extensions: Sparse Group Lasso and Overlapping Groups

Standard Group Lasso has limitations. Two extensions address common real-world scenarios:

Sparse Group Lasso: Sparsity Both Across and Within Groups

Group Lasso enforces sparsity across groups but not within them. Once a group is selected, all its coefficients are non-zero (though shrunk). But sometimes you want sparsity at both levels: select a few groups, and within those groups, select a few features.

Sparse Group Lasso adds an L1 penalty on individual coefficients in addition to the group penalty:

minimize: (1/2n)||y - Xβ||₂² + λ₁∑√(pₐ)||βₐ||₂ + λ₂∑|βⱼ|

The λ₁ term encourages group sparsity (entire groups go to zero). The λ₂ term encourages individual sparsity within groups (individual coefficients within active groups go to zero). You now have two tuning parameters to control sparsity at both levels.

When to use: Categorical variables with many levels where you suspect only a few levels matter. Example: "industry" has 50 levels, but you believe only 5-10 industries genuinely affect the outcome. Sparse Group Lasso can select the "industry" group while zeroing out irrelevant industries.

Overlap Group Lasso: Features Belonging to Multiple Groups

Standard Group Lasso assumes non-overlapping groups: each feature belongs to exactly one group. But real-world features often have multiple conceptual roles.

Example: "Customer age" might belong to both a "demographic" group and a "lifecycle stage" group. "Transaction amount" might belong to both a "financial metrics" group and a "engagement signals" group. These are overlapping groups.

Overlap Group Lasso (OGL) allows features to belong to multiple groups. The penalty becomes more complex, each coefficient β_j can appear in multiple group norms ||βg||₂. The optimization is harder (requires specialized algorithms like ADMM), but the flexibility matches real-world feature semantics.

When to use: Features with multiple conceptual roles that can't be cleanly partitioned into non-overlapping groups. Caution: OGL is computationally expensive and less widely supported in standard libraries. Only use if overlapping structure is genuinely important and simpler approaches fail.