When to Use Breast Cancer Classification with PCA + Logistic Regression

You have 30 measurements from each of 569 breast masses, and a label for each one: malignant or benign. Many of the 30 columns move together. Radius and perimeter rise and fall as one, and area follows radius squared. Feed all 30 into logistic regression and the coefficients can swing with small changes in the data, because correlated inputs share the same signal.

Principal Component Analysis (PCA) is one way to handle this. It rotates the correlated columns into uncorrelated components, and logistic regression then fits the components. This page runs that pipeline once on the breast cancer data bundled with scikit-learn, shows the full script and its printed output, and says what that one run does and does not show. It is a method demo, not a screening tool.

Worked Example: One Test Split

Question. Can PCA plus logistic regression sort malignant and benign cases in this file?

Inputs. These are real data, not simulated: the Wisconsin Diagnostic Breast Cancer file that ships with scikit-learn, with 569 rows and 30 numeric columns (357 benign, 212 malignant). The script splits it 70/30 with seed 42, keeping the class mix in both parts: 398 rows for training and 171 for testing.

Method. Scaling and PCA are fit on the training rows only. Five-fold cross-validation (CV) on the training rows picks the number of components and the logistic regression setting C, scored by AUC. AUC is the chance that a random malignant case gets a higher score than a random benign one. The chosen model is then scored once on the 171 test rows at a fixed 0.5 cutoff. In the matrix, rows are the true labels and columns are the model's picks.

Result. CV chose 15 components, which keep 0.9873 of the variance, and C=0.1. Both are the edge of the grid the script searched (components 2, 3, 5, 7, 10, 15 and C 0.1, 1, 10), so a wider search might have chosen differently. Test AUC was 0.9990. The matrix shows 107 benign cases called benign, no benign case called malignant, 62 malignant cases found, and 2 malignant cases called benign. That is sensitivity 0.9688 (62 of 64) and specificity 1.0000 (107 of 107). The largest PC1 loadings were mean concave points, mean concavity and worst concave points. Loadings show how the measures feed a component. They do not show that a measure causes cancer.

Limit. This is one test split of 171 rows, and it has 2 errors. A different seed could give different numbers. The 2 missed malignant cases are the ones that matter: a miss means a malignant case labelled benign. The run does not show how the model would behave on another clinic's data, and it should not guide care.

Complete script (run with python3; needs numpy and scikit-learn):

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score, confusion_matrix
import numpy as np

# Offline public data bundled with scikit-learn; fixed split seed 42.
data = load_breast_cancer()
X = data.data
# scikit-learn labels malignant as 0 and benign as 1; use malignant=1 here.
y = (data.target == 0).astype(int)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.30, random_state=42, stratify=y
)
pipe = Pipeline([
    ('scale', StandardScaler()),
    ('pca', PCA()),
    ('lr', LogisticRegression(max_iter=5000, solver='liblinear',
                              random_state=42))
])
search = GridSearchCV(
    pipe,
    {'pca__n_components': [2, 3, 5, 7, 10, 15], 'lr__C': [0.1, 1.0, 10.0]},
    scoring='roc_auc',
    cv=StratifiedKFold(n_splits=5, shuffle=True, random_state=42),
    refit=True
)
search.fit(X_train, y_train)
model = search.best_estimator_
prob = model.predict_proba(X_test)[:, 1]
pred = (prob >= 0.5).astype(int)
cm = confusion_matrix(y_test, pred, labels=[0, 1])
best = search.best_params_
pca = model.named_steps['pca']
print('Dataset: scikit-learn Wisconsin Diagnostic Breast Cancer; '
      '569 rows, 30 features')
print(f'Split: train={len(y_train)} (malignant={y_train.sum()}, '
      f'benign={len(y_train)-y_train.sum()}), test={len(y_test)} '
      f'(malignant={y_test.sum()}, benign={len(y_test)-y_test.sum()}); '
      'seed=42')
print(f'CV choice: components={best["pca__n_components"]}, '
      f'C={best["lr__C"]}, mean 5-fold training AUC='
      f'{search.best_score_:.4f}')
print('Variance retained by chosen components='
      f'{pca.explained_variance_ratio_.sum():.4f}')
print('Test threshold=0.5000; rows=true [benign, malignant], '
      'columns=predicted [benign, malignant]')
print(cm)
print(f'Test AUC={roc_auc_score(y_test, prob):.4f}')
print(f'Test sensitivity={cm[1,1]/cm[1].sum():.4f}; '
      f'specificity={cm[0,0]/cm[0].sum():.4f}')
coef = model.named_steps['lr'].coef_[0]
print('Logistic coefficients by PC=' + ', '.join(f'{v:.4f}' for v in coef))
for pc in range(min(2, len(coef))):
    loading = pca.components_[pc]
    idx = np.argsort(np.abs(loading))[-3:][::-1]
    print(f'PC{pc+1} largest absolute loadings: ' + ', '.join(
        f'{data.feature_names[i]}={loading[i]:.4f}' for i in idx))

Printed output:

Dataset: scikit-learn Wisconsin Diagnostic Breast Cancer; 569 rows, 30 features
Split: train=398 (malignant=148, benign=250), test=171 (malignant=64, benign=107); seed=42
CV choice: components=15, C=0.1, mean 5-fold training AUC=0.9937
Variance retained by chosen components=0.9873
Test threshold=0.5000; rows=true [benign, malignant], columns=predicted [benign, malignant]
[[107   0]
 [  2  62]]
Test AUC=0.9990
Test sensitivity=0.9688; specificity=1.0000
Logistic coefficients by PC=1.3779, -0.6988, -0.2654, 0.4234, 0.4269, -0.0971, 0.0339, -0.2081, -0.2315, -0.0861, 0.3391, 0.0207, 0.0453, -0.1791, -0.0783
PC1 largest absolute loadings: mean concave points=0.2631, mean concavity=0.2591, worst concave points=0.2516
PC2 largest absolute loadings: mean fractal dimension=0.3608, worst fractal dimension=0.2788, fractal dimension error=0.2770

Sources

• Street, Wolberg and Mangasarian published a paper titled "Nuclear feature extraction for breast tumor diagnosis" in 1993 (SPIE Proceedings).

Why Correlated Columns Cause Trouble

The 30 columns are ten nucleus measurements (radius, texture, perimeter, area, smoothness, compactness, concavity, concave points, symmetry and fractal dimension), each given as a mean, a standard error and a worst value. Some of these are linked by geometry. Perimeter is about 2πr for a round nucleus, and area scales with radius squared.

When predictors are correlated, small changes in the data can cause large changes in the coefficients. The predictions may stay steady while the story each coefficient tells does not. PCA rotates the columns into components ranked by variance. By construction the components are uncorrelated, so the logistic regression coefficients on them do not trade off against each other the way coefficients on raw correlated columns can. The cost is that each component mixes many measures.

To see which factors predict an outcome class in your own data, and where the predictions fail, try the free classification tool.

The Experimental Setup

The research question is: Can a logistic regression model trained on principal components of nucleus measurements classify these tumors as malignant or benign better than guessing? The null hypothesis is that the measurements carry no diagnostic information (AUC 0.5). The script above follows this protocol:

1. Data: 569 rows, each labelled malignant or benign in the file.
2. Split: 398 training rows and 171 test rows, stratified by class.
3. Preprocessing: standardize on the training rows and apply the same transform to the test rows.
4. PCA: fit on the training rows; the number of components is a setting chosen by CV.
5. Logistic regression: L2-regularized, with the strength C chosen by CV.
6. Evaluation: one pass over the test rows at a 0.5 cutoff. The cutoff was fixed in advance and the test rows were not used to tune anything.

Diagnosis Class Balance

The file has 357 benign and 212 malignant tumors, a 62.7% / 37.3% split. A model that called every case benign would be right 62.7% of the time and would find no malignant case. That is why this page reports AUC, sensitivity and specificity and not accuracy alone. The script did not reweight the classes; it kept the 0.5 cutoff. Class weights or other cutoffs are options to compare in cross-validation, not something this run tested.

The stratified split keeps the class mix the same in training and test rows. Without it, a small test set can drift away from the full file's mix and bias the performance estimate.

How Many Components

There is more than one way to choose the number of components:

• Elbow: look for the point where the variance curve flattens.
• Cumulative variance: keep enough components to reach a chosen share of the variance.
• Cross-validation: fit the classifier with different component counts and compare the CV score. This is the method the script used.

On all 569 rows, the first three components hold about 72% of the variance, and seven are needed to reach 90%. The script above did not print these figures. Its CV search chose 15, the top of its grid, which is a reminder to widen the grid and see whether the choice moves. Components are uncorrelated by construction, so using many of them does not bring back the correlation problem, but it does add parameters to estimate.

Reading the Coefficients

The script prints one logistic coefficient per component (15 of them). The first, 1.3779, is the largest in size, so a higher PC1 score raised the model's score for malignant in this fit. The sign of a component is arbitrary from one fit to the next, so compare sizes and check the loadings before reading a direction into any coefficient.

Reading the Confusion Matrix

On the 171 test rows the matrix was [[107, 0], [2, 62]]. Sensitivity is 62 of 64 malignant cases, 0.9688. Specificity is 107 of 107 benign cases, 1.0000. The two errors are both malignant cases called benign. With only 64 malignant test rows, two more or fewer errors would change sensitivity by a visible amount, so treat the figure as a rough guide. Which errors a team can accept is a judgment about the cost of each kind of mistake, and one test split cannot settle it.

How to Interpret Your Results

If you run this on your own biopsy-style data, here is what to look at.

Step 1: Check Class Balance

Look at the class counts. If one class is small, accuracy will mislead you and the estimates for that class will be noisy. Compare AUC, sensitivity and specificity, and consider class weights or a different cutoff.

Step 2: Check the Separation

Plot the first two components colored by class. If the classes overlap heavily, a linear boundary may not be enough and you can try a nonlinear model. Also look at the top PC1 loadings: do they make sense for your domain?

Step 3: Choose the Number of Components

Use the variance curve together with cross-validation. If the best value sits at the edge of your grid, extend the grid and run it again.

Step 4: Interpret Coefficients With Care

Components with large coefficients should be read through their loadings. A component with a large coefficient but little variance deserves a second look: it may be signal or it may be overfitting.

Step 5: Choose the Cutoff Without the Test Set

The 0.5 cutoff is a convention, not a result. If you need a different balance between misses and false alarms, choose the cutoff with cross-validation on training data, and then score the test rows once.

Step 6: Read the Confusion Matrix Cell by Cell

Count the false negatives and the false positives separately. A missed malignant case and an unneeded follow-up are different costs, and a single AUC value hides which one you got.

Common Mistakes and How to Avoid Them

Fitting PCA on all rows before the split. Test-set statistics then shape the rotation, and the test score looks better than it should. Fit on the training rows, then transform the test rows.

Forgetting to standardize. PCA maximizes variance, so columns with large numeric ranges dominate. Use the training mean and standard deviation for both train and test rows.

Using accuracy as the only metric. With 62.7% benign cases, a model that says "benign" every time scores 62.7% accuracy and finds no malignant case. Report AUC, sensitivity and specificity.

Tuning the cutoff on the test set. That uses the test rows twice. Pick the cutoff on training data only.

Reading loadings as causes. A large loading on "worst concave points" does not mean that measure causes cancer. PCA describes variance, not cause. Check any insight against domain knowledge before drawing conclusions.

Choosing the component count by eye only. Use cross-validation, and widen the grid if the best value sits at its edge.

When PCA + Logistic Regression Fits

This method suits you when you have:

• Many correlated features: measurement sets, sensor arrays and image features often move together.
• A two-class outcome: logistic regression models the log-odds of one class against the other. For more classes, use a multinomial form.
• A roughly linear boundary: if the classes separate along a line or plane in component space, logistic regression can work. If the boundary is curved, try kernel PCA or a nonlinear classifier.
• A need to explain the model: coefficients on components, read through loadings, are easier to explain than a large ensemble.

When not to use it:

• Features are already uncorrelated: PCA adds little and makes the coefficients harder to read.
• Nonlinear patterns: if malignant cases are marked by extreme values on both ends, a linear model can miss the pattern.
• Categorical features: PCA wants continuous inputs. One-hot columns often give components that are hard to read.
• Very sparse data: standard PCA can dilute the signal; consider sparse PCA.

Extending the Model: What to Try Next

Regularization: Compare L1 and L2 penalties with cross-validation. L1 can drive some component coefficients to exactly zero.

Polynomial features: Add terms such as PC1 squared or PC1 times PC2 to let the boundary curve, and check whether the CV score improves.

Ensemble methods: Fit a random forest or boosted trees on the components and compare CV scores. Weigh any gain against the loss of interpretability.

Cutoff selection: Build a grid of cutoffs, score each with cross-validation on training data, and choose the one that meets your target for misses with the fewest false alarms.

Independent test: Score the model on data from a different source or period. A drop there tells you that the first split was optimistic.

Calibration: Check whether predicted probabilities match observed rates by binning predictions and comparing. If they do not, consider Platt scaling or isotonic regression.

Your turn

Bring your own data and the question you actually need answered.

MCP AnalyticsData science by email Send us your data and your question, and we will reply. ds@mcpanalytics.ai

FAQ: Breast Cancer Classification with PCA + Logistic Regression

Why use PCA before logistic regression for breast cancer classification?

The 30 columns are correlated: each of ten nucleus measurements appears as a mean, a standard error and a worst value. PCA turns them into uncorrelated components, which can make logistic regression coefficients steadier. The cost is that each component mixes many measures, so you read it through its loadings. In the worked example, scaling and PCA were fit on training data only.

What does a test AUC of 0.9990 mean in this example?

In this run, a random malignant test case got a higher score than a random benign test case almost every time. It is one split of 171 rows with 2 malignant cases called benign, so it is a method demo and not a claim that the model is ready for clinical use.

How many principal components should I use for tumor classification?

Choose with cross-validation. In the worked example the search covered 2, 3, 5, 7, 10 and 15 components and chose 15, the top of that grid, so a wider search might choose differently. On all 569 rows, the first three components hold about 72% of the variance and about seven are needed to reach 90%.

Should I balance the benign/malignant class distribution before training?

The file is 357 benign and 212 malignant, 62.7% benign, so guessing benign every time would be right 62.7% of the time. The worked example did not reweight the classes. Class weights or resampling are options to compare in cross-validation.

What cutoff should I use for classification?

The worked example used a 0.5 cutoff, fixed before testing, and 2 of 64 malignant test cases were called benign. If misses matter more to you than false alarms, choose a different cutoff with cross-validation on training data, not on the test rows. This page does not test one.

Run This Analysis on Your Own Data

If you have morphology measurements, imaging features or other correlated columns and a two-class label, MCP Analytics can run a similar pipeline on your CSV. Upload the file and name the label column. You get PCA plots, logistic regression coefficients, ROC curves and confusion matrices in an interactive report.

The platform handles feature standardization, train/test splitting, PCA fitting, cross-validation and performance metrics, and shows which features load on each component.

Before you draw conclusions about which features predict a class in your data, check the design: did you stratify by class, fit PCA on training data only, and test on held-out samples?