Machine Learning Methods: Classification, Clustering & Ensembles

Worked Comparison on Public Data

Here is a small, reproducible comparison you can run in a few seconds. It uses two datasets that ship with scikit-learn, so there is nothing to download.

• Classification: the Breast Cancer Wisconsin (Diagnostic) data, 569 rows, 30 numeric features, 2 classes (a copy of the UCI dataset).
• Clustering: the iris data, 150 flowers, 4 measurements, 3 species. We hide the species from the clustering and use it only to score the result afterwards.

Evaluation setup. Each classifier is scored with 5-fold stratified cross-validation (the data is split into 5 parts with the same class mix; each part is held out once while the model trains on the other 4). Scaling sits inside the pipeline, so it is learned on the training folds only. We report accuracy (share of rows labelled correctly) and ROC AUC (how well the model ranks positive rows above negative ones; 1.0 is perfect, 0.5 is chance). Default settings are used throughout; nothing was tuned.

from sklearn.datasets import load_breast_cancer, load_iris
from sklearn.model_selection import StratifiedKFold, cross_validate
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier, HistGradientBoostingClassifier
from sklearn.svm import SVC
from sklearn.naive_bayes import GaussianNB
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score, adjusted_rand_score

# Part 1: classification
X, y = load_breast_cancer(return_X_y=True)
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
models = {
    "Logistic regression": make_pipeline(StandardScaler(), LogisticRegression(max_iter=5000)),
    "Random forest": RandomForestClassifier(n_estimators=300, random_state=0),
    "Gradient boosting (HistGB)": HistGradientBoostingClassifier(random_state=0),
    "SVM (RBF kernel)": make_pipeline(StandardScaler(), SVC(random_state=0)),
    "Gaussian naive Bayes": GaussianNB(),
}
for name, m in models.items():
    r = cross_validate(m, X, y, cv=cv, scoring=["accuracy", "roc_auc"])
    acc, auc = r["test_accuracy"], r["test_roc_auc"]
    print(f"{name:28s} acc {acc.mean():.3f} (sd {acc.std():.3f})  AUC {auc.mean():.3f} (sd {auc.std():.3f})")

# Part 2: clustering (species hidden)
Xi, yi = load_iris(return_X_y=True)
Xs = StandardScaler().fit_transform(Xi)
for k in range(2, 7):
    km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(Xs)
    print(f"K={k}  inertia {km.inertia_:7.1f}  silhouette {silhouette_score(Xs, km.labels_):.3f}  "
          f"ARI vs species {adjusted_rand_score(yi, km.labels_):.3f}")

Run with Python 3, scikit-learn 1.8.0 and NumPy 2.4.0 on 2026-11-02. Other versions can shift the last digit.

Classification results (5-fold cross-validation)

How to read it:

• The simple baseline was not beaten. Plain logistic regression had the highest mean accuracy (0.979). The SVM was within 0.002, and gradient boosting within 0.009. These gaps are smaller than the spread between folds (fold SD 0.007 to 0.020), so on this data the top three are hard to tell apart.
• Naive Bayes was lowest on accuracy (0.939) but still ranked rows well (AUC 0.989). Its independence assumption is a poor fit for features that are strongly related, such as a tumour's radius, perimeter and area.
• This is one small, clean dataset. It says nothing general about which family wins. It shows why you run the baseline first: a more complex model has to earn its place on your own data and metric.

Clustering results (K-Means on scaled iris data)

Inertia is the within-cluster sum of squared distances (lower means tighter clusters; it generally falls as K grows). The drop column is simple subtraction of the printed inertia values. ARI, the adjusted Rand index, scores agreement with the hidden species: 1 is a perfect match, about 0 is chance.

• The elbow points to K=3. Going from 2 to 3 clusters cuts inertia by 82.6; each step after that cuts it by 25.7 or less.
• The silhouette score points to K=2 (0.582, the highest). Two of the three species overlap in these measurements, so a two-group split is the most cleanly separated one.
• The real species count is 3, and K=3 matches it most closely (ARI 0.620). Even then the match is partial, because K-Means cannot fully separate the two overlapping species.

The lesson: the elbow and the silhouette score are aids, not answers. Check the candidate groupings against what they will be used for before you pick K.