Quantum machine learning for defect detection: a quantum kernel SVM against classical models on steel plate faults
We tested a quantum kernel SVM on 4 and 8 qubits, simulated exactly, against tuned classical models on the public Steel Plates Faults data (1,941 plates, 27 features, 7 fault types), over five splits. At this size the quantum kernel classifies surface defects about as well as a classical RBF SVM given the same PCA components, and slightly worse on average; the best models are classical, an RBF SVM and an untuned random forest on all 27 features. This page gives the benchmark, which faults are hard for every model, what would have to change for the result to flip, and how to run the same test on your own inspection data with eigenkit.
Updated
The short answer
- Best models
- Classical, on all 27 features: RBF SVM with balanced accuracy 0.779, random forest with default settings 0.777
- Best quantum model
- QSVC with a ZZ feature map on 8 qubits, 0.764, against the RBF SVM's 0.770 on the same 8 PCA components
- Quantum minus classical, same split
- Below the RBF SVM on average in all four pairings, from -0.006 to -0.021 balanced accuracy, inside the seed-to-seed spread
- Hardest fault
- Pastry: recall 0.59 for the 8-qubit QSVC and 0.59 for the RBF SVM, the same for both kernels
- Hybrid quantum network, not tuned
- Balanced accuracy 0.551 on 8 qubits, below every other model on the same 8 components
The data: steel surface faults as measured features
Automated surface inspection usually starts with a camera over the line. The Steel Plates Faults dataset starts one step later: each of its 1,941 plates is a fault that has already been found and measured, described by 27 numbers such as the position and size of the fault, its perimeter, the sum, minimum and maximum of luminosity over it, the length of the conveyer, the steel type, the plate thickness and a set of shape indices. The label is one of seven fault types: Pastry, Z_Scratch, K_Scatch, Stains, Dirtiness, Bumps and Other_Faults[1]. The classes are uneven, from 673 Other_Faults down to 55 Dirtiness plates, so the headline metrics are balanced accuracy and macro F1, which weigh a rare fault type as much as a common one.
That makes it the tabular half of quality inspection: given the features an inspection system extracts, which fault is it? It is also the form of the problem a quantum kernel can take today, because a feature map needs one qubit per input and a camera image has far more pixels than any simulator or device can encode. The question we asked: does eigenkit's quantum kernel SVM classify these faults better than tuned classical models, when every model gets the same split, the same features and the same tuning budget?
The quantum model is an ordinary support vector machine whose kernel is the overlap of two quantum states, each prepared from one plate's features by a feature-map circuit : [2]. The 27 features are reduced with PCA to 4 or 8 components so the circuit fits on 4 or 8 qubits, and the classical models get the same components as well as all 27 features. The method and the benchmarking rules are explained in the QSVM vs SVM comparison; the breast cancer benchmark runs the same protocol on a binary medical dataset. This page is the multi-class manufacturing case.
Every model on the same five splits
Five seeds, each a stratified split of 1,552 training and 389 test plates. Every SVM and the logistic regression was tuned by 5-fold cross-validation on the training rows, scored by balanced accuracy, over the same C grid; the quantum kernels and the RBF kernel also searched the same six bandwidths. Mean ± standard deviation over seeds. Chance level for balanced accuracy is one in seven. Fit time is the whole grid search plus the final fit on a shared laptop CPU; for the untuned models it is one training run.
| Model and inputs | Balanced accuracy | Macro F1 | ROC-AUC | Fit time |
|---|---|---|---|---|
| QSVC, ZZ map, 4 qubits, PCA 4 | 0.646 ± 0.043 | 0.641 ± 0.047 | 0.878 ± 0.016 | 21.7 s |
| QSVC, AngleCNOT map, 4 qubits, PCA 4 | 0.641 ± 0.045 | 0.632 ± 0.054 | 0.872 ± 0.018 | 9.3 s |
| RBF SVM, PCA 4 | 0.659 ± 0.041 | 0.650 ± 0.046 | 0.882 ± 0.028 | 4.9 s |
| Logistic regression, PCA 4 | 0.417 ± 0.028 | 0.417 ± 0.028 | 0.856 ± 0.005 | 0.2 s |
| QSVC, ZZ map, 8 qubits, PCA 8 | 0.764 ± 0.029 | 0.755 ± 0.037 | 0.917 ± 0.009 | 41.4 s |
| QSVC, AngleCNOT map, 8 qubits, PCA 8 | 0.749 ± 0.028 | 0.740 ± 0.031 | 0.911 ± 0.008 | 23.8 s |
| HQNC, 8 qubits, not tuned, PCA 8 | 0.551 ± 0.034 | 0.566 ± 0.038 | 0.902 ± 0.004 | 87.7 s |
| RBF SVM, PCA 8 | 0.770 ± 0.005 | 0.764 ± 0.019 | 0.925 ± 0.005 | 4.4 s |
| Linear SVM, PCA 8 | 0.560 ± 0.030 | 0.557 ± 0.028 | 0.896 ± 0.008 | 0.1 s |
| Logistic regression, PCA 8 | 0.593 ± 0.017 | 0.599 ± 0.024 | 0.913 ± 0.011 | 0.2 s |
| RBF SVM, all 27 features | 0.779 ± 0.022 | 0.768 ± 0.036 | 0.930 ± 0.011 | 5.1 s |
| Logistic regression, all 27 features | 0.726 ± 0.024 | 0.722 ± 0.020 | 0.930 ± 0.009 | 0.9 s |
| Random forest, defaults, all 27 features | 0.777 ± 0.030 | 0.791 ± 0.028 | 0.958 ± 0.004 | 0.2 s |
| k-NN, defaults, all 27 features | 0.761 ± 0.018 | 0.732 ± 0.019 | 0.913 ± 0.008 | 0.0 s |
Run defect-classification-steel-plates of 2026-11-03, mean and spread over 5 seeds, on Steel Plates Faults (UCI Machine Learning Repository, dataset 198; Buscema, Terzi, Tastle; Semeion). On the same components the quantum and RBF kernels are within the spread of each other; more components help every model far more than the choice of kernel does. ROC-AUC is one-vs-rest, averaged over the seven classes. ZZ is a nearest-neighbour version of the feature map of Havlicek et al.; AngleCNOT is eigenkit's built-in angle map with a CNOT ladder.
Which faults each model finds
Recall per fault type on the test split, mean over the five seeds. A test split holds about 32 Pastry, 38 Z_Scratch, 78 K_Scatch, 14 or 15 Stains, 11 Dirtiness, 80 Bumps and 135 Other_Faults plates, so one plate moves the Dirtiness recall by about nine points.
| Model and inputs | Pastry | Z_Scratch | K_Scatch | Stains | Dirtiness | Bumps | Other_Faults |
|---|---|---|---|---|---|---|---|
| QSVC, ZZ map, 4 qubits, PCA 4 | 0.33 | 0.58 | 0.89 | 0.91 | 0.60 | 0.60 | 0.61 |
| RBF SVM, PCA 4 | 0.32 | 0.57 | 0.91 | 0.90 | 0.67 | 0.63 | 0.61 |
| QSVC, ZZ map, 8 qubits, PCA 8 | 0.59 | 0.84 | 0.94 | 0.91 | 0.75 | 0.65 | 0.66 |
| RBF SVM, PCA 8 | 0.59 | 0.84 | 0.94 | 0.93 | 0.75 | 0.66 | 0.68 |
| Linear SVM, PCA 8 | 0.21 | 0.78 | 0.93 | 0.79 | 0.00 | 0.56 | 0.65 |
| HQNC, 8 qubits, not tuned, PCA 8 | 0.19 | 0.74 | 0.88 | 0.79 | 0.11 | 0.43 | 0.73 |
| RBF SVM, all 27 features | 0.57 | 0.88 | 0.96 | 0.93 | 0.76 | 0.68 | 0.67 |
| Random forest, defaults, all 27 features | 0.53 | 0.86 | 0.95 | 0.90 | 0.78 | 0.65 | 0.76 |
Run defect-classification-steel-plates of 2026-11-03, mean and spread over 5 seeds, on Steel Plates Faults (UCI Machine Learning Repository, dataset 198; Buscema, Terzi, Tastle; Semeion). The quantum kernel and the RBF kernel make nearly the same mistakes, fault by fault. The linear SVM and the untuned hybrid network miss most Dirtiness plates, which plain accuracy would hide.
What the numbers say
The quantum kernel behaves like an RBF kernel on the same inputs. Per fault type, the recalls of the QSVC and the RBF SVM agree to within a few points at 4 and at 8 components. On the same split, the 8-qubit ZZ kernel was ahead of the RBF SVM on 2 of the five seeds and behind on the rest, with a mean difference of -0.006; the 4-qubit ZZ kernel was ahead on 1 of five. With five seeds no paired test can call a gap that small, and none of the gaps favours the quantum kernel on average.
Feature reduction costs more than the kernel choice. Going from 4 to 8 components lifts the QSVC from 0.646 to 0.764, and Pastry and Z_Scratch go from hard for every model to the level of the full-feature models. The best rows use all 27 features, which a quantum kernel would need 27 qubits to see. Against the RBF SVM on all 27 features, the mean paired difference of the best quantum kernel is -0.015.
A non-linear kernel is what helps, and a classical one provides it. On PCA components the linear SVM and the logistic regression fall far behind both kernel SVMs and almost never recognise Dirtiness (recall 0.00 and 0.07 on 8 components). The useful property of the quantum kernel here is that it is non-linear, which the RBF kernel already is.
Bandwidth decides whether the kernel can learn at all. The quantum kernels' bandwidth is a factor that multiplies the inputs before encoding, and it moves a quantum kernel between overfitting and underfitting as the width does for an RBF kernel[3]. Cross-validation chose mid-range values and never the two largest at 8 qubits. On one training split, the 8-qubit ZZ kernel's mean off-diagonal entry is 0.11 at scale 0.5 and 0.013 at scale 1: almost every pair of different plates looks unrelated, the exponential concentration that limits expressive quantum kernels as qubits are added[4]. It is the kernel cousin of the barren plateau.
The hybrid network is a floor, not its best. The hybrid quantum neural network classifier (HQNC) used the architecture of eigenkit's breast cancer example with default training settings and was not tuned, because one 8-qubit training run took longer than the whole grid search of any SVM. It mostly predicts the large classes. A tuned version would do better; the predictive maintenance page tests the same kind of model with an ablation of its circuit.
What would change this result
The signal to look for is a quantum kernel ahead of a tuned RBF SVM on the same components, on the same splits, by more than the seed-to-seed spread. Here none of the four pairings was. Three things could move that, and each can be tested rather than assumed.
More of the features, on more qubits. PCA cost more than the kernel choice, so a kernel that sees more of the 27 features has room to gain. Wider feature maps concentrate, as the off-diagonal values above show[4], but the bandwidth study found that tuning the scale keeps a quantum kernel learning as qubits are added[3]. The test is to widen the circuit one step at a time, retune the scale and watch both the accuracy and the mean off-diagonal value.
A kernel shaped to the data. Both maps here are fixed. The original quantum kernel study separated data generated for its own feature map[2]; on real faults the question is whether a trained map, with weights fitted to the labels, finds structure an RBF kernel misses. eigenkit's kernel alignment trains one, and it belongs next to the same RBF baseline.
Real hardware. Every kernel entry on a QPU is a circuit run, estimated from shots, and gate noise moves it away from the exact value. Before planning on a device, check which QPUs can run an 8-qubit feature map within their error budget and what a run costs.
Run it on a real QPU with EigenOrbit RouterA sample of 8-qubit kernel entries between plates estimated from shots on a real QPU and set against the exact values, to measure how far hardware noise moves the kernel.In eigenkit: pip install "eigenkit[router]", then provider=RouterConfig(device=...) (docs).
How to test a quantum kernel on your own inspection data
The same protocol works on any table of features an inspection system writes out.
Start from features, not pixels
Use the measurements your vision system already extracts per defect: geometry, intensity statistics, texture or shape descriptors. Standardise them and reduce them with PCA to as many components as you will have qubits, fitted on the training rows only inside a pipeline.
Give the classical models the same inputs
Run an RBF SVM on the same components with the same split, then add classical models on all the features. The first comparison tells you whether the quantum kernel adds anything; the second tells you what the reduction cost, which on this data was the larger number.
Tune both bandwidths over the same range
Search the quantum
scaleand the RBF with grids of the same size. For small distances an angle-encoded kernel behaves like an RBF kernel with for scale , so one grid covers both. Check the mean off-diagonal kernel value at each scale: near 1 the kernel is almost constant, near 0 it has concentrated.Score per fault type
Report balanced accuracy, macro F1 and recall per class over several seeds, with the paired per-seed difference between the quantum and the classical row. Rare fault types are the ones a plain accuracy figure hides, and often the ones that matter on the line.
Count the hardware cost before you submit
On a simulator a Gram matrix is one state evolution per sample. On a QPU it is one circuit per pair: for training samples, which for 1,552 plates is about circuits, plus for the test kernel, each with many shots and repeated for every point in the grid. Tune on the simulator, then send one fixed kernel on a small subsample.
Run the comparison on one split
import io
import urllib.request
import zipfile
import numpy as np
from sklearn.decomposition import PCA
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import balanced_accuracy_score, recall_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from eigenkit import QSVC
URL = "https://archive.ics.uci.edu/static/public/198/steel+plates+faults.zip"
with zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(URL).read())) as zf:
table = np.loadtxt(io.StringIO(zf.read("Faults.NNA").decode()))
X, y = table[:, :27], table[:, 27:].argmax(axis=1) # 7 one-hot fault columns
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)
def pca8(clf):
# 8 PCA components on 8 qubits, fitted on the training plates only
return make_pipeline(StandardScaler(), PCA(n_components=8), StandardScaler(), clf)
models = {
"QSVC, 8 qubits": pca8(QSVC(feature_map="AngleCNOT", reps=2, scale=0.25, C=1000)),
"RBF SVM, PCA 8": pca8(SVC(kernel="rbf", gamma=0.25, C=10)),
"Random forest, all 27": make_pipeline(StandardScaler(), RandomForestClassifier(random_state=0)),
}
for name, model in models.items():
pred = model.fit(X_train, y_train).predict(X_test)
dirtiness = recall_score(y_test, pred, labels=[4], average=None)[0] # class 4 is Dirtiness
print(f"{name}: balanced accuracy {balanced_accuracy_score(y_test, pred):.3f}, Dirtiness recall {dirtiness:.2f}")It downloads the UCI archive (about 100 kB), makes the seed-0 split of 1,552 training and 389 test plates and fits three models with settings the tuned runs selected, in a few seconds on a laptop CPU with pip install eigenkit[qml]. It prints the balanced accuracy and the Dirtiness recall of each model. On the run we checked, the two kernel SVMs nearly tied and the random forest on all 27 features was on top. Change random_state to see how far one split moves the scores, and replace the download with your own feature table.
Questions
Can quantum machine learning detect manufacturing defects better than classical models?
Not in our benchmark, with 4- and 8-qubit kernels on a simulator. On the UCI Steel Plates Faults data, a tuned quantum kernel SVM matched an RBF SVM on the same PCA components and was slightly below it on average, and an RBF SVM and a random forest on all 27 features were the best models. It is one dataset at one scale, so it shows no advantage here rather than proving none can exist; the page lists what would have to change for that to flip.
What is the steel plates faults dataset?
It is a public dataset in the UCI Machine Learning Repository, donated in 2010 by M. Buscema, S. Terzi and W. Tastle. It has 1,941 steel plates, each described by 27 measured features of a surface fault, and a label of one of seven fault types: Pastry, Z_Scratch, K_Scatch, Stains, Dirtiness, Bumps and Other_Faults. It is licensed CC BY 4.0.
Can a quantum kernel classify defects straight from inspection images?
Not in practice today. A feature map uses about one qubit per input, so an image has to be turned into a small feature vector first, by a vision system or by PCA. Image-based inspection is the domain of classical convolutional networks, and eigenkit has no image pipeline; the quantum kernel works on the extracted features.
Why use PCA before a quantum SVM on defect data?
Because each component becomes one qubit, and 27 qubits per kernel entry is slow to simulate and expensive on hardware. In our run the reduction cost more than the choice of kernel: every model scored lower on 4 components than on 8, and the best scores came from classical models on all 27 features.
Do I need a quantum computer to run this benchmark?
No. Everything here ran on eigenkit's built-in statevector simulator on a laptop, with exact kernel values. On a QPU, the training kernel for 1,552 plates alone would be about 1.2 million circuits, each repeated for many shots, so tune on the simulator first and send only a small subsample to hardware.
Sources
- M. Buscema, S. Terzi, W. Tastle. Steel Plates Faults. UCI Machine Learning Repository, dataset 198, CC BY 4.0, doi:10.24432/C5J88N, 2010.
- V. Havlicek, A. D. Córcoles, K. Temme, A. W. Harrow, A. Kandala, J. M. Chow, J. M. Gambetta. Supervised learning with quantum enhanced feature spaces. Nature 567, 209–212, 2019. arXiv:1804.11326.
- R. Shaydulin, S. M. Wild. Importance of kernel bandwidth in quantum machine learning. Physical Review A 106, 042407, 2022. arXiv:2111.05451.
- S. Thanasilp, S. Wang, M. Cerezo, Z. Holmes. Exponential concentration in quantum kernel methods. Nature Communications 15, 5200, 2024. arXiv:2208.11060.
- J. Bowles, S. Ahmed, M. Schuld. Better than classical? The subtle art of benchmarking quantum machine learning models. arXiv preprint, 2024. arXiv:2403.07059.
Run it yourself with eigenkit
eigenkit is EigenOrbit's open-source Python library: write a circuit once, run it on any simulator or QPU, and test it against a classical baseline on your own data before you invest in it or rule it out.
$ pip install eigenkit