Know which task the system performs
Image classification assigns a label to a picture or crop. Object detection places boxes around candidate features. Segmentation marks pixels believed to belong to a feature. Anomaly detection may identify patterns unlike the examples used to learn “normal.” These outputs have different meanings, but none automatically measures material strength. A neat outline is a location estimate, not a boundary established by physical examination.
The distinction matters when selecting a tool. A whole-image classifier may use a background cue rather than the suspected mark. A detector may miss a feature outside its expected size range. A segmentation system can draw a persuasive mask around glare. Ask what output was validated, on which equipment and under which capture conditions, before relying on its presentation.
Visible evidence has physical limits
Ordinary photographs describe light reflected from accessible surfaces. They do not directly reveal every internal bond, foam cell, embedded fibre or fatigue process. Composite delamination, internal helmet damage and hidden corrosion can exist without a clear external sign. Some concerns require manufacturer-directed inspection or specialist methods. An AI model cannot recover a reliable physical measurement from information the image never contained.
This does not make image review useless. It can help prioritise photographs, draw attention to visible changes and make documentation more consistent. The benefit is narrower than certification: better organisation of evidence, with explicit boundaries around the conclusion.
False negatives are not the only error
A false negative leaves a relevant feature unflagged. A false positive flags a harmless seam or reflection. Both matter, but their consequences depend on the workflow. Repeated false alarms may lead staff to ignore warnings; missed features may produce unjustified reassurance. A useful evaluation reports both kinds of error and explains how uncertain or unreadable images are handled.
“Accuracy” alone can be misleading when most photographs show no labelled defects. A system can appear successful by mostly predicting the common class. Review precision and recall for the relevant defect families, as well as item-level outcomes. If the intended use concerns protective equipment, do not substitute a broad average across unrelated product types.
Confidence is not severity
A score attached to “crack” usually reflects the model’s own prediction process. It is not necessarily a calibrated probability, and it does not measure crack depth, residual strength or urgency. A model can be confidently wrong on a new surface finish. It can also be uncertain about an obvious concern because the image differs from its training examples.
Threshold changes alter the balance of missed detections and false alarms. They do not create a universal safety boundary. Record the model version, threshold and preprocessing settings so a later reviewer can understand the result. Do not compare scores from different tools as though they share a common measurement scale.
Test on equipment the model has not seen
Data leakage can make a system look better than it is. If nearly identical photographs of the same racket appear in both training and testing, the test may reward recognition of that item rather than generalisation to new damage. Split data by physical item and, where relevant, by site, session and equipment family. Keep a genuinely held-out evaluation set.
Domain shift occurs when deployment differs from development: a different phone, darker workshop, new composite weave or unusual coating. Evaluate those changes explicitly. Human label quality also limits performance. An expert disagreement should be recorded, not erased to manufacture a single confident answer. Reference labels need a documented basis appropriate to the claim being tested.
Standards are context, not an AI shortcut
NIST’s AI Risk Management Framework provides a way to think about AI risks and governance; it does not certify a particular inspection model. ISO/IEC 23894 addresses AI risk management guidance. Neither turns a detector output into a product conformity finding.
Likewise, ISO publishes the ISO 4210 bicycle safety series, CEN develops European standards including EN 1078 for certain cycling and skating helmets, and the U.S. Consumer Product Safety Commission explains its bicycle helmet requirements. These concern defined products and test conditions. A photograph of an in-service item does not reproduce those tests or renew a certification.