Membership Inference Attack

A membership inference attack asks a model a question it was never supposed to answer: was this specific record part of your training data? The attacker probes the model — often just via confidence scores or loss on a candidate example — and exploits the fact that models behave measurably differently on data they memorized than on data they have never seen. Overfitted models are the most vulnerable, so the attack doubles as a practical privacy audit. The stakes are concrete: confirming that a person's record was in a hospital's training set discloses that they were a patient, and regulators increasingly treat such leakage as a personal-data breach. Defenses include differential privacy during training (the principled fix), regularization and deduplication to curb memorization, and limiting the precision of confidence outputs exposed through your API. Any SaaS product that fine-tunes on customer data should test for this before shipping.