Introduction

Native Language Identification (NLI) seeks to infer an author’s first language (L1) from their writing in a second language (L2). While earlier studies reported strong performance on curated learner corpora, contemporary deployments confront a markedly different landscape: user‑generated content (UGC) that is informal, topical, and noisy. In such settings, conventional accuracy metrics can obscure a critical issue – models may succeed by exploiting spurious topical cues rather than genuine cross‑linguistic transfer. This blog post explains the evaluation framework that treats performance, fairness, and generalization as co‑equal objectives, with explicit tests for topic leakage and mechanisms for rejecting unseen languages.

The practical question is not only “How accurate is the model?” but “Accurate on what basis, and under what distributional shifts?” We therefore emphasize (i) cross‑topic evaluation to decouple linguistic signal from domain content, (ii) bias‑leakage auditing to quantify spurious correlations, and (iii) open‑set recognition so the system can state “unknown” when confronted with L1s absent from training. Together, these components support trustworthy NLI suitable for research and pedagogical use.