The dashboard says "perfect:" a systematic audit of label-free model monitoring and test-time adaptation under temporal distribution shift in network intrusion detection

The dashboard says "perfect:" a systematic audit of label-free model monitoring and test-time adaptation under temporal distribution shift in network intrusion detection. INTRODUCTION: Machine-learning classifiers deployed in security operations rarely receive timely ground-truth labels, so their health is monitored on an operational dashboard built from label-free proxies: confidence-based accuracy estimators, cross-model agreement, and conformal abstention. Performance degradation under distribution shift is countered with test-time adaptation and label-shift correction. This toolbox is validated predominantly on synthetic image-corruption benchmarks in which the shift is covariate-only and label-preserving, conditions that do not hold in the deployment regime that motivates the tools. METHODS: We present a systematic, four-iteration audit of this entire toolbox on a naturally occurring temporal shift of high practical importance: network intrusion detectors trained on Canadian Institute for Cybersecurity Intrusion Detection System 2017 (CIC-IDS2017) and deployed on Communications Security Establishment-Canadian Institute for Cybersecurity Intrusion Detection System 2018 (CSE-CIC-IDS2018) traffic recorded one year later, with evaluation also performed in the reverse direction, over seven harmonized classes (benign plus six attack families) and 77 shared flow features, under a strict protocol in which no target-year labels influence training, adaptation, hyperparameters, or model selection. Sixteen method variants are evaluated, spanning marginal quantile alignment, BatchNorm (BN) adaptation, entropy-minimization adaptation with uniform and source-prior anchoring, conformal-filtered and agreement-filtered pseudo-labeling, drift-weighted feature subspaces, expectation-maximization (EM) label-shift correction, per-domain rank representations, dimensionless ratio features, and invariance training. RESULTS: No evaluated variant outperforms the unadapted source model in…