Background. Machine-learning intrusion detection is routinely reported with near-perfect accuracy, yet deployments disappoint. Frameworks that promise adaptivity, explainability and resilience compound the problem: each property is usually demonstrated in isolation and in-distribution, where it is easiest to satisfy. Objective. We specify AERCF, an architecture that fuses unsupervised and supervised detectors, sets thresholds by extreme-value theory (EVT), explains decisions with SHAP, hardens against adversarial perturbation, adapts its threshold from feedback, and maps scores onto a tiered containment lattice. We then ask which of these properties survive the move from the training distribution to new data and new attacks. Methods. We implemented the detection, calibration, explanation and adaptation layers in Python and evaluated them on NSL-KDD with five seeds, using the held-out KDDTest+ (22,544 records, 17 attack types absent from training) and the harder KDDTest-21. Seven experiments measured detection under shift, EVT calibration, explanation faithfulness and stability, white-box and transferred PGD attacks, four threshold-adaptation policies on a constructed non-stationary stream, tiered containment operating points, and inference cost. The cloud-native containment layer (eBPF telemetry, pod quarantine, token revocation) is specified but not evaluated. Results. In-distribution, the supervised detectors and the fusion reached AUROC 1.000 and detection rates above 99.9% at 1% FPR. On KDDTest+, the best detector (XGBoost) detected 81.3% of attacks while its false-positive rate rose to 3.85% against a 1% target; on KDDTest-21 it rose to 16.4%. Stacked fusion did not beat its best member (AUROC 0.958 vs 0.967) and its weights changed sign across seeds. EVT thresholds met their targets on held-out validation data but exceeded them 29-fold on KDDTest+ at α = 0.1%. The variational autoencoder lost only 3.5 percentage points on novel attack types, against 13.3 for XGBoost. SHAP explanations were exact, faithful (removing the top three features flipped 53.7% of detections, against 1.3% for random features) and stable across retraining (global rank correlation 0.97–0.99). Adversarial training raised detection under PGD at ε = 0.1 from 49.8% to 63.1%, and an OR-combination of XGBoost and the autoencoder kept 79.0% detection under transferred attacks that cut XGBoost alone to 61.0%. All three adaptive threshold policies underperformed a fixed threshold; one, a feedback controller on a 2% audit sample, drove detection to zero. At an isolation threshold calibrated for 0.1% FPR, 2.78% of benign test records would have been isolated; requiring corroboration by an unsupervised detector halved this to 1.35% while retaining 54.1% of attacks. Conclusions. Explanation and robustness to perturbation held up; calibration, fusion and naive adaptivity did not. A credible adaptive defence must treat its own false-positive rate as unknown after deployment, prefer diverse detector families over learned fusion, and gate disruptive containment on corroboration. Code and raw results are released.

REFERENCES

[1]. H. M. Melaku, ―Context-based and adaptive cybersecurity risk management framework,‖ Risks, vol. 11, no. 6, art. 101, 2023.

[2]. M. Tavallaee, E. Bagheri, W. Lu, and A. A. Ghorbani, ―A detailed analysis of the KDD CUP 99 data set,‖ in Proc. IEEE Symp. Computational Intelligence for Security and Defense Applications (CISDA), 2009, pp. 1–6.

[3]. R. Sommer and V. Paxson, ―Outside the closed world: On using machine learning for network intrusion detection,‖ in Proc. IEEE Symp. Security and Privacy, 2010, pp. 305–316.

[4]. D. Arp et al., ―Dos and don‘ts of machine learning in computer security,‖ in Proc. 31st USENIX Security Symp., 2022.

[5]. G. Engelen, V. Rimmer, and W. Joosen, ―Troubleshooting an intrusion detection dataset: The CICIDS2017 case study,‖ in Proc. IEEE Security and Privacy Workshops (SPW), 2021.

[6]. F. T. Liu, K. M. Ting, and Z.-H. Zhou, ―Isolation forest,‖ in Proc. 8th IEEE Int. Conf. Data Mining (ICDM), 2008, pp. 413 422.

[7]. D. P. Kingma and M. Welling, ―Auto-encoding variational Bayes,‖ in Proc. Int. Conf. Learning Representations (ICLR), 2014.

[8]. J. An and S. Cho, ―Variational autoencoder based anomaly detection using reconstruction probability,‖ SNU Data Mining Center, Tech. Rep., 2015.

[9]. T. Chen and C. Guestrin, ―XGBoost: A scalable tree boosting system,‖ in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2016, pp. 785–794.

[10]. D. H. Wolpert, ―Stacked generalization,‖ Neural Networks, vol. 5, no. 2, pp. 241–259, 1992.

[11]. S. Bai, J. Z. Kolter, and V. Koltun, ―An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,‖ arXiv:1803.01271, 2018.

[12]. J. Pickands III, ―Statistical inference using extreme order statistics,‖ Ann. Statist., vol. 3, no. 1, pp. 119–131, 1975.

[13]. S. Coles, An Introduction to Statistical Modeling of Extreme Values. London: Springer, 2001.

[14]. A. Siffer, P.-A. Fouque, A. Termier, and C. Largouët, ―Anomaly detection in streams with extreme value theory,‖ in Proc. 23rd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2017, pp. 1067–1075.

[15]. S. M. Lundberg and S.-I. Lee, ―A unified approach to interpreting model predictions,‖ in Advances in Neural Information Processing Systems 30, 2017.

[16]. S. M. Lundberg et al., ―From local explanations to global understanding with explainable AI for trees,‖ Nature Machine Intelligence, vol. 2, pp. 56–67, 2020.

[17]. M. T. Ribeiro, S. Singh, and C. Guestrin, ―‗Why should I trust you?‘: Explaining the predictions of any classifier,‖ in Proc. 22nd ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining, 2016, pp. 1135–1144.

[18]. S. Bach et al., ―On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation,‖ PLoS ONE, vol. 10, no. 7, e0130140, 2015.

[19] S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim, ―A benchmark for interpretability methods in deep neural networks,‖ in Advances in Neural Information Processing Systems 32, 2019.

[20]. A. Warnecke, D. Arp, C. Wressnegger, and K. Rieck, ―Evaluating explanation methods for deep learning in security,‖ in Proc. IEEE European Symp. Security and Privacy (EuroS&P), 2020, pp. 158–174.

[21]. I. J. Goodfellow, J. Shlens, and C. Szegedy, ―Explaining and harnessing adversarial examples,‖ in Proc. ICLR, 2015.

[22] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, ―Towards deep learning models resistant to adversarial attacks,‖ in Proc. ICLR, 2018.

[23]. N. Carlini and D. Wagner, ―Towards evaluating the robustness of neural networks,‖ in Proc. IEEE Symp. Security and Privacy, 2017, pp. 39–57.

[24]. A. Athalye, N. Carlini, and D. Wagner, ―Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,‖ in Proc. 35th Int. Conf. Machine Learning (ICML), 2018, pp. 274–283.

[25]. F. Tramèr, N. Carlini, W. Brendel, and A. Madry, ―On adaptive attacks to adversarial example defenses,‖ in Advances in Neural Information Processing Systems 33, 2020.

[26] F. Pierazzi, F. Pendlebury, J. Cortellazzi, and L. Cavallaro, ―Intriguing properties of adversarial ML attacks in the problemspace,‖ in Proc. IEEE Symp. Security and Privacy, 2020, pp. 1332–1349.

[27]. G. Apruzzese et al., ―‗Real attackers don‘t compute gradients‘: Bridging the gap between adversarial ML research and practice,‖ in Proc. IEEE Conf. Secure and Trustworthy Machine Learning (SaTML), 2023.

[28]. .J. Cohen, E. Rosenfeld, and J. Z. Kolter, ―Certified adversarial robustness via randomized smoothing,‖ in Proc. 36th ICML, 2019, pp. 1310–1320.

[29]. J. Gama, I. Žliobaitė, A. Bifet, M. Pechenizkiy, and A. Bouchachia, ―A survey on concept drift adaptation,‖ ACM Comput. Surv., vol. 46, no. 4, art. 44, 2014.

[30]. R. Jordaney et al., ―Transcend: Detecting concept drift in malware classification models,‖ in Proc. 26th USENIX Security Symp., 2017, pp. 625–642.

[31]. S. Rabanser, S. Günnemann, and Z. C. Lipton, ―Failing loudly: An empirical study of methods for detecting dataset shift,‖ in Advances in Neural Information Processing Systems 32, 2019.

[32].A. N. Angelopoulos and S. Bates, ―Conformal prediction: A gentle introduction,‖ Foundations and Trends in Machine Learning, vol. 16, no. 4, pp. 494–591, 2023.

[33].V. Mnih et al., ―Human-level control through deep reinforcement learning,‖ Nature, vol. 518, pp. 529–533, 2015.

[34]. L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, ―Planning and acting in partially observable stochastic domains,‖ Artificial Intelligence, vol. 101, pp. 99–134, 1998.

[35].K. J. Åström and R. M. Murray, Feedback Systems: An Introduction for Scientists and Engineers. Princeton, NJ: Princeton Univ. Press, 2008.

[36]. M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu, ―Safe reinforcement learning via shielding,‖ in Proc. AAAI Conf. Artificial Intelligence, 2018, pp. 2669–2678.

[37].S. Rose, O. Borchert, S. Mitchell, and S. Connelly, ―Zero trust architecture,‖ NIST SP 800-207, 2020, doi: 10.6028/NIST.SP.800-207.

[38]. National Institute of Standards and Technology, ―The NIST Cybersecurity Framework (CSF) 2.0,‖ NIST CSWP 29, 2024, doi: 10.6028/NIST.CSWP.29.

[39]. B. Gregg, BPF Performance Tools: Linux System and Application Observability. Boston, MA: Addison-Wesley, 2019.

[40]. The Kubernetes Authors, ―Network policies‖ and ―Pod security standards,‖ Kubernetes documentation. [Online]. Available: https://kubernetes.io/docs/

[41]. NIST National Vulnerability Database, ―CVE-2022-0492,‖ 2022. [Online]. Available: https://nvd.nist.gov/vuln/detail/CVE 2022-0492

[42].NIST National Vulnerability Database, ―CVE-2022-0847,‖ 2022. [Online]. Available: https://nvd.nist.gov/vuln/detail/CVE 2022-0847

[43] .Y. Mirsky, T. Doitshman, Y. Elovici, and A. Shabtai, ―Kitsune: An ensemble of autoencoders for online network intrusion detection,‖ in Proc. Network and Distributed System Security Symp. (NDSS), 2018.

[44].M. Ring, S. Wunderlich, D. Scheuring, D. Landes, and A. Hotho, ―A survey of network-based intrusion detection data sets,‖ Computers & Security, vol. 86, pp. 147–167, 2019.