Multi-Model Forensic Ensemble and Partially Fine-Tuned CLIP Detector for Identifying AI-Generated Images: The Xabarnavis System Case Study
Keywords:
AI-generated image, Deepfake, Media forensics, CLIP, Vision Transformer, Ensemble, Calibration, Disinformation
Abstract
Photorealistic images generated by generative models (Midjourney, DALL·E, Flux, Stable Diffusion, GPT-image) are increasingly becoming a means of spreading disinformation in the mass-media information flow. This paper proposes a two-stage approach developed for the Xabarnavis media-forensics platform: (i) a degradation-robust binary detector in which the last 8 blocks of the CLIP ViT-L/14 encoder are fine-tuned, and (ii) an ensemble that performs weighted fusion of the outputs of 16 independent detectors. The model was trained on 134,108 images from the open-source OpenFake and CommunityForensics datasets (including 14,404 images from commercial generators) and evaluated on four held-out test sets: AUC 0.968 on new generators, AUC 0.958 on social-media images, and AUC 0.984 and 0.998 on the external GenImage++ and DRAGON datasets, respectively. The mathematical formulation of the task, loss function, temperature calibration, ensemble fusion formula, and model scheduling under constrained GPU memory are described.
References
1. Agarwal, A.K., Hamid, A.B.B.A., Ather, D., Tiwari, R.G., De, I., Krishna, K.R.: Lesion-aware ordinal transformer for diabetic retinopathy classification from fundus images. Biomedical and Pharmacology Journal 19(1), 233–245 (2026). https://doi.org/10.13005/bpj/3349 2. Bertazzini, G., Baracchi, D., Shullani, D., Echizen, I., Piva, A.: DRAGON: A large-scale dataset of realistic images generated by diffusion models. arXiv preprint (2025). https://doi.org/10.48550/arXiv.2505.11257 3. Cheng, S., et al.: CO-SPY: Combining semantic and pixel features to detect synthetic images by AI. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025) 4. Corvi, R., Cozzolino, D., Zingarini, G., Poggi, G., Nagano, K., Verdoliva, L.: On the detection of synthetic images generated by diffusion models. In: Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1–5. IEEE (2023). https://doi.org/10.1109/ICASSP49357.2023.10095167 5. Donchenko, D., Ovchar, N., Sadovnikova, N., Parygin, D., Shabalina, O., Ather, D.: Analysis of comments of users of social networks to assess the level of social tension. Procedia Computer Science 119, 359–367 (2017). https://doi.org/10. 1016/j.procs.2017.11.195 6. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16×16 words: Transformers for image recognition at scale. In: Proc. International Conference on Learning Representations (ICLR) (2021). https://doi.org/10.48550/arXiv.2010.11929 7. Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: Proc. International Conference on Machine Learning (ICML). pp. 1321–1330 (2017). https://doi.org/10.48550/arXiv.1706.04599
8. Gupta, S., Abdul Hamid, A.B., Nyamasvisva, T.E., Jain, V., Tyagi, N., Mun, N.K., Ather, D.: Cotton disease prediction using deep transfer learning: Comparative analysis of ResNet50, VGG16 and InceptionV3 models. International Journal of Engineering, Science and Information Technology 5(3), 588–594 (2025). https: //doi.org/10.52088/ijesty.v5i3.1116 9. Livernoche, V., Arodi, A., Musulan, A., Yang, Z., Salvail, A., Caron, G., Godbout, J.-F., Rabbany, R.: OpenFake: An open dataset and platform toward large-scale deepfake detection. arXiv preprint (2025). https://doi.org/10.48550/ arXiv.2509.09495 10. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: Proc. International Conference on Learning Representations (ICLR) (2019) 11. Ojha, U., Li, Y., Lee, Y.J.: Towards universal fake image detectors that generalize across generative models. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 24480–24489 (2023). https://doi.org/10. 48550/arXiv.2302.10174 12. Park, J., Owens, A.: Community Forensics: Using thousands of generators to train fake image detectors. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025) 13. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: Proc. International Conference on Machine Learning (ICML). pp. 8748–8763 (2021) 14. Saleem, M., Chawla, C., Ather, D.: Ethical considerations in the deployment of generative AI systems in academia. In: Generative AI: Current Trends and Applications. Studies in Computational Intelligence, vol. 1177, pp. 245–256. Springer, Singapore (2024). https://doi.org/10.1007/978-981-97-8460-8_12 15. Sattarov, K., Shakhriyorbek, B., Chaudhary, N., Ather, D., Kler, R., Shukla, B.: Automated sentiment analysis and emotion recognition of Steam game reviews using ensemble machine learning techniques. In: 2024 1st International Conference on Sustainable Computing and Integrated Communication in Changing Landscape of AI (ICSCAI). pp. 1–10. IEEE (2024). https://doi.org/10.1109/ICSCAI61790. 2024.10867053 16. Shukla, R.S., Ather, D., Raj, D., Jain, A., Sarkar, T., Rakhra, M.: A secure and intelligent framework for cyber threat detection using machine learning. Journal of Discrete Mathematical Sciences and Cryptography 29(6), 2441–2457 (2026). https://doi.org/10.47974/JDMSC-2752 17. Tripathi, R.P., Khatri, S.K., Van Greunen, D., Ather, D.: Enhancing breast cancer diagnosis through segmentation-driven generative adversarial networks for synthetic mammogram generation. In: Proc. 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS). pp. 1078–1082. IEEE (2023). https://doi.org/10.1109/ICTACS59847.2023.10389935 18. Wang, S.-Y., Wang, O., Zhang, R., Owens, A., Efros, A.A.: CNN-generated images are surprisingly easy to spot... for now. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 8692–8701 (2020). https: //doi.org/10.1109/CVPR42600.2020.00872 19. Yan, S., Li, O., Cai, J., Hao, Y., Jiang, X., Hu, Y., Xie, W.: A sanity check for AI-generated image detection. In: Proc. International Conference on Learning Representations (ICLR) (2025). https://doi.org/10.48550/arXiv.2406.19435 20. Yan, Z., et al.: Orthogonal subspace decomposition for generalizable AI-generated image detection (Effort). In: Proc. International Conference on Machine Learning (ICML) (2025)
21. Zhou, Y., He, X., Lin, K., Fan, B., Ding, F., Li, B.: Breaking latent prior bias in detectors for generalizable AIGC image detection. In: Advances in Neural Information Processing Systems (NeurIPS). pp. 34606–34636 (2025). https: //doi.org/10.52202/085713-1030
8. Gupta, S., Abdul Hamid, A.B., Nyamasvisva, T.E., Jain, V., Tyagi, N., Mun, N.K., Ather, D.: Cotton disease prediction using deep transfer learning: Comparative analysis of ResNet50, VGG16 and InceptionV3 models. International Journal of Engineering, Science and Information Technology 5(3), 588–594 (2025). https: //doi.org/10.52088/ijesty.v5i3.1116 9. Livernoche, V., Arodi, A., Musulan, A., Yang, Z., Salvail, A., Caron, G., Godbout, J.-F., Rabbany, R.: OpenFake: An open dataset and platform toward large-scale deepfake detection. arXiv preprint (2025). https://doi.org/10.48550/ arXiv.2509.09495 10. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: Proc. International Conference on Learning Representations (ICLR) (2019) 11. Ojha, U., Li, Y., Lee, Y.J.: Towards universal fake image detectors that generalize across generative models. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 24480–24489 (2023). https://doi.org/10. 48550/arXiv.2302.10174 12. Park, J., Owens, A.: Community Forensics: Using thousands of generators to train fake image detectors. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2025) 13. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: Proc. International Conference on Machine Learning (ICML). pp. 8748–8763 (2021) 14. Saleem, M., Chawla, C., Ather, D.: Ethical considerations in the deployment of generative AI systems in academia. In: Generative AI: Current Trends and Applications. Studies in Computational Intelligence, vol. 1177, pp. 245–256. Springer, Singapore (2024). https://doi.org/10.1007/978-981-97-8460-8_12 15. Sattarov, K., Shakhriyorbek, B., Chaudhary, N., Ather, D., Kler, R., Shukla, B.: Automated sentiment analysis and emotion recognition of Steam game reviews using ensemble machine learning techniques. In: 2024 1st International Conference on Sustainable Computing and Integrated Communication in Changing Landscape of AI (ICSCAI). pp. 1–10. IEEE (2024). https://doi.org/10.1109/ICSCAI61790. 2024.10867053 16. Shukla, R.S., Ather, D., Raj, D., Jain, A., Sarkar, T., Rakhra, M.: A secure and intelligent framework for cyber threat detection using machine learning. Journal of Discrete Mathematical Sciences and Cryptography 29(6), 2441–2457 (2026). https://doi.org/10.47974/JDMSC-2752 17. Tripathi, R.P., Khatri, S.K., Van Greunen, D., Ather, D.: Enhancing breast cancer diagnosis through segmentation-driven generative adversarial networks for synthetic mammogram generation. In: Proc. 3rd International Conference on Technological Advancements in Computational Sciences (ICTACS). pp. 1078–1082. IEEE (2023). https://doi.org/10.1109/ICTACS59847.2023.10389935 18. Wang, S.-Y., Wang, O., Zhang, R., Owens, A., Efros, A.A.: CNN-generated images are surprisingly easy to spot... for now. In: Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 8692–8701 (2020). https: //doi.org/10.1109/CVPR42600.2020.00872 19. Yan, S., Li, O., Cai, J., Hao, Y., Jiang, X., Hu, Y., Xie, W.: A sanity check for AI-generated image detection. In: Proc. International Conference on Learning Representations (ICLR) (2025). https://doi.org/10.48550/arXiv.2406.19435 20. Yan, Z., et al.: Orthogonal subspace decomposition for generalizable AI-generated image detection (Effort). In: Proc. International Conference on Machine Learning (ICML) (2025)
21. Zhou, Y., He, X., Lin, K., Fan, B., Ding, F., Li, B.: Breaking latent prior bias in detectors for generalizable AIGC image detection. In: Advances in Neural Information Processing Systems (NeurIPS). pp. 34606–34636 (2025). https: //doi.org/10.52202/085713-1030
Published
2026-10-29
How to Cite
Solijonov Abduxoliq. (2026). Multi-Model Forensic Ensemble and Partially Fine-Tuned CLIP Detector for Identifying AI-Generated Images: The Xabarnavis System Case Study. MATRIX Academic International Online Journal Of Engineering And Technology, 9(2), 57-68. Retrieved from https://maiojet.com/index.php/matrix/article/view/121
Section
Articles
