Abstract：

The accelerating growth of malware variants and the increasing sophistication of evasion strategies pose significant challenges for traditional security mechanisms. Signature-based detection is unable to keep pace with polymorphic and metamorphic malware, while dynamic analysis is resource-intensive and vulnerable to anti-sandbox techniques. Malware visualization, which transforms binary executables into images, has emerged as a promising approach that enables the application of advanced deep learning models originally designed for computer vision. Among these, the YOLO (You Only Look Once) family has attracted considerable attention due to its balance of speed, accuracy, and capacity for localization. This review expands upon recent developments in applying YOLO to malware visualization, analyzing visualization techniques, architectural evolution across YOLO versions, classification and fine-grained detection strategies, comparative performance, limitations, and prospects for future research. The review shows that YOLO-based approaches consistently reach above 98% accuracy on standard datasets while offering superior real-time performance compared to conventional CNNs and hybrid models. Remaining challenges include lack of granular annotations, vulnerability to obfuscation and adversarial evasion, and inadequate interpretability. Potential solutions—including weakly supervised learning, multimodal fusion, explainable AI, and lightweight model deployment—are proposed to guide future work in this growing field.

Key Words：

review; YOLO algorithm applications; malware visualization detection