Recent cybersecurity research has highlighted a significant security concern within machine learning (ML) model sharing platforms. Two malicious ML models, discovered on the popular Hugging Face repository, employed a novel technique using “broken” pickle files to evade detection. This method has been dubbed nullifAI, reflecting the models’ attempts to circumvent existing security protocols.
The Discovery of Malicious Models
Cybersecurity experts from ReversingLabs uncovered these malicious models, revealing that the exploit leveraged faulty pickle files to embed harmful Python code. Karlo Zanki, a researcher at ReversingLabs, detailed the discovery in a recent report. Zanki explained that the malicious payloads consisted of platform-specific reverse shells configured to connect to predetermined IP addresses.
The malicious models in question were hosted at specific Hugging Face repositories:
- glockr1/ballr7
- who-r-u0000/0000000000000000000000000000000000000
While these models were likely intended as proof-of-concept demonstrations rather than active supply chain attacks, they underscore significant vulnerabilities in current security measures.
Understanding the NullifAI Technique
The nullifAI technique primarily hinges on the exploitation of pickle serialization. Typically used to distribute ML models, pickle files can pose a substantial security risk due to their ability to execute arbitrary code upon deserialization. In this case, the malicious payloads were strategically positioned at the beginning of the pickle streams, making evasive maneuvers easier.
The models were stored in the PyTorch format, typically a compressed pickle file using ZIP for compression. However, these specific models utilized the less common 7z format, eluding detection by Picklescan, a security tool employed by Hugging Face.
When these models were deserialized, the malicious code executed immediately, followed by a failure in object deserialization due to the intentional disruption of the pickle stream.
The Impact on Security Protocols
The discovery of these malicious models highlights critical gaps in ML model security. Traditional security measures, such as those used by Hugging Face, often struggle with such sophisticated evasion tactics. The use of broken pickle files exploits the sequential nature of object deserialization, where each opcode in the stream is executed as encountered.
Zanki noted, “Pickle opcodes are executed in sequence, and the malicious payload, placed at the beginning, executes before any security checks can flag the model as unsafe.”
Addressing the Vulnerability
In response to these findings, the developers of Picklescan have released an update to enhance the tool’s detection capabilities. The security patch aims to resolve the identified bug that allowed partial deserialization of broken pickle files, improving the overall security posture for users of Hugging Face’s repository.
The incident underscores the need for continuous monitoring and updating of security tools to counteract emerging threats in the field of machine learning.
Conclusion
The advent of nullifAI and similar techniques exemplify the evolving nature of cybersecurity threats. As machine learning becomes increasingly integrated into various sectors, protecting ML models from malicious exploitation must remain a top priority. This incident serves as a reminder of the critical importance of robust security practices and the need for ongoing research to anticipate and mitigate future threats.
This article provides a comprehensive understanding of the recent discovery of malicious machine learning models on Hugging Face, highlighting the use of the nullifAI technique and the vulnerabilities it exposes. It also discusses the steps taken to address the issue and emphasizes the importance of robust security measures in the field of machine learning. The structure includes clear subheadings and shorter paragraphs for better readability and SEO optimization.
