NullifAI: How Malicious Models Bypass AI Security Scanners
In February 2025, researchers at ReversingLabs published findings that should have set off alarms across every enterprise ML team: they had discovered malicious machine learning models uploaded to Hugging Face that used a novel technique to bypass Picklescan, the primary security scanner used to detect malicious code in Python Pickle files. The technique was dubbed “NullifAI” — a name that captures what it does to the security tools meant to protect against exactly this kind of attack.
The discovery was reported by Infosecurity Magazine, The Hacker News, and multiple security research outlets. It represents a genuinely new category of supply chain attack: one that targets model artifacts rather than software packages, and one that exploits the specific technical assumptions of the tools designed to prevent it.
What Happened
Hugging Face is the largest open repository for machine learning models, hosting hundreds of thousands of model files that researchers and organizations download and run. Many of these models are distributed as Python Pickle files — a serialization format that can execute arbitrary code when deserialized. This is a known risk. Hugging Face deployed Picklescan, an open-source scanner, to detect malicious Pickle files before users download them.
The ReversingLabs team discovered models on Hugging Face that contained reverse shell payloads — code that, when the model file is loaded, opens a network connection back to an attacker-controlled server, giving the attacker remote access to the machine that loaded the model. These payloads were functional and dangerous. And they had passed Picklescan without triggering any alerts.
The Bypass Technique
The NullifAI technique exploited two specific weaknesses in Picklescan's scanning approach.
First, broken Pickle file formats. Picklescan parses Pickle files according to the standard Pickle protocol specification. The malicious models used intentionally malformed Pickle files that deviated from the specification in ways that caused Picklescan's parser to fail or skip sections, while Python's own Pickle deserializer — which is more permissive — still processed them successfully. The scanner saw a broken file. The Python runtime saw a functional file with an embedded payload.
Second, 7z compression instead of ZIP. Pickle files in machine learning models are typically stored inside ZIP archives (the standard format used by PyTorch's torch.save). Picklescan was designed to handle ZIP-compressed Pickle files. The malicious models used 7z compression instead. Picklescan did not recognize the archive format and could not extract and scan the Pickle files inside. The compression change was trivial for the attacker and invisible to the scanning infrastructure.
Why This Matters for Enterprise ML
The NullifAI discovery matters not because of the specific models found — Hugging Face removed them after disclosure — but because of what it reveals about the security assumptions underlying enterprise ML pipelines.
Model Artifacts Are Not Safe Just Because Source Code Is Clean
Most enterprise security teams focus their supply chain security efforts on source code and software packages. They scan npm packages, PyPI packages, Maven artifacts, and container images. They review code before deploying it. These are necessary practices, but they completely miss the model artifact attack surface.
A machine learning model file is not source code. It is a serialized binary artifact that, in the case of Pickle-based formats, can contain arbitrary executable code. When your ML pipeline downloads a model from Hugging Face, a model registry, or an internal artifact store and loads it into a Python process, it is executing that artifact. The security implications are identical to downloading and running an unknown binary — but the organizational perception is that it is “just loading a model.”
Single-Scanner Reliance Is Insufficient
Picklescan is a valuable tool, and the Hugging Face team has been responsive in updating it to address the NullifAI bypass. But the broader lesson is that relying on a single scanner for a critical security boundary is insufficient. The NullifAI technique was not sophisticated cryptographic evasion — it was a format mismatch between how the scanner parsed files and how the Python runtime parsed them. This category of bypass applies to any scanner that makes assumptions about file format compliance.
Defense in depth applies to model artifacts the same way it applies to every other security domain. Multiple scanning approaches, behavioral analysis, sandboxed execution during validation, and network monitoring during model loading are all necessary layers.
The Supply Chain Extends to Model Registries
Enterprise organizations that use pre-trained models — whether from Hugging Face, internal registries, or vendor-provided model stores — need to treat their model supply chain with the same rigor they apply to their software supply chain. This means provenance tracking for every model artifact. It means scanning with multiple tools, not just one. It means sandboxed environments for model validation before any model touches production data. And it means monitoring network behavior when models are loaded, because a reverse shell payload makes outbound connections that network monitoring can detect even when file scanners miss the payload.
Implications for Agentic AI
The NullifAI attack is particularly concerning in the context of agentic AI systems, where models are not just generating text but taking actions — calling APIs, accessing databases, modifying files, and interacting with production infrastructure.
An agentic system that loads a compromised model is not just leaking information through a reverse shell. It is potentially giving an attacker access to every tool and credential the agent has access to. If the agent can query your production database, the attacker can query your production database. If the agent can send Slack messages on behalf of employees, the attacker can send Slack messages on behalf of employees. The attack surface of a compromised agent is the union of every tool and permission granted to that agent.
This is why sandboxed execution environments are not optional for production agentic AI. When models execute inside isolated sandboxes with explicit network policies, credential scoping, and resource boundaries, a compromised model's blast radius is contained. The reverse shell cannot reach the attacker if the sandbox blocks outbound connections that are not on the explicit allowlist. The payload cannot access production credentials if the sandbox uses short-lived, scoped tokens that are only issued after human approval.
What to Do About It
For enterprise ML and AI teams, the NullifAI discovery should trigger a review of model artifact security practices. Specific actions include:
- Audit your model supply chain. Identify every source of model artifacts in your pipeline — public repositories, vendor-provided models, internally trained models, fine-tuned variants. Each source needs a validation process.
- Deploy multiple scanning approaches. Do not rely solely on Picklescan or any single scanner. Layer static analysis, format validation, behavioral analysis, and runtime monitoring.
- Prefer safer serialization formats. Where possible, use SafeTensors or other formats that do not support arbitrary code execution during deserialization. Many model providers now offer SafeTensors as an alternative to Pickle-based formats.
- Load models in sandboxed environments. Model validation should happen in isolated environments with no access to production data, no outbound network access except to known-good endpoints, and full network monitoring.
- Monitor network behavior during model loading. A reverse shell payload makes outbound connections. Network monitoring during model deserialization can detect payloads that file scanners miss.
Model artifacts are the next frontier of supply chain attacks. The NullifAI technique demonstrated that the scanning tools the community relies on can be bypassed with relatively simple format manipulation. The enterprise response needs to match the threat: defense in depth, sandboxed execution, and the assumption that any model artifact could be hostile until proven otherwise.
How Our Platform Protects Against Model-Level Attacks
The NullifAI attack exploits the moment a model artifact is loaded and code executes. Our platform breaks this attack chain at multiple points. Every AI workload runs inside K8s-native sandboxed execution with explicit network policies — a reverse shell payload cannot reach an attacker-controlled server because the sandbox blocks outbound connections that are not on the explicit allowlist. The DLP Scanner monitors all data flowing in and out of the execution environment in real time, detecting credential exfiltration attempts and PII exposure across a configurable set of detection patterns before data leaves the boundary.
Critically, the platform's OBO (On-Behalf-Of) Authentication ensures that AI agents never hold persistent credentials. Instead, agents receive scoped, short-lived tokens tied to the specific user identity they are acting on behalf of. Even if a compromised model achieves code execution inside the sandbox, there are no standing credentials to steal — every token is task-scoped and time-limited. Combined with the platform's Audit System logging every execution with cryptographic hashing, organizations get both containment and forensic visibility for model-level threats.