Browse / Unsafe Model Loading and Supply Chain
Unsafe Model Loading and Supply Chain
Malicious model files, pickle deserialisation, poisoned packages and artifacts. 280 records. Reference: OWASP LLM03 · ATLAS AML.T0010.
- Code and command execution 98
- Unsafe deserialisation 77
- Denial of service 39
- Path traversal and file access 20
- Memory corruption 18
- Real-world incidents 6
- Data exfiltration 5
- Publish Poisoned AI Artifacts: Models 5
- Other weaknesses 3
- Supply Chain Compromise: Software 3
- Information exposure 2
- Supply Chain Compromise: Data Repository 2
- Cross-site scripting 1
- Publish Poisoned AI Artifacts: AI Agent Tools 1
- DeepJavaLibrary API absolute path traversal DJL extracts archive entries with absolute paths to their literal location, so a malicious model artifact can overwrite system files.
- onnx allows Arbitrary File Overwrite in download_model_with_test_data A malicious model tar archive with absolute or traversing member paths is extracted without checks, letting it overwrite arbitrary files on the host.
- MLFlow unsafe deserialization A malicious scikit-learn model uploaded to MLflow deserialises into code execution when a user loads and interacts with it.
- MLFlow unsafe deserialization A malicious scikit-learn model uploaded to MLflow is deserialised with pickle, so attacker code runs when the model is loaded or used.
- MLFlow unsafe deserialization A malicious serialised scikit-learn or LightGBM model uploaded to MLflow executes code when a user loads and interacts with it.
- MLFlow unsafe deserialization A maliciously crafted MLflow recipe is deserialised unsafely, so loading and running it executes attacker code.
- MLFlow unsafe deserialization A malicious LangChain AgentExecutor model uploaded to MLflow is deserialised when used, executing attacker-chosen code on the consumer's machine.
- MLFlow unsafe deserialization A malicious PyFunc model artifact is deserialised when loaded, so its embedded code runs on the machine that uses the model.
- MLFlow unsafe deserialization Loading a maliciously crafted TensorFlow model through MLflow triggers unsafe deserialisation that runs attacker code on the user's system.
- MLFlow improper input validation A crafted MLproject file passes unfiltered input into execution, so attacker-supplied commands run on the machine that runs the project.
- MLFlow unsafe deserialization A maliciously uploaded PyTorch model in MLflow is deserialised on load, running attacker-chosen code on the machine that interacts with it.
- MLFlow unsafe deserialization A malicious pmdarima model uploaded to MLflow runs arbitrary code on the machine that loads and uses it.
- Skops unsafe deserialization Loading a maliciously crafted skops model file deserialises untrusted data, which runs attacker code on the machine that loads it.
- ChatGPT Package Hallucination
- llama-cpp-python vulnerable to Remote Code Execution by Server-Side Template Injection in Model Metadata A malicious GGUF file's chat template is rendered by an unsandboxed Jinja2 environment, so template code runs on the host.
- sagemaker-python-sdk vulnerable to Deserialization of Untrusted Data NumpyDeserializer unpickles object arrays from untrusted input, so a crafted pickle payload can execute code on the client.
- Keras code injection vulnerability Loading a crafted Keras model file lets embedded code run with the permissions of the host application.
- LiteLLM has Server-Side Template Injection vulnerability in /completions endpoint A malicious tokenizer_config.json supplies a chat_template that LiteLLM renders with Jinja unsafely, executing code on the server when the model is used.
- Transformers Deserialization of Untrusted Data vulnerability A malicious checkpoint containing a pickle payload is loaded by the TensorFlow model repository loader, which deserialises it and runs attacker code when loaded.
- Cross-site Scripting in MLFlow Running an untrusted MLflow recipe renders template variables without sanitisation, so script injected through them executes in the notebook client.
- Onnx Directory Traversal vulnerability An ONNX tensor's external_data path can point outside the model directory, getting around the earlier fix and reading arbitrary files during model loading.
- Allegro AI ClearML vulnerable to deserialization of untrusted data The client SDK deserialises artifacts without safety checks, so a maliciously uploaded artifact runs code when an end user interacts with it.
- transformers has a Deserialization of Untrusted Data vulnerability Transformers releases before 4.36 deserialise untrusted data without adequate safeguards, so loading a crafted model artifact can create attacker-chosen objects.
- mlflow Command Injection vulnerability A crafted MLflow configuration file, once downloaded and loaded, injects shell commands that run on the victim's machine.
- transformers has a Deserialization of Untrusted Data vulnerability Untrusted serialised data is deserialised in transformers releases before 4.36.0, so a crafted input can trigger unsafe object loading.
- Purportedly Hallucinated Software Packages with Potential Malware Reportedly Downloaded Thousands of Times by Developers Attackers registered package names that LLMs had hallucinated, and developers downloaded the malicious packages over 15,000 times, introducing potential malware into codebases through slopsquatting.
- Organization Confusion on Hugging Face
- langchain vulnerable to arbitrary code execution A crafted prompt JSON file passed to load_prompt can reach Python internals through template and __subclasses__ tricks, executing code when loaded.
- PoisonGPT
- Langchain vulnerable to arbitrary code execution Loading a crafted prompt through LangChain's load_prompt function causes attacker-supplied code to run on the host.
- TensorFlow has Floating Point Exception in TFLite in conv kernel A crafted TFLite model with a non-positive input channel count triggers a floating-point exception during convolution kernel setup.
- Directory Traversal in onnx An ONNX model's external tensor data path is not confined to the model directory, so a crafted model can read files outside it.
- Compromised PyTorch Dependency Chain
- Arbitrary Code Execution with Google Colab
- scikit-learn Deserialization of Untrusted Data Loading a crafted joblib model file runs attacker-chosen commands embedded through a reduce method during unpickling.
- scikit-learn Denial of Service A crafted serialized SVM model with a large support-vector count value causes libsvm to segfault during prediction.
- `CHECK`-failures in `TensorByteSize` in Tensorflow A crafted SavedModel causes a tensor size helper to hit internal assertions, aborting the TensorFlow process when the model is loaded.
- `CHECK`-failures during Grappler's `SafeToRemoveIdentity` in Tensorflow A tampered SavedModel causes the Grappler identity-removal check to hit assertion failures, crashing the process when the model is optimized.
- Null-dereference in Tensorflow A crafted SavedModel with a missing nested signature key makes the loader dereference a null pointer and crash when the model is opened.
- `CHECK`-failures in binary ops in Tensorflow A tampered SavedModel with mismatched tensor dtypes makes binary ops hit CHECK failures, crashing the process when the model runs.
- Stack overflow in TensorFlow A SavedModel containing a self-recursive function in its GraphDef causes unbounded recursion during loading or execution, overflowing the stack.
- Read and Write outside of bounds in TensorFlow A crafted TFLite model with sparse tensor data is converted without validation, causing out-of-bounds reads and writes when the model is loaded.
- `CHECK`-failures in Tensorflow A tampered SavedModel with duplicated operation attribute definitions triggers an assertion failure on load, crashing the process.
- Integer overflow in TFLite A crafted TFLite model overflows size multiplications in embedding lookup, leading to out-of-bounds heap reads and writes.
- Integer overflow in TFLite array creation A crafted TFLite model makes an int-sized size calculation overflow during array creation, causing an undersized heap allocation.
- Null pointer dereference in Grappler's `IsConstant` A malformed SavedModel causes Grappler's constant-folding and identity checks to dereference missing node pointers.
- Out of bounds read in Tensorflow A modified SavedModel on disk breaks assumptions in TFG import, causing heap out-of-bounds reads when converted to MLIR.
- `CHECK`-fails due to attempting to build a reference tensor A modified SavedModel makes Grappler constant folding build a tensor with a reference dtype, which trips a CHECK failure and crashes the process.
- Division by zero in TFLite A crafted TFLite model supplies a bias size of zero, so the bias kernel divides by zero and the inference process crashes.
- Crash due to erroneous `StatusOr` in TensorFlow A SavedModel's graph definition can be altered so that forcibly extracting an error status during node creation crashes the TensorFlow process.