PyPI · nltk
NLTK: Allowlisted pickle loaders still permit code execution in current source
The current source tree still allows arbitrary code execution during supposedly safer allowlisted pickle loading. The allowlist trusts whole module namespaces instead of exact safe globals, so crafted pickles can invoke dangerous in-namespace callables through pickle REDUCE.
nltk.picklesec.allowlisted_pickle_load, nltk.tokenize.punkt.punkt_pickle_load, nltk.parse.transitionparser.TransitionParser.parsev3.10.0-rc2; published 3.9.4 was not the claim target for this bypass.nltk.tokenize.repp.ReppTokenizer._execute and numpy.f2py.crackfortran.myeval.punkt_pickle_load() allowlists both nltk.tokenize.punkt and the whole nltk.tokenize namespace, which exposes ReppTokenizer._execute() and its subprocess.Popen(...) sink during unpickling. TransitionParser.parse() uses allowlisted_pickle_load(..., allowed_modules=("numpy", "scipy", "sklearn")), which permits numpy.f2py.crackfortran.myeval() and its attacker-controlled eval(...) path. I confirmed both gadgets create marker files before the caller returns or later aborts on type misuse.
Preconditions
Steps
ReppTokenizer._execute and point its command to a harmless marker-file write.punkt_pickle_load(BytesIO(payload)) and observe the marker file is created during unpickling.numpy.f2py.crackfortran.myeval and load it through TransitionParser.parse().TransitionParser.parse() later fails on the returned object type.Minimal reproducible excerpt
{'punkt_marker': 'PUNKT_RCE', 'transitionparser_marker': 'TP_RCE'}
Any caller that trusts these current allowlisted loaders can still execute attacker-controlled commands while loading model or tokenizer artifacts. This defeats the protection mechanism that replaced unrestricted pickle loading and creates a dangerous false sense of safety.
Replace broad module-prefix allowlists with exact (module, qualname) pairs for the few safe classes or functions genuinely required. Do not allow entire namespaces such as nltk.tokenize or numpy, and keep post-load type validation only as a secondary defense.
find_class now, before the allowlists:name → closes 4489 with zero legit impact.os, subprocess, sys, builtins, numpy.f2py, nltk.tokenize.repp, …) even under a broad allowed_modules — a defense-in-depth backstop so a future too-broad allowlist can't silently reopen RCE.builtins denied wholesale; safe primitives (int, str, …) must be named exactly via allowed_globals.Callers tightened: punkt drops the broad nltk.tokenize (keeps nltk.tokenize.punkt + exact collections.defaultdict/builtins.int); transitionparser keeps numpy/scipy/sklearn (array unpickling needs their submodules) with the new guards blocking the gadgets.
Every deserialization sink in the tree was reviewed: no raw pickle.load anywhere, and no joblib/numpy/torch/dill/yaml/marshal loaders. data.load + wordnet_app use RestrictedUnpickler (blocks all globals — safe); the remaining pickle_load sites (chartparser_app, tbl/demo) load user-selected or self-written files and keep their warning.
=== EXPLOITS blocked ===
4489 sklearn.os.system (dotted) -> BLOCKED
x99w numpy.f2py.crackfortran.myeval -> BLOCKED
x99w nltk.tokenize.repp._execute -> BLOCKED
backstop os.system (os allowlisted) -> BLOCKED
backstop builtins.eval (exact global)-> BLOCKED
=== LEGIT loads still work ===
punkt round-trip via punkt_pickle_load -> OK
builtins.int (safe primitive) -> OK
test_pickle_allowlist_security.py — added 5 regressions (dotted traversal, both namespace gadgets, denied-module backstop, legit round-trip). Suite: 122 passed / 9 skipped (sklearn-dependent) across pickle/punkt/transition/tokenize. pre-commit (black/isort/ruff) clean.
Is your project exposed to this? Stateward checks every dependency on every pull request and flags it only if your code actually reaches it.
Check my repoSources: CISA KEV (public domain), OSV.dev & GitHub Advisory Database (CC-BY-4.0), FIRST EPSS, NVD/CWE (public domain). Served live from the Stateward advisory database.