Unpatched versions of Python

Tools are allowed to ignore this PEP when running on Python without tarfile filters.

The feature has been backported to all versions of Python supported by Python.org.Vendoring it in third-party libraries is tricky, and we should not force all tools to do so. This shifts the responsibility to keep up with security updates from the tools to the users.

Permissions

Common tools (git,zip) don’t preserve Unix permissions (mode bits). Telling users to not rely on them insdists,and allowing tools to handle them relatively freely, seems fair.

The only exception is theexecutablepermission. We recommend, but not require, that tools preserve it. Given that scripts are generally platform-specific, it seems fitting to say that keeping them executable is tool-specific behaviour.

Note that whilegitpreserves executability,zip(and thuswheel) doesn’t do it natively. (It is possible to encode it in “external attributes”, but Python’sZipFile.extractdoes not honour that.)

Unpacking with the data filter

When extracting a source distribution, tools MUST either use tarfile.data_filter(e.g.TarFile.extractall(...,filter='data')), OR follow theUnpacking without the data filtersection below.

As an exception, on Python interpreters withouthasattr(tarfile,'data_filter') (PEP 706), tools that normally use that filter (directly on indirectly) MAY warn the user and ignore this specification. The trade-off between usability (e.g. fully trusting the archive) and security (e.g. refusing to unpack) is left up to the tool in this case.

Unpacking without the data filter

Tools that do not use thedatafilter directly (e.g. for backwards compatibility, allowing additional features, or not using Python) MUST follow this section. (At the time of this writing, thedatafilter also follows this section, but it may get out of sync in the future.)

The following files are invalid in ansdistarchive. Upon encountering such an entry, tools SHOULD notify the user, MUST NOT unpack the entry, and MAY abort with a failure:

• Files that would be placed outside the destination directory.
• Links (symbolic or hard) pointing outside the destination directory.
• Device files (including pipes).

The following are also invalid. Tools MAY treat them as above, but are NOT REQUIRED to do so:

• Files with a..component in the filename or link target.
• Links pointing to a file that is not part of the archive.

Tools MAY unpack links (symbolic or hard) as regular files, using content from the archive.

When extractingsdistarchives:

• Leading slashes in file names MUST be dropped. (This is nowadays standard behaviour fortarunpacking.)
• For eachmode(Unix permission) bit, tools MUST either:
  • use the platform’s default for a new file/directory (respectively),
  • set the bit according to the archive, or
  • use the bit fromrw-r--r--(0o644) for non-executable files or rwxr-xr-x(0o755) for executable files and directories.
• Highmodebits (setuid, setgid, sticky) MUST be cleared.
• It is RECOMMENDED to preserve the userexecutablebit.

Further hints

Tool authors are encouraged to consider howhints for further verificationintarfiledocumentation apply for their tool.