mmap— Memory-mapped file support

---

Availability:not WASI.

This module does not work or is not available on WebAssembly. See WebAssembly platformsfor more information.

Memory-mapped file objects behave like bothbytearrayand like file objects.You can use mmap objects in most places wherebytearrayare expected; for example, you can use there module to search through a memory-mapped file. You can also change a single byte by doingobj[index]=97,or change a subsequence by assigning to a slice:obj[i1:i2]=b'...'.You can also read and write data starting at the current file position, andseek()through the file to different positions.

A memory-mapped file is created by themmapconstructor, which is different on Unix and on Windows. In either case you must provide a file descriptor for a file opened for update. If you wish to map an existing Python file object, use itsfileno()method to obtain the correct value for the filenoparameter. Otherwise, you can open the file using the os.open()function, which returns a file descriptor directly (the file still needs to be closed when done).

Note

If you want to create a memory-mapping for a writable, buffered file, you shouldflush()the file first. This is necessary to ensure that local modifications to the buffers are actually available to the mapping.

For both the Unix and Windows versions of the constructor,accessmay be specified as an optional keyword parameter.accessaccepts one of four values:ACCESS_READ,ACCESS_WRITE,orACCESS_COPYto specify read-only, write-through or copy-on-write memory respectively, or ACCESS_DEFAULTto defer toprot.accesscan be used on both Unix and Windows. Ifaccessis not specified, Windows mmap returns a write-through mapping. The initial memory values for all three access types are taken from the specified file. Assignment to anACCESS_READ memory map raises aTypeErrorexception. Assignment to an ACCESS_WRITEmemory map affects both memory and the underlying file. Assignment to anACCESS_COPYmemory map affects memory but does not update the underlying file.

Changed in version 3.7:AddedACCESS_DEFAULTconstant.

To map anonymous memory, -1 should be passed as the fileno along with the length.

classmmap.mmap(fileno,length,tagname=None,access=ACCESS_DEFAULT,offset=0)

(Windows version)Mapslengthbytes from the file specified by the file handlefileno,and creates a mmap object. Iflengthis larger than the current size of the file, the file is extended to containlength bytes. Iflengthis0,the maximum length of the map is the current size of the file, except that if the file is empty Windows raises an exception (you cannot create an empty mapping on Windows).

tagname,if specified and notNone,is a string giving a tag name for the mapping. Windows allows you to have many different mappings against the same file. If you specify the name of an existing tag, that tag is opened, otherwise a new tag of this name is created. If this parameter is omitted orNone,the mapping is created without a name. Avoiding the use of thetagnameparameter will assist in keeping your code portable between Unix and Windows.

offsetmay be specified as a non-negative integer offset. mmap references will be relative to the offset from the beginning of the file.offset defaults to 0.offsetmust be a multiple of theALLOCATIONGRANULARITY.

Raises anauditing eventmmap.__new__with argumentsfileno,length,access,offset.

classmmap.mmap(fileno,length,flags=MAP_SHARED,prot=PROT_WRITE|PROT_READ,access=ACCESS_DEFAULT,offset=0,*,trackfd=True)

(Unix version)Mapslengthbytes from the file specified by the file descriptorfileno,and returns a mmap object. Iflengthis0,the maximum length of the map will be the current size of the file when mmapis called.

flagsspecifies the nature of the mapping.MAP_PRIVATEcreates a private copy-on-write mapping, so changes to the contents of the mmap object will be private to this process, andMAP_SHAREDcreates a mapping that’s shared with all other processes mapping the same areas of the file. The default value isMAP_SHARED.Some systems have additional possible flags with the full list specified in MAP_* constants.

prot,if specified, gives the desired memory protection; the two most useful values arePROT_READandPROT_WRITE,to specify that the pages may be read or written.protdefaults to PROT_READ|PROT_WRITE.

accessmay be specified in lieu offlagsandprotas an optional keyword parameter. It is an error to specify bothflags,protand access.See the description ofaccessabove for information on how to use this parameter.

offsetmay be specified as a non-negative integer offset. mmap references will be relative to the offset from the beginning of the file.offset defaults to 0.offsetmust be a multiple ofALLOCATIONGRANULARITY which is equal toPAGESIZEon Unix systems.

IftrackfdisFalse,the file descriptor specified byfilenowill not be duplicated, and the resultingmmapobject will not be associated with the map’s underlying file. This means that thesize()andresize() methods will fail. This mode is useful to limit the number of open file descriptors.

To ensure validity of the created memory mapping the file specified by the descriptorfilenois internally automatically synchronized with the physical backing store on macOS.

Changed in version 3.13:Thetrackfdparameter was added.

This example shows a simple way of usingmmap:

importmmap

# write a simple example file
withopen("hello.txt","wb")asf:
f.write(b"Hello Python!\n")

withopen("hello.txt","r+b")asf:
# memory-map the file, size 0 means whole file
mm=mmap.mmap(f.fileno(),0)
# read content via standard file methods
print(mm.readline())# prints b "Hello Python!\n"
# read content via slice notation
print(mm[:5])# prints b "Hello"
# update content using slice notation;
# note that new content must have same size
mm[6:]=b"world!\n"
#... and read again using standard file methods
mm.seek(0)
print(mm.readline())# prints b "Hello world!\n"
# close the map
mm.close()

mmapcan also be used as a context manager in awith statement:

importmmap

withmmap.mmap(-1,13)asmm:
mm.write(b"Hello world!")

Added in version 3.2:Context manager support.

The next example demonstrates how to create an anonymous map and exchange data between the parent and child processes:

importmmap
importos

mm=mmap.mmap(-1,13)
mm.write(b"Hello world!")

pid=os.fork()

ifpid==0:# In a child process
mm.seek(0)
print(mm.readline())

mm.close()

Raises anauditing eventmmap.__new__with argumentsfileno,length,access,offset.

Memory-mapped file objects support the following methods:

close()

Closes the mmap. Subsequent calls to other methods of the object will result in a ValueError exception being raised. This will not close the open file.

closed

Trueif the file is closed.

Added in version 3.2.

find(sub[,start[,end]])

Returns the lowest index in the object where the subsequencesubis found, such thatsubis contained in the range [start,end]. Optional argumentsstartandendare interpreted as in slice notation. Returns-1on failure.

Changed in version 3.5:Writablebytes-like objectis now accepted.

flush([offset[,size]])

Flushes changes made to the in-memory copy of a file back to disk. Without use of this call there is no guarantee that changes are written back before the object is destroyed. Ifoffsetandsizeare specified, only changes to the given range of bytes will be flushed to disk; otherwise, the whole extent of the mapping is flushed.offsetmust be a multiple of the PAGESIZEorALLOCATIONGRANULARITY.

Noneis returned to indicate success. An exception is raised when the call failed.

Changed in version 3.8:Previously, a nonzero value was returned on success; zero was returned on error under Windows. A zero value was returned on success; an exception was raised on error under Unix.

madvise(option[,start[,length]])

Send adviceoptionto the kernel about the memory region beginning at startand extendinglengthbytes.optionmust be one of the MADV_* constantsavailable on the system. If startandlengthare omitted, the entire mapping is spanned. On some systems (including Linux),startmust be a multiple of the PAGESIZE.

Availability: Systems with themadvise()system call.

Added in version 3.8.

move(dest,src,count)

Copy thecountbytes starting at offsetsrcto the destination index dest.If the mmap was created withACCESS_READ,then calls to move will raise aTypeErrorexception.

read([n])

Return abytescontaining up tonbytes starting from the current file position. If the argument is omitted,Noneor negative, return all bytes from the current file position to the end of the mapping. The file position is updated to point after the bytes that were returned.

Changed in version 3.3:Argument can be omitted orNone.

read_byte()

Returns a byte at the current file position as an integer, and advances the file position by 1.

readline()

Returns a single line, starting at the current file position and up to the next newline. The file position is updated to point after the bytes that were returned.

resize(newsize)

Resizes the map and the underlying file, if any.

Resizing a map created withaccessofACCESS_READor ACCESS_COPY,will raise aTypeErrorexception. Resizing a map created with withtrackfdset toFalse, will raise aValueErrorexception.

On Windows:Resizing the map will raise anOSErrorif there are other maps against the same named file. Resizing an anonymous map (ie against the pagefile) will silently create a new map with the original data copied over up to the length of the new size.

Changed in version 3.11:Correctly fails if attempting to resize when another map is held Allows resize against an anonymous map on Windows

rfind(sub[,start[,end]])

Returns the highest index in the object where the subsequencesubis found, such thatsubis contained in the range [start,end]. Optional argumentsstartandendare interpreted as in slice notation. Returns-1on failure.

Changed in version 3.5:Writablebytes-like objectis now accepted.

seek(pos[,whence])

Set the file’s current position.whenceargument is optional and defaults toos.SEEK_SETor0(absolute file positioning); other values areos.SEEK_CURor1(seek relative to the current position) andos.SEEK_ENDor2(seek relative to the file’s end).

Changed in version 3.13:Return the new absolute position instead ofNone.

seekable()

Return whether the file supports seeking, and the return value is alwaysTrue.

Added in version 3.13.

size()

Return the length of the file, which can be larger than the size of the memory-mapped area.

tell()

Returns the current position of the file pointer.

write(bytes)

Write the bytes inbytesinto memory at the current position of the file pointer and return the number of bytes written (never less than len(bytes),since if the write fails, aValueErrorwill be raised). The file position is updated to point after the bytes that were written. If the mmap was created withACCESS_READ,then writing to it will raise aTypeErrorexception.

Changed in version 3.5:Writablebytes-like objectis now accepted.

Changed in version 3.6:The number of bytes written is now returned.

write_byte(byte)

Write the integerbyteinto memory at the current position of the file pointer; the file position is advanced by1.If the mmap was created withACCESS_READ,then writing to it will raise aTypeErrorexception.

MADV_* Constants

mmap.MADV_NORMAL

mmap.MADV_RANDOM

mmap.MADV_SEQUENTIAL

mmap.MADV_WILLNEED

mmap.MADV_DONTNEED

mmap.MADV_REMOVE

mmap.MADV_DONTFORK

mmap.MADV_DOFORK

mmap.MADV_HWPOISON

mmap.MADV_MERGEABLE

mmap.MADV_UNMERGEABLE

mmap.MADV_SOFT_OFFLINE

mmap.MADV_HUGEPAGE

mmap.MADV_NOHUGEPAGE

mmap.MADV_DONTDUMP

mmap.MADV_DODUMP

mmap.MADV_FREE

mmap.MADV_NOSYNC

mmap.MADV_AUTOSYNC

mmap.MADV_NOCORE

mmap.MADV_CORE

mmap.MADV_PROTECT

mmap.MADV_FREE_REUSABLE

mmap.MADV_FREE_REUSE

These options can be passed tommap.madvise().Not every option will be present on every system.

Availability: Systems with the madvise() system call.

Added in version 3.8.

MAP_* Constants

mmap.MAP_SHARED

mmap.MAP_PRIVATE

mmap.MAP_32BIT

mmap.MAP_ALIGNED_SUPER

mmap.MAP_ANON

mmap.MAP_ANONYMOUS

mmap.MAP_CONCEAL

mmap.MAP_DENYWRITE

mmap.MAP_EXECUTABLE

mmap.MAP_HASSEMAPHORE

mmap.MAP_JIT

mmap.MAP_NOCACHE

mmap.MAP_NOEXTEND

mmap.MAP_NORESERVE

mmap.MAP_POPULATE

mmap.MAP_RESILIENT_CODESIGN

mmap.MAP_RESILIENT_MEDIA

mmap.MAP_STACK

mmap.MAP_TPRO

mmap.MAP_TRANSLATED_ALLOW_EXECUTE

mmap.MAP_UNIX03

These are the various flags that can be passed tommap.mmap().MAP_ALIGNED_SUPER is only available at FreeBSD andMAP_CONCEALis only available at OpenBSD. Note that some options might not be present on some systems.

Changed in version 3.10:AddedMAP_POPULATEconstant.

Added in version 3.11:AddedMAP_STACKconstant.

Added in version 3.12:AddedMAP_ALIGNED_SUPERandMAP_CONCEALconstants.

Added in version 3.13:AddedMAP_32BIT,MAP_HASSEMAPHORE,MAP_JIT, MAP_NOCACHE,MAP_NOEXTEND,MAP_NORESERVE, MAP_RESILIENT_CODESIGN,MAP_RESILIENT_MEDIA, MAP_TPRO,MAP_TRANSLATED_ALLOW_EXECUTE,and MAP_UNIX03constants.