Tagged pointer

Incomputer science,atagged pointeris apointer(concretely amemory address) with additional data associated with it, such as anindirection bitorreference count.This additional data is often "folded" into the pointer, meaning stored inline in the data representing the address, taking advantage of certain properties of memory addressing. The name comes from "tagged architecture"systems, which reserved bits at the hardware level to indicate the significance of each word; the additional data is called a" tag "or" tags ", though strictly speaking" tag "refers to data specifying atype,not other data; however, the usage "tagged pointer" is ubiquitous.

Folding tags into the pointer

There are various techniques for folding tags into a pointer.

Most architectures arebyte-addressable(the smallest addressable unit is a byte), but certain types of data will often bealignedto the size of the data, often awordor multiple thereof. This discrepancy leaves a few of theleast significant bitsof the pointer unused, which can be used for tags – most often as abit field(each bit a separate tag) – as long as code that uses the pointermasks outthese bits before accessing memory. E.g., on a32-bitarchitecture (for both addresses and word size), a word is 32 bits = 4 bytes, so word-aligned addresses are always a multiple of 4, hence end in 00, leaving the last 2 bits available; while on a64-bitarchitecture, a word is 64 bits = 8 bytes, so word-aligned addresses end in 000, leaving the last 3 bits available. In cases where data is aligned at a multiple of word size, further bits are available. In case ofword-addressablearchitectures, word-aligned data does not leave any bits available, as there is no discrepancy between alignment and addressing, but data aligned at a multiple of word size does.

Conversely, in some operating systems,virtual addressesare narrower than the overall architecture width, which leaves themost significant bitsavailable for tags; this can be combined with the previous technique in case of aligned addresses. This is particularly the case on 64-bit architectures, as 64 bits of address space are far above the data requirements of all but the largest applications, and thus manypractical 64-bit processorshave narrower addresses. Note that the virtual address width may be narrower than thephysical addresswidth, which in turn may be narrower than the architecture width; for tagging of pointers inuser space,the virtual address space provided by the operating system (in turn provided by thememory management unit) is the relevant width. In fact, some processors specifically forbid use of such tagged pointers at the processor level, notablyx86-64,which requires the use ofcanonical form addressesby the operating system, with most significant bits all 0s or all 1s.

Lastly, thevirtual memorysystem in most modernoperating systemsreserves a block of logical memory around address 0 as unusable. This means that, for example, a pointer to 0 is never a valid pointer and can be used as a specialnull pointervalue. Unlike the previously mentioned techniques, this only allows a single special pointer value, not extra data for pointers generally.

Examples

One of the earliest examples of hardware support for tagged pointers in a commercial platform was theIBM System/38.IBM later added tagged pointer support to thePowerPCarchitecture to support theIBM ioperating system, which is an evolution of the System/38 platform.

A significant example of the use of tagged pointers is theObjective-Cruntime oniOS 7onARM64,notably used on theiPhone 5S.In iOS 7, virtual addresses only contain 33 bits of address information but are 64-bits long leaving 31 bits for tags. Objective-C class pointers are 8-byte aligned freeing up an additional 3 bits of address space, and the tag fields are used for many purposes, such as storing a reference count and whether the object has adestructor.

Early versions of macOS used tagged addresses called Handles to store references to data objects. The high bits of the address indicated whether the data object was locked, purgeable, and/or originated from a resource file, respectively. This caused compatibility problems, when macOS addressing advanced from 24 bits to 32 bits in System 7.

Null versus aligned pointer

Use of zero to represent a null pointer is extremely common, with many programming languages (such asAda) explicitly relying on this behavior. In theory, other values in an operating system-reserved block of logical memory could be used to tag conditions other than a null pointer, but these uses appear to be rare, perhaps because they are at bestnon-portable.It is generally accepted practice in software design that if a special pointer value distinct from null (such as asentinelin certaindata structures) is needed, the programmer should explicitly provide for it.

Taking advantage of the alignment of pointers provides more flexibility than null pointers/sentinels because it allows pointers to be tagged with information about the type of data pointed to, conditions under which it may be accessed, or other similar information about the pointer's use. This information can be provided along with every valid pointer. In contrast, null pointers/sentinels provide only a finite number of tagged values distinct from valid pointers.

In atagged architecture,a number of bits in every word of memory are reserved to act as a tag. Tagged architectures, such as theLisp machines,often have hardware support for interpreting and processing tagged pointers.

GNU libcmalloc()provides 8-byte aligned memory addresses for 32-bit platforms, and 16-byte alignment for 64-bit platforms.Larger alignment values can be obtained withposix_memalign().

Examples

Example 1

In the following C code, the value of zero is used to indicate a null pointer:

voidoptionally_return_a_value(int*optional_return_value_pointer){
/*... */
intvalue_to_return=1;

/* is it non-NULL? (note that NULL, logical false, and zero compare equally in C) */
if(optional_return_value_pointer)
/* if so, use it to pass a value to the calling function */
*optional_return_value_pointer=value_to_return;

/* otherwise, the pointer is never dereferenced */
}

Example 2

Here, the programmer has provided a global variable, whose address is then used as a sentinel:

#define SENTINEL &sentinel_s

node_tsentinel_s;

voiddo_something_to_a_node(node_t*p){
if(NULL==p)
/* do something */
elseif(SENTINEL==p)
/* do something else */
else
/* treat p as a valid pointer to a node */
}

Example 3

Assume we have a data structuretable_entrythat is always aligned to a 16 byte boundary. In other words, the least significant 4 bits of a table entry's address are always 0(2⁴= 16).We could use these 4 bits to mark the table entry with extra information. For example, bit 0 might mean read only, bit 1 might mean dirty (the table entry needs to be updated), and so on.

If pointers are 16-bit values, then:

• 0x3421is a read-only pointer to thetable_entryat address0x3420
• 0xf472is a pointer to a dirtytable_entryat address0xf470

Advantages

The major advantage of tagged pointers is that they take up less space than a pointer along with a separate tag field. This can be especially important when a pointer is a return value from afunction.It can also be important in large tables of pointers.

A more subtle advantage is that by storing a tag in the same place as the pointer, it is often possible to guarantee theatomicityof an operation that updates both the pointer and its tag without externalsynchronizationmechanisms.This can be an extremely large performance gain, especially in operating systems.

Disadvantages

Tagged pointers have some of the same difficulties asxor linked lists,although to a lesser extent. For example, not alldebuggerswill be able to properly follow tagged pointers; however, this is not an issue for a debugger that is designed with tagged pointers in mind.

The use of zero to represent a null pointer does not suffer from these disadvantages: it is pervasive, most programming languages treat zero as a special null value, and it has thoroughly proven its robustness. An exception is the way that zero participates inoverload resolutionin C++, where zero is treated as an integer rather than a pointer; for this reason the special valuenullptris preferred over the integer zero. However, with tagged pointers zeros are usually not used to represent null pointers.