dm-cache

dm-cache

Developer(s)	Joe Thornber, Heinz Mauelshagen, Mike Snitzer and others
Initial release	April 28, 2013(2013-04-28)(Linux 3.9)
Written in	C
Operating system	Linux
Type	Linux kernel feature
License	GNU GPL
Website	kernel.org

dm-cacheis a component (more specifically, a target) of theLinux kernel'sdevice mapper,which is aframeworkfor mappingblock devicesonto higher-level virtual block devices. It allows one or more fast storage devices, such as flash-basedsolid-state drives(SSDs), to act as acachefor one or more slower storage devices such ashard disk drives(HDDs); this effectively createshybrid volumesand providessecondary storageperformance improvements.

The design of dm-cache requires three physical storage devices for the creation of a single hybrid volume; dm-cache uses those storage devices to separately store actual data, cache data, and requiredmetadata.Configurable operating modes and cache policies, with the latter in the form of separate modules, determine the way data caching is actually performed.

dm-cache is licensed under the terms ofGNU General Public License(GPL), with Joe Thornber, Heinz Mauelshagen and Mike Snitzer as its primary developers.

Overview

dm-cache uses solid-state drives (SSDs) as an additional level of indirection while accessing hard disk drives (HDDs), improving the overall performance by using fastflash-based SSDs as caches for the slower mechanical HDDs based on rotationalmagnetic media.As a result, the costly speed of SSDs becomes combined with the storage capacity offered by slower but less expensive HDDs.Moreover, in the case ofstorage area networks(SANs) used incloudenvironments as shared storage systems forvirtual machines,dm-cache can also improve overall performance and reduce the load of SANs by providing data caching using client-side local storage.

dm-cache is implemented as a component of the Linux kernel'sdevice mapper,which is avolume managementframework that allows various mappings to be created between physical and virtual block devices. The way a mapping between devices is created determines how the virtualblocksare translated into underlying physical blocks, with the specific translation types referred to astargets.Acting as a mapping target, dm-cache makes it possible for SSD-based caching to be part of the created virtual block device, while the configurable operating modes and cache policies determine how dm-cache works internally. The operating mode selects the way in which the data is kept in sync between an HDD and an SSD, while the cache policy, selectable from separate modules that implement each of the policies, provides thealgorithmfor determining which blocks are promoted (moved from an HDD to an SSD), demoted (moved from an SSD to an HDD), cleaned, etc.

When configured to use themultiqueue(mq) orstochastic multiqueue(smq) cache policy, with the latter being the default, dm-cache uses SSDs to store the data associated with performedrandom reads and writes,capitalizing on near-zeroseek timesof SSDs and avoiding suchI/Ooperations as typical HDD performance bottlenecks. The data associated withsequential readsand writes is not cached on SSDs, avoiding undesirablecache invalidationduring such operations; performance-wise, this is beneficial because the sequential I/O operations are suitable for HDDs due to their mechanical nature. Not caching the sequential I/O also helps in extending thelifetime of SSDsused as caches.

History

Another dm-cache project with similar goals was announced by Eric Van Hensbergen and Ming Zhao in 2006, as the result of an internship work atIBM.

Later, Joe Thornber, Heinz Mauelshagen and Mike Snitzer provided their own implementation of the concept, which resulted in the inclusion of dm-cache into the Linux kernel. dm-cache was merged into theLinux kernel mainlinein kernel version 3.9, which was released on April 28, 2013.

Design

In dm-cache, creating a mapped virtual block device that acts as ahybrid volumerequires three physical storage devices:

• Origin device– provides slow primary storage (usually an HDD)
• Cache device– provides a fast cache (usually an SSD)
• Metadata device– records the placement of blocks and their dirty flags, as well as other internal data required by a cache policy, including per-block hit counts; a metadata device cannot be shared between multiple cache devices, and it is recommended to bemirrored

Internally, dm-cache references to each of the origin devices through a number of fixed-size blocks; the size of these blocks, equaling to the size of a cachingextent,is configurable only during the creation of a hybrid volume. The size of a caching extent must range between 32KBand 1GB,and it must be a multiple of 32 KB; typically, the size of a caching extent is between 256 and 1024 KB. The choice of the caching extents bigger thandisk sectorsacts a compromise between the size ofmetadataand the possibility for wasting cache space. Having too small caching extents increases the size of metadata, both on the metadata device and in kernel memory, while having too large caching extents increases the amount of wasted cache space due to caching whole extents even in the case of highhit ratesonly for some of their parts.

Operating modes supported by dm-cache arewrite-back,which is the default,write-through,andpass-through.In the write-back operating mode, writes to cached blocks go only to the cache device, while the blocks on origin device are only marked as dirty in the metadata. For the write-through operating mode, write requests are not returned as completed until the data reaches both the origin and cache devices, with no clean blocks becoming marked as dirty. In the pass-through operating mode, all reads are performed directly from the origin device, avoiding the cache, while all writes go directly to the origin device; any cache write hits also cause invalidation of the cached blocks. The pass-through mode allows a hybrid volume to be activated when the state of a cache device is not known to be consistent with the origin device.

The rate of data migration that dm-cache performs in both directions (i.e., data promotions and demotions) can bethrottleddown to a configured speed so regular I/O to the origin and cache devices can be preserved. Decommissioning a hybrid volume or shrinking a cache device requires use of thecleanerpolicy, which effectively flushes all blocks marked in metadata as dirty from the cache device to the origin device.

Cache policies

As of August 2015and version 4.2 of the Linux kernel,the following three cache policies are distributed with the Linux kernel mainline, out of which dm-cache by default uses thestochastic multiqueuepolicy:

multiqueue (mq)

Themultiqueue(mq) policy has three sets of 16queues,using the first set for entries waiting for the cache and the remaining two sets for entries already in the cache, with the latter separated so the clean and dirty entries belong to each of the two sets. The age of cache entries in the queues is based on their associated logical time. The selection of entries going into the cache (i.e., becoming promoted) is based on variable thresholds, and queue selection is based on the hit count of an entry. This policy aims to take differentcache misscosts into account, and to make automatic adjustments to different load patterns.

This policy internally trackssequential I/Ooperations so they can be routed around the cache, with different configurable thresholds for the differentiation betweenrandom I/Oand sequential I/O operations. As a result, large contiguous I/O operations are left to be performed by the origin device because such data access patterns are suitable for HDDs, and because they avoid undesirable cache invalidation.

stochastic multiqueue (smq)

Thestochastic multiqueue(smq) policy performs in a similar way as themultiqueuepolicy, but requires fewer resources to operate; in particular, it uses substantially smaller amounts ofmain memoryto track cached blocks. It also replaces the hit counting from themultiqueuepolicy with a "hotspot" queue, and decides on data promotion and demotion on aleast-recently used(LRU) basis. As a result, this policy provides better performance compared to themultiqueuepolicy, adjusts better automatically to different load patterns, and eliminates the configuration of various thresholds.

cleaner

Thecleanerpolicy writes back to the origin device all blocks that are marked as dirty in the metadata. After the completion of this operation, a hybrid volume can be decommissioned or the size of a cache device can be reduced.

Use with LVM

Logical Volume Managerincludeslvmcache,which provides a wrapper fordm-cacheintegrated with LVM.

See also

• bcache– a Linux kernel's block layer cache, developed by Kent Overstreet
• Flashcache– a disk cache component for the Linux kernel, initially developed by Facebook
• Hybrid drive– a storage device that combines flash-based and spinning magnetic media storage technologies
• ReadyBoost– a disk caching software component of Windows Vista and later Microsoft operating systems
• Smart Response Technology(SRT) – a proprietary disk storage caching mechanism, developed by Intel for its chipsets
• ZFS– a cross-OS storage management system that has a similar integrated caching device support (L2ARC)

External links

• Linux Block Caching Choices in Stable Upstream Kernel(PDF),Dell,December 2013
• Performance Comparison among EnhanceIO, bcache and dm-cache,LKML,June 11, 2013
• EnhanceIO, Bcache & DM-Cache Benchmarked,Phoronix,June 11, 2013, by Michael Larabel
• SSD Caching Using dm-cache Tutorial,July 2014, by Kyle Manna
• Re: [dm-devel] [PATCH 8/8] [dm-cache] cache target,December 14, 2012 (guidelines for metadata device sizing)