Get best of the week

Zone data storage


Western Digital recently announced its zonal storage initiative, an initiative aimed at an open data infrastructure designed to take advantage of zonal storage devices.

Zoned Storage is a new storage paradigm motivated by incredible data growth. Our society is increasingly dependent on data in everyday life, and data management on an extreme scale is becoming increasingly necessary. Already today, in large-scale data processing infrastructures, tens of thousands of hard drives and solid state drives are used. But even so, out of more than 30 ZB data that is created per year, less than 20% remains in storage. The goal of our industry is to ensure that the consumer can afford not to throw out data that can be beneficial.

In this article we will discuss:

  • What are zonal storage devices and why is this technology being implemented
  • SMR Hard Drives and ZNS Solid State Drives
  • Data Infrastructure - What is Required to Implement Zonal Data Storage

What are zone storage devices?


In its simplest form, zone storage devices are block storage devices whose address space is divided into zones. ZSD impose unconventional recording rules: zones can be recorded sequentially, only starting from the beginning of the zone. In addition, data within a zone cannot be arbitrarily overwritten.


The only way to overwrite an already recorded zone is to reset the zone record pointer, which effectively deletes all the data in the zone, and overwrite it from the beginning of the zone. On the other hand, reading data is largely unlimited, and can be read in the same way as on traditional storage devices.

The concept of zonal storage devices is standardized:

  • For devices with SAS interface, work with zones is prescribed in the T10 ZBC standard
  • SATA also standardized under T13 ZAC
  • For NVMe SSDs, a technical proposal is under development called ZNS

Why zonal storage?


The motivation for applying zonal storage technology is the high efficiency of the storage infrastructure. Thanks to this technology, higher capacity devices can be used in a more economical way. More details on how this economic efficiency is achieved for each type of zonal devices will be explained later in the article.

It all started with SMR


Over the past few years, the technology of tiled magnetic recording (SMR) has been introduced in hard drives, which has increased the recording density, increase the capacity and cost-effectiveness of hard drives. In SMR, unlike conventional recording, tracks are recorded overlapping, like tiles on a house roof. This allows you to more densely pack tracks and, therefore, provides a higher recording density. In fact, the use of SMR gives us “free” terabytes of capacity: on the basis of the usual 12 TB disk, you can make 14 or even 15 TB SMR drive, without increasing the cost of production.


The main limitation of this technology is the inability to write data to tracks in random order. As we cannot replace arbitrary tiles on the roof of the house, so with the data in the SMR - we cannot arbitrarily overwrite several blocks without hooking adjacent tracks. To control recording, the disc surface is divided into zones with a gap between them. This allows you to record and erase each zone independently of each other (the size of the zone in the current generation of disks is usually 256 MB).

Several approaches can be used to control recording restriction. The traditional approach (drive-managed - controlled by the drive itself) assumes that the device processes the internal write restrictions on its own using algorithms in microcode and buffers, and provides a normal interface to the host. This type of SMR is commonly used in client devices such as laptop HDDs. However, for large-scale corporate or cloud systems, where performance and space utilization should be predictable, relying on the disc itself to control sequential recording is not possible - performance “floats” too much depending on the structure and intensity of the load and buffer fullness.

Host-managed SMR is an almost mandatory requirement for large data storage systems and is the preferred option for data centers.

Host SMR control is standardized in INCITS T10 / T13 as part of the Zone Block Command (ZBC) and Zone ATA (ZAC) device standards for SAS and SATA, respectively.

NVMe and Zone Namespaces in Solid State Drives


For solid state drives (SSDs), the limitation on the presence of regions that can only be written sequentially (NAND pages) and must be deleted before new data is written (NAND blocks) is an integral property of NAND flash memory operation.

When SSDs were first introduced, they introduced an internal management system, called the Flash Translation Layer (FTL), which addressed this recording limitation. FTL allowed the use of solid state drives instead of hard drives without the need to immediately rewrite the software stack. For the end user, the SSD looks like a hard drive with fast random access to information - and this is the merit of FTL.

However, local control within an SSD is far from ideal in terms of efficiency. In particular, there are factors such as:

  • Write Amplification: To cope with the limitations of not overwriting data, the SSD must move the data on its own to clean unused storage areas called Garbage Collection (GC). The garbage collection process, when the FTL "plays tag" with data blocks, causes multiple recordings of the same information (hence the term "excess recording"), which leads to increased wear on the flash memory and shortened the life of the solid state drive. The WA coefficient shows how many write operations must be done inside the drive for one write operation received from the host.



  • (overprovisioning — OP): ( 28% ) .
  • DRAM : FTL - DRAM. , DRAM . — 1 SSD. , SSD 15.36 — 16 «» . , !
  • QoS, . FTL , , , .

These shortcomings were acceptable in the first wave of the introduction of solid state drives, since the software stacks and interfaces were designed for hard drives, and the response time to the HDD was much longer. For the end user, switching from HDD to SSD was still noticeable acceleration.

Over time, however, the industry moved on to creating more efficient interfaces and software stacks that can take advantage of lower latency and higher flash bandwidth. In particular, the NVMe (Non-Volatile Memory Express) interface specification was created and the corresponding low-cost software stack developed.

The current state of the industry is such that the local optimizations that occurred in the FTL SSD are now detrimental to the deployment of the data infrastructure. In particular, many customers have the desire to create a host-driven sequential workload that matches the size of the NAND flash blocks. In response, NVMe standardizes zone namespaces (ZNSs), which allows the host to direct I / O to share workloads, while also improving latency, bandwidth, and cost-effectiveness by moving the main control to the host.

If we talk about economic efficiency, the transition to ZNS SSD will reduce the cost of the drive by using a smaller amount of NAND (reduction of redundancy), the possibility of using QLC in server drives, reducing the amount of DRAM in SSD controllers.

Data Infrastructure - Implementing a Zonal Data Warehouse


We have already found out that zonal block devices increase storage efficiency, however, to take advantage of these new devices, some efforts are required, since they do not have backward compatibility with the usual block program stack. While traditional storage devices do not have recording restrictions, zone block devices have recording restrictions that must be consistent within the zone. The main consequence of this is the need to update the software stack.

The first component that needs to be updated is the operating system. This is not a trivial task in modern multi-tasking operating systems running on multi-core and multi-socket servers typical of data centers. The Linux community has made significant efforts to support zone block devices in general, and SMR in particular.

In addition to supporting the Linux kernel, there are a number of utilities and applications that support Zone Block Devices, such as fio, Blktests, and util-linux.

Linux Kernel and Zone Block Devices


Work on support for zone storage in the Linux kernel began back in 2014 with the smallest amount of support built into the 3.18 kernel. The first kernel release with functional support for ZBC / ZAC commands was the 4.10 kernel in early 2017. Support continues to improve, and the latest kernels have support for displaying zone devices in the form of block / dev / sd, as well as support for some file systems (f2fs, btrfs).

The figure below shows a high-level image of the structure of the Linux kernel and how zone devices can be integrated with traditional block devices. Support in the Linux kernel was implemented by modifying some existing components, introducing new interfaces, such as the ZBD interface at the block level, and introducing new components, such as the device-mapper dm-zoned.


As shown in the figure, there are many ways in which the deployment of a data infrastructure can use zone block devices. For example (i) Using the old file system on top of the dm-zoned LVM volume, (ii) using the ZBC-enabled file system, (iii) using ZBD-enabled applications that interact directly with block devices through user libraries such as libzbc, and etc.

Area Storage Initiative - Promoting broader support and implementation of applied technologies


Despite the successes made by the open source community of developers in supporting Zone Block Devices, information technology engineers and application developers are still making efforts to use this technology, for example, finding out what level of support exists in each release of the Linux kernel, which versions of applications have the necessary support and which version of the kernel is necessary for their work.

The zonal storage initiative was launched in part to address this problem and to promote wider application support for zonal storage technologies. You can visit the site ZonedStorage.ioto learn more about zonal storage technologies and the use of ZNS SSDs and ZBC / ZAC SMRs. There you will find information on the following issues:

  • Zone block devices
  • Getting Started Guides
  • Linux kernel support and features
  • Applications and Libraries
  • Zone device emulation on a regular
  • Performance testing
  • System Compliance Tests

Using this platform, infrastructure engineers and storage application developers now have a centralized location to get all the information and resources needed to use area storage technologies.

Are you ready


Visit ZonedStorage.io and see if your servers are ready to work with zone drives. Try to deploy software emulation over a regular device, and it may turn out that the load structure of your application fits well with the concept of zonal storage, and you should consider deploying a test environment.

More articles:


All Articles