Linux NVMe rig

Good day.

I wanted to draw the attention of the community to a characteristic feature of Linux when working with multiple NVMe SSDs in one system. It will be especially important for those who like to make software RAID arrays from NVMe.

I hope that the information below will help protect your data and save you from annoying errors.

We are all used to the following Linux logic when working with block devices:
If the device is called / dev / sda then the partitions on it will be / dev / sda1, / dev / sda2, etc.
To view SMART attributes, we use something like smartctl -a / dev / sda, but we format it and add partitions to arrays, such as / dev / sda1.

We are all used to the axiom that / dev / sda1 is located on / dev / sda. And, if in one day SMART shows that / dev / sda is almost dead, it is / dev / sda1 that we will throw out from the RAID array for replacement.

It turns out that when working with NVMe Namespaces this rule does not work. Proof:

nvme list && ( smartctl -a /dev/nvme0 && smartctl -a /dev/nvme1  && smartctl -a /dev/nvme2 ) | grep Serial
Node             SN                   Model                                    Namespace Usage                      Format           FW Rev  
---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- --------
/dev/nvme0n1     S466NX0K72XX06M      Samsung SSD 970 EVO 500GB                1          96.92  GB / 500.11  GB    512   B +  0 B   1B2QEXE7
/dev/nvme1n1     S466NX0K43XX48W      Samsung SSD 970 EVO 500GB                1          91.00  GB / 500.11  GB    512   B +  0 B   1B2QEXE7
/dev/nvme2n1     S466NX0K72XX01A      Samsung SSD 970 EVO 500GB                1           0.00   B / 500.11  GB    512   B +  0 B   1B2QEXE7
Serial Number:                      S466NX0K72XX06M
Serial Number:                      S466NX0K72XX01A
Serial Number:                      S466NX0K43XX48W

An attentive reader when comparing serial numbers will notice that / dev / nvme1n1 is actually located on / dev / nvme2, and vice versa.

P.S.

I wish you never to remove the last live NVMe SSD from the RAID array.

All Articles