Simple Recovery of LUKS-encrypted Partitions

LUKS encryption, while convenient, can become a major pain when the drive gets corrupted. Many things can lead to it:

accidental wipe/format of the boot partition
accidental dd to a wrong drive (sure, you pressed ^C right away but your grub is gone, sorry)
exceedingly “smart” utilities overwriting MBR or re-initializing encrypted partitions
drive/memory failures (power outage, random bit flip, etc.)

There are plenty of manuals on how to restore an accidentally wiped boot partition. However, more complicated cases (e.g. when vgscan can’t find any LVM partitions) are not described as well, or sometimes wrongfully described as unrecoverable. The data is lost only when the LUKS header or encrypted bytes themselves are corrupted or overwritten, and encrypted data is usually recoverable even when the partitioning information is lost.

Mandatory disclaimer: perform your data recovery on a cloned copy of the drive. If you don’t know what you are doing – contact data recovery professionals. I am not responsible for any loss of data or loss of revenue or profit or any other damages, however caused.

Test environment

Debian 10.3 is installed with a standard full disk encryption as a test environment:

debian luks installation

Here’s how partitions look after installation:

user@debian:~$ sudo fdisk -l
Disk /dev/sda: 8 GiB, 8589934592 bytes, 16777216 sectors
Disk model: VBOX HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xf7251e65

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sda1  *      2048   499711   497664  243M 83 Linux
/dev/sda2       501758 16775167 16273410  7.8G  5 Extended
/dev/sda5       501760 16775167 16273408  7.8G 83 Linux


Disk /dev/mapper/sda5_crypt: 7.8 GiB, 8315207680 bytes, 16240640 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/debian--vg-root: 6.7 GiB, 7226785792 bytes, 14114816 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/debian--vg-swap_1: 1020 MiB, 1069547520 bytes, 2088960 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

As a test file for recovery, 1GB file with random data is created in the home directory:

user@debian:~$ head -c 1G </dev/urandom >~/test_file
user@debian:~$ md5sum ~/test_file
20b731225ecb7260c1155da750f810d3  /home/user/test_file

In order to test the worst (but still recoverable) failure, LVM partitioning metadata is erased. LUKS reserves 2 MiB for its headers and that area is preserved. The size of LVM metadata area is calculated using the output of fdisk from above:

user@debian:~$ sudo dd if=/dev/zero of=/dev/sda5 bs=512 seek=4096 count=$(((8315207680-7226785792-1069547520)/512-4096))
32768+0 records in
32768+0 records out
16777216 bytes (17 MB, 16 MiB) copied, 0.796235 s, 21.1 MB/s

Finally, MBR is wiped:

user@debian:~$ sudo dd if=/dev/zero of=/dev/sda bs=512 count=1
1+0 records in
1+0 records out
512 bytes copied, 0.00791088 s, 64.7 kB/s

Note that /dev/sda has no partition information left:

user@debian:~$ sudo fdisk -l
Disk /dev/sda: 8 GiB, 8589934592 bytes, 16777216 sectors
Disk model: VBOX HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/sda5_crypt: 7.8 GiB, 8315207680 bytes, 16240640 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/debian--vg-root: 6.7 GiB, 7226785792 bytes, 14114816 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/debian--vg-swap_1: 1020 MiB, 1069547520 bytes, 2088960 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

After reboot, as expected, the system stops with an error:

FATAL: No bootable media found! System halted.

Finding LUKS partitions

The recovery is performed using gparted live-cd environment (iso-images are available from https://gparted.org).

First, it is verified that there are indeed no partitions left on the drive (an extra 2GB disk represents a USB thumb drive in the output below):

user@debian:~$ sudo fdisk -l
Disk /dev/sda: 8 GiB, 8589934592 bytes, 16777216 sectors
Disk model: VBOX HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 2 GiB, 2147483648 bytes, 4194304 sectors
Disk model: HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x54c0d141

Device     Boot Start     End Sectors Size Id Type
/dev/sdb1        2048 4194303 4192256   2G  7 HPFS/NTFS/exFAT


Disk /dev/loop0: 303.5 MiB, 317767680 bytes, 620640 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

But the drive still contains the data. In order to locate partitions, we can use the fact that the first four bytes of any LUKS partition are always 0x4c 0x55 0x4b 0x53 or LUKS in ascii. It might take awhile to scan the entire drive, so as soon as the offset of LUKS partition is found, hexdump can be killed via ^C.

user@debian:~$ sudo hexdump -C /dev/sda | grep LUKS
0f500000  4c 55 4b 53 ba be 00 02  00 00 00 00 00 00 40 00  |LUKS..........@.|
^C

Next, a loop device can be set up using the found offset. Note the -r option – it is safer to set up the device as read-only.

user@debian:~$ sudo losetup -o 0xf500000 -r -f /dev/sda
user@debian:~$ sudo losetup -a
/dev/loop1: [0006]:10205 (/dev/sda), offset 256901120
/dev/loop0: [2816]:4883 (/run/live/medium/live/filesystem.squashfs)

Loop device /dev/loop1 now contains bytes from the drive /dev/sda, starts at the offset 0xf500000 and spans to the rest of the drive. The beginning of LUKS partition is caught correctly, but not necessarily the end of it. In practice, this is not an issue.

Decrypting LUKS partitions

user@debian:~$ sudo cryptsetup luksOpen /dev/loop1 recovery
Enter passphrase for /dev/sda:

At this stage, the passphrase is prompted and, if entered correctly, decrypted LVM partition is mapped to /dev/mapper/recovery:

user@debian:~$ sudo fdisk -l
Disk /dev/sda: 8 GiB, 8589934592 bytes, 16777216 sectors
Disk model: VBOX HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 2 GiB, 2147483648 bytes, 4194304 sectors
Disk model: HARDDISK
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x54c0d141

Device     Boot Start     End Sectors Size Id Type
/dev/sdb1        2048 4194303 4192256   2G  7 HPFS/NTFS/exFAT


Disk /dev/loop0: 303.5 MiB, 317767680 bytes, 620640 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop1: 7.78 GiB, 8333033472 bytes, 16275456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/recovery: 7.76 GiB, 8316256256 bytes, 16242688 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

It is unfortunately not mountable directly (even if LVM partition metadata wasn’t damaged), since it contains multiple LVM logical volumes and therefore cannot be treated as a single partition.

LVM partition metadata was cleared so LVM tools cannot identify any volumes or groups:

user@debian:~$ sudo pvscan
  No matching physical volumes found
user@debian:~$ sudo vgscan
user@debian:~$ sudo lvscan

However, this volume can be scanned directly and the files can be recovered using testdisk in few easy steps:

mount external drive and launch testdisk:

user@debian:~$ sudo mkdir /media/external
user@debian:~$ sudo mount -t ntfs /dev/sdb1 /media/external
user@debian:~$ sudo testdisk

recover the test file:

testdisk recovery

and verify its integrity:

user@debian:~$ md5sum /media/external/home/user/test_file
20b731225ecb7260c1155da750f810d3  /media/external/home/user/test_file

This approach is of course overkill in case of minor failures. E.g. if GRUB doesn’t boot due to damaged /boot partition, it could be restored without loosing the base system using grub-install. However, if the partitioning metadata is lost, testdisk can help to quickly recover the files.

P.S. Set up you backups already!