No container superblock with an XID that doesn't exceed 0xffffffffffffffff exists in the checkpoint descriptor area. #11

ldsalomone · 2020-12-22T03:03:07Z

ldsalomone
Dec 22, 2020

Hi, I'm wondering if you could provide some guidance on restoring mounting ability to an external drive that is currently formatted as apfs (I won't make that mistake again).

I successfully built your tools on an M1 MacBook Pro (incase you were wondering about compatibility) but am now running into an error I think has to do with the B-tree node since this is the output from fsck_apfs:

sudo fsck_apfs -y /dev/disk4s1

(base) luke@My-MacBook-Pro ~ % sudo fsck_apfs -y /dev/disk4s1
Password:
** Checking the container superblock.
** Checking the EFI jumpstart record.
** Checking the space manager.
** Checking the space manager free queue trees.
warning: Unable to read apfs keylocker ranges: No such process
** Checking the object map.
** Checking volume.
** Checking the APFS volume superblock.
** The volume Torch was formatted by hfs_convert (748.57.19) and last modified by apfs_kext (1677.60.23).
** Checking the object map.
error: btn: invalid key order (163) oid 4209151 / oxid 0 / level 1 / flags 0x5
	previous key: 0d 3a 00 00 00 00 00 00 70 8b 00 00 00 00 00 00
	current key:  1d 04 00 00 00 00 00 00 2f 07 00 00 00 00 00 00
	next key:     ef 3a 00 00 00 00 00 00 41 8a 00 00 00 00 00 00
   Object map is invalid.
** The volume /dev/disk4s1 could not be verified completely.

Here is the output from sudo ./apfs-inspect /dev/disk4s1

Opening file at `/dev/disk4s1` in read-only mode ... OK.
Simulating a mount of the APFS container.
Validating checksum of block 0x0 ... FAILED.
!! APFS ERROR !! Checksum of block 0x0 should validate, but it doesn't. Proceeding as if it does.

Details of block 0x0:
--------------------------------------------------------------------------------
Stored checksum:    0x006513adb556683c
OID:                0x1
XID:                0x8c27
Storage type:       Ephemeral
Type flags:         (none)
Type:               Container superblock
Subtype:            (none/invalid)
Keybag location: starts at 0x811d89, spans 0x1 blocks
Magic string:       NXSB
Block size:         8192 bytes
Block count:        488331131
Supported features:
- No feature flags are set.
Supported read-only compatible features:
- No read-only compatible feature flags are set.
Backward-incompatible features:
- No backward-incompatible feature flags are set.
UUID:       0x7238fcd169ed3aa82049687876943b5f
Next OID:                       0x3b7d
Next XID:                       0x8c28
Space manager Ephemeral OID:    0x400
Object map Physical OID:        0x403a25
Reaper Ephemeral OID:           0x401
Other flags:
- No other flags are set.
--------------------------------------------------------------------------------

Locating the checkpoint descriptor area:
- Its length is 136 blocks.
- It is contiguous.
- The address of its first block is 0x1661e.
Loading the checkpoint descriptor area into memory ... OK.
Locating the most recent well-formed container superblock in the checkpoint descriptor area:
- !! APFS WARNING !! Block at index 0 within this area failed checksum validation. Skipping it.
[OMITTED: SAME LINE AS ABOVE AND BELOW FROM 0-135 WITH JUST THE NUMBER CHANGING]
- !! APFS WARNING !! Block at index 135 within this area failed checksum validation. Skipping it.
No container superblock with an XID that doesn't exceed 0xffffffffffffffff exists in the checkpoint descriptor area.

Also if it's helpful...

Here is the output of `diskutil apfs list`:

(base) luke@My-MacBook-Pro ~ % diskutil apfs list                   
APFS Containers (4 found)
|
+-- Container disk3 7BBFE2B4-A53F-40DA-9A23-B7F93F023391
|   ====================================================
|   APFS Container Reference:     disk3
|   Size (Capacity Ceiling):      994662584320 B (994.7 GB)
|   Capacity In Use By Volumes:   133843386368 B (133.8 GB) (13.5% used)
|   Capacity Not Allocated:       860819197952 B (860.8 GB) (86.5% free)
|   |
|   +-< Physical Store disk0s2 FFE7F386-3F12-4521-A960-CD58EE0D38F9
|   |   -----------------------------------------------------------
|   |   APFS Physical Store Disk:   disk0s2
|   |   Size:                       994662584320 B (994.7 GB)
|   |
|   +-> Volume disk3s1 A473740A-7503-4953-BBAC-DF597AA612B6
|   |   ---------------------------------------------------
|   |   APFS Volume Disk (Role):   disk3s1 (System)
|   |   Name:                      Macintosh HD (Case-insensitive)
|   |   Mount Point:               Not Mounted
|   |   Capacity Consumed:         15053828096 B (15.1 GB)
|   |   Sealed:                    Broken
|   |   FileVault:                 No (Encrypted at rest)
|   |   |
|   |   Snapshot:                  2FA71488-D63A-403D-9650-DDA3923EA697
|   |   Snapshot Disk:             disk3s1s1
|   |   Snapshot Mount Point:      /
|   |   Snapshot Sealed:           Yes
|   |
|   +-> Volume disk3s2 DA6B5982-740B-4BD8-8D6F-ABBA714CE061
|   |   ---------------------------------------------------
|   |   APFS Volume Disk (Role):   disk3s2 (Preboot)
|   |   Name:                      Preboot (Case-insensitive)
|   |   Mount Point:               /System/Volumes/Preboot
|   |   Capacity Consumed:         305635328 B (305.6 MB)
|   |   Sealed:                    No
|   |   FileVault:                 No
|   |
|   +-> Volume disk3s3 63144F31-6220-41D2-A929-A7F4F5993914
|   |   ---------------------------------------------------
|   |   APFS Volume Disk (Role):   disk3s3 (Recovery)
|   |   Name:                      Recovery (Case-insensitive)
|   |   Mount Point:               Not Mounted
|   |   Capacity Consumed:         1072377856 B (1.1 GB)
|   |   Sealed:                    No
|   |   FileVault:                 No
|   |
|   +-> Volume disk3s5 6956C502-FC9F-47A4-B673-05D2D93913B4
|   |   ---------------------------------------------------
|   |   APFS Volume Disk (Role):   disk3s5 (Data)
|   |   Name:                      Data (Case-insensitive)
|   |   Mount Point:               /System/Volumes/Data
|   |   Capacity Consumed:         117204049920 B (117.2 GB)
|   |   Sealed:                    No
|   |   FileVault:                 No (Encrypted at rest)
|   |
|   +-> Volume disk3s6 EB20ECBE-DE85-48F5-B8C3-60A12C07D5ED
|       ---------------------------------------------------
|       APFS Volume Disk (Role):   disk3s6 (VM)
|       Name:                      VM (Case-insensitive)
|       Mount Point:               /System/Volumes/VM
|       Capacity Consumed:         20480 B (20.5 KB)
|       Sealed:                    No
|       FileVault:                 No
|
+-- Container ERROR -69808
    ======================
    APFS Container Reference:     disk5
    Size (Capacity Ceiling):      ERROR -69620
    Capacity In Use By Volumes:   ERROR -69620
    Capacity Not Allocated:       ERROR -69620
    |
    +-< Physical Store disk4s1 7CC5E36F-BD03-48D6-835F-963FD194BB6C
    |   -----------------------------------------------------------
    |   APFS Physical Store Disk:   disk4s1
    |   Size:                       4000408625152 B (4.0 TB)
    |
    +-> No Volumes

Here is the output of `diskutil list`:

(base) luke@My-MacBook-Pro ~ % diskutil list                        
/dev/disk0 (internal):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                         1.0 TB     disk0
   1:             Apple_APFS_ISC ⁨⁩                        524.3 MB   disk0s1
   2:                 Apple_APFS ⁨Container disk3⁩         994.7 GB   disk0s2
   3:        Apple_APFS_Recovery ⁨⁩                        5.4 GB     disk0s3

/dev/disk3 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +994.7 GB   disk3
                                 Physical Store disk0s2
   1:                APFS Volume ⁨Macintosh HD⁩            15.1 GB    disk3s1
   2:              APFS Snapshot ⁨com.apple.os.update-...⁩ 15.1 GB    disk3s1s1
   3:                APFS Volume ⁨Preboot⁩                 305.6 MB   disk3s2
   4:                APFS Volume ⁨Recovery⁩                1.1 GB     disk3s3
   5:                APFS Volume ⁨Data⁩                    117.2 GB   disk3s5
   6:                APFS Volume ⁨VM⁩                      20.5 KB    disk3s6

/dev/disk4 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *4.0 TB     disk4
   1:                 Apple_APFS ⁨Container disk5⁩         4.0 TB     disk4s1

/dev/disk5 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +ERROR      disk5
                                 Physical Store disk4s1

If you could even point me in the direction of some documentation that might let show me how to manually edit the B Tree to match what is expected that would be great. Thanks in advance!

jivanpal · 2020-12-22T21:28:54Z

jivanpal
Dec 22, 2020
Maintainer

We don't currently support APFS block sizes other than 4096 bytes (this is the default given in the spec), but it appears that your partition uses an APFS block size of 8192 bytes, which is why checksum verification and block seeking is not working as expected. I'll add better support for this to the roadmap, but for now, you can try editing line 16 of src/apfs/io.h to size_t nx_block_size = 8192;, then recompiling and trying apfs-inspect again.

0 replies

ldsalomone · 2020-12-23T00:08:17Z

ldsalomone
Dec 23, 2020
Author

You're right, that seemed to mostly resolve my issues exploring my drive. I will continue troubleshooting using the additional tools that you've provided. The drive was encrypted with FileVault before I started having issues. Are there any flags or code I should alter to pass in the encryption key?

apfs-inspect looks all good up until the following:

Details of the volume object map B-tree:
--------------------------------------------------------------------------------
Stored checksum:    0x6162bbf1788cf936
OID:                0x4039ff
XID:                0x8c26
Storage type:       Physical
Type flags:         (none)
Type:               B-tree (root node)
Subtype:            Object map
Flags:                          Root node, Fixed size for keys and values
Number of child levels:         1
Number of keys in this node:    165
Location of table of contents:
- Offset from start of node data area:  0x0 = 0
- Length (bytes):                       0x488 = 1160
Location of key–value shared free space:
- Offset from start of keys area:       0xa50 = 2640
- Length (bytes):                       0xba0 = 2976

Info relating to the entire B-tree:
- Flags:
  - This B-tree is currently undergoing a series of sequential inserts --- optimise operations if possible
  - Child nodes are referred to using Physical OIDs
- Node size:                8192 bytes
- Key size:                 16 bytes
- Value size:               16 bytes

- Length of longest key:    16 bytes
- Length of longest value:  16 bytes
- Number of keys:           26342
- Number of nodes:          166
--------------------------------------------------------------------------------

The file-system tree root for this volume has Virtual OID 0x404.
Looking up this Virtual OID in the volume object map ... corresponding block address is 0x55d.
Reading ... validating ... FAILED.
Going back to look at the previous checkpoint instead.
END: Handling of this case has not yet been implemented.

I can write something to explore the case handling if you point me in the right direction.

0 replies

jivanpal · 2020-12-23T16:35:27Z

jivanpal
Dec 23, 2020
Maintainer

Unfortunately, we don't support encryption at all currently — sorry about that! — but it's on the roadmap. Only the filesystem B-tree and the file data/metadata are encrypted, so what you get is expected. The case handling mentioned there is in relation to using an older checkpoint/XID if data in the current one appears to be malformed. In your case, trying to use an older checkpoint is futile because we need encryption support first.

0 replies

ldsalomone · 2020-12-23T16:56:00Z

ldsalomone
Dec 23, 2020
Author

Thanks for that. I will keep following to see when you implement the encryption support.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

No container superblock with an XID that doesn't exceed 0xffffffffffffffff exists in the checkpoint descriptor area. #11

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

No container superblock with an XID that doesn't exceed 0xffffffffffffffff exists in the checkpoint descriptor area. #11

ldsalomone Dec 22, 2020

Replies: 4 comments

jivanpal Dec 22, 2020 Maintainer

ldsalomone Dec 23, 2020 Author

jivanpal Dec 23, 2020 Maintainer

ldsalomone Dec 23, 2020 Author

ldsalomone
Dec 22, 2020

jivanpal
Dec 22, 2020
Maintainer

ldsalomone
Dec 23, 2020
Author

jivanpal
Dec 23, 2020
Maintainer

ldsalomone
Dec 23, 2020
Author