2

I have a computer with 64 physical cores (AMD Ryzen Threadripper PRO 3995WX 64-Cores), with two threads per core capability (that is, 128 logical cores), running on Ubuntu 20.04.3 LTS. Recently, I launched 100 instances of a program that involves some I/O usage on a secondary HDD (new, 18 TB, model ST18000NM000J-2TV103), which is independent of the SSD from which the program instances are launched. I had done this several times before, but this time the HDD entered in writing protected mode showing I/O errors, and many of the 100 instances could not end successfully.

These are the relevant lines of dmesg output:

[1634026.731362] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00000 flags=0x0020]
[1634026.731375] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00180 flags=0x0020]
[1634026.731383] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00280 flags=0x0020]
[1634026.731390] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00380 flags=0x0020]
[1634026.731396] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00400 flags=0x0020]
[1634026.731403] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00500 flags=0x0020]
[1634026.731409] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00600 flags=0x0020]
[1634026.731416] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00700 flags=0x0020]
[1634026.731427] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00800 flags=0x0020]
[1634026.731436] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00900 flags=0x0020]
[1634026.731444] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe00a00 flags=0x0020]
[1634026.731453] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe00b00 flags=0x0020]
[1634026.731460] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe00c00 flags=0x0020]
[1634026.731468] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe00d00 flags=0x0020]
[1634026.731475] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe00e00 flags=0x0020]
[1634026.731482] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe00f00 flags=0x0020]
[1634026.731490] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe01000 flags=0x0020]
[1634026.731497] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe01180 flags=0x0020]
[1634026.731504] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe01280 flags=0x0020]
[1634026.731512] AMD-Vi: Event logged [IO_PAGE_FAULT device=26:00.0 domain=0x0032 address=0x7ffffe01300 flags=0x0020]
[1634285.885038] ata1.00: exception Emask 0x0 SAct 0x38f00000 SErr 0x0 action 0x6 frozen
[1634285.885052] ata1.00: failed command: WRITE FPDMA QUEUED
[1634285.885122] ata1.00: status: { DRDY }
[1634285.885126] ata1: hard resetting link
[1634286.361002] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[1634291.484362] ata1.00: qc timeout (cmd 0x47)
[1634291.484380] ata1.00: READ LOG DMA EXT failed, trying PIO
[1634291.484382] ata1.00: failed to get NCQ Send/Recv Log Emask 0x40
[1634291.484388] ata1.00: configured for UDMA/133
[1634301.484249] ata1: softreset failed (1st FIS failed)
[1634311.484744] ata1: softreset failed (1st FIS failed)
[1634346.484183] ata1: softreset failed (1st FIS failed)
[1634346.484194] ata1: limiting SATA link speed to 3.0 Gbps
[1634351.484345] ata1: softreset failed (1st FIS failed)
[1634351.484356] ata1: reset failed, giving up
[1634351.484359] ata1.00: disabled
[1634351.484378] ata1: EH complete
[1634351.484428] sd 0:0:0:0: [sda] tag#12 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=97s
[1634351.484442] sd 0:0:0:0: [sda] Read Capacity(16) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[1634351.484443] sd 0:0:0:0: [sda] tag#12 CDB: Write(16) 8a 00 00 00 00 02 87 54 40 80 00 00 04 80 00 00
[1634351.484448] sd 0:0:0:0: [sda] Sense not available.
[1634351.484448] blk_update_request: I/O error, dev sda, sector 10860380288 op 0x1:(WRITE) flags 0x0 phys_seg 81 prio class 0
[1634351.484459] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843302 starting block 1357547552)
[1634351.484465] sd 0:0:0:0: [sda] Read Capacity(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[1634351.484467] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843303 starting block 1357547568)
[1634351.484469] sd 0:0:0:0: [sda] Sense not available.
[1634351.484474] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843304 starting block 1357547584)
[1634351.484480] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843305 starting block 1357547600)
[1634351.484484] sd 0:0:0:0: [sda] 0 512-byte logical blocks: (0 B/0 B)
[1634351.484486] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843306 starting block 1357547616)
[1634351.484491] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843307 starting block 1357547632)
[1634351.484495] sd 0:0:0:0: [sda] 4096-byte physical blocks
[1634351.484512] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843308 starting block 1357547648)
[1634351.484517] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843309 starting block 1357547664)
[1634351.484521] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 84843310 starting block 1357547680)
[1634351.484521] sda: detected capacity change from 35156656128 to 0
[1634351.484525] Buffer I/O error on device sda1, logical block 1357547280
[1634351.484536] sd 0:0:0:0: [sda] tag#13 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=97s
[1634351.484541] sd 0:0:0:0: [sda] tag#13 CDB: Write(16) 8a 00 00 00 00 02 87 40 52 e8 00 00 00 08 00 00
[1634351.484543] blk_update_request: I/O error, dev sda, sector 10859074280 op 0x1:(WRITE) flags 0x103000 phys_seg 1 prio class 0
[1634351.484547] Buffer I/O error on device sda1, logical block 1357547281
[1634351.484549] Buffer I/O error on dev sda1, logical block 1357384029, lost async page write
[1634351.484552] Buffer I/O error on device sda1, logical block 1357547282
[1634351.484556] Buffer I/O error on device sda1, logical block 1357547283
[1634351.484558] Buffer I/O error on device sda1, logical block 1357547284
[1634351.484561] Buffer I/O error on device sda1, logical block 1357547285
[1634351.484562] Buffer I/O error on device sda1, logical block 1357547286

and the relevant line of df -h

/dev/sdb1        17T  796G   15T   6% /data

After rebooting, this problem seemed to disappear (the HDD does not show I/O errors anymore).

The HDD had a similar problem a weeks ago, entering in writing protected mode with I/O errors and returning to normal state after reboot. The computer was performing different tasks, but still involving many threads and reading/writing files. The dmesg output of that first problem was somewhat different (also, the HDD was in /dev/sda at that time):

[1024558.271654] ahci 0000:26:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0032 address=0x7ffffe00000 flags=0x0000]
[1024589.483643] ata1.00: exception Emask 0x0 SAct 0x1f800 SErr 0x80000 action 0x6 frozen
[1024589.483654] ata1: SError: { 10B8B }
[1024589.483718] ata1.00: status: { DRDY }
[1024589.483722] ata1: hard resetting link
[1024589.959619] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[1024595.087563] ata1.00: qc timeout (cmd 0x47)
[1024595.087581] ata1.00: READ LOG DMA EXT failed, trying PIO
[1024595.087583] ata1.00: failed to get NCQ Send/Recv Log Emask 0x40
[1024595.087591] ata1.00: configured for UDMA/133
[1024605.087857] ata1: softreset failed (1st FIS failed)
[1024615.087044] ata1: softreset failed (1st FIS failed)
[1024650.087438] ata1: softreset failed (1st FIS failed)
[1024650.087450] ata1: limiting SATA link speed to 3.0 Gbps
[1024655.087628] ata1: softreset failed (1st FIS failed)
[1024655.087639] ata1: reset failed, giving up
[1024655.087642] ata1.00: disabled
[1024655.087659] ata1: EH complete
[1024655.087709] sd 0:0:0:0: [sda] tag#13 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=96s
[1024655.087725] sd 0:0:0:0: [sda] tag#13 CDB: Write(16) 8a 00 00 00 00 03 3a 07 12 00 00 00 0a 00 00 00
[1024655.087729] blk_update_request: I/O error, dev sda, sector 13858443776 op 0x1:(WRITE) flags 0x4000 phys_seg 85 prio class 0
[1024655.087754] sd 0:0:0:0: [sda] tag#14 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=96s
[1024655.087759] sd 0:0:0:0: [sda] Read Capacity(16) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[1024655.087761] sd 0:0:0:0: [sda] tag#14 CDB: Write(16) 8a 00 00 00 00 03 3a 07 08 00 00 00 0a 00 00 00
[1024655.087765] blk_update_request: I/O error, dev sda, sector 13858441216 op 0x1:(WRITE) flags 0x4000 phys_seg 86 prio class 0
[1024655.087768] sd 0:0:0:0: [sda] Sense not available.
...
[1024655.088120] Buffer I/O error on device sda1, logical block 1732303796
[1024655.088119] blk_update_request: I/O error, dev sda, sector 13405843408 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
[1024655.088120] blk_update_request: I/O error, dev sda, sector 13363431088 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0
[1024655.088125] Buffer I/O error on device sda1, logical block 1732303797
[1024655.088127] sd 0:0:0:0: [sda] tag#20 access beyond end of device
[1024655.088128] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 103547010 starting block 1670428887)
[1024655.088131] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 103546995 starting block 1675730427)
[1024655.088133] Buffer I/O error on device sda1, logical block 1732303798
[1024655.088134] sd 0:0:0:0: [sda] tag#29 access beyond end of device
[1024655.088136] EXT4-fs warning (device sda1): ext4_end_bio:342: I/O error 10 writing to inode 103547000 starting block 1671125037)
[1024655.088140] Buffer I/O error on device sda1, logical block 1732303799
[1024655.088143] Buffer I/O error on device sda1, logical block 1732303800
[1024655.088145] Buffer I/O error on device sda1, logical block 1732303801
[1024655.088147] Buffer I/O error on device sda1, logical block 1732303802
[1024655.088150] Buffer I/O error on device sda1, logical block 1732303803
[1024655.088152] Buffer I/O error on device sda1, logical block 1732303804
...
[1024655.088929] Buffer I/O error on dev sda1, logical block 0, lost sync page write
[1024655.088937] EXT4-fs (sda1): I/O error while writing superblock
[1024655.088938] EXT4-fs (sda1): I/O error while writing superblock
[1024655.088941] EXT4-fs (sda1): failed to convert unwritten extents to written extents -- potential data loss!  (inode 103546991, error -30)
[1024655.088959] sd 0:0:0:0: [sda] tag#13 access beyond end of device
[1024655.088963] Buffer I/O error on dev sda1, logical block 0, lost sync page write
[1024655.088979] sd 0:0:0:0: [sda] tag#15 access beyond end of device
[1024655.088982] Buffer I/O error on dev sda1, logical block 0, lost sync page write
[1024655.089053] sd 0:0:0:0: [sda] tag#28 access beyond end of device
[1024655.089062] Buffer I/O error on dev sda1, logical block 0, lost sync page write
[1024655.724098] sd 0:0:0:0: [sda] tag#8 access beyond end of device
[1024655.724137] sd 0:0:0:0: [sda] tag#2 access beyond end of device
[1024655.724146] sd 0:0:0:0: [sda] tag#3 access beyond end of device
[1024655.724148] Buffer I/O error on dev sda1, logical block 0, lost sync page write
[1024655.724165] sd 0:0:0:0: [sda] tag#9 access beyond end of device
...
[1025036.714171] sd 0:0:0:0: [sda] tag#10 access beyond end of device
[1025036.714180] blk_update_request: I/O error, dev sda, sector 13254019344 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[1025036.714197] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #103546884: lblock 0: comm ls: error -5 reading directory block
[1025045.177176] sd 0:0:0:0: [sda] tag#28 access beyond end of device
[1025045.177186] blk_update_request: I/O error, dev sda, sector 13254019344 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[1025045.177198] EXT4-fs error: 10160 callbacks suppressed
[1025045.177200] EXT4-fs error (device sda1): __ext4_find_entry:1524: inode #103546884: comm mkdir: reading directory lblock 0
[1025045.177520] sd 0:0:0:0: [sda] tag#29 access beyond end of device
[1025045.177524] blk_update_request: I/O error, dev sda, sector 2048 op 0x1:(WRITE) flags 0x3800 phys_seg 1 prio class 0
[1025045.177529] buffer_io_error: 5245 callbacks suppressed
[1025045.177530] Buffer I/O error on dev sda1, logical block 0, lost sync page write
[1025045.177536] EXT4-fs: 5245 callbacks suppressed
[1025045.177538] EXT4-fs (sda1): I/O error while writing superblock

Does anybody know why these problems happened? Could it be a bug in the Linux kernel?

Thanks!

Update: A similar error happened recently, but this time just launching simple bash commands (grep and ls). Again, the HDD entered in writing protected mode showing I/O errors, and it went back to normal after rebooting. The relevant lines of dmesg output were:

[1027135.049335] ata1.00: exception Emask 0x0 SAct 0x4000700 SErr 0x0 action 0x6 frozen
[1027135.049350] ata1.00: failed command: READ FPDMA QUEUED
[1027135.049363] ata1.00: status: { DRDY }
[1027135.049402] ata1: hard resetting link
[1027135.525212] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[1027140.649269] ata1.00: qc timeout (cmd 0x47)
[1027140.649289] ata1.00: READ LOG DMA EXT failed, trying PIO
[1027140.649292] ata1.00: failed to get NCQ Send/Recv Log Emask 0x40
[1027140.649299] ata1.00: configured for UDMA/133
[1027150.649291] ata1: softreset failed (1st FIS failed)
[1027160.649202] ata1: softreset failed (1st FIS failed)
[1027195.648363] ata1: softreset failed (1st FIS failed)
[1027195.648376] ata1: limiting SATA link speed to 3.0 Gbps
[1027200.648819] ata1: softreset failed (1st FIS failed)
[1027200.648831] ata1: reset failed, giving up
[1027200.648834] ata1.00: disabled
[1027200.648854] sd 0:0:0:0: [sda] tag#8 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=102s
[1027200.648859] sd 0:0:0:0: [sda] tag#8 Sense Key : Illegal Request [current] 
[1027200.648861] sd 0:0:0:0: [sda] tag#8 Add. Sense: Unaligned write command
[1027200.648864] sd 0:0:0:0: [sda] tag#8 CDB: Read(16) 88 00 00 00 00 02 7e 09 bb 00 00 00 01 00 00 00
[1027200.648866] blk_update_request: I/O error, dev sda, sector 10704501504 op 0x0:(READ) flags 0x80700 phys_seg 3 prio class 0
[1027200.648890] sd 0:0:0:0: [sda] tag#9 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=102s
[1027200.648892] sd 0:0:0:0: [sda] tag#9 Sense Key : Illegal Request [current] 
[1027200.648894] sd 0:0:0:0: [sda] tag#9 Add. Sense: Unaligned write command
[1027200.648896] sd 0:0:0:0: [sda] tag#9 CDB: Read(16) 88 00 00 00 00 02 7e 09 bc 00 00 00 01 00 00 00
[1027200.648897] blk_update_request: I/O error, dev sda, sector 10704501760 op 0x0:(READ) flags 0x80700 phys_seg 3 prio class 0
[1027200.648906] ata1: EH complete
[1027200.648958] sd 0:0:0:0: [sda] tag#13 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=102s
[1027200.648967] sd 0:0:0:0: [sda] tag#14 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=95s
[1027200.648975] sd 0:0:0:0: [sda] tag#13 CDB: Write(16) 8a 00 00 00 00 04 17 8a ea 38 00 00 02 78 00 00
[1027200.648980] blk_update_request: I/O error, dev sda, sector 17574849080 op 0x1:(WRITE) flags 0x800 phys_seg 79 prio class 0
[1027200.648982] sd 0:0:0:0: [sda] tag#14 CDB: Read(16) 88 00 00 00 00 02 83 00 57 18 00 00 00 08 00 00
[1027200.648987] sd 0:0:0:0: [sda] tag#16 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
[1027200.648990] blk_update_request: I/O error, dev sda, sector 10787772184 op 0x0:(READ) flags 0x3000 phys_seg 1 prio class 0
[1027200.648996] sd 0:0:0:0: [sda] tag#16 CDB: Read(16) 88 00 00 00 00 02 7e 09 bb 00 00 00 00 08 00 00
[1027200.649000] blk_update_request: I/O error, dev sda, sector 10704501504 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[1027200.649007] sd 0:0:0:0: [sda] Read Capacity(16) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[1027200.649016] sd 0:0:0:0: [sda] Sense not available.
[1027200.649035] sd 0:0:0:0: [sda] tag#22 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
[1027200.649040] sd 0:0:0:0: [sda] Read Capacity(10) failed: Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK
[1027200.649043] sd 0:0:0:0: [sda] tag#22 CDB: Read(16) 88 00 00 00 00 02 7e 09 bb 00 00 00 00 08 00 00
[1027200.649044] sd 0:0:0:0: [sda] Sense not available.
[1027200.649047] blk_update_request: I/O error, dev sda, sector 10704501504 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[1027200.649051] EXT4-fs warning (device sda1): dx_probe:757: inode #84279410: lblock 0: comm bash: error -5 reading directory block
[1027200.649062] sd 0:0:0:0: [sda] 0 512-byte logical blocks: (0 B/0 B)
[1027200.649065] sd 0:0:0:0: [sda] 4096-byte physical blocks
[1027200.649079] Aborting journal on device sda1-8.
[1027200.649091] sda: detected capacity change from 35156656128 to 0
[1027200.649110] sd 0:0:0:0: [sda] tag#10 access beyond end of device
[1027200.649119] blk_update_request: I/O error, dev sda, sector 10704501504 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[1027200.649124] sd 0:0:0:0: [sda] tag#11 access beyond end of device
[1027200.649126] blk_update_request: I/O error, dev sda, sector 10787752240 op 0x1:(WRITE) flags 0x103000 phys_seg 1 prio class 0
[1027200.649136] Buffer I/O error on dev sda1, logical block 1348468774, lost async page write
[1027200.649180] sd 0:0:0:0: [sda] tag#27 access beyond end of device
[1027200.649183] blk_update_request: I/O error, dev sda, sector 17574397952 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
[1027200.649188] Buffer I/O error on dev sda1, logical block 2196799488, lost sync page write
[1027200.649198] JBD2: Error -5 detected when updating journal superblock for sda1-8.
[1027200.649209] sd 0:0:0:0: [sda] tag#6 access beyond end of device
[1027200.649215] blk_update_request: I/O error, dev sda, sector 10833889536 op 0x1:(WRITE) flags 0x103000 phys_seg 1 prio class 0
[1027200.649221] Buffer I/O error on dev sda1, logical block 1354235936, lost async page write
[1027200.649237] sd 0:0:0:0: [sda] tag#7 access beyond end of device
[1027200.649240] Buffer I/O error on dev sda1, logical block 1354235940, lost async page write
[1027200.649243] Buffer I/O error on dev sda1, logical block 1354235941, lost async page write
[1027200.649252] sd 0:0:0:0: [sda] tag#8 access beyond end of device
[1027200.649253] Buffer I/O error on dev sda1, logical block 1354235943, lost async page write
[1027200.649263] sd 0:0:0:0: [sda] tag#9 access beyond end of device
[1027200.649265] Buffer I/O error on dev sda1, logical block 1354235949, lost async page write
[1027200.649267] Buffer I/O error on dev sda1, logical block 1354235950, lost async page write
[1027200.649276] sd 0:0:0:0: [sda] tag#12 access beyond end of device
[1027200.649278] Buffer I/O error on dev sda1, logical block 1354235953, lost async page write
[1027200.649287] sd 0:0:0:0: [sda] tag#13 access beyond end of device
[1027200.649289] Buffer I/O error on dev sda1, logical block 1354236143, lost async page write
[1027200.649298] sd 0:0:0:0: [sda] tag#14 access beyond end of device
[1027200.649316] sd 0:0:0:0: [sda] tag#15 access beyond end of device
[1027200.649332] sd 0:0:0:0: [sda] tag#16 access beyond end of device
[1027200.649338] sd 0:0:0:0: [sda] tag#17 access beyond end of device
[1027200.649350] sd 0:0:0:0: [sda] tag#18 access beyond end of device
[1027200.649357] sd 0:0:0:0: [sda] tag#19 access beyond end of device
[1027200.649363] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #84677243: lblock 1: comm grep: error -5 reading directory block
[1027200.649367] sd 0:0:0:0: [sda] tag#20 access beyond end of device
[1027200.649372] EXT4-fs error (device sda1): ext4_journal_check_start:83: comm grep: Detected aborted journal
[1027200.649379] sd 0:0:0:0: [sda] tag#21 access beyond end of device
[1027200.649392] sd 0:0:0:0: [sda] tag#22 access beyond end of device
[1027200.649404] sd 0:0:0:0: [sda] tag#23 access beyond end of device
[1027200.649416] sd 0:0:0:0: [sda] tag#24 access beyond end of device
[1027200.649435] sd 0:0:0:0: [sda] tag#12 access beyond end of device
[1027200.649444] EXT4-fs (sda1): I/O error while writing superblock
[1027200.649446] EXT4-fs (sda1): Remounting filesystem read-only
[1027200.649457] sd 0:0:0:0: [sda] tag#25 access beyond end of device
[1027200.649472] sd 0:0:0:0: [sda] tag#26 access beyond end of device
[1027200.649484] sd 0:0:0:0: [sda] tag#13 access beyond end of device
[1027200.649491] sd 0:0:0:0: [sda] tag#14 access beyond end of device
[1027200.649504] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #84677260: lblock 1: comm grep: error -5 reading directory block
[1027200.649528] sd 0:0:0:0: [sda] tag#28 access beyond end of device
[1027200.649536] sd 0:0:0:0: [sda] tag#29 access beyond end of device
[1027200.649544] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #84639805: lblock 1: comm grep: error -5 reading directory block
[1027200.649555] sd 0:0:0:0: [sda] tag#30 access beyond end of device
[1027200.649569] sd 0:0:0:0: [sda] tag#0 access beyond end of device
[1027200.649575] sd 0:0:0:0: [sda] tag#1 access beyond end of device
[1027200.649586] sd 0:0:0:0: [sda] tag#2 access beyond end of device
[1027200.649592] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #84677245: lblock 1: comm grep: error -5 reading directory block
[1027200.649597] sd 0:0:0:0: [sda] tag#3 access beyond end of device
[1027200.649615] sd 0:0:0:0: [sda] tag#4 access beyond end of device
[1027200.649624] sd 0:0:0:0: [sda] tag#5 access beyond end of device
[1027200.649634] sd 0:0:0:0: [sda] tag#6 access beyond end of device
[1027200.649636] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #84639816: lblock 1: comm grep: error -5 reading directory block
[1027200.649645] sd 0:0:0:0: [sda] tag#7 access beyond end of device
[1027200.649656] sd 0:0:0:0: [sda] tag#8 access beyond end of device
[1027200.649685] sd 0:0:0:0: [sda] tag#9 access beyond end of device
[1027200.649696] sd 0:0:0:0: [sda] tag#15 access beyond end of device
[1027200.649706] sd 0:0:0:0: [sda] tag#10 access beyond end of device
[1027200.649713] sd 0:0:0:0: [sda] tag#11 access beyond end of device
...
[1027200.650422] sd 0:0:0:0: [sda] tag#4 access beyond end of device
[1027200.650512] sd 0:0:0:0: [sda] tag#5 access beyond end of device
[1027200.650520] EXT4-fs error (device sda1): __ext4_get_inode_loc_noinmem:4445: inode #83891181: block 1342177630: comm grep: unable to read itable block
[1027200.650546] sd 0:0:0:0: [sda] tag#6 access beyond end of device
[1027200.650550] EXT4-fs (sda1): I/O error while writing superblock
[1027200.650591] sd 0:0:0:0: [sda] tag#7 access beyond end of device
[1027200.650599] sd 0:0:0:0: [sda] tag#8 access beyond end of device
...
[1027200.656698] sd 0:0:0:0: [sda] tag#10 access beyond end of device
[1027200.656705] sd 0:0:0:0: [sda] tag#11 access beyond end of device
[1027200.656740] sd 0:0:0:0: [sda] tag#12 access beyond end of device
[1027200.656744] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #83886134: lblock 1: comm grep: error -5 reading directory block
[1027200.656771] sd 0:0:0:0: [sda] tag#13 access beyond end of device
[1027200.656774] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #84639812: lblock 1: comm grep: error -5 reading directory block
[1027200.656799] sd 0:0:0:0: [sda] tag#14 access beyond end of device
[1027200.656803] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #84639778: lblock 1: comm grep: error -5 reading directory block
[1027200.656829] sd 0:0:0:0: [sda] tag#15 access beyond end of device
[1027200.656832] EXT4-fs warning (device sda1): htree_dirblock_to_tree:993: inode #83886156: lblock 1: comm grep: error -5 reading directory block
[1027200.656861] sd 0:0:0:0: [sda] tag#16 access beyond end of device
[1027200.656890] sd 0:0:0:0: [sda] tag#17 access beyond end of device
...
[1027200.691566] sd 0:0:0:0: [sda] tag#0 access beyond end of device
[1027200.691578] sd 0:0:0:0: [sda] tag#1 access beyond end of device
[1027200.691591] sd 0:0:0:0: [sda] tag#2 access beyond end of device
[1027208.503984] sd 0:0:0:0: [sda] tag#3 FAILED Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK cmd_age=0s
[1027208.503993] sd 0:0:0:0: [sda] tag#3 CDB: ATA command pass through(16) 85 06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00
6
  • 1
    You might try turning off NCQ with a command-line option as a test. I've seen problems in the past with some controller-disk combinations with too many queued commands.
    – stark
    Jan 3, 2022 at 17:35
  • This post recommends not to use ext4 for HDD over 16 Tb, but it is probably outdated, so not sure if it is related at all with this error. unix.stackexchange.com/questions/365355/… Jan 4, 2022 at 10:27
  • Also potentially related: linuxquestions.org/questions/linux-general-1/… Jan 4, 2022 at 10:46
  • @stark Thanks for the answer, I tried it by modifying /sys/block/sda/device/queue_depth. Unfortunately, it didn't work. After launching the instances, it completely collapsed the RAM, and the HDD entered again in writing protected mode, this time showing an "Bad message" error. Jan 17, 2022 at 9:22

1 Answer 1

1

You could try formatting your drive as XFS instead of EXT4 and check if there is any improvement.

See: https://unix.stackexchange.com/questions/365355/why-is-ext4-only-recommended-up-to-16-tb and also: https://github.com/openzfs/zfs/issues/66

3
  • I have done this and, at the moment, the HDD works correctly with high multithreading and I/O usage. Thanks! Feb 10, 2022 at 18:15
  • Finally, we saw that it failed again (took longer to happen), and at some point the ASUS SAGE WIFI motherboard stopped recognizing (any) HDD on Linux (weirdly not on windows), so the vendor replaced the complete motherboard and for the moment it looks ok. May 27, 2022 at 11:20
  • Confirmed, no more errors. It was a faulty motherboard. Oct 13, 2022 at 13:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .