Improve wipe I/O throughput with large aligned buffers and optional O_DIRECT

This change reworks the pass/verify I/O path and adds optional direct I/O
support to reduce syscall overhead and better utilize modern storage
performance.

pass.c:
- Introduce NWIPE_BUFFER_SIZE (default 16 MiB) as a generic scratch buffer size
  and NWIPE_IO_BLOCKSIZE (default 4 MiB) as the target read/write block size.
- Add nwipe_effective_io_blocksize() to compute an effective I/O block size
  per device:
  - At least device_stat.st_blksize
  - Rounded down to a multiple of st_blksize for O_DIRECT compatibility
  - Never larger than the device size
- Add nwipe_alloc_io_buffer(), which allocates I/O buffers using
  posix_memalign() aligned to the device block size (>= 512 B). This makes the
  same code safe for both buffered I/O and O_DIRECT.
- Rework nwipe_random_pass():
  - Use a large, aligned scratch buffer (default 16 MiB) instead of tiny
    st_blksize-sized buffers.
  - Generate and write data in large chunks (default 4 MiB) to drastically
    reduce the number of write() syscalls.
  - Keep the original PRNG init/read interface and the “PRNG wrote something”
    sanity check (still checks within the first st_blksize bytes).
  - Preserve existing error handling, progress accounting and periodic
    fdatasync() logic.
- Rework nwipe_random_verify():
  - Use the same large I/O block logic for read/compare.
  - Generate the expected random stream in large blocks and compare against
    data read from the device.
  - Maintain the original semantics for partial reads and error counters.
- Rework nwipe_static_pass() and nwipe_static_verify():
  - Build large pattern buffers that repeat the user-specified pattern and
    support a sliding window (w) into the pattern.
  - Perform writes/reads in large blocks (default 4 MiB) while keeping the
    pattern alignment consistent via the window offset.
  - Preserve original behaviour regarding partial I/O, logging and counters.

nwipe.c:
- Add support for optional direct I/O when NWIPE_USE_DIRECT_IO is defined:
  - Include <fcntl.h> and ensure O_DIRECT is available (fallback to 0 on
    platforms that do not define it).
  - Open devices with O_RDWR|O_DIRECT, and transparently fall back to O_RDWR
    if O_DIRECT is not supported (e.g. EINVAL/EOPNOTSUPP).
- Enable GNU extensions (e.g. _GNU_SOURCE) so that O_DIRECT is visible on
  glibc-based systems.

Behavioural impact:
- The wiping/verification algorithms and patterns are unchanged; only the I/O
  strategy is modified to use larger, aligned buffers.
- The number of read()/write() syscalls per pass is reduced by orders of
  magnitude (e.g. 4 MiB vs. 4 KiB), which should significantly increase
  throughput on fast disks/NVMe.
- When NWIPE_USE_DIRECT_IO is enabled and supported by the device, the same
  code path uses direct I/O to avoid unnecessary page cache pollution; when
  unsupported, behaviour gracefully falls back to buffered I/O.
This commit is contained in:
Fabian Druschke
2025-11-17 19:37:31 +01:00
parent 96127687cd
commit 2dbdaf447c
2 changed files with 433 additions and 345 deletions

View File

@@ -27,6 +27,11 @@
#define _POSIX_SOURCE
#endif
/* Enable GNU extensions so that O_DIRECT is visible from <fcntl.h>. */
#ifndef _GNU_SOURCE
#define _GNU_SOURCE 1
#endif
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
@@ -59,6 +64,7 @@
#include "hpa_dco.h"
#include "conf.h"
#include <libconfig.h>
#include <fcntl.h> /* O_DIRECT, O_RDWR, ... */
int terminate_signal;
int user_abort;
@@ -478,8 +484,34 @@ int main( int argc, char** argv )
/* Initialise the wipe_status flag, -1 = wipe not yet started */
c2[i]->wipe_status = -1;
/* Open the file for reads and writes. */
c2[i]->device_fd = open( c2[i]->device_name, O_RDWR );
/* Open the file for reads and writes. Optionally use O_DIRECT. */
int open_flags = O_RDWR;
#ifdef NWIPE_USE_DIRECT_IO
open_flags |= O_DIRECT;
#endif
c2[i]->device_fd = open( c2[i]->device_name, open_flags );
#ifdef NWIPE_USE_DIRECT_IO
/* If O_DIRECT is not supported (or rejected by the FS), fall back to buffered I/O. */
if( c2[i]->device_fd < 0 && ( errno == EINVAL || errno == EOPNOTSUPP ) )
{
nwipe_log( NWIPE_LOG_WARNING,
"O_DIRECT not supported on '%s', retrying without O_DIRECT.",
c2[i]->device_name );
open_flags &= ~O_DIRECT;
c2[i]->device_fd = open( c2[i]->device_name, open_flags );
}
#endif
/* Check the open() result. */
if( c2[i]->device_fd < 0 )
{
nwipe_perror( errno, __FUNCTION__, "open" );
nwipe_log( NWIPE_LOG_WARNING, "Unable to open device '%s'.", c2[i]->device_name );
c2[i]->select = NWIPE_SELECT_DISABLED;
continue;
}
/* Check the open() result. */
if( c2[i]->device_fd < 0 )

File diff suppressed because it is too large Load Diff