This commit enhances the handling of the `-P /path` / `--PDFreportpath` option by
ensuring that nwipe can create the specified directory if it does not already
exist. Previously, nwipe simply called `access(path, W_OK)` and exited with a
generic “not a writeable directory” error if the directory did not exist or was
not writable. This caused ambiguity and prevented the use of custom report paths
without pre-creating them externally.
Key improvements:
- Added a new helper function `nwipe_ensure_directory()` that:
- Differentiates between “non-existent”, “not a directory”, and “not writable”.
- Attempts to create the directory recursively (`mkdir -p` style) when absent.
- Creates directories with mode 0755 so other users can read/list directory contents.
- Performs final verification that the directory exists, is a directory, and writable.
- Replaced the previous simple `access()` check in `nwipe.c` with the new
directory-ensure logic.
- Introduces clearer and more helpful error messages when directory creation or
permission checks fail.
Benefits:
- Users can now safely specify custom report paths (e.g. `-P /reports` or
USB-mounted paths) without requiring manual pre-creation.
- Eliminates ambiguous error reporting and improves overall user experience.
- Maintains backward-compatible behavior when the target directory already exists.
This patch fixes several issues that could cause garbage or control
characters to appear in the ncurses UI when displaying device serial
numbers.
Key changes:
- Added nwipe_normalize_serial() to strip control characters, non-ASCII
bytes and trim whitespace from all serial numbers before they are
shown in the UI.
- Initialize the serialnumber buffer in
nwipe_get_device_bus_type_and_serialno() to avoid passing undefined
data back to check_device() when no valid "Serial Number:" field is
found.
- Prevent ioctl(HDIO_GET_IDENTITY) from being called on an invalid file
descriptor when open() fails.
- Ensure consistent null termination and sanitize the final
device_serial_no regardless of whether it came from HDIO, smartctl
output, or quiet-mode anonymization.
These fixes resolve cases where devices (especially virtual/QEMU or
USB-attached drives) could report malformed or unexpected serial
strings, resulting in UI corruption such as extra characters, ^A, or
line wrapping.
the effective I/O block size before any sync-rate logic is executed.
- Add new helper function `nwipe_compute_sync_rate_for_device()` to `pass.c`,
converting legacy `--sync` semantics (sync * st_blksize) into a per-write
sync interval based on the actual `io_blocksize`, and disabling periodic
syncing when using direct I/O.
- Update both `nwipe_random_pass()` and `nwipe_static_pass()` to use the new
helper, ensuring consistent and correct sync behaviour for all cached-I/O
passes and removing duplicated sync-calculation logic.
This commit updates both README.md and the nwipe(8) manpage to reflect the
features and behaviour introduced in the upcoming v0.40 release. Changes include:
- Added documentation for the new AES-256-CTR PRNG and its hardware-accelerated
implementation.
- Updated erasure method list to include the BMB21-2019 State Secrets Bureau
sanitisation standard.
- Added full documentation for large, aligned I/O buffers and their impact on
performance.
- Documented the new I/O mode system (`--io-mode=auto|direct|cached`,
`--directio`, `--cachedio`) and the interaction with O_DIRECT fallback logic.
- Updated sync behaviour description to match the new byte-accurate scaling for
cached I/O.
- Updated PRNG section to remove the “future release” note for AES-CTR.
- Documented improved device exclusion with `/dev/disk/by-id/*` support.
- Updated seeding description to reflect the use of `getrandom()` instead of
`/dev/urandom`.
- Refreshed dependency lists and provided concise installation instructions for
multiple Linux distributions.
- Minor stylistic cleanup, clarification of SSD limitations, and improved README
structure for readability and accuracy.
After migrating nwipe to large aligned write buffers (multi-MB blocks), the
existing `sync` option unintentionally changed behaviour. The original logic
performed an fdatasync() every `sync * device_block_size` bytes, which for the
default `sync = 100000` resulted in ~50–400 MB between syncs.
With the new 4 MB I/O blocks, the same value produced syncs only every ~390 GB,
causing extremely delayed I/O error detection in cached I/O mode (errors appear
at fsync time, not on write). This was observed during testing on USB HDDs,
where no sync occurred even after several percent of the wipe.
This commit resolves the issue by:
- Restoring the original “bytes between syncs” behaviour.
The effective sync interval is recalculated based on the new large block size
so that fdatasync() again occurs every few hundred megabytes, not hundreds of
gigabytes.
- Disabling periodic sync entirely when direct I/O (`O_DIRECT`) is forced.
Direct I/O returns hardware errors at write() time, so syncs are unnecessary
and provide no safety benefit.
- Keeping cached I/O safe by ensuring timely detection of device failures,
stalled writeback caches, USB disconnects, and similar hardware conditions.
The large-block write path remains unchanged; only the scheduling of sync
operations is corrected to maintain practical error detection behaviour
consistent with the original nwipe design.
device ID (major/minor), allowing persistent identifiers in /dev/disk/by-id/
and /dev/disk/by-path/ to be used safely. Legacy string-based matching is
preserved.
This change extends the recently-added large-block, aligned I/O path with a
user-selectable I/O mode, allowing nwipe to choose between direct I/O
(O_DIRECT) and kernel cached I/O at runtime.
The goal is to:
- Make it easy to benchmark and compare cached vs. direct I/O.
- Provide an operational escape-hatch if Direct I/O causes issues on some
systems or devices.
- Keep a sensible default that "does the right thing" automatically.
Summary of changes
*options.h / options.c*
- Introduce a new enum `nwipe_io_mode_t`:
- `NWIPE_IO_MODE_AUTO` (default)
Try to use `O_DIRECT` for device access. If the kernel rejects
`O_DIRECT` with `EINVAL` or `EOPNOTSUPP`, nwipe automatically falls
back to cached I/O and logs a warning.
- `NWIPE_IO_MODE_DIRECT`
Force direct I/O. Devices are opened with `O_DIRECT` and there is
no fallback. If `O_DIRECT` is not supported for a device, it is
treated as a fatal condition and the device is marked disabled.
- `NWIPE_IO_MODE_CACHED`
Force kernel cached I/O. Devices are always opened without
`O_DIRECT` and no attempt is made to use direct I/O.
- Extend `nwipe_options_t` with a new field:
- `nwipe_io_mode_t io_mode;`
and initialize it to `NWIPE_IO_MODE_AUTO` in the default options setup so
that existing usage (no new flags) preserves the current behaviour.
- Add new command-line options to control the I/O mode:
- `--directio`
Sets `io_mode = NWIPE_IO_MODE_DIRECT`. This explicitly requests
`O_DIRECT` and disables the auto-fallback.
- `--cachedio`
Sets `io_mode = NWIPE_IO_MODE_CACHED`. This disables `O_DIRECT`
completely and forces classic cached I/O.
- `--io-mode=MODE`
Accepts `auto`, `direct`, or `cached` and sets `io_mode`
accordingly. Any other value results in a clear error message:
`Error: Unknown I/O mode 'X' (expected auto|direct|cached).`
- Integrate the new options into the existing long-option parsing logic
in `options.c` (`case 0:`), ensuring that:
- `--directio` and `--cachedio` are handled alongside other long
options (method, prng, verify, etc.).
- The previous unconditional `exit(EINVAL)` placeholder at the end
of `case 0` is moved to the end of the chain so that known options
(`directio`, `cachedio`, `io-mode`) are parsed correctly.
- Unknown long options still terminate with a clear error instead
of silently being ignored.
- Update `display_help()` to document the new flags:
- `--directio` Force direct I/O (O_DIRECT); fail if not supported
- `--cachedio` Force kernel cached I/O; never attempt O_DIRECT
- `--io-mode=MODE` I/O mode: auto (default), direct, cached
*nwipe.c*
- Update the device open path to honour `nwipe_options.io_mode` whenever
`NWIPE_USE_DIRECT_IO` is enabled at build time:
- Compute `open_flags` starting from `O_RDWR`.
- In `AUTO` and `DIRECT` modes, append `O_DIRECT` to `open_flags`.
- In `CACHED` mode, never add `O_DIRECT` (pure cached I/O).
- Implement mode-specific handling for `O_DIRECT` failures:
- In `AUTO` mode:
If `open()` fails with `EINVAL` or `EOPNOTSUPP`, log a warning and
retry without `O_DIRECT` (cached I/O). This preserves the previous
behaviour of “try direct I/O if available, but keep working if it
isn’t.”
- In `DIRECT` mode:
If `open()` fails with `EINVAL` or `EOPNOTSUPP`, treat this as a
fatal condition for that device. We log a clear error stating that
`O_DIRECT` was explicitly requested via `--directio` but is not
supported, mark the device as disabled, and do not silently fall
back.
- In `CACHED` mode:
Devices are always opened without `O_DIRECT`; no additional logic
is required, and behaviour matches classic buffered I/O.
- Add informational logging for the chosen I/O mode per device:
- On successful open, nwipe logs whether it is using:
- `"Using direct I/O (O_DIRECT) on device '...'.` or
- `"Using cached I/O on device '...'."`
This helps benching and debugging by making the actual mode
visible in the logs.
- For portability, ensure that builds on non-Linux / non-glibc systems
remain possible by defining `O_DIRECT` as `0` if it is not provided
by the system headers and `NWIPE_USE_DIRECT_IO` is set. This turns
`O_DIRECT` into a no-op flag on such platforms while keeping the API
intact.
Behavioural impact
- The actual wipe patterns, verification behaviour, and large-block
aligned I/O path remain unchanged. The new I/O mode only controls
how devices are opened (with or without `O_DIRECT`) and how we react
if direct I/O is not supported by the kernel or underlying filesystem.
- Default behaviour (`AUTO`) continues to “do the right thing”:
try direct I/O where available and fall back to kernel cached I/O
otherwise, with a clear log message.
- Advanced users and testers now have fine-grained control:
- `--directio` / `--io-mode=direct` for hard-fail direct I/O,
- `--cachedio` / `--io-mode=cached` to force buffered I/O,
- `--io-mode=auto` (or no flag) for the previous automatic behaviour.
- Combined with the existing large I/O buffers and aligned allocations
in `pass.c`, all three modes share the same fast, O_DIRECT-safe I/O
implementation. The new options simply toggle whether direct I/O is
requested and how strictly that requirement is enforced, which is
particularly useful for benchmarking and for diagnosing any potential
Direct I/O issues in the field.
You can now specify --pdftag to enable the
display of system UUID and system serial
number information on the PDF report.
Nwipe defaults to not displaying system IDs
but some users like to record the system UUID
or serial number on the Erasure Report along
with the disk information.
This change reworks the pass/verify I/O path and adds optional direct I/O
support to reduce syscall overhead and better utilize modern storage
performance.
pass.c:
- Introduce NWIPE_BUFFER_SIZE (default 16 MiB) as a generic scratch buffer size
and NWIPE_IO_BLOCKSIZE (default 4 MiB) as the target read/write block size.
- Add nwipe_effective_io_blocksize() to compute an effective I/O block size
per device:
- At least device_stat.st_blksize
- Rounded down to a multiple of st_blksize for O_DIRECT compatibility
- Never larger than the device size
- Add nwipe_alloc_io_buffer(), which allocates I/O buffers using
posix_memalign() aligned to the device block size (>= 512 B). This makes the
same code safe for both buffered I/O and O_DIRECT.
- Rework nwipe_random_pass():
- Use a large, aligned scratch buffer (default 16 MiB) instead of tiny
st_blksize-sized buffers.
- Generate and write data in large chunks (default 4 MiB) to drastically
reduce the number of write() syscalls.
- Keep the original PRNG init/read interface and the “PRNG wrote something”
sanity check (still checks within the first st_blksize bytes).
- Preserve existing error handling, progress accounting and periodic
fdatasync() logic.
- Rework nwipe_random_verify():
- Use the same large I/O block logic for read/compare.
- Generate the expected random stream in large blocks and compare against
data read from the device.
- Maintain the original semantics for partial reads and error counters.
- Rework nwipe_static_pass() and nwipe_static_verify():
- Build large pattern buffers that repeat the user-specified pattern and
support a sliding window (w) into the pattern.
- Perform writes/reads in large blocks (default 4 MiB) while keeping the
pattern alignment consistent via the window offset.
- Preserve original behaviour regarding partial I/O, logging and counters.
nwipe.c:
- Add support for optional direct I/O when NWIPE_USE_DIRECT_IO is defined:
- Include <fcntl.h> and ensure O_DIRECT is available (fallback to 0 on
platforms that do not define it).
- Open devices with O_RDWR|O_DIRECT, and transparently fall back to O_RDWR
if O_DIRECT is not supported (e.g. EINVAL/EOPNOTSUPP).
- Enable GNU extensions (e.g. _GNU_SOURCE) so that O_DIRECT is visible on
glibc-based systems.
Behavioural impact:
- The wiping/verification algorithms and patterns are unchanged; only the I/O
strategy is modified to use larger, aligned buffers.
- The number of read()/write() syscalls per pass is reduced by orders of
magnitude (e.g. 4 MiB vs. 4 KiB), which should significantly increase
throughput on fast disks/NVMe.
- When NWIPE_USE_DIRECT_IO is enabled and supported by the device, the same
code path uses direct I/O to avoid unnecessary page cache pollution; when
unsupported, behaviour gracefully falls back to buffered I/O.
This fixes an issue where a default blanking pass was added to methods which do not support it when in --nogui mode.
Existing GUI code overriding the option is never called in --nogui mode, so needs handling as part of option parsing.
Signed-off-by: desertwitch <24509509+desertwitch@users.noreply.github.com>
The internal PRNGs are now seeded via getrandom(2) rather than through a
long-lived file descriptor to /dev/urandom.
Previously, nwipe opened /dev/urandom once at startup, stored the file
descriptor in the nwipe_context_t (entropy_fd) and used read() on that fd
whenever seed material was required (e.g. DoD, Gutmann, OPS-II and generic
random passes). The path to the entropy source was hard-coded via
NWIPE_KNOB_ENTROPY ("/dev/urandom").
This change introduces a small helper in method.c:
- nwipe_read_entropy(buf, len)
which calls the getrandom(2) syscall in a loop until the requested number
of bytes has been filled, handling short reads and EINTR/EAGAIN. All
former uses of read(ctx->entropy_fd, ...) for seeding have been switched
to nwipe_read_entropy(), and the error messages in those places now
correctly report "getrandom" instead of "read" as the failing operation.
The nwipe_context_t no longer carries an entropy_fd and nwipe.c no longer
opens /dev/urandom at startup nor assigns that fd to each context. The
NWIPE_KNOB_ENTROPY macro and its only use site were removed. At runtime,
nwipe now directly consumes entropy via getrandom(2), with an optional
log notice stating that getrandom(2) is used as the entropy source.
Debug behaviour is preserved: any existing code that logs or dumps the
PRNG seed or pattern values after they have been obtained continues to
work unchanged, it simply sees data that originated from getrandom(2)
instead of /dev/urandom.
Rationale for preferring getrandom(2) over /dev/urandom via fd:
- getrandom(2) is a dedicated kernel API for CSPRNG output; it does not
depend on any device node or path existing in /dev. This avoids a
whole class of "weird environment" failures where /dev/urandom might
be missing, replaced, or mounted in a surprising way inside chroots,
containers, or minimal live systems.
- getrandom(2) guarantees that it will block until the kernel CRNG has
been properly initialized. Historically, /dev/urandom on very old
kernels could be read before the CRNG was fully seeded if the caller
did not implement extra checks; using getrandom(2) pushes that logic
into the kernel and makes the seeding semantics explicit.
- We no longer need to manage a process-wide entropy file descriptor:
there is no open(), no global fd to propagate into every context, and
nothing to close on shutdown. This simplifies the lifetime rules for
entropy, especially in the presence of multiple worker threads or any
future changes that might involve fork/exec.
- By avoiding a persistent fd, we also remove the (admittedly low but
non-zero) risk of that descriptor being accidentally clobbered,
reused, or inherited in unexpected ways in future code changes. Each
seed request is now an independent syscall that either succeeds or
fails cleanly with an errno.
- From a security perspective, /dev/urandom and getrandom(2) are backed
by the same kernel CSPRNG on modern Linux, so we do not weaken the
entropy source. Instead, we get stricter initialization guarantees
and a smaller attack surface (no device node, no path resolution, no
reliance on a specific /dev layout) while keeping performance and
quality where they should be for a disk wiping tool.
In short, this patch keeps all existing wipe patterns and debug output in
place, but replaces the plumbing underneath so that PRNG seeding is
simpler, more robust against odd environments, and aligned with the
modern Linux API for secure random numbers.
The fifteen lines of code that creates the header
and footer text in the PDF appear in two separate
places. The first occurrence in the create_pdf(..)
function and once in the create_header_and_footer(..)
function.
This duplicated code was combined into a third
function pdf_header_footer_text(..) and is now called
from the other functions.
This was done as I need to add some user selectable
changes to the header text that will include host
identification such as system tag, UUID, hostname
without creating further duplicated code.
The purpose of this commit is to add an additional
identifying piece of information to the pdf filename.
This was found to be necessary in the case of a user
wiping partitions as opposed to the whole disc. Currently
when wiping a partition the model name and serial number
is missing from the pdf content and pdf filename so by adding
the device name, it make it less likely that an existing pdf
will get overwritten. This is a stop gap fix as preferably
the disk model and serial no needs to be retrieved even
wiping just one partition.
Additional functions were added including retrieval of UUID,
however UUID was found to not be available for some USB
devices when wiping partitions. The UUID function remains
in the code and the UUID if available is output to the log
but is not used anywhere else at the moment.
Replaced the old memmove-based stash buffer with a true circular (ring) buffer
for the thread-local AES-CTR PRNG prefetch mechanism Increased Buffers to 1M stash and 128 KiB block.
Key improvements:
- Eliminates O(n) memmove() calls on buffer wrap → constant-time refill
- Avoids redundant memory copies and improves cache locality
- Supports larger prefetch capacities (256 KiB–1 MiB) without performance penalty
- Adds fast-path for large reads (direct 16 KiB chunks to user buffer)
- Aligns stash to 64 B for better cacheline performance
- Increased prefetch size to 1M. Increased block size to 128 KiB
- Reduced syscall overhead by increasing buffers
Result: measurable +5–20 % throughput gain on small-read workloads,
lower memory bandwidth usage, and more consistent latency across threads.
