This fixes a issue related to konsole and terminals based on Konsole,
like cool retro terminal. If you exit the terminal before exiting nwipe,
nwipe will then continue running in the background but detached from any
terminals input/output. This causes a call to halfdelay()/getch() to
return immediately, thus removing the delay and causing the thread to
run at full speed causing 100% CPU in a core. This requires nwipe to
then be killed manually. This patch is related to the last patch in
that both patches do much the same thing but the previous patch fixes
the problem when nwipe is sitting at the drive selection screen, while
this patch fixes the problem during a wipe.
This problem would only be seen in specific KDE konsole related
terminals and only if you do not exit nwipe by using nwipe's control c
to abort or use the space bar on completion of the wipe.
Much like the same check we perform in the nwipe_gui_select() function,
here we check that we are not looping any faster than as defined by the
halfdelay() function, typically this loop runs at 10 times a second.
This check makes sure that if the loop runs faster than double this
value i.e 20 times a second then the program exits. This check is
therefore determining whether the getch() function is returning
immediately rather than blocking for the defined period of 100ms.
Why is this necessary? Some terminals (konsole & deriviatives) that are
exited while nwipe is still running fail to terminate nwipe this causes
the halfdelay()/getch() functions to immediately fail causing the loop
frequency to drastically increase. We detect that speed increase here
and therefore close down nwipe. This doesn't affect the use of the tmux
terminal by which you can detach and reattach to running nwipe
processes. The tmux terminal will still work correctly when a nwipe
session is detached.
when Konsole terminal exited while nwipe is sitting at the
drive selection screen.
To avoid 100% CPU usage, check for a runaway condition caused by the
function "keystroke = getch() that immediately returns an error
condition. We check for an error condition because getch() returns a
ERR value when the timeout value "timeout( 250 );" expires as well as
when a real error occurs. We can't differentiate from normal operation
and a failure of the getch function to block for the specified period
of timeout. So here we check the while loop hasn't exceeded the number
of expected iterations per second ie. a timeout(250) block value of
250ms means we should not see any more than (1000/250) = 4 iterations.
We double this to 8 to allow a little tolerance. Why is this necessary?
It's been found that in KDE konsole and other terminals based on the QT
terminal engine exiting the terminal without first existing nwipe
results in nwipe remaining running but detached from any interface
which causes getch to fail and its associated timeout. So the CPU or
CPU core rises to 100%. Here we detect that failure and exit nwipe
gracefully with the appropriate error. This does not affect use of
tmux for attaching or detaching from a running nwipe session when
sitting at the selection screen. All other terminals correctly
terminate nwipe when the terminal itself is exited.
This fixes a problem where the drive temperature is not updated
automatically in the drive selection window only. The temperature
is however updated every correctly every 60 seconds during a wipe in
the wipe status window.
This bug would probably never be noticed by most people as usually the
drive temperature changes slowly and only rises once a wipe has started.
The only time I imagine it would have been noticed would have been if
the drive temperature was already high and you were trying to reduce the
temperature by cooling before starting a wipe.
This has now been corrected so that the temperature in the drive
selection window is updated every 60 seconds.
This patch fixes a minor display issue that occurs when
a user aborts a wipe before a wipe has started. It only occurs
if the user had selected one or more drives for wipe and then
aborted before starting the wipe. The spurious message only
occurs in a virtual terminal, i.e. /dev/tty1, /dev/tty2, /dev/console
It does not occur in terminal applications such as konsole, xterm,
terminator etc.
The spurious message that appears in the main window, states that
"/dev/sdxyz 100% complete" along with garbage values in the statistics
window. The message appears for a fraction of a second before being
replaced with the textual log information that correctly states that
the user aborted and no wipe was started.
Basically the gui status information update function tries to update
the data when the wipe hasn't even started. The fix is to only update
the statistics information only if a wipe has started by checking the
'global_wipe_status' value which indicates whether any wipe started. '1'
indicates that a wipe has started, else '0' if no wipe has started.
If the drive becomes non responsive during the wipe, the MB/s will
slowly drop towards 0MB/s and will display a FAILURE -1 error. The logs
will display errors and nwipe's return status will be non zero, however
the summary table may display erased rather than FAILURE, this is because
the wipe thread exited prematurely without setting the pass error.
This fixes the error by checking the context's result status, i.e non zero
on failure and if pass equals zero it makes pass equal to one. This is
then picked up by the summary table log code which then marks the status
correctly as FAILURE in the summary table.
This was caused by inconsistent labeling of the
serial number by smartctl. In a majority of cases
smartctl would output serial number data using
the label "Serial Number:" however on a SAS
drive we found that smartctl output the label
as "Serial number:" i.e. differs with a lower
case 'n'. Unfortunately we were doing a case
sensitive search for "Serial Number" so the result
being the serial number was not found.
This patch converts both strings in the search
to lower case before searching.
In addition a new field was added to the
anonymize list, "logical unit id:" so when the
-q option is used "serial number", "lu wwn device id:"
and "logical unit id:" are all now anonymized in the
smartctl debug data.
If pthread_join failed, nwipe would report an error
but would also report that the thread had been
cancelled. This has been corrected so that only
the error message is displayed in a fault condition.
The format specifier didn't match the size of the variable that holds
c2[i]->device_sector_size which is an int. This didn't appear to cause
a problem with reporting in a 64 bit build. It does cause a problem in
a 32 bit build displaying a very large & incorrect number for sector,
block and device sizes.
This doesn't cause any issues with the overall function, simply
incorrect sector, block and device sizes in 32 bit builds as displayed
in the nwipe log.
I also changed the signed long long for c2[i]->device _size to a
unsigned long long as there is little point in a negative device size.
Add /sys/class/hwmon/hwmonX/device/ to the list
of directories to search for a given device.
Some nvme devices/controllers put the device name
in /sys/class/hwmon/hwmonX/device/ rather than
/sys/class/hwmon/hwmonX/device/nvme/nvme0/
Update debug messaging.
For sdX devices we look in
/sys/class/hwmon/hwmonX/block for the device
name.
However, for nvme we access the device name
by looking in ..
/sys/class/hwmon/hwmonX/device/nvme/nvme0
nvme0n1
Multiple nvme drives/controllers will need
to be tested.
If a faulty drive fails mid wipe and it's
throughput drops until eventually reaching
zero. The ETA calculation grows to an enormously
high value.
This patch prevents the ETA being calculated if
the throughput for a given drive drops below
100,000 bytes per second. In this way we can still
see that something is wrong because the ETA is much
higher than normal but prevents the sort of calculation
that looks like this ! 90867213:29:12 i.e ..
90,867,213 hours, 29 minutes and 12 seconds, or put
another way, 3,786,133 days or 10,372 years.
When many verification or pass errors are detected
the status line can wrap on a 80 column display.
This patch makes the error message more succinct
which will free up about 10 characters & prevents
the line wrapping.
Anonymize the serial numbers in the gui, the
log and the summary table.
If a serial number was obtained from the device,
it is replaced with "XXXXXXXXXXXXXXX".
If the serial number could not be obtained from the
device, it's replaced with "???????????????".
In both selection and wipe status windows I
moved the temperature readout to the right
of the disk size column.
Also in the wipe status window, by moving to
the right of the disk size column the
temperature is no longer on the same line as
the disk pass progress. This reduces the line
length as I found that in tests if you had
many verification errors the temperature
being on the end of the line would disappear
of the right of the screen.
Also removed 1. 2. etc from drive selection
to reduce the line length. Also removed the
space between > and [, ie "> [wipe]" becomes
">[wipe]" These changes remove 3 characters
and help to reduce the affect of the additional
temperature field [30C] which add 5 characters.
Therefore the line length overall, increased by
5-3 = 2 characters.
This helps to reduce line wrapping on 80
character terminals, when the drive model length
exceeds 24 characters.
For some controllers/drivers the readlink method of
obtaining the bus type for GUI display does not work.
If we haven't already resolved the bus type, we then
also check smartctl for the transport protocol
for SAS.
Using the 'b' key you can now toggle between
dark/blank/default screens during a wipe.
Added the dark option to prevent TFT/LCD image
persistence.
See:
https://en.wikipedia.org/wiki/Image_persistence
1. Allows zero fill to be verified when blanking off.
Prior to this if you wanted to verify a zero pass,
blanking had to be on. This meant a zero pass, then a
blanking pass then a verify, effectively two zero passes
and a verify. This is now fixed so you can now do a zero
pass with verification without a blanking pass. This knocks
a third off the wipe time of a zero fill with verification.
This also means all other methods can have either all their
passes or just the last pass verified without blanking being
enabled.
2. OPS2 method requires the last pass to be random, the GUI
now disables the use of the blanking option for this message
and displays a warning message that a final blanking pass is
not allowed for OPS2. It never did a final blanking pass
anyway, even if it was selected, but this makes it clearer in
the GUI.
3. The caculate_round_size() function was improved by reducing
some duplicated code and bring the full calculation into this function.
4. On completion of each pass or verification the total number of
bytes written or read for each pass or verification is logged.
Although the blanking pass verification is
completed correctly the GUI status message
did not show the status as 'Verifying', the
status message continued to say blanking.
This patch fixes the GUI message so if
verification and blanking pass are enabled,
then when blanking is complete and verification
starts the drive status message changes from
'blanking' to 'Verifying'.
Since at least 2013 (the initial nwipe commit),
isaac has never functioned. When the issac prng
was selected in the GUI, nwipe used the mersenne
twister prng instead. Not that you would ever
have known, as there were no log entries saying which
prng was being actively used.
However, I don't believe this was just an nwipe
issue, looking at the code for DBAN's dwipe the
same function nwipe_isaac_read( NWIPE_PRNG_READ_SIGNATURE )
exists as it does in nwipe. In both cases the function
has no code that actually does anything.
This patch populates this function and brings isaac
back to life !
This bug was also responsible for verification errors
when the option prng=isaac was used on the command
line. Worse still, if you used prng=isaac on the
command line then wiped using method=prng, no verification
and no blanking you would expect to see random data. You
don't, instead you would see either all zeros or mainly
zeros because the uninitialised buffer that should have
contained random data instead contained initialised text
data such as partial log entries. This patch and previously
submitted patches fix all these problems related to the
isaac implementation.
A separate commit will fix the GUI prng selection which
was leading everybody to believe isaac was being used
when in fact it was mersenne all along.
Additional log messages are produced and a failed prng
causes a wipe failure. The buffer that outputs prng data
to the disk is now initialised with zeros (calloc rather than malloc)
to avoid uninitialised memory leakage to the disk in the event of some
unforeseen bug. This initialised buffer is also required for the check
process.
Why do this check? At a future date more prng methods may be added. This
is a very basic check that they produce some output although the output
is not verified in terms of it's randomness. This check was also
implemented to show an existing bug in the Isaac implementation in nwipe.
See the example log below that shows a failed prng. This would be caused
by a bug in the prng implementation. See the last few messages after
pass 3/3 starts. 3/3 in DoD short is the prng pass.
[2021/05/29 20:30:27] notice: Invoking method 'DoD Short' on /dev/loop29
[2021/05/29 20:30:27] notice: Starting round 1 of 1 on /dev/loop29
[2021/05/29 20:30:27] notice: Starting pass 1/3, round 1/1, on /dev/loop29
[2021/05/29 20:30:39] notice: 1073741824 bytes written to /dev/loop29
[2021/05/29 20:30:39] notice: Finished pass 1/3, round 1/1, on /dev/loop29
[2021/05/29 20:30:39] notice: Starting pass 2/3, round 1/1, on /dev/loop29
[2021/05/29 20:30:58] notice: 1073741824 bytes written to /dev/loop29
[2021/05/29 20:30:58] notice: Finished pass 2/3, round 1/1, on /dev/loop29
[2021/05/29 20:30:58] notice: Starting pass 3/3, round 1/1, on /dev/loop29
[2021/05/29 20:30:58] notice: Initialising Isaac prng
[2021/05/29 20:30:58] fatal: ERROR, prng wrote nothing to the buffer
[2021/05/29 20:30:58] notice: 0 bytes written to /dev/loop29
[2021/05/29 20:31:03] error: Nwipe exited with fatal errors on device = /dev/loop29
********************************************************************************
! Device | Status | Thru-put | HH:MM:SS | Model/Serial Number
--------------------------------------------------------------------------------
! loop29 |-FAILED-| 69 MB/s | 00:00:31 | Loopback device/
--------------------------------------------------------------------------------
[2021/05/29 20:31:03] Total Throughput 69 MB/s, DoD Short, 1R+NB+NV
********************************************************************************
A message is also shown for a successful prng output. i.e "prng is active"
See example below.
[2021/05/29 20:04:30] notice: Invoking method 'DoD Short' on /dev/loop29
[2021/05/29 20:04:30] notice: Starting round 1 of 1 on /dev/loop29
[2021/05/29 20:04:30] notice: Starting pass 1/3, round 1/1, on /dev/loop29
[2021/05/29 20:04:44] notice: 1073741824 bytes written to /dev/loop29
[2021/05/29 20:04:44] notice: Finished pass 1/3, round 1/1, on /dev/loop29
[2021/05/29 20:04:44] notice: Starting pass 2/3, round 1/1, on /dev/loop29
[2021/05/29 20:04:59] notice: 1073741824 bytes written to /dev/loop29
[2021/05/29 20:04:59] notice: Finished pass 2/3, round 1/1, on /dev/loop29
[2021/05/29 20:04:59] notice: Starting pass 3/3, round 1/1, on /dev/loop29
[2021/05/29 20:04:59] notice: Initialising Mersenne Twister prng
[2021/05/29 20:04:59] notice: prng stream is active
[2021/05/29 20:05:25] notice: 1073741824 bytes written to /dev/loop29
1. The log reported verification errors while also
showing an entry in the log that said "[SUCCESS] Blanked /dev/...".
The blanked device message now shows [FAILURE] /dev/... may not be blanked"
if any verification errors are detected for a specific drive.
2. If a verification error occurred, the error would be correctly
shown in the GUI and in the logs but the summary table drive status
would show 'ERASED' not 'FAILED'. Now corrected so that the tables
drive status field shows 'FAILED' if any verification error is detected.
Prior to this it was marking the status as 'FAILED' only if the O.S
detected write errors. In practise most drive errors are detected
by the write I/O process on syncing but this will now detect errors not
recognised by the O.S. and found by the verification process. Despite
this the textual log and GUI correcty reported verification errors.
3. The final log message "Nwipe exited successfully" was checking for
fatal errors but ignoring non fatal errors despite being reported in
the log.
The final message now reads
either "Nwipe Exited Succesfully" if no fatal and non fatal errors were
detected. Alternatively it displays ...
"Nwipe exited with errors, check the log & summary table for individual drive status."
if any fatal OR non fatal errors are detected.
1. Prior to this fix, if a user aborted a multi drive using control-C,
and if some of the drives had completed their wipe either
successfully or with a I/O failures, the summary log would say
UABORTED for all drives. UABORTED is acceptable status for drives that
hadn't completed their wipe but for drives that had completed the wipe,
the summary table should have gave either a ERASED or FAILED status
message. This fix corrects the summary status when using control-c to
abort a multi drive wipe where some drives have completed and some have
not.
2. If I wipe was selected but not started and then the user used
control-C to abort nwipe, the summary table would possibly
intermittently be displayed showing the last drive wiped. The correct
behaviour is that no summary table is displayed if the wipe hasn't been
started yet. This fix corrects the behaviour of the summary table when
control-c is used and a drive has been selected for wiping but not
started, in this scenario the summary table is NOT displayed.
3. When nwipe exits it always prints the message "nwipe successfully
exited". To make this single line message more informative it now
reports a different message depending upon the wipe status.
These four messages are:
"Nwipe was aborted by the user. Check the summary table for the drive
status" - When the user aborts during a wipe.
"Nwipe was aborted by the user prior to the wipe starting."
"Nwipe exited with fatal errors, check the summary table for individual
drive status." - When one or more drives failed.
"Nwipe successfully completed. See summary table for details." - When
selected drives were all erased without error.
If you do a ctrl c to exit nwipe before you have
even started a wipe, it will be reported in the logs
that there are fatal or non fatal errors. This is
incorrect as no wipe has even been started.
This error is only relevant to when using ctrl c
before a wipe has started and not after a wipe has
started.
Basically the variables that hold the errors are
in an indeterminate state before wiping begins and
in the case of a user abort we mistakenly checked
those values and reported. They should only be used
after a thread has completed or failed when their
values are then valid.
Now if you exit nwipe before any wiping started
there are no fatal/non fatal errors reported.
