Categories

[solved] "find / -type f" processes with unknown origin cause High CPU and IO Loads - Webmin DNSSEC

Donate with

Donate BAT now

Verified BAT publisher

Webmin DNSSEC Bug: How resign.pl Cron Job Spawns Multiple "find / -type f" Processes Causing Extreme CPU and I/O Load

If you're running a Linux server with Webmin and BIND DNS with DNSSEC enabled, you may have encountered a mysterious performance issue: your server suddenly becomes sluggish, disk I/O spikes dramatically, and when you check your running processes, you find multiple instances of find / -type f running simultaneously. This article explains the root cause of this problem, which lies deep within Webmin's BIND8 module, and provides several solutions to fix it.

The Symptoms: Server Performance Degradation

The issue typically manifests as follows:

• Multiple find / -type f processes running as root
• Each process consuming significant CPU time (often 2-3% each)
• Processes running for hours or even days on systems with large filesystems
• High disk I/O load affecting other services
• Processes running with nice value of -19 (high priority)

When examining your process list using htop or top, you might see something alarming like this:

In the screenshot above, you can see multiple find / -type f processes that have been running for over 10 minutes each, consuming CPU resources and generating massive I/O load. On systems with terabytes of data, these processes can run for hours or even days.

Root Cause Analysis: The DNSSEC Entropy Generation Code

The culprit is located in Webmin's BIND8 module, specifically in the file bind8/bind8-lib.pl. This code was originally introduced in November 2008 as part of DNSSEC implementation to generate entropy for cryptographic key generation.

The Problematic Code

The issue exists in two functions within bind8-lib.pl:

1. create_dnssec_key() at line 3461-3467
2. resign_dnssec_key() at line 3592-3598

Both functions contain identical code blocks:

# Fork a background job to do lots of IO, to generate entropy
my $pid;
if (!&get_rand_flag()) {
    $pid = fork();
    if (!$pid) {
        exec("find / -type f >/dev/null 2>&1");
        exit(1);
    }
}

The logic depends on the get_rand_flag() function defined at lines 68-80:

# get_rand_flag()
# Return a flag to read from a randomness source
sub get_rand_flag
{
if ($gconfig{'os_type'} =~ /-linux$/ &&
    $config{'force_random'} eq '0' &&
    -r "/dev/urandom" &&
    &compare_version_numbers($bind_version, 9) >= 0 &&
    &compare_version_numbers($bind_version, '9.14.2') < 0) {
    # Version: 9.14.2 deprecated the use of -r option
    # in favor of using /dev/random [bugs:#5370]
    return "-r /dev/urandom";
    }
return "";
}

Understanding the Flawed Logic

The function get_rand_flag() returns the -r /dev/urandom flag only when ALL of the following conditions are met:

• Operating system is Linux
• Configuration option force_random is set to '0'
• /dev/urandom is readable
• BIND version is >= 9.0 AND < 9.14.2

If any condition fails, the function returns an empty string. Then, the entropy generation code checks:

if (!&get_rand_flag()) {   # In Perl: !"" = !false = TRUE
    exec("find / -type f >/dev/null 2>&1");  # EXECUTES!
}

In Perl, an empty string is "falsy", so !"" evaluates to TRUE, triggering the find command.

The Three-Case Problem: Why Modern BIND Is Affected

This is not a configuration issue - it is a logic bug in the code that affects ALL servers running BIND 9.14.2 or later with DNSSEC enabled.

The code was designed to handle two cases but there are actually three:

Real-World Example: BIND 9.16.50

Consider a typical modern Debian server running BIND 9.16.50-1~deb11u4. Here's exactly what happens:

$bind_version = "9.16" (parsed from "9.16.50-1~deb11u4")

get_rand_flag() evaluates:
├─ Linux?                              → TRUE  ✓
├─ force_random eq '0'?                → TRUE  ✓ (default setting)
├─ /dev/urandom readable?              → TRUE  ✓
├─ version >= 9?                       → TRUE  ✓ (9.16 >= 9)
└─ version < 9.14.2?                   → FALSE ✗ (9.16 is NOT less than 9.14.2)

Result: One condition fails → Returns "" (empty string)
        → !&get_rand_flag() = !"" = TRUE
        → find / -type f EXECUTES!

The critical flaw: For BIND versions 9.14.2 and later, the -r option was deprecated because BIND now handles entropy generation internally using /dev/random. However, the Webmin code incorrectly assumes that when it cannot use the -r flag, it must generate entropy manually by scanning the entire filesystem.

This assumption is fundamentally wrong. For modern BIND versions (9.14.2+), no external entropy generation is needed at all. The find / -type f command serves absolutely no purpose and only wastes system resources. BIND 9.14.2+ was specifically updated to use /dev/random internally, making external entropy generation obsolete.

Important: This means if you are running any modern Linux distribution with a recent BIND version (Debian 11+, Ubuntu 20.04+, RHEL 8+, etc.), you are likely affected by this bug if you have DNSSEC zones with automatic re-signing enabled in Webmin.

Why Multiple Processes Are Spawned

The problem is compounded when the DNSSEC re-signing cron job runs. The script bind8/resign.pl is designed to be executed periodically to re-sign DNSSEC zones before they expire:

my @zones = &list_zone_names();
foreach my $z (@zones) {
    # ... zone checks ...
    if ($old > $period) {
        # Too old .. signing
        my $err = &resign_dnssec_key($z);
        # ...
    }
}

The script iterates through all DNSSEC-enabled zones and calls resign_dnssec_key() for each zone that needs re-signing. Since each call to this function spawns a new find / -type f process, you end up with multiple concurrent processes if you have multiple zones requiring re-signing.

Historical Timeline: When Was This Introduced?

Based on Git history analysis:

• November 24, 2008: Initial introduction of the find / -type f entropy generation code (commit 749a49b2e)
• March 5, 2010: Added /dev/urandom flag support for Linux to avoid the find command (commit 893f838)
• January 3, 2013: Made urandom usage optional and conditional (commit e190aef)
• April 20, 2025: Refactored version checking into get_rand_flag() function (commit e87fc560c)

The code has existed for over 16 years, but the problem became more prevalent as users upgraded to BIND 9.14.2 or later, which deprecated the -r option and inadvertently triggered the fallback entropy generation code.

Affected Systems: Who Is Impacted?

This bug affects virtually all modern Linux servers running Webmin with DNSSEC. Specifically, you are affected if:

• Webmin version: 2.610 (current) and likely all previous versions with the BIND8/DNS module
• BIND version: 9.14.2 or later (released May 2019) - this includes virtually all current Linux distributions:
  • Debian 10+ (ships with BIND 9.16+)
  • Ubuntu 20.04+ (ships with BIND 9.16+)
  • RHEL/CentOS/Rocky 8+ (ships with BIND 9.11+, some have 9.16+)
  • Any distribution with BIND updated in the last 5+ years
• DNSSEC: Any zones with DNSSEC enabled and automatic re-signing configured

Key Point: The severity of the impact scales with your filesystem size. On servers with terabytes of data, each find / -type f process can run for hours or even days. If you have multiple DNSSEC zones, multiple processes spawn simultaneously, compounding the problem.

Solutions and Workarounds

Solution 1: Patch the Source Code (Recommended)

The most effective solution is to modify the bind8-lib.pl file to skip the entropy generation for BIND versions 9.14.2 and later. Here's the proposed fix:

Modify the get_rand_flag() function to also indicate when entropy generation should be skipped entirely:

# get_rand_flag()
# Return a flag to read from a randomness source, or undef if no entropy generation needed
sub get_rand_flag
{
# BIND 9.14.2+ uses /dev/random internally, no entropy generation needed
if (&compare_version_numbers($bind_version, '9.14.2') >= 0) {
    return undef;  # Signal that no entropy generation is needed
    }

if ($gconfig{'os_type'} =~ /-linux$/ &&
    $config{'force_random'} eq '0' &&
    -r "/dev/urandom" &&
    &compare_version_numbers($bind_version, 9) >= 0) {
    return "-r /dev/urandom";
    }
return "";
}

# Helper function to check if entropy generation is needed
sub needs_entropy_generation
{
my $flag = &get_rand_flag();
return 0 if (!defined($flag));  # Modern BIND, no entropy needed
return 0 if ($flag ne "");       # Can use -r flag
return 1;                        # Need manual entropy generation
}

Then modify the entropy generation blocks in both create_dnssec_key() and resign_dnssec_key():

# Fork a background job to do lots of IO, to generate entropy
my $pid;
if (&needs_entropy_generation()) {
    $pid = fork();
    if (!$pid) {
        exec("find / -type f >/dev/null 2>&1");
        exit(1);
    }
}

Solution 2: Minimal Fix (Alternative)

If you prefer a smaller change without adding a new helper function, simply skip entropy generation for BIND 9.14.2+ by adding an additional version check at lines 3461 and 3592:

# Fork a background job to do lots of IO, to generate entropy
my $pid;
if (!&get_rand_flag() &&
    &compare_version_numbers($bind_version, '9.14.2') < 0) { $pid = fork(); if (!$pid) { exec("find / -type f >/dev/null 2>&1");
        exit(1);
        }
    }

This adds a second condition: only run the find command if get_rand_flag() returns empty AND the BIND version is less than 9.14.2. For modern BIND versions, the second condition fails, and the unnecessary entropy generation is skipped.

Solution 3: Quick Fix - Comment Out the Code

If you need an immediate fix and are running BIND 9.14.2 or later, you can simply comment out the entropy generation blocks:

# Fork a background job to do lots of IO, to generate entropy
# DISABLED: Not needed for BIND 9.14.2+ which uses /dev/random internally
# my $pid;
# if (!&get_rand_flag()) {
#     $pid = fork();
#     if (!$pid) {
#         exec("find / -type f >/dev/null 2>&1");
#         exit(1);
#     }
# }

Apply this change at both locations: lines 3459-3467 and lines 3590-3598 in bind8-lib.pl.

Solution 4: Disable the Cron Job

If you don't need automatic DNSSEC re-signing, you can disable the cron job:

# Find and remove the resign.pl cron job
crontab -l | grep -v "resign.pl" | crontab -

# Or check for system cron entries
ls -la /etc/cron.d/ | grep webmin
ls -la /etc/cron.daily/ | grep webmin

You can also disable it through the Webmin interface: Navigate to Servers → BIND DNS Server → Module Config → DNSSEC Settings and disable automatic re-signing.

Solution 5: Set force_random Configuration

As a workaround, you can try setting the force_random configuration option in the BIND module configuration. However, this may not work for all BIND versions and is not a recommended long-term solution.

Comparison: Entropy Generation Methods

How to Verify You're Affected

Run the following commands to check if your system is experiencing this issue:

# Check for running find processes
ps aux | grep "find / -type f"

# Check BIND version
named -v

# Check if resign.pl cron job exists
grep -r "resign.pl" /etc/cron* /var/spool/cron/

# Check Webmin BIND module logs
tail -f /var/webmin/miniserv.log | grep -i dnssec

Prevention: Best Practices for DNSSEC in Webmin

1. Apply the Patch Immediately: If you're running BIND 9.14.2+ (which is virtually all modern systems), apply one of the fixes described above as soon as possible
2. Monitor for Long-Running Processes: Set up monitoring to alert on find processes running longer than expected
3. Check After Webmin Updates: Be aware that Webmin updates may overwrite your patched bind8-lib.pl file - verify the fix is still in place after updates
4. Review Re-signing Frequency: If you have many DNSSEC zones, stagger their re-signing to avoid multiple simultaneous triggers
5. Consider Alternatives: For high-performance or large-storage environments, consider using BIND's native DNSSEC tools (like dnssec-keygen and dnssec-signzone directly) instead of Webmin's interface
6. Subscribe to Webmin Updates: Watch the Webmin GitHub repository for an official fix to this issue

Report This Issue

This bug has been reported to the Webmin development team. If you're experiencing this issue, please add your feedback to help prioritize a fix:

View GitHub Issue Report

Conclusion

The Webmin DNSSEC find / -type f bug is a logic flaw that affects all modern Linux servers running Webmin with DNSSEC and BIND 9.14.2 or later. The code correctly identifies that the -r /dev/urandom flag cannot be used with modern BIND, but incorrectly falls back to manual entropy generation via filesystem scanning - which is completely unnecessary since BIND 9.14.2+ handles entropy internally.

This is not a configuration issue or edge case - it is a bug that will trigger on any properly configured modern system. The entropy generation method using find / -type f was a reasonable approach in 2008 when it was introduced, but the code was never updated to recognize that BIND 9.14.2+ eliminated the need for external entropy generation entirely.

If you're experiencing severe performance degradation on your Linux server with multiple find / -type f processes running, you are almost certainly affected by this bug. The recommended approach is to patch the source code to properly detect modern BIND versions (9.14.2+) and skip the unnecessary entropy generation entirely.

Until an official fix is released by the Webmin team, server administrators should immediately apply one of the solutions outlined in this article, especially on systems with large filesystems or multiple DNSSEC-enabled zones where the impact can be catastrophic to server performance.

References and Further Reading

• Webmin GitHub Repository
• BIND 9 DNSSEC Guide
• ISC Knowledge Base: DNSSEC Key Generation