Menü schliessen
Created: December 19th 2019
Last updated: November 7th 2023
Categories: Linux
Author: Marcus Fleuti

[SOLVED] Postfix Mail Bounce Statistics Script - Receive hourly bounced mail statistics

Tags:  bounce,  bounced,  E-Mail,  hourly statistics,  mail bounce,  Postfix,  statistic
Donation Section: Background
Monero Badge: QR-Code
Monero Badge: Logo Icon Donate with Monero Badge: Logo Text
82uymVXLkvVbB4c4JpTd1tYm1yj1cKPKR2wqmw3XF8YXKTmY7JrTriP4pVwp2EJYBnCFdXhLq4zfFA6ic7VAWCFX5wfQbCC

Postfix bounce E-Mail statistics

What's the purpose?

We wanted to receive notifications from Postfix about bounced e-mails send by our customers. The reason is that larger amounts of e-mail bounces may be an indicator of a hacked account or a user abusing the system. We did not find a tool or script which can handle this simple task out of the box. Thus we wrote our own neat little program to do that.

How does it work?

Simple:

  1. The script needs to run hourly as a cronjob
  2. It will fetch a list of all bounced entries in the postfix mail log which are from the past hour
  3. By using REGEX we are filtering out the relevant information about each bounce within a LOOP, structure everything and create a table-based HTML e-mail
  4. The mail is being sent through regular mailx command (mail)

Optimization potential

  1. Adding a limit which would trigger the mail sending after a certain amount of bounces / hour only. Right now we want to see all bounces and rather use e-mail rules to sort out the e-mails (but be informed about everything what's going on).

The Subject is not being displayed

In case you don't see any subject most probably Postfix does not write the Subject into the mail.log file. This is how you can enable the Subject function in Postfix:

  1. Create a PCRE file (or add to an existing): nano /etc/postfix/header_checks.pcre
  2. Add the following line to it:
    /^Subject:/ WARN
  3. Save the file (CTRL + x , confirm with "y")
  4. Edit the Postfix main.cf configuration file: nano /etc/postfix/main.cf
  5. Add the following line in that file (or make sure the existing file is listed like this):
    header_checks = pcre:/etc/postfix/header_checks.pcre
  6. Initialize the header_checks.pcre file: postmap /etc/postfix/header_checks.pcre
  7. Restart Postfix: /etc/init.d/postfix restart (or service postfix restart)

The script

Copy paste it from below or download it here.

#!/bin/bash
export LC_ALL=en_US.utf8

MAILLOG=/var/log/mail.log
LOGMAILFROM="Your sender name<websupport@yourdomain.tld>"       ## Set the FROM for the e-mail. You can write "Bounce Mail Report<youremail@address.tld>"
LOGMAILTO="your@yourdomain.tld"                                 ## Set the TO for the e-mail (single e-mail address).

# Set the amount of hours you want the system to check your logs for bounce messages. 
# Example: If it's 18:13 hours now and you enter 6 in the variable below, the system will check for all bounce messages which occured between 12:00-13:00, 13:00-14:00, 14:00-15:00, 15:00-16:00, 16:00-17:00 and 17:00-18:00
AMOUNTOFHOURSTOCHECK=6

TIME_START=$(date +"%s")

# Initialize the regex pattern which is used to fetch all bounce messages from the past X hours
BOUNCEMESSAGERGXPATTERN=""

# Loop through the last 6 hours and build the regex pattern
for (( i = ${AMOUNTOFHOURSTOCHECK}; i > 0; i-- )); do
    hour=$(date -d "-${i} hour" '+%b %e %H')
    # Append to the pattern, separated by the OR operator |
    BOUNCEMESSAGERGXPATTERN+="(${hour}.*postfix/smtp.*status=bounced)|"
done

# Remove the trailing '|'
BOUNCEMESSAGERGXPATTERN=${BOUNCEMESSAGERGXPATTERN%|}

# Fetch all bounce messages from the mail log and store the result in a variable for later use
ALLBOUNCES=`cat ${MAILLOG} | egrep "$BOUNCEMESSAGERGXPATTERN"`

COUNTBOUNCES=$( [ -n "$ALLBOUNCES" ] && echo "$ALLBOUNCES" | wc -l || echo 0 )

if [ ${COUNTBOUNCES} -gt 0 ]; then
        MAILINFO='<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><html><head><title></title>'
        MAILINFO+='<style>'
        MAILINFO+='.mainTable { border-collapse: collapse; } .mainTable th, .mainTable td { border:1px dotted #cccccc; text-align:left; vertical-align:top; padding:5px 10px; } .mainTable th { border-bottom: 2px solid #cccccc; }'
        MAILINFO+='.additionalTable { border-collapse: collapse; } .additionalTable td { border: 0px; text-align: left; vertical-align: top; padding: 5px 10px; }'
        MAILINFO+='</style>'
        MAILINFO+='</head><body><table class="mainTable">'
        MAILINFO+="<tr><th>DATE & TIME</th><th>MAIL ID</th><th>CLIENT PTR</th><th>CLIENT IP</th><th>USERNAME</th><th>MAIL FROM</th><th>MAIL TO</th><th>HOST</th><th>HOST IP</th><th>REASON</th><th>SUBJECT</th></tr>"

        while IFS= read -r BOUNCE
                do
                ## Clean the bounce variable (remove all line-breaks)
                BOUNCE="${BOUNCE//$'\n'/ }"

                ## Get the mail ID from the current log line
                MAILID=$(perl -pe "s/.*?postfix\/smtp\[\w+\]:\s(\w+).*/\1/g" <<< ${BOUNCE})
                MAILTO=$(perl -pe "s/.*to=<(.*?)>.*/\1/g" <<< ${BOUNCE})
                DATETIME=$(perl -pe "s/^(\w+\s+\w+\s+\w+:\w+:\w+)\s.*/\1/g" <<< ${BOUNCE})

                ## Check if there's information about the host (there is non when the delivery has been done locally)
                NEEDLE=".*?\(host\s.*?\[.*"
                if [[ "${BOUNCE}" =~ ${NEEDLE} ]]; then
                        HOST=$(perl -pe "s/.*?\(host\s(.*?)\[.*/\1/g" <<< ${BOUNCE})
                        HOSTIP=$(perl -pe "s/.*?\(host\s.*?\[(.*?)\]\s.*/\1/g" <<< ${BOUNCE})
                else
                        HOST="<span style='color:#888888;'><i>No host found</i></span>"
                        HOSTIP="<span style='color:#888888;'><i>No IP found</i></span>"
                fi

                ## Try to fetch the bounce reason
                NEEDLE=".*\ssaid:\s.*"
                if [[ "${BOUNCE}" =~ ${NEEDLE} ]]; then
                        REASON=$(perl -pe "s/.*\ssaid:\s(.*)/\1/g" <<< ${BOUNCE})
                else
                        ## Check if perhaps the domain could not be resolved (Host not found message). In that case there would be no said
                        NEEDLE=".*\(Host or domain name not found.*"
                        BOUNCE_MSG_PRESENT=".*?status=bounced\s\(.*"
                        if [[ "${BOUNCE}" =~ ${NEEDLE} ]]; then
                                REASON=$(perl -pe "s/.*?\((Host or domain name not found.*)/\1/g" <<< ${BOUNCE})
                        elif [[ "${BOUNCE}" =~ ${BOUNCE_MSG_PRESENT} ]]; then
                                REASON=$(perl -pe "s/.*?status=bounced\s\((.*?)\)/\1/g" <<< ${BOUNCE})
                        else
                                REASON="<span style='color:#888888;'><i>Reject reason not found.</i></span>"
                        fi
                fi

                ## Fetch additional information about the bounced message based on the MAILID
                MESSAGEDATA=$(cat ${MAILLOG} |grep ${MAILID})
                MESSAGEDATA="${MESSAGEDATA//$'\n'/ }"

                ## Check if there's a username
                NEEDLE=".*sasl_username.*"
                if [[ "${MESSAGEDATA}" =~ ${NEEDLE} ]]; then
                        USERNAME=$(perl -pe "s/.*?sasl_username=(.*?)\s.*/\1/gm" <<< ${MESSAGEDATA})
                        SASL_CLIENT_PTR=$(perl -pe "s/.*?client=(.*?)\[.*/\1/gm" <<< ${MESSAGEDATA})
                        SASL_CLIENT_IP=$(perl -pe "s/.*?client=.*?\[(.*?)\].*/\1/gm" <<< ${MESSAGEDATA})
                else
                        USERNAME="<span style='color:#888888;'><i>local delivery (non-delivery notification)</i></span>"
                        SASL_CLIENT_PTR="<span style='color:#888888;'><i>No PTR found</i></span>"
                        SASL_CLIENT_IP="<span style='color:#888888;'><i>No IP found</i></span>"
                fi

                ## Check if there's a subject line
                NEEDLE=".*?header\sSubject:\s.*?\sfrom\s.*"
                if [[ "${MESSAGEDATA}" =~ ${NEEDLE} ]]; then
                        SUBJECT=$(perl -pe "s/.*?header\sSubject:\s(.*?)\sfrom\s.*/\1/gm" <<< ${MESSAGEDATA})
                else
                        NEEDLE=".*sender non-delivery notification.*"
                        if [[ "${MESSAGEDATA}" =~ ${NEEDLE} ]]; then
                                SUBJECT="<span style='color:#888888;'><i>No subject (delivery notification)</i></span>"
                        else
                                SUBJECT="<span style='color:#888888;'><i>No subject (reason unknown)</i></span>"
                        fi
                fi

                MAILFROM=$(perl -pe "s/.*?from=<(.*?)>.*/\1/gm" <<< ${MESSAGEDATA})

                MAILINFO+="<tr><td>${DATETIME}</td><td>${MAILID}</td><td>${SASL_CLIENT_PTR}</td><td>${SASL_CLIENT_IP}</td><td>${USERNAME}</td><td>${MAILFROM}</td><td>${MAILTO}</td><td>${HOST}</td><td>${HOSTIP}</td><td>${REASON}</td><td>${SUBJECT}</td></tr>"


        done <<< "$ALLBOUNCES"

        MAILINFO+="</table></body></html>"
        MAILINFO+="<br/><br/><h3>Additional information</h3>"
        MAILINFO+="<table class='additionalTable'>"
        TIME_DIFF=$(($(date +"%s")-${TIME_START}))
        MAILINFO+="<tr><td><strong>Script runtime:</strong></td><td>$((${TIME_DIFF} / 60)) Minutes</td><td>$((${TIME_DIFF} % 60)) Seconds</td><td></td></tr>"
        MAILINFO+="</table></body></html>"

        if [ ${COUNTBOUNCES} -gt 6 ]; then BOUNCEWARNING="WARNING | "; else BOUNCEWARNING=""; fi
        echo ${MAILINFO} | mail -a "From: ${LOGMAILFROM}" -a "MIME-Version: 1.0" -a "Content-Type: text/html; charset=utf-8" -s "${BOUNCEWARNING}${COUNTBOUNCES} Mail Bounce(s) Registered" ${LOGMAILTO}
fi

Output preview