[SOLVED] Postfix Mail Bounce Statistics Script - Receive hourly bounced mail statistics
Postfix bounce E-Mail statistics
What's the purpose?
We wanted to receive notifications from Postfix about bounced e-mails send by our customers. The reason is that larger amounts of e-mail bounces may be an indicator of a hacked account or a user abusing the system. We did not find a tool or script which can handle this simple task out of the box. Thus we wrote our own neat little program to do that.
How does it work?
Simple:
- The script needs to run hourly as a cronjob
- It will fetch a list of all bounced entries in the postfix mail log which are from the past hour
- By using REGEX we are filtering out the relevant information about each bounce within a LOOP, structure everything and create a table-based HTML e-mail
- The mail is being sent through regular mailx command (mail)
Optimization potential
- Adding a limit which would trigger the mail sending after a certain amount of bounces / hour only. Right now we want to see all bounces and rather use e-mail rules to sort out the e-mails (but be informed about everything what's going on).
The Subject is not being displayed
In case you don't see any subject most probably Postfix does not write the Subject into the mail.log file. This is how you can enable the Subject function in Postfix:
- Create a PCRE file (or add to an existing): nano /etc/postfix/header_checks.pcre
- Add the following line to it:
/^Subject:/ WARN - Save the file (CTRL + x , confirm with "y")
- Edit the Postfix main.cf configuration file: nano /etc/postfix/main.cf
- Add the following line in that file (or make sure the existing file is listed like this):
header_checks = pcre:/etc/postfix/header_checks.pcre - Initialize the header_checks.pcre file: postmap /etc/postfix/header_checks.pcre
- Restart Postfix: /etc/init.d/postfix restart (or service postfix restart)
The script
Copy paste it from below or download it here.
#!/bin/bash
export LC_ALL=en_US.utf8
MAILLOG=/var/log/mail.log
LOGMAILFROM="Your sender name<websupport@yourdomain.tld>" ## Set the FROM for the e-mail. You can write "Bounce Mail Report<youremail@address.tld>"
LOGMAILTO="your@yourdomain.tld" ## Set the TO for the e-mail (single e-mail address).
# Set the amount of hours you want the system to check your logs for bounce messages.
# Example: If it's 18:13 hours now and you enter 6 in the variable below, the system will check for all bounce messages which occured between 12:00-13:00, 13:00-14:00, 14:00-15:00, 15:00-16:00, 16:00-17:00 and 17:00-18:00
AMOUNTOFHOURSTOCHECK=6
TIME_START=$(date +"%s")
# Initialize the regex pattern which is used to fetch all bounce messages from the past X hours
BOUNCEMESSAGERGXPATTERN=""
# Loop through the last 6 hours and build the regex pattern
for (( i = ${AMOUNTOFHOURSTOCHECK}; i > 0; i-- )); do
hour=$(date -d "-${i} hour" '+%b %e %H')
# Append to the pattern, separated by the OR operator |
BOUNCEMESSAGERGXPATTERN+="(${hour}.*postfix/smtp.*status=bounced)|"
done
# Remove the trailing '|'
BOUNCEMESSAGERGXPATTERN=${BOUNCEMESSAGERGXPATTERN%|}
# Fetch all bounce messages from the mail log and store the result in a variable for later use
ALLBOUNCES=`cat ${MAILLOG} | egrep "$BOUNCEMESSAGERGXPATTERN"`
COUNTBOUNCES=$( [ -n "$ALLBOUNCES" ] && echo "$ALLBOUNCES" | wc -l || echo 0 )
if [ ${COUNTBOUNCES} -gt 0 ]; then
MAILINFO='<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "https://www.w3.org/TR/html4/loose.dtd"><html><head><title></title>'
MAILINFO+='<style>'
MAILINFO+='.mainTable { border-collapse: collapse; } .mainTable th, .mainTable td { border:1px dotted #cccccc; text-align:left; vertical-align:top; padding:5px 10px; } .mainTable th { border-bottom: 2px solid #cccccc; }'
MAILINFO+='.additionalTable { border-collapse: collapse; } .additionalTable td { border: 0px; text-align: left; vertical-align: top; padding: 5px 10px; }'
MAILINFO+='</style>'
MAILINFO+='</head><body><table class="mainTable">'
MAILINFO+="<tr><th>DATE & TIME</th><th>MAIL ID</th><th>CLIENT PTR</th><th>CLIENT IP</th><th>USERNAME</th><th>MAIL FROM</th><th>MAIL TO</th><th>HOST</th><th>HOST IP</th><th>REASON</th><th>SUBJECT</th></tr>"
while IFS= read -r BOUNCE
do
## Clean the bounce variable (remove all line-breaks)
BOUNCE="${BOUNCE//$'\n'/ }"
## Get the mail ID from the current log line
MAILID=$(perl -pe "s/.*?postfix\/smtp\[\w+\]:\s(\w+).*/\1/g" <<< ${BOUNCE})
MAILTO=$(perl -pe "s/.*to=<(.*?)>.*/\1/g" <<< ${BOUNCE})
DATETIME=$(perl -pe "s/^(\w+\s+\w+\s+\w+:\w+:\w+)\s.*/\1/g" <<< ${BOUNCE})
## Check if there's information about the host (there is non when the delivery has been done locally)
NEEDLE=".*?\(host\s.*?\[.*"
if [[ "${BOUNCE}" =~ ${NEEDLE} ]]; then
HOST=$(perl -pe "s/.*?\(host\s(.*?)\[.*/\1/g" <<< ${BOUNCE})
HOSTIP=$(perl -pe "s/.*?\(host\s.*?\[(.*?)\]\s.*/\1/g" <<< ${BOUNCE})
else
HOST="<span style='color:#888888;'><i>No host found</i></span>"
HOSTIP="<span style='color:#888888;'><i>No IP found</i></span>"
fi
## Try to fetch the bounce reason
NEEDLE=".*\ssaid:\s.*"
if [[ "${BOUNCE}" =~ ${NEEDLE} ]]; then
REASON=$(perl -pe "s/.*\ssaid:\s(.*)/\1/g" <<< ${BOUNCE})
else
## Check if perhaps the domain could not be resolved (Host not found message). In that case there would be no said
NEEDLE=".*\(Host or domain name not found.*"
BOUNCE_MSG_PRESENT=".*?status=bounced\s\(.*"
if [[ "${BOUNCE}" =~ ${NEEDLE} ]]; then
REASON=$(perl -pe "s/.*?\((Host or domain name not found.*)/\1/g" <<< ${BOUNCE})
elif [[ "${BOUNCE}" =~ ${BOUNCE_MSG_PRESENT} ]]; then
REASON=$(perl -pe "s/.*?status=bounced\s\((.*?)\)/\1/g" <<< ${BOUNCE})
else
REASON="<span style='color:#888888;'><i>Reject reason not found.</i></span>"
fi
fi
## Fetch additional information about the bounced message based on the MAILID
MESSAGEDATA=$(cat ${MAILLOG} |grep ${MAILID})
MESSAGEDATA="${MESSAGEDATA//$'\n'/ }"
## Check if there's a username
NEEDLE=".*sasl_username.*"
if [[ "${MESSAGEDATA}" =~ ${NEEDLE} ]]; then
USERNAME=$(perl -pe "s/.*?sasl_username=(.*?)\s.*/\1/gm" <<< ${MESSAGEDATA})
SASL_CLIENT_PTR=$(perl -pe "s/.*?client=(.*?)\[.*/\1/gm" <<< ${MESSAGEDATA})
SASL_CLIENT_IP=$(perl -pe "s/.*?client=.*?\[(.*?)\].*/\1/gm" <<< ${MESSAGEDATA})
else
USERNAME="<span style='color:#888888;'><i>local delivery (non-delivery notification)</i></span>"
SASL_CLIENT_PTR="<span style='color:#888888;'><i>No PTR found</i></span>"
SASL_CLIENT_IP="<span style='color:#888888;'><i>No IP found</i></span>"
fi
## Check if there's a subject line
NEEDLE=".*?header\sSubject:\s.*?\sfrom\s.*"
if [[ "${MESSAGEDATA}" =~ ${NEEDLE} ]]; then
SUBJECT=$(perl -pe "s/.*?header\sSubject:\s(.*?)\sfrom\s.*/\1/gm" <<< ${MESSAGEDATA})
else
NEEDLE=".*sender non-delivery notification.*"
if [[ "${MESSAGEDATA}" =~ ${NEEDLE} ]]; then
SUBJECT="<span style='color:#888888;'><i>No subject (delivery notification)</i></span>"
else
SUBJECT="<span style='color:#888888;'><i>No subject (reason unknown)</i></span>"
fi
fi
MAILFROM=$(perl -pe "s/.*?from=<(.*?)>.*/\1/gm" <<< ${MESSAGEDATA})
MAILINFO+="<tr><td>${DATETIME}</td><td>${MAILID}</td><td>${SASL_CLIENT_PTR}</td><td>${SASL_CLIENT_IP}</td><td>${USERNAME}</td><td>${MAILFROM}</td><td>${MAILTO}</td><td>${HOST}</td><td>${HOSTIP}</td><td>${REASON}</td><td>${SUBJECT}</td></tr>"
done <<< "$ALLBOUNCES"
MAILINFO+="</table></body></html>"
MAILINFO+="<br/><br/><h3>Additional information</h3>"
MAILINFO+="<table class='additionalTable'>"
TIME_DIFF=$(($(date +"%s")-${TIME_START}))
MAILINFO+="<tr><td><strong>Script runtime:</strong></td><td>$((${TIME_DIFF} / 60)) Minutes</td><td>$((${TIME_DIFF} % 60)) Seconds</td><td></td></tr>"
MAILINFO+="</table></body></html>"
if [ ${COUNTBOUNCES} -gt 6 ]; then BOUNCEWARNING="WARNING | "; else BOUNCEWARNING=""; fi
echo ${MAILINFO} | mail -a "From: ${LOGMAILFROM}" -a "MIME-Version: 1.0" -a "Content-Type: text/html; charset=utf-8" -s "${BOUNCEWARNING}${COUNTBOUNCES} Mail Bounce(s) Registered" ${LOGMAILTO}
fi
Output preview
