amavisd configure & usage

由 datahunter 在三, 26/08/2020 - 10:02 發表

最後更新: 2018-07-23

介紹
處理 E-Mail 的過程
Install
Amavisd Cli
基本設定
include_config_files
postfix setting
設置2
debian config
log
Mysql Backend
local_domains
阻隔某類附件
Quarantine
Notify
Filtering E-Mails per Recipient Domain
turn off checking mails for sending
Acting on mail checks results
E-Mail add Header
housekeeping
av_scanners
Email address 的 lookups 次序
容許 exe 檔通過
".docx" often incorrectly detected as ".exe"
Troubleshoot
Doc

前言

Amavisd-new 不會應用 spamassassin 的 Daemon(spamd),

因為它會直接載入 spamassassin 作為一個模組去運行.

介紹

Programing Lang:

Perl

Daemon

amavisd

Version:

amavisd -V

amavisd-new-2.6.6 (20110518)

Help

amavisd -h

功能:

Acting on mail checks results (amavis 係會看 spamassassin 及 clamav 返回的 result)
Spam mail Tag, tag2 and kill levels
Quarantine Mail, Releasing from a Quarantine
Redirecting malware to a different mailbox -- plus addressing
Hard black- and whitelisting senders regarding spam
Soft black- and whitelisting senders regarding spam -- @score_sender_maps
Policy banks
Setting up DKIM mail signing and verification
Per domain / user Disclaimer

處理 E-Mail 的過程

Incoming mail

Postfix (MTA-IN)(port 25)

Amavisd (Port: 10024)

Postfix (MTA-OUT)(Port: 10025)

Install

# Centos7

yum install spamassassin

yum install clamav clamav-lib clamav-data clamav-update clamav-filesystem \
clamav-server clamav-scanner-systemd clamav-server-systemd

yum install p7zip unzip

yum install amavisd

Amavisd Cli

amavisd [Options]

start
stop
reload # ln -s /etc/amavisd/amavisd.conf /etc/amavisd.conf
debug
debug-sa
foreground

amavisd 基本設定

/etc/amavisd/amavisd.conf

$max_servers = 1;                                   <-- 要與 postfix 的 master 內的 maxproc 對應, 
                                                        否則 postfix side 會有 timeout

$mydomain = 'x.x';                                  <-- 會 display 在 "X-Virus-Scanned: "

$inet_socket_port = [10024, 10026];                 <-- MTA-IN, 在這裡雖然 Listen 了兩個 port, 
                                                        不過只用 10025, 10026 用來做另一個 bank 

$forward_method = 'smtp:[127.0.0.1]:10025';         <-- MTA-OUT
$notify_method  = 'smtp:[127.0.0.1]:10025';         <-- MTA-OUT, 用來出警訴信.

$unix_socketname = "/var/run/amavisd/amavisd.sock"; <-- amavisd-release or amavis-milter 用的 socket

當 $max_servers = 1; 時會以以下 pstree 結構:

init───amavisd-new───amavisd-new

amavis disable ipv6

$inet_socket_bind = '127.0.0.1';

當 interface 沒有 ipv6 時, 就會有以下 log:

... server /usr/sbin/amavisd[20297]: Net::Server: Binding to TCP port 10024 on host 127.0.0.1 with IPv4
... server /usr/sbin/amavisd[20297]: Net::Server: Binding to TCP port 10024 on host ::1 with IPv6
... server /usr/sbin/amavisd[20297]: 
(!)Net::Server: 2015/04/13-15:24:00 Can't connect to TCP port 10024 on ::1 
[Cannot assign requested address]\n  at line 68 in file /usr/share/perl5/vendor_perl/Net/Server/Proto/TCP.pm

Other tools

# Disables use of BerkeleyDB/libdb (SNMP(amavisd-agent) and nanny(amavisd-nanny))
# falling back to memory-based cache and
# loss of amavisd-nanny and amavisd-agent functionality.

$enable_db=1

# SNMP-like counters updated by amavisd-new.

amavisd-agent

# a program to show the status and keep an eye on the health of child processes in amavisd-new.

amavisd-nanny

process-id task-id     elapsed in    elapsed-bar (dots indicate idle)
           or state   idle or busy

PID 24231:               0:02:53 .........:.........:.........:.....

PID 24231:               0:02:55 .........:.........:.........:.....

# a DKIM signing service daemon for amavisd.

amavisd-signer

include_config_files

use strict;
...
include_config_files('/etc/amavisd/amavisd-custom.conf');
1;

Postfix Setting

master:

smtp-amavis unix -  -   -   -   1  smtp            // 只開兩個 amavisd-daemon
    -o smtp_data_done_timeout=1200                 // 等 amavisd reply DONE, 如果等不到, 就入 deferred queue
    -o smtp_send_xforward_command=yes              // forward the original clients HELO name and IP address
    -o disable_dns_lookups=yes                     // Disable DNS lookups in the Postfix SMTP and LMTP clients
    -o max_use=20                                  // 每個 amavisd-daemon 只限用 20 次就開另一個新的


127.0.0.1:10025 inet n - - - - smtpd
        -o content_filter=
        -o cleanup_service_name=pcleanup
        -o local_recipient_maps=
        -o relay_recipient_maps=
        -o smtpd_restriction_classes=
        -o smtpd_delay_reject=no
        -o smtpd_tls_security_level=none
        -o smtpd_client_restrictions=
        -o smtpd_helo_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_end_of_data_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks=127.0.0.0/8
        -o strict_rfc821_envelopes=yes
        -o smtpd_error_sleep_time=0
        -o smtpd_soft_error_limit=1001
        -o smtpd_hard_error_limit=1000
        -o smtpd_client_connection_count_limit=0
        -o smtpd_client_connection_rate_limit=0
        -o receive_override_options=no_unknown_recipient_checks
        -o smtp_bind_address=127.0.0.1

pcleanup  unix  n       -       -       -       0       cleanup
        -o header_checks=pcre:/etc/postfix/amavis_header_checks

/etc/postfix/amavis_header_checks

/^Received: from localhost/                     IGNORE

main.cf

myhostname = x.x

content_filter = amavis:[127.0.0.1]:10024

# The default maximal number of recipients per message delivery(default: 50)
# 1: It changes the meaning of the corresponding per-destination concurrency limit,
# from concurrency of deliveries to the same domain into concurrency of deliveries to the same recipient.
# Different recipients are delivered in parallel,
# subject to the process limits specified in master.cf.
amavis_destination_recipient_limit = 1

receive_override_options = no_address_mappings

smtp-amavis_destination_recipient_limit = 1

# The default maximal number of recipients per message delivery.
# 當它是 1 時, meaning: Same domain ---> same recipient (Different recipients are delivered in parallel)

receive_override_options = no_address_mappings

# disables address manipulation before the content filter,
# so that the content filter sees the original mail addresses instead of the result of virtual alias expansion,
# canonical mapping, automatic bcc, address masquerading, etc.

設置2

相關工具:

SpamAssassin <--- 不用設置 (/etc/mail/spamassassin/local.cf)
ClamAV <--- 設定用 socket 就可以 (LocalSocket /var/run/clamav/clamd.sock)

停用某些檢查(bypass):

# @bypass_virus_checks_maps = (1);  # controls running of anti-virus code
# @bypass_spam_checks_maps  = (1);  # controls running of anti-spam code
# $bypass_decode_parts = 1;         # controls running of decoders & dearchivers

SpamAssassin settings:

# add spam info headers if at, or above that level (-999 = all messages)
$sa_tag_level_deflt = -999;

# add 'spam detected' headers at that level
$sa_tag2_level_deflt = 5.0;

# triggers spam evasive actions (e.g. blocks mail)
# spam level beyond which a DSN is not sent
$sa_kill_level_deflt = 8.0;

# spam level beyond which a DSN is not sent
# Any mail that scores at 12 or higher will effectively turn D_BOUNCE into D_DISCARD
$sa_dsn_cutoff_level = 12.0;

# credibel(可信的), likewise, but for a likely valid From, 它的值應該 > $sa_dsn_cutoff_level
# "$msginfo->sender_credible = true" => sa_crediblefrom_dsn_cutoff_level
$sa_crediblefrom_dsn_cutoff_level = $sa_dsn_cutoff_level

# spam level beyond which quarantine is off
$sa_quarantine_cutoff_level = 15;

# don't waste time on SA if mail is larger (bytes)
$sa_mail_body_size_limit = 3*1024*1024;

# no SA tests that require internet access will be performed.
$sa_local_tests_only = 0;

$sa_spam_subject_tag = '***SPAM*** ';

my setting

#### sa setting ####
$sa_tag_level_deflt  = -999;
$sa_tag2_level_deflt = 5;
$sa_kill_level_deflt = 7;
$sa_dsn_cutoff_level = 12;
$sa_crediblefrom_dsn_cutoff_level = 13;
$sa_quarantine_cutoff_level = 15;

$sa_mail_body_size_limit = 3*1024*1024;

$sa_local_tests_only = 0;

$sa_spam_subject_tag = '***SPAM*** ';

Cleanup quarantine weekly

/root/scripts/cleanup-quarantine.sh

#!/bin/bash
find /var/spool/amavisd/quarantine -mtime +7 -exec rm -f '{}' \;

cron job

1 1 * * *       /root/scripts/cleanup-quarantine.sh

"pen pals" lookup Setting

# Use a pen pals lookup to check inbound DSN for a corresponding outbound message;

# if a Message-ID contained within the inbound DSN doesn't match a valid Message-ID from the apparent sender,

# such message receives $bounce_killer_score spam score points (100 by default) and can be blocked as spam.

$penpals_bonus_score = 8;                        # (no effect without a @storage_sql_dsn database)

$penpals_threshold_high = $sa_kill_level_deflt;  # don't waste time on hi spam

# inbound DSN doesn't match a valid outbound Message-ID 時, 那 mail 獲得的 score
$bounce_killer_score = 100;

Debian Config Files

01-debian
05-domain_id
05-node_id
15-av_scanners
15-content_filter_mode
20-debian_defaults
25-amavis_helpers
30-template_localization
50-user

15-av_scanners

@av_scanners = (
 ['ClamAV-clamd',
   \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd.ctl"],
   qr/\bOK$/m, qr/\bFOUND$/m,
   qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ]
);

@av_scanners_backup = (
    ['ClamAV-clamscan', 'clamscan',
    "--stdout --disable-summary -r --tempdir=$TEMPBASE {}", [0], [1],
    qr/^.*?: (?!Infected Archive)(.*) FOUND$/ ],
);

05-node_id

問題:

The value of variable $myhostname is "", but should have been a fully qualified domain name

解決:

$myhostname = "mail.example.com";

20-debian_defaults

$inet_socket_port = 10024;     # default listening socket

$QUARANTINEDIR = "$MYHOME/virusmails";  # 當 '= undef;' 時即是沒隔離
$quarantine_subdir_levels = 1;          # enable quarantine dir hashing

$enable_db = 1;                # enable use of BerkeleyDB/libdb (SNMP and nanny)
$enable_global_cache = 1;      # enable use of libdb-based cache if $enable_db=1

$enable_dkim_verification = 0; #disabled to prevent warning

50-user

dsn:

# @lookup_sql_dsn  <-- lookups value 時用, 如果無用此功能, 一定要 "#" 無佢, 否則會每次 lookup

# @storage_sql_dsn <-- reporting and quarantining 時用

@lookup_sql_dsn =
   ( ['DBI:mysql:database=dbispconfig;host=127.0.0.1;port=3306', 'ispconfig', '?????????????'] );

$sql_select_policy =
   'SELECT *,spamfilter_users.id'.
   ' FROM spamfilter_users LEFT JOIN spamfilter_policy ON spamfilter_users.policy_id=spamfilter_policy.id'.
   ' WHERE spamfilter_users.email IN (%k) ORDER BY spamfilter_users.priority DESC';

sql_white_black_list:

# sender in per-recipient whitelist/blacklist

$sql_select_white_black_list = 'SELECT wb FROM spamfilter_wblist'.
    ' WHERE (spamfilter_wblist.rid=?) AND (spamfilter_wblist.email IN (%k))' .
    ' ORDER BY spamfilter_wblist.priority DESC';

# * empty list disables the function and is a default
# '?' will be users.id from recipient SQL lookup, the
# '%k': will be sender addresses
# 'IN': full address, domain only, catchall).

DB:

CREATE TABLE IF NOT EXISTS `spamfilter_wblist` (
  `wblist_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
  `sys_userid` int(11) unsigned NOT NULL DEFAULT '0',
  `sys_groupid` int(11) unsigned NOT NULL DEFAULT '0',
  `sys_perm_user` varchar(5) NOT NULL DEFAULT '',
  `sys_perm_group` varchar(5) NOT NULL DEFAULT '',
  `sys_perm_other` varchar(5) NOT NULL DEFAULT '',
  `server_id` int(11) unsigned NOT NULL DEFAULT '0',
  `wb` enum('W','B') NOT NULL DEFAULT 'W',
  `rid` int(11) unsigned NOT NULL DEFAULT '0',
  `email` varchar(255) NOT NULL DEFAULT '',
  `priority` tinyint(3) unsigned NOT NULL DEFAULT '0',
  `active` enum('y','n') NOT NULL DEFAULT 'y',
  PRIMARY KEY (`wblist_id`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8;

# whitelist x@y

INSERT INTO `spamfilter_wblist` (`wblist_id`, `sys_userid`, `sys_groupid`, 
`sys_perm_user`, `sys_perm_group`, `sys_perm_other`, `server_id`, `wb`, `rid`, 
`email`, `priority`, `active`) VALUES
(1, 2, 2, 'riud', 'riud', '', 0, 'W', 0, 'x@y', 5, 'y');

sql_policy

$sql_select_policy =
   'SELECT *,spamfilter_users.id'.
   ' FROM spamfilter_users LEFT JOIN spamfilter_policy ON spamfilter_users.policy_id=spamfilter_policy.id'.
   ' WHERE spamfilter_users.email IN (%k) ORDER BY spamfilter_users.priority DESC';

bypass

#@bypass_virus_checks_maps = (
#   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
#

# 停止檢查病毒
@bypass_virus_checks_maps = (1);  # controls running of anti-virus code

@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);

final

$final_virus_destiny = D_BOUNCE;
$final_spam_destiny = D_DISCARD;
$final_banned_destiny = D_BOUNCE;
$final_bad_header_destiny = D_PASS;

D_REJECT 與 D_BOUNCE 的分別

D_REJECT 與 D_BOUNCE are similar, the difference is in who is responsible

D_REJECT

MTA may reject original SMTP, or send DSN (delivery status notification, colloquially called 'bounce') [depending on MTA]

Best suited for sendmail milter, especially for spam.

log

... postfix/smtp[29962]: 78A3F13837A: to=<user@domain>, 
 relay=127.0.0.1[127.0.0.1]:10024,
 delay=1, delays=0.45/0/0/0.59, dsn=5.7.0, status=bounced (host 127.0.0.1[127.0.0.1]
 said: 554 5.7.0 Reject, id=29512-07 - spam (in reply to end of DATA command))
... postfix/bounce[30027]: 78A3F13837A: sender non-delivery notification: 22BF113837D

D_BOUNCE

amavisd-new (not MTA) sends DSN

(can better explain the reason for mail non-delivery or even suppress DSN, but unable to reject the original SMTP session).

which can't reject original client SMTP session, as the mail has already been enqueued.

... postfix/qmgr[2625]: 2E5EF13837E: from=<S@S>, size=310, nrcpt=1 (queue active)
... amavis[30620]: (30620-01) Blocked SPAM {NoBounceInternal}, ... , Hits: 892.606, size: 310, 636 ms
... postfix/smtp[30633]: 2E5EF13837E: to=<D@D>,
 relay=127.0.0.1[127.0.0.1]:10024, delay=1.2,
 delays=0.51/0.01/0.01/0.63, dsn=2.5.0,
 status=sent (250 2.5.0 Ok, id=30620-01, DISCARD(bounce.suppressed))

spam

$sa_spam_subject_tag = '***SPAM*** ';
$sa_tag_level_deflt  = 20.0;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 60.0; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 60.0; # triggers spam evasive actions
$sa_dsn_cutoff_level = 100;   # spam level beyond which a DSN is not sent

LOG

Amavisd log configure (Centos7)

# mkdir /var/log/amavisd
# touch /var/log/amavisd/amavisd.log
# chmod 660 /var/log/amavisd/amavisd.log
# chown amavis.adm /var/log/amavisd/amavisd.log

#### log setting ####

# log via rsyslogd
$do_syslog = 0;

# facility.priority, default 'mail.info'
$syslog_facility = 'mail';

# if not using syslog, defaults to empty, no log
$LOGFILE = "/var/log/amavisd/amavisd.log";

# verbosity 0..5 (defaults to 0)
$log_level = 1;

# built-in default at the end of file amavisd
$log_templ = undef;
$log_recip_templ = undef;

service amavisd restart

log 的資訊

$log_templ = $log_short_templ; # 用 $log_short_templ format 去 log 東西

$log_recip_templ = undef; # ?

$log_level = N;

0: startup/exit/failure messages, viruses detected
1: args passed from client, some more interesting messages
2: virus scanner output, timing
3: server, client
4: decompose parts
5: more debug details

$log_templ = '...';

$log_templ = undef; # undef = disables

$log_templ = $log_short_templ; # 設定用 $log_short_templ 的 log format

$log_templ = $log_verbose_templ; # log 到 Subject, User-Agent,

i.e.

$log_templ = '
[?%#D|#|Passed #
[? [:ccat|major] |#
OTHER|CLEAN|MTA-BLOCKED|OVERSIZED|BAD-HEADER-[:ccat|minor]|SPAMMY|SPAM|\
UNCHECKED[?[:ccat|minor]||-ENCRYPTED|]|BANNED (%F)|INFECTED (%V)]#
 {[:actions_performed]}#
,[?%p|| %p][?%a||[?%l|| LOCAL] [:client_addr_port]][?%e|| \[%e\]] %s -> [%D|,]#
[? %q ||, quarantine: %q]#
[? %Q ||, Queue-ID: %Q]#
[? %m ||, Message-ID: [:mail_addr_decode_octets|%m]]#
[? %r ||, Resent-Message-ID: [:mail_addr_decode_octets|%r]]#
[? %i ||, mail_id: %i]#
, Hits: [:SCORE]#
, size: %z#
[? [:partition_tag] ||, pt: [:partition_tag]]#
[~[:remote_mta_smtp_response]|["^$"]||[", queued_as: "]]\
[remote_mta_smtp_response|[~%x|["queued as ([0-9A-Za-z]+)$"]|["%1"]|["%0"]]|/]#
#, Subject: [:dquote|[:mime2utf8|[:header_field_octets|Subject]|100|1]]#
#, From: [:uquote|[:mail_addr_decode_octets|[:rfc2822_from]]]#
[? [:dkim|sig_sd]    ||, dkim_sd=[:dkim|sig_sd]]#
[? [:dkim|newsig_sd] ||, dkim_new=[:dkim|newsig_sd]]#
, %y ms#
[? %#T ||, Tests: \[[%T|,]\]]#
]
[?%#O|#|Blocked #
[? [:ccat|major|blocking] |#
OTHER|CLEAN|MTA-BLOCKED|OVERSIZED|BAD-HEADER-[:ccat|minor]|SPAMMY|SPAM|\
UNCHECKED[?[:ccat|minor]||-ENCRYPTED|]|BANNED (%F)|INFECTED (%V)]#
 {[:actions_performed]}#
,[?%p|| %p][?%a||[?%l|| LOCAL] [:client_addr_port]][?%e|| \[%e\]] %s -> [%D|,]#
[? %q ||, quarantine: %q]#
[? %Q ||, Queue-ID: %Q]#
[? %m ||, Message-ID: [:mail_addr_decode_octets|%m]]#
[? %r ||, Resent-Message-ID: [:mail_addr_decode_octets|%r]]#
[? %i ||, mail_id: %i]#
, Hits: [:SCORE]#
, size: %z#
[? [:partition_tag] ||, pt: [:partition_tag]]#
#, Subject: [:dquote|[:mime2utf8|[:header_field_octets|Subject]|100|1]]#
#, From: [:uquote|[:mail_addr_decode_octets|[:rfc2822_from]]]#
[? [:dkim|sig_sd]    ||, dkim_sd=[:dkim|sig_sd]]#
[? [:dkim|newsig_sd] ||, dkim_new=[:dkim|newsig_sd]]#
, %y ms#
[? %#T ||, Tests: \[[%T|,]\]]#
]';

logrotate

/etc/logrotate.d/amavisd

/var/log/amavisd/amavisd.log {
    weekly
    rotate 4
    compress
    delaycompress
    missingok
    copytruncate
    notifempty
    create 660 amavis adm
}

Log Level:

0: startup / exit / failure messages / viruses detected
1: args passed from client / some more interesting messages
2: virus scanner output, timing (要這 level 才 log 到 "Jan 30 17:39:46 vm amavis[23367]: (23367-01) wbl: whitelisted sender <x@y>")
3: server, client
4: decompose parts
5: more debug details

bypass_spam_checks 的 log 要 "$log_level = 4;" 才 log 到

$log_level = 4;

output example:

Apr 13 16:39:32 server /usr/sbin/amavisd[24313]: (24313-01) lookup [bypass_spam_checks] => undef, 
    "x@y" does not match
Apr 13 16:41:24 server /usr/sbin/amavisd[24432]: (24432-01) lookup [bypass_spam_checks] => true,  
    "tim@x" matches, result="1", matching_key="x"

START LOG

Sep 26 15:20:42 vps8.domain /usr/sbin/amavisd-new[788]: starting.  /usr/sbin/amavisd-new at 
vps8.domain amavisd-new-2.6.4 (20090625), Unicode aware, LANG="en_HK.UTF-8"
Sep 26 15:20:42 vps8.domain /usr/sbin/amavisd-new[788]: user=, EUID: 108 (108);  group=, EGID: 112 112 (112 112)
Sep 26 15:20:42 vps8.domain /usr/sbin/amavisd-new[788]: Perl version               5.010001
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[788]: SpamControl: init_pre_chroot on SpamAssassin done
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Net::Server: Process Backgrounded
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Net::Server: 2013/09/26-15:20:43 
Amavis (type Net::Server::PreForkSimple) starting! pid(794)
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Net::Server: Binding to UNIX socket file 
/var/lib/amavis/amavisd.sock using SOCK_STREAM
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Net::Server: Binding to TCP port 10024 on host 127.0.0.1
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Net::Server: Group Not Defined.  Defaulting to EGID '112 112'
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Net::Server: User Not Defined.  Defaulting to EUID '108'
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Module Amavis::Conf        2.207
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Module Archive::Zip        1.30
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Module BerkeleyDB          0.42
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Module Compress::Zlib      2.02
.........................................
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Using primary internal av scanner code for ClamAV-clamd
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Found secondary av scanner ClamAV-clamscan at /usr/bin/clamscan
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: Creating db in /var/lib/amavis/db/; BerkeleyDB 0.42, libdb 4.8
Sep 26 15:20:43 vps8.domain /usr/sbin/amavisd-new[794]: initializing Mail::SpamAssassin
Sep 26 15:20:44 vps8.domain /usr/sbin/amavisd-new[794]: SpamControl: init_pre_fork on SpamAssassin done
Sep 26 15:20:44 vps8.domain /usr/sbin/amavisd-new[794]: extra modules loaded after daemonizing/chrooting: 
Mail/SpamAssassin/Plugin/FreeMail.pm

STOP LOG

Sep 26 15:20:08 vps8.domain /usr/sbin/amavisd-new[768]:
 Valid PID file (younger than sys uptime 266 16:44:00)
Sep 26 15:20:08 vps8.domain /usr/sbin/amavisd-new[643]:
 Net::Server: 2013/09/26-15:20:08 Server closing!

log_level

[0]

... (05448-01) Passed CLEAN, MYUSERS LOCAL [192.168.123.31] [192.168.123.31] <[email protected]> -> <tim@mydomain>, Message-ID: <[email protected]>, mail_id: MQiX+H0kRxCE, Hits: -10.001, size: 231664, queued_as: 74F5D45F05, 2256 ms

[1]

... SpamControl: init_pre_fork on SpamAssassin done
... extra modules loaded after daemonizing/chrooting: Mail/SpamAssassin/Plugin/FreeMail.pm

... (05507-01) ESMTP::10024 /var/spool/amavisd/tmp/amavis-20160217T125610-05507: <[email protected]> -> <tim@mydomain>
    SIZE=231666 Received: from mail.datahunter.org ([127.0.0.1]) by localhost (mail.datahunter.org [127.0.0.1]) (amavisd-new, port 10024)
    with ESMTP for <tim@mydomain>;
    Wed, 17 Feb 2016 12:56:10 +0800 (HKT)
... (05507-01) Checking: Cvf6bz3pJL8N MYUSERS [192.168.123.31] <[email protected]> -> <tim@mydomain>
... (05507-01) mangling by altermime (disclaimer) done, new size: 228538, orig 231666 bytes
... (05507-01) FWD via SMTP: <[email protected]> -> <tim@mydomain>,
     BODY=7BIT 250 2.0.0 from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 6103345F05
... (05507-01) Passed CLEAN, MYUSERS LOCAL [192.168.123.31] [192.168.123.31] <[email protected]> -> <tim@mydomain>,
     Message-ID: <[email protected]>, mail_id: Cvf6bz3pJL8N, Hits: -10.001, size: 231666, queued_as: 6103345F05, 2235 ms

[2]

... SpamAssassin loaded plugins: AutoLearnThreshold, Bayes, BodyEval, Check, DKIM, DNSEval, FreeMail,
HTMLEval, HTTPSMismatch, Hashcash, HeaderEval, ImageInfo, MIMEEval, MIMEHeader, Pyzor, Razor2,
RelayEval, ReplaceTags, SPF, SpamCop, URIDNSBL, URIDetail, URIEval, VBounce, WLBLEval, WhiteListSubject
...
... storage and lookups will use separate connections to SQL
...
... (05559-01) p003 1 Content-Type: multipart/mixed
... (05559-01) p001 1/1 Content-Type: text/plain, size: 7 B, name:
... (05559-01) p002 1/2 Content-Type: image/bmp, size: 168354 B, name: test.bmp
...
... (05559-01) dkim: candidate originators: 2822.From:<[email protected]>, 2821.mail_from:<[email protected]>
...
... (05559-01) TIMING-SA total 1784 ms - ...
...
... (05559-01) TIMING [total 2187 ms] - ...

Log Checking

SPAMMY is above tag2 level
SPAM is above kill level

Dec 14 13:19:15 mail amavis[26861]: (26861-09) Blocked INFECTED (), LOCAL [Client IP] [Client IP]
<tim@sender> -> <test@recipients>, quarantine: virus-9M0L+iQKExwu,
Message-ID: <50CAB709.5030404@sender>, mail_id: 9M0L+iQKExwu, Hits: -, size: 2868, 150 ms

Mysql Backend

/etc/amavisd/amavisd.conf

# use the appropriate data type in SQL commands

# 比如在 DB 的 maddr.email colume: 0 => VARCHAR 1 => VARBINARY

$sql_allow_8bit_address = 1;

# time_iso 用的 format 設定

$timestamp_fmt_mysql = 1;

Table:

mailaddr
users
wblist
policy
msgs
msgrcpt
quarantine

常見的 feild

priority: 0 is low priority
id AUTO_INCREMENT PRIMARY KEY

users <-- local user

id
priority // default: 7
policy_id // default: 1
email // NOT NULL UNIQUE
fullname // DEFAULT NULL

INSERT INTO `users`(`email`) VALUES ('@datahunter.org');

mailaddr

id
priority DEFAULT '7'
email varbinary(255) NOT NULL UNIQUE

INSERT INTO `mailaddr`(`priority`, `email`) VALUES (9, '@gmail.com');

wblist

rid // 對應 recipient: users.id
sid // 對應 sender: mailaddr.id
wb // W | B

INSERT INTO `wblist`(`rid`, `sid`) VALUES (1, 1);

policy

virus_lover // Y | N
spam_lover
................
virus_quarantine_to // NULL

msgrcpt

mail_id
bl // blacklisted by this recip
wl

quarantine ($*_quarantine_method='sql:')

mail_id
mail_text // 類型: blob

Example: Whitelist

create tables

-- local users
CREATE TABLE users (
  id         int unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,  -- unique id
  priority   integer      NOT NULL DEFAULT '7',  -- sort field, 0 is low prior.
  policy_id  integer unsigned NOT NULL DEFAULT '1',  -- JOINs with policy.id
  email      varbinary(255) NOT NULL UNIQUE,
  fullname   varchar(255) DEFAULT NULL    -- not used by amavisd-new
  -- local   char(1)      -- Y/N  (optional field, see note further down)
);

-- any e-mail address (non- rfc2822-quoted), external or local, used as senders in wblist
CREATE TABLE mailaddr (
  id         int unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
  priority   integer      NOT NULL DEFAULT '7',  -- 0 is low priority
  email      varbinary(255) NOT NULL UNIQUE
);

-- per-recipient whitelist and/or blacklist, puts sender and recipient in relation wb  
(white or blacklisted sender)
CREATE TABLE wblist (
  rid        integer unsigned NOT NULL,  -- recipient: users.id
  sid        integer unsigned NOT NULL,  -- sender: mailaddr.id
  wb         varchar(10)  NOT NULL,  -- W or Y / B or N / space=neutral / score
  PRIMARY KEY (rid,sid)
);

ALTER TABLE `wblist` ADD UNIQUE( `rid`, `sid`);

users / mailaddr

-- local domain
INSERT INTO `users`(`email`) VALUES ('@datahunter.org');
-- sender / domain
INSERT INTO `mailaddr`(`priority`, `email`) VALUES (9, '@gmail.com');

rule (rid, sid, 'W'/'B')

-- rule By id
INSERT INTO `wblist`(`rid`, `sid`) VALUES (1, 1);

-- rule By Domain
INSERT INTO wblist VALUES (
    (SELECT id FROM users where email=BINARY('@datahunter.org')),
    (SELECT id FROM mailaddr where email=BINARY('@sender.domain')),
    "W"
)

check

log

# 要 $log_level = 2; 才 log 到 wbl

tail -f maillog | grep wbl

Dec 23 17:46:57 mail amavis[5724]: (05724-06) wbl: blacklisted sender <[email protected]>
Dec 23 17:50:09 mail amavis[5727]: (05727-07) wbl: whitelisted sender <[email protected]>

mail header

X-Spam-Status: No, score=x tagged_above=-999 required=5 WHITELISTED tests=[]
    autolearn=unavailable

Example: policy

Table:

CREATE TABLE policy (
  id  int unsigned NOT NULL AUTO_INCREMENT PRIMARY KEY,
                                    -- 'id' this is the _only_ required field
  policy_name      varchar(32),     -- not used by amavisd-new, a comment

  virus_lover           char(1) default NULL,     -- Y/N
  spam_lover            char(1) default NULL,     -- Y/N
  unchecked_lover       char(1) default NULL,     -- Y/N
  banned_files_lover    char(1) default NULL,     -- Y/N
  bad_header_lover      char(1) default NULL,     -- Y/N

  bypass_virus_checks   char(1) default NULL,     -- Y/N
  bypass_spam_checks    char(1) default NULL,     -- Y/N
  bypass_banned_checks  char(1) default NULL,     -- Y/N
  bypass_header_checks  char(1) default NULL,     -- Y/N

  virus_quarantine_to      varchar(64) default NULL,
  spam_quarantine_to       varchar(64) default NULL,
  banned_quarantine_to     varchar(64) default NULL,
  unchecked_quarantine_to  varchar(64) default NULL,
  bad_header_quarantine_to varchar(64) default NULL,
  clean_quarantine_to      varchar(64) default NULL,
  archive_quarantine_to    varchar(64) default NULL,

  spam_tag_level  float default NULL, -- higher score inserts spam info headers
  spam_tag2_level float default NULL, -- inserts 'declared spam' header fields
  spam_tag3_level float default NULL, -- inserts 'blatant spam' header fields
  spam_kill_level float default NULL, -- higher score triggers evasive actions
                                      -- e.g. reject/drop, quarantine, ...
                                     -- (subject to final_spam_destiny setting)
  spam_dsn_cutoff_level        float default NULL,
  spam_quarantine_cutoff_level float default NULL,

  addr_extension_virus      varchar(64) default NULL,
  addr_extension_spam       varchar(64) default NULL,
  addr_extension_banned     varchar(64) default NULL,
  addr_extension_bad_header varchar(64) default NULL,

  warnvirusrecip      char(1)     default NULL, -- Y/N
  warnbannedrecip     char(1)     default NULL, -- Y/N
  warnbadhrecip       char(1)     default NULL, -- Y/N
  newvirus_admin      varchar(64) default NULL,
  virus_admin         varchar(64) default NULL,
  banned_admin        varchar(64) default NULL,
  bad_header_admin    varchar(64) default NULL,
  spam_admin          varchar(64) default NULL,
  spam_subject_tag    varchar(64) default NULL,
  spam_subject_tag2   varchar(64) default NULL,
  spam_subject_tag3   varchar(64) default NULL,
  message_size_limit  integer     default NULL, -- max size in bytes, 0 disable
  banned_rulenames    varchar(64) default NULL, -- comma-separated list of ...
        -- names mapped through %banned_rules to actual banned_filename tables
  disclaimer_options  varchar(64) default NULL,
  forward_method      varchar(64) default NULL,
  sa_userconf         varchar(64) default NULL,
  sa_username         varchar(64) default NULL
);

Table Data:

INSERT INTO policy (id, policy_name,
  virus_lover, spam_lover, banned_files_lover, bad_header_lover,
  bypass_virus_checks, bypass_spam_checks,
  bypass_banned_checks, bypass_header_checks, spam_modifies_subj,
  spam_tag_level, spam_tag2_level, spam_kill_level) VALUES
  (1, 'Non-paying',    'N','N','N','N', 'Y','Y','Y','N', 'Y', 3.0,   7, 10),
  (2, 'Uncensored',    'Y','Y','Y','Y', 'N','N','N','N', 'N', 3.0, 999, 999),
  (3, 'Wants all spam','N','Y','N','N', 'N','N','N','N', 'Y', 3.0, 999, 999),
  (4, 'Wants viruses', 'Y','N','Y','Y', 'N','N','N','N', 'Y', 3.0, 6.9, 6.9),
  (5, 'Normal',        'N','N','N','N', 'N','N','N','N', 'Y', 3.0, 6.9, 6.9),
  (6, 'Trigger happy', 'N','N','N','N', 'N','N','N','N', 'Y', 3.0,   5, 5),
  (7, 'Permissive',    'N','N','N','Y', 'N','N','N','N', 'Y', 3.0,  10, 20),
  (8, '6.5/7.8',       'N','N','N','N', 'N','N','N','N', 'N', 3.0, 6.5, 7.8),
  (9, 'userB',         'N','N','N','Y', 'N','N','N','N', 'Y', 3.0, 6.3, 6.3),
  (10,'userC',         'N','N','N','N', 'N','N','N','N', 'N', 3.0, 6.0, 6.0),
  (11,'userD',         'Y','N','Y','Y', 'N','N','N','N', 'N', 3.0,   7, 7);

local_domains 與 MYUSERS

設定 local domain 的方法

/etc/amavisd/amavisd.conf

@local_domains_maps = 1;                                # 所有 sender 都係 local domain
@local_domains_maps = [];                               # 沒有人係 local domain
@local_domains_maps = ( ["$mydomain","domain1.com"] );  # list of all local domains
@local_domains_maps = read_hash("/etc/amavisd/all_hosted_domains");

all_hosted_domains

domain1.com
domain2.com
...

## the name 'MYUSERS' has special semantics: this policy bank gets loaded
## whenever the sender matches @local_domains_maps. This only makes sense
## if local sender addresses can be trusted
# Apply to mails which coming from internal networks or authenticated users.
# mail supposedly originating from our users

* It would require you to configure Postfix to reject non-authenticated mail addressed from any of your domains;

阻隔某類附件

有關要 block 的類型設定

$banned_filename_re
$banned_namepath_re

$bypass_decode_parts

# Disabling decoding also causes banned_files checking to only see MIME names and MIME content types, 
# not the content classification types as provided by the file(1) utility.

#$bypass_decode_parts = 1;

$banned_filename_re

All nodes (mail parts) of the fully recursively decoded mail and embedded archives are checked,

each node independently from remaining nodes.

The search for a node stops at the first match

Object $banned_filename_re provides a list of Perl regular expressions to be matched against each part's:

Content-Type: application/x-zip-compressed;

Content-Disposition: attachment;
	filename="=?utf-8?B?5pel6KqM5qqULnppcA==?="

* file content type as guessed by 'file(1)' utility

* $banned_filename_re > whitelist

$banned_filename_re = new_RE(
    qr'.\.(exe|vbs|pif|scr|cpl|js)$'i,      # banned extension
    qr'^\.(exe|zip|lha|tnef)$'i,            # file(1) type
    qr'^application/x-msdos-program$'i,     # block these MIME types
    ..............
);

$banned_namepath_re

此設定提供 "bird's eye view"

make more complex decision, based on the environment in which each mail part is

e.g.

a.txt within z.zip is allowed, while b.txt whithin a z.zip is not

The banned_filename only sees one node at a time (no parent nodes of each leaf node)

Remark

# use old or new style of banned lookup table; not both to avoid confusion

# $banned_filename_re =undef;   # to disable old-style
# $banned_namepath_re = undef;  # to disable new-style

$final_banned_destiny

當 $final_banned_destiny = D_PASS;

在 log 會見到

Dec 19 17:08:19 mail amavis[10826]: (10826-15) Passed BANNED (.exe,.exe-ms,pietty0.327.exe)

改成 $final_banned_destiny = D_DISCARD;

在 log 會見到

Dec 19 17:19:02 mail amavis[16243]: (16243-13) Blocked BANNED (.exe,.exe-ms,pietty0.327.exe)

zip 檔內可以什麼都有

i.e. 如果想 exe 可以存在於 zip 內

在 amavisd.con 要有

$banned_filename_re = new_RE(
    // 在第一行
    [ qr'^\.zip$'=> 0 ],
    .........
);

amavisd reload

我的 Setting

# MySetting
$banned_filename_re = new_RE(
  # block certain double extensions in filenames
  qr'^(?!cid:).*\.[^./]*[A-Za-z][^./]*\.\s*(exe|vbs|pif|scr|bat|cmd|com|cpl|dll)[.\s]*$'i,
  # banned extension
  qr'.\.(ade|adp|app|bas|bat|chm|cmd|com|cpl|crt|emf|exe|fxp|grp|hlp|hta|
         inf|ini|ins|isp|js|jse|lib|lnk|mda|mdb|mde|mdt|mdw|mdz|msc|msi|
         msp|mst|ocx|ops|pcd|pif|prg|reg|scr|sct|shb|shs|sys|vb|vbe|vbs|vxd|
         wmf|wsc|wsf|wsh)$'ix,                # banned extensions - long
  qr'.\.(asd|asf|asx|url|vcs|wmd|wmz)$'i,     # consider also
  qr'.\.(mim|b64|bhx|hqx|xxe|uu|uue)$'i,      # banned extension - WinZip vulnerab.
);

Log:

$log_level = 1;

Oct 12 18:07:57 sf-server /usr/sbin/amavisd[25189]: 
(25189-01) p.path BANNED:1 [email protected]:
"P=p004,L=1,M=multipart/mixed | P=p002,L=1/2,M=application/octet-stream,T=zip,N=test.zip | 
P=p005,L=1/2/1,T=asc,N=test.js",
matching_key="(?^ix:.\\.(ade|adp|app|bas|bat|chm|cmd|com|cpl|crt|emf|exe|fxp|grp|hlp|hta|\n
inf|ini|ins|isp|js|jse|lib|lnk|mda|mdb|mde|mdt|mdw|mdz|msc|msi|\n
msp|mst|ocx|ops|pcd|pif|prg|reg|scr|sct|shb|shs|sys|vb|vbe|vbs|vxd|\n         
wmf|wsc|wsf|wsh)$)"

Extension

app        # runs under Mac OS X.
exe
com
hlp
chm
inf        # Setup Information File
           # defines what files are installed with a certain software program or update
bat
jar
js
jse        # Script written in JScript
vbs
wsf        # Windows Script File
wsh        # Windows Scripting Host
wsc        # XML-formatted scripting object containing properties and/or methods;
sct        # Script used to create a Component Object Model (.COM) component; 
           # may be written in various scripting languages such as VBScript, JavaScript, or JScript
ocx
sys
cpl
reg
lnk
msi
msp        # Windows Installer Patch (run by Hotfix.exe and Update.exe)
msc        # Microsoft Management Console Snap-in
scr        # Screensaver file for Windows

Quarantine

Trigger quarantine 的條件

* enabled for a given contents category (sapm, virus ...)

* at least one of its recipients at or above his kill level

* quarantining of clean messages for archiving or troubleshooting purposes

設定

*_quarantine_to (支援 per-recipient settings)

*_quarantine_method (a static and a site-wide setting)

設置隔離的方式

#chown amavis. /var/quarantine
#chmod 770 /var/quarantine
$QUARANTINEDIR = "/var/quarantine";                     # Quarantine Directory

#### quarantine spam setting ####
$spam_quarantine_method = 'local:spam-%i-%n-%m';        # local quarantine method, set filename in $QUARANTINEDIR

# $spam_quarantine_to = "postmaster\@$mydomain";        # Send Spam to Adminstrator
# $spam_quarantine_to = undef;                          # disables quarantine
$spam_quarantine_to = 'spam-quarantine';                # predefined aliases are 'virus-quarantine' and 'spam-quarantine'

$spam_admin = "postmaster\@$mydomain";                  # notifications to admin about spam

#### quarantine virus setting ####
$virus_quarantine_method = 'sql:';                      # sql:anything, Default: 'local:virus-%i-%m';
$virus_quarantine_to = 'virus-quarantine';

#### quarantine other setting ####
#$banned_files_quarantine_method                        # Do nothing with banned
#$bad_header_quarantine_method                          # Do nothing with bad_header

隔離的位置(*_quarantine_method)

"local:[filename-template]" # /path/to/folder (if a template file name ends in .gz the message will be gzip-compressed)
"sql:" # stored into SQL database specified by @storage_sql_dsn
"smtp:[hostname:port]" # xxx@xxx

*_quarantine_to

undef/empty # not quarantined

filename-template

%b     $msginfo->body_digest
%P     $msginfo->partition_tag
%m     $msginfo->mail_id                               # 在 mail 內容內, "X-Quarantine-ID: <xxxxxxxx>"
%n     $msginfo->log_id                                # 在 /var/log/maillog 及 return mail內, 格式: id=21825-02, 
                                                       # amavis 回覆 postfix 時有的 msg 來
%i     ISO 8601 timestamp of a mail reception time     # 20161109T183133
%%     a single %

在 Quarantine 放出 Mail

Server Setting

$unix_socketname = "/var/spool/amavisd/amavisd.sock";

$policy_bank{'AM.PDP-SOCK'} = {
  protocol => 'AM.PDP',
  auth_required_release => 0,  # do not require secret_id for amavisd-release
};

# 如果是以 local 的方式隔離, 那以下指令可放出 Mail

amavisd-release mail_file

* release 後那個 mail_file 依然存在, 不會被 Delete

Example:

log:

quarantine: spam/U/UM3XM3XDbN52.gz,

cmd:

amavisd-release spam/U/UM3XM3XDbN52.gz

# 如是是以 sql 的話, 那要在 mysql 內找出 mail_id 及secret_id 之後 telnet server

select mail_id,secret_id,quar_type from msgs where mail_id="???????";

telnet localhost 9998

Trying 127.0.0.1...
Connected to localhost
Escape character is '^]'.

request=release mail_id=??????? secret_id=??????? quar_type=Q

我的設定

#### quarantine setting ####
$QUARANTINEDIR = '/var/quarantine';
$virus_quarantine_method        = '';
$spam_quarantine_method         = 'local:spam-%i-%n-%m';      # %m 一定要在尾, 否則 amavisd-release 唔到
$banned_files_quarantine_method = '';
$bad_header_quarantine_method   = '';

archive_quarantine_method

archive incoming mail with Postfix's bcc maps

amavisd-new (因為 amavisd-release 方便)

# clean 的 mail
# $clean_quarantine_method   = undef;
# 所有 mail
# $archive_quarantine_method = undef;

Log

amavis[29406]: (29406-01) Blocked SPAM {RejectedInternal,Quarantined}, ...

$sa_quarantine_cutoff_level = 25; # spam level beyond which quarantine is off

amavis[29511]: (29511-01) Blocked SPAM {RejectedInternal}, ...

Notify

#### notify ####

# msg header
$hdr_encoding = 'UTF-8';       # MIME charset (default: 'iso-8859-1')    
$hdr_encoding_qb = 'B';        # MIME encoding: 'B': base64, 'Q': quoted-printable
$bdy_encoding = 'UTF-8';       # (default: 'iso-8859-1')

# Notify spam / virus / banned / invalid_header sender?
$warnvirussender = 0;
$warnspamsender = 0;
$warnbannedsender = 0;
$warnbadhsender = 0;

# 通知邊個
# 當是 undef 時, 那什麼人都不通知
$virus_admin = undef;
$spam_admin = undef;
$banned_admin = undef;
$bad_header_admin = undef;

# 通知信的 template
#$notify_sender_templ      = read_text('/var/amavis/notify_sender.txt');
#$notify_virus_sender_templ= read_text('/var/amavis/notify_virus_sender.txt');
#$notify_virus_admin_templ = read_text('/var/amavis/notify_virus_admin.txt');
#$notify_virus_recips_templ= read_text('/var/amavis/notify_virus_recips.txt');
#$notify_spam_sender_templ = read_text('/var/amavis/notify_spam_sender.txt');
#$notify_spam_admin_templ  = read_text('/etc/amavisd/notify_spam_admin.txt');

# 在 bank 內可以另外設定
# 沒有 $var 及不用 "="
$policy_bank{'ORIGINATING'} = {  # mail supposedly originating from our users
  originating => 1,  # declare that mail was submitted by our smtp client
  allow_disclaimers => 1,  # enables disclaimer insertion if available

  # notify administrator of locally originating malware
  virus_admin_maps => ['postmaster@serveradmin'],
  spam_admin_maps  => ['postmaster@serveradmin'],
  $banned_admin = ['postmaster@serveradmin'],
  $bad_header_admin = [],

  # 等 sender 知自己出左事
  warnvirussender => 1,
  warnspamsender = >1,
  warnbannedsender => 1,
  warnbadhsender => 1,

  # forward to a smtpd service providing DKIM signing service
  #forward_method => 'smtp:[127.0.0.1]:10027',

  # force MTA conversion to 7-bit (e.g. before DKIM signing)
  #smtpd_discard_ehlo_keywords => ['8BITMIME'],

  #bypass_spam_checks_maps => [1],
  #bypass_virus_checks_maps => [1],

  terminate_dsn_on_notify_success => 0,  # don't remove NOTIFY=SUCCESS option
};

map usage

# 用來設定不同 domain 有不同的 admin

$virus_admin = "virusalert\@$mydomain";
# $virus_admin = '[email protected]';
# $virus_admin = undef;   # do not send virus admin notifications (default)

@virus_admin_maps = (    # by-recipient maps
  {'not.example.com' => '',
  {'an.example.com' => "postmaster\@an.example.com",
   '.' => '[email protected]'},
  $virus_admin,                                         # the usual default
);

Notification Template 類別

Template text = run-time macro

administrator notifications
sender (non)delivery notifications
recipient warnings

notify_spam_admin.txt

Date: %d
From: %f
Subject: SPAM FROM <%o> - \[%c\]
To: [? %#T |undisclosed-recipients: ;|[<%T>|, ]]
Message-ID: <SA%i@%h>

---------------------------------------------------------------------
ID Quarantaine : %i
---------------------------------------------------------------------
Policy  : %p
Hit     : %c sur _REQD_
From    : %o
To      : %T
CC      : %C
Subject : %j
Date    : %d
Client  : %a - %g

The message has been quarantined as: %q

[%A|\n]

macros

%A a list of SpamAssassin report lines for body report (a single string)

Macros

Call macro

_NAME_ or _NAME(...)_

i.e.

_SUMMARY_

Built-in macros selector

A macro is evaluated only in non-quoted context. Enclosing strings between tokens [" and "] prevents its evaluation.

[? arg1 | arg2 | ... ]    a selector
[~ arg1 | arg2 | ... ]    a regexp selector
[  arg1 | arg2 | ... ]    an iterator

[? 2   | zero | one | two | three ]  -> two
[? foo | none | any | two | three ]  -> any
[? 24  | 0    | one | many ]         -> many
[? 2   |No recipients]               -> (empty string)

SELECTOR

If it is a non-numeric string, it is treated as 0 for all-whitespace and as 1 otherwise.

If there is only one (the first) alternative available but
the value is greater than 0, an empty string is returned.

ITERATOR

All iterator's arguments are implicitly quoted

Examples:

[%V| ]     a space-separated list of virus names
[%V|\n]    a newline-separated list of virus names
[%V|
]          same thing: a newline-separated list of virus names

msg template DOC

https://www.ijs.si/software/amavisd/README.customize.txt

Turn off checking mails for sending

Sender:

The email address the sending machine gives in the MAIL FROM: command

# SOURCE 由 mynets

$policy_bank{'MYNETS'} = {
  originating => 1,
  os_fingerprint_method => undef,
  allow_disclaimers => 1,
  bypass_virus_checks_maps => [1],    # <-- Add this line.
};

Amavisd-new uses @bypass_*_checks_maps static maps as a way to bypass checks for listed recipients/domains.

# Amavisd 不同的 PORT 不同的 bank

$inet_socket_port = [10024, 10026];

$interface_policy{'10026'} = 'INTERNAL';

$policy_bank{'INTERNAL'} = {  # mail originating from the internal server
  bypass_spam_checks_maps   => [1],  # don't spam-check outgoing mail
  bypass_banned_checks_maps => [1],  # don't banned-check outgoing mail

#    bypass_header_checks_maps => [1],  # don't header-check this mail
#    bypass_banned_checks_maps => [1],

  final_spam_destiny   => D_PASS, # insure spam passes
  final_banned_destiny => D_PASS, # insure banned files pass
};

Postfix bypress filter (By IP)

When the FILTER mechanism is used in an access map, the result is DUNNO => continue on to the next (bypress)

master.cf

smtpd_client_restrictions =
    .....
    check_client_access hash:/etc/postfix/amavis_bypass
    .....

contents of /etc/postfix/amavis_bypass:

192.168.1.41 FILTER smtp-amavis:[127.0.0.1]:10026

Remark

Actually: configure Postfix to use a content filter on port 10026
for authenticated mail, and on port 10024 otherwise
(or pick any two unused port numbers to your liking).

Postfix to NOT check mails from sasl authenticated users

# Use a amavisd filter after sasl authentication

postfix(smtp-amavis)->amavis(10024)->postfix(10025)

master.cf:

smtp-amavis unix -  -   -   -   4  smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20

127.0.0.1:10025 inet n  -   n   -   -  smtpd
    -o syslog_name=postfix/10025
    -o content_filter=
    -o mynetworks_style=host
    -o mynetworks=127.0.0.0/8
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o strict_rfc821_envelopes=yes
    -o smtp_tls_security_level=none
    -o smtpd_tls_security_level=none
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_end_of_data_restrictions=
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks

main.cf:

# Default 沒有 filter
content_filter =

# check_sender_access 在 permit_sasl_authenticated 後
smtpd_sender_restrictions =
  permit_sasl_authenticated,
  check_sender_access regexp:/etc/postfix/amavisd.regexp

amavisd.regexp:

/^/ FILTER smtp-amavis:[127.0.0.1]:10024

Remark

max_use (default: 100)

The maximal number of incoming connections that a Postfix daemon process will service before terminating voluntarily.

Filtering E-Mails per Recipient Domain(Local)

cd /etc/postfix/

main.cf:

# Default filter (咩都掃)
content_filter = smtp-amavis:[127.0.0.1]:10024

# check_recipient_access 要在 reject_unauth_destination 之後 !!!
smtpd_recipient_restrictions =
    permit_mynetworks,
    reject_unauth_destination,
    check_recipient_access pcre:/etc/postfix/filter_recipient_domains.cf,
    ...............

filter_recipient_domains.cf:

/example.com/               FILTER amavisfeed:[127.0.0.1]:10026

postmap /etc/postfix/filter_recipient_domains

Checking:

postmap -q "example.com" /etc/postfix/filter_recipient_domains.cf

FILTER amavisfeed:[127.0.0.1]:10026

SASL users connect to port 25 use different bank

cd /etc/postfix/

master.cf

# Default filter (咩都掃)
content_filter = smtp-amavis:[127.0.0.1]:10024

smtpd_sender_restrictions =
 reject_unknown_sender_domain,
 reject_non_fqdn_sender,
 reject_unlisted_sender,
 check_sasl_access pcre:/etc/postfix/per_user_filter.cf,
 permit_mynetworks,
 check_client_access hash:/etc/postfix/mxserver.cf,
 reject_sender_login_mismatch,
 permit_sasl_authenticated

Remark

* check_sasl_access 要在 permit_mynetworks 之前 !! 因為 webmail 未必用 permit_sasl_authenticated

* check_sasl_access use the remote SMTP client SASL user name as lookup key for the specified access database.

sasl_access.cf:

# 以下 login 用另一個 FILTER 
/[email protected]/     FILTER smtp-amavis:[127.0.0.1]:10026

Acting on mail checks results

$final_*_destiny

$final_*_destiny = D_BOUNCE / D_REJECT / D_PASS / D_DISCARD / @*_lovers_maps

D_BOUNCE

MTA receives a 2xx status (success)

A non-delivery notification (bounce) will be created by amavisd-new and sent to the sender by amavisd-new.

bounce (DSN) will not be sent if a virus name matches @viruses_that_fake_sender_maps

or for spam level that exceeds the $sa_dsn_cutoff_level.

Mail will not be delivered to its recipients.

D_REJECT

Amavisd-new will send the typical 550 (or 554) reject response to the upstream MTA

Mail will not be delivered to its recipients.

D_DISCARD

Mail will not be delivered to its recipients and the sender normally will NOT be notified (會去 $virus_quarantine_to)

D_PASS

除了原本的收件者外, @*_lovers_maps 的人都收到

我的設定

#### final_*_destiny ####

$final_virus_destiny      = D_DISCARD;
$final_banned_destiny     = D_REJECT;
$final_spam_destiny       = D_REJECT;
$final_bad_header_destiny = D_PASS;

官方 pre-cleanup:

      .......................................
      :                Postfix              :
   ----->smtpd \                            :
      :         -pre-cleanup-\       /local---->
   ---->pickup /              -queue-       :
      :             -cleanup-/   |   \smtp----->
      :     bounces/    ^        v          :
      : and locally     |        v          :
      :   forwarded   smtpd  amavisfeed     :
      :    messages   10025      |          :
      ...........................|...........
                        ^        |
                        |        v
            ............|...............................
            :           |   $inet_socket_port=10024    :
            :           |                              :
            : $forward_method='smtp:[127.0.0.1]:10025' :
            : $notify_method ='smtp:[127.0.0.1]:10025' :
            :                                          :
            :    amavisd-new                           :
            ............................................

pre-cleanup unix    n       -       n       -       0       cleanup
    -o virtual_alias_maps=

cleanup unix    n       -       n       -       0       cleanup
    -o mime_header_checks=
    -o nested_header_checks=
    -o body_checks=
    -o header_checks=

pickup    fifo  n       -       n       60      1       pickup
    -o cleanup_service_name=pre-cleanup
smtp      inet  n       -       n       -       -       smtpd
    -o cleanup_service_name=pre-cleanup
submission inet n       -       n       -       -       smtpd
    -o cleanup_service_name=pre-cleanup

E-Mail add Header

X-Spam-* headers

X-Spam-Flag
X-Spam-Score
X-Spam-Level
X-Spam-Status <-- 很長

Example: mail header

# Default

X-Spam-Flag: NO
X-Spam-Score: 1.059
X-Spam-Level: *
X-Spam-Status: No, score=1.059 tagged_above=-999 required=5
    tests=[MISSING_MID=0.14, SPF_FAIL=0.919]
    autolearn=no autolearn_force=no

P.S.

X-Spam-Score: -1.789

Force ALL domain adding of the X-Spam-* headers

# 必須要有 "@local_domains_maps" setting 才會加 headers

# 懶做法是

@local_domains_maps = 1;

Add header

$allowed_added_header_fields{lc('X-Spam-Flag')} = 1;
$allowed_added_header_fields{lc('X-Spam-Score')} = 1;
$allowed_added_header_fields{lc('X-Spam-Level')} = 1;
$allowed_added_header_fields{lc('X-Spam-Status')} = 1;

X-Virus-Scanned:

修改 X-Virus-Scanned header field to mail

$X_HEADER_LINE = "By $myproduct_name using ClamAV & SpamAssassin at $mydomain";

Default:

X-Virus-Scanned: amavisd-new at $mydomain

Remove Existing X header

# 0 => leave existing X-Virus-Scanned alone (defaults)

# 1 => remove existing headers

$remove_existing_x_scanned_headers = 1;
$remove_existing_spam_headers = 1;

Amavisd 的 MTA header

$allowed_added_header_fields{lc('Received')} = 0;

我的設定

#### HEADER SETTING ####
$remove_existing_x_scanned_headers = 1;
$remove_existing_spam_headers = 1;

@local_domains_maps = ( "." );

$X_HEADER_LINE = "By $myproduct_name using ClamAV & SpamAssassin at $mydomain";

# Add header
$allowed_added_header_fields{lc('X-Spam-Status')} = 1;
$allowed_added_header_fields{lc('X-Spam-Level')} = 1;
$allowed_added_header_fields{lc('X-Spam-Flag')} = 1;
$allowed_added_header_fields{lc('X-Spam-Score')} = 1;

Housekeeping

* Discarding indexes makes deletion faster;

# 刪除東西, 之後再 Create 返

DROP INDEX msgs_idx_sid         ON msgs;
DROP INDEX msgrcpt_idx_rid      ON msgrcpt;
DROP INDEX msgrcpt_idx_mail_id  ON msgrcpt;

CREATE INDEX msgs_idx_sid        ON msgs    (sid);
CREATE INDEX msgrcpt_idx_rid     ON msgrcpt (rid);
CREATE INDEX msgrcpt_idx_mail_id ON msgrcpt (mail_id);

* delete unreferenced records from tables msgrcpt and quarantine

DELETE msgrcpt FROM msgrcpt LEFT JOIN msgs USING(mail_id)
  WHERE msgs.mail_id IS NULL;
DELETE quarantine FROM quarantine LEFT JOIN msgs USING(mail_id)
  WHERE msgs.mail_id IS NULL;

# 很久的 msgs

DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP() - 21*24*3600;

# sender domains with >100 messages, sorted on sender.domain:

SELECT count(*) as cnt, avg(bspam_level) as spam_avg, sender.domain
  FROM msgs
  LEFT JOIN msgrcpt ON msgs.mail_id=msgrcpt.mail_id
  LEFT JOIN maddr AS sender ON msgs.sid=sender.id
  LEFT JOIN maddr AS recip ON msgrcpt.rid=recip.id
  GROUP BY sender.domain HAVING count(*) > 100
  ORDER BY sender.domain DESC LIMIT 100;

最後

// Optimize tables

OPTIMIZE TABLE msgs, msgrcpt, quarantine, maddr;

amavisd connect SQL

/etc/amavisd/amavisd.conf

@lookup_sql_dsn = ...

# tells amavisd how to fetch per-recipient policy and whitelist/blacklist settings
#$sql_select_policy = ...
#$sql_select_white_black_list = ...

Default

SELECT *,users.id FROM users,policy
  WHERE (users.policy_id=policy.id) AND (users.email IN (%k))
  ORDER BY users.priority DESC;

SELECT wb FROM wblist,mailaddr
  WHERE (wblist.rid=?) AND (wblist.sid=mailaddr.id) AND (mailaddr.email IN (%k))
  ORDER BY mailaddr.priority DESC;

Tables

maddr                     // provide unique id for each e-mail address (id, email, domain )
mailaddr                 // used as senders in wblist (external or local) (id, priority, email)
msgrcpt                   // per-recipient information related to each processed message (與 msgs.mail_id 有關聯)
msgs
policy                      // 'id' this is the _only_ required field
quarantine               // mail_id, chunk_ind, mail_text
users                      // local users, 0 is low prior, policy_id default: 1 (id, priority, policy_id, email, fullname, local)
wblist                     // per-recipient whitelist (rid: users.id, sid: mailaddr.id, wb: W or Y / B or N)

Example:

INSERT INTO users VALUES ( 1, 9, 5, '[email protected]','Name1 Surname1', 'Y');
INSERT INTO users VALUES (10, 3, 8, 'userA', 'NameA SurnameA anywhere', 'Y');
INSERT INTO users VALUES (14, 3, 0, '@sub1.example.net', NULL, 'Y');
INSERT INTO users VALUES (16, 3, 5, '@example.net',      NULL, 'Y');
-- INSERT INTO users VALUES (20, 0, 5, '@.',             NULL, 'N');  -- catchall
 
INSERT INTO mailaddr VALUES (1, 5, '@example.com');
INSERT INTO mailaddr VALUES (2, 9, '[email protected]');
INSERT INTO mailaddr VALUES (3, 9, '[email protected]');

INSERT INTO wblist VALUES (14, 1, 'W');
INSERT INTO wblist VALUES (14, 3, 'W');
INSERT INTO wblist VALUES (17, 2, 'W');
INSERT INTO wblist VALUES (17, 3, 'W');

INSERT INTO policy (id, policy_name,
  virus_lover, spam_lover, banned_files_lover, bad_header_lover,
  bypass_virus_checks, bypass_spam_checks,
  bypass_banned_checks, bypass_header_checks, spam_modifies_subj,
  spam_tag_level, spam_tag2_level, spam_kill_level) VALUES
  (1, 'Non-paying',    'N','N','N','N', 'Y','Y','Y','N', 'Y', 3.0,   7, 10),
  (2, 'Uncensored',    'Y','Y','Y','Y', 'N','N','N','N', 'N', 3.0, 999, 999),
  (3, 'Wants all spam','N','Y','N','N', 'N','N','N','N', 'Y', 3.0, 999, 999),
  (4, 'Wants viruses', 'Y','N','Y','Y', 'N','N','N','N', 'Y', 3.0, 6.9, 6.9),
  (5, 'Normal',        'N','N','N','N', 'N','N','N','N', 'Y', 3.0, 6.9, 6.9),
  (6, 'Trigger happy', 'N','N','N','N', 'N','N','N','N', 'Y', 3.0,   5, 5),
  (7, 'Permissive',    'N','N','N','Y', 'N','N','N','N', 'Y', 3.0,  10, 20),
  (8, '6.5/7.8',       'N','N','N','N', 'N','N','N','N', 'N', 3.0, 6.5, 7.8),
  (9, 'userB',         'N','N','N','Y', 'N','N','N','N', 'Y', 3.0, 6.3, 6.3),
  (10,'userC',         'N','N','N','N', 'N','N','N','N', 'N', 3.0, 6.0, 6.0),
  (11,'userD',         'Y','N','Y','Y', 'N','N','N','N', 'N', 3.0,   7, 7);

-- $sql_select_policy setting in amavisd.conf tells amavisd how to fetch per-recipient policy settings.
-- SELECT *,users.id FROM users,policy
--   WHERE (users.policy_id=policy.id) AND (users.email IN (%k))
--   ORDER BY users.priority DESC;
--
-- $sql_select_white_black_list in amavisd.conf tells amavisd how to check sender in per-recipient whitelist/blacklist.
-- See comments there. Example:
-- SELECT wb FROM wblist,mailaddr
--   WHERE (wblist.rid=?) AND (wblist.sid=mailaddr.id) AND (mailaddr.email IN (%k))
--   ORDER BY mailaddr.priority DESC;

NOTE:

The SELECT, INSERT and UPDATE clauses as used by the amavisd-new program are configurable through %sql_clause

reset Bayes 學錯的 eml

[1] 看看 bayes 的 DB 是否在 Default 位置

ls -l ~amavis/.spamassassin

[2] 用 amavis 的身份 reset 佢

su amavis

sa-learn --forget tmp.eml

av_scanners

@av_scanners = (
  ### http://www.clamav.net/
  ['ClamAV-clamd',
    \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamd.amavisd/clamd.sock"],
    qr/\bOK$/m, qr/\bFOUND$/m,
    qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],
  # NOTE: run clamd under the same user as amavisd - or run it under its own
  #   uid such as clamav, add user clamav to the amavis group, and then add
  #   AllowSupplementaryGroups to clamd.conf;
  # NOTE: match socket name (LocalSocket) in clamav.conf to the socket name in
  #   this entry; when running chrooted one may prefer a socket under $MYHOME.
);

Email address 的 lookups 次序

Hash lookups

following order:
- lookup for [email protected]
- lookup for [email protected] (only if $recipient_delimiter is '+')
- lookup for user+foo@
- lookup for user@ (only if $recipient_delimiter is '+')
- lookup for sub.example.com
- lookup for .sub.example.com
- lookup for .example.com
- lookup for .com
- lookup for .

SQL LOOKUPS

9 - lookup for [email protected]
8 - lookup for [email protected]                (only if $recipient_delimiter is '+')
7 - lookup for user+foo                                      (only if domain part is local)
6 - lookup for user                                             (only local; only if $recipient_delimiter is '+')
5 - lookup for @sub.example.com
3 - lookup for @.sub.example.com
2 - lookup for @.example.com
1 - lookup for @.com
0 - lookup for @.                                                (catchall)

容許 exe 檔通過

要同時刪了以下兩句內的 exe-ms 及 exe, 否則部份 exe 不能通過

  qr'^\.(exe-ms|dll|js)$',                   # banned file(1) types, rudimentary
# qr'^\.(exe|lha|cab|dll)$',              # banned file(1) types

原因是

log

... "... | P=p004,L=1/2/1,T=exe,T=exe-ms,N=software.exe", matching_key="(?-xism:^\134.(exe-ms|dll|js)$)"
... Blocked BANNED (.exe,.exe-ms,software.exe)

".docx" often incorrectly detected as ".exe"

log

Nov 29 23:03:47 mail amavis[11917]: ...
 | P=p026,L=1/2/21,T=exe,T=exe-ms,N=[trash]/0000.dat", matching_key="(?-xism:^\\.(exe-ms|dll)$)"

fix

$banned_filename_re = new_RE(
    [qr'^\[trash\]/[0-9a-f]{4}\.dat$' => 0 ], # allow trash sections of docx files
    ...
);

Troubleshoot

Error 1: 誤 block 了 PDF 檔

amavis[13049]: (13049-16) Blocked BANNED (application/x-msdownload,.pdf,KV-6521 20131202.pdf)

原因: MIME type "application/x-msdownload" instead of "application/pdf"

原來 Mail Client "JavaMail.open-xchange" 出的信 MINE 是這樣 ...

Error 2: 有 mail Incoming server 時會有以下 log

Mar 11 11:48:39 mymail amavis[16435]: (16435-17) Open relay?
Nonlocal recips but not originating: user at somewhere.net

在 configure file 加入

$policy_bank{'MYNETS'} = {
  originating                 => 1,
  allow_disclaimers           => 0,
  log_level                   => 1,
};

$interface_policy{'10024'} = 'SASL_AUTH';
$policy_bank{'SASL_AUTH'} = {
  originating => 1,  # indicates client is ours, allows signing
};

P.S.

For more complex setups where your users submit mail from foreign networks,

you need to set up a dedicated policy bank with originating=>1,

attach it to a dedicated TCP port, then configure Postfix to pass authenticated mail from MSA to such port.

Error 3: spamassassin

所有 mail 都中 "FH_DATE_PAST_20XX" 被加 3 分

describe FH_DATE_PAST_20XX The date is grossly in the future.

/usr/share/spamassassin/72_active.cf

header   FH_DATE_PAST_20XX      Date =~ /20[1-9][0-9]/ [if-unset: 2006]

Fix

/etc/mail/spamassassin/local.cf

score FH_DATE_PAST_20XX 0.0

service amavisd restart

Doc

瀏覽次數： 2744

夢想家

amavisd configure & usage

目錄

前言

介紹

處理 E-Mail 的過程

Install

Amavisd Cli

amavisd 基本設定

Other tools

include_config_files

Postfix Setting

設置2

Debian Config Files

LOG

Mysql Backend

Example: Whitelist

Example: policy

local_domains 與 MYUSERS

阻隔某類附件

Quarantine

在 Quarantine 放出 Mail

Notify

Turn off checking mails for sending

Acting on mail checks results

官方 pre-cleanup:

E-Mail add Header

Housekeeping

最後

amavisd connect SQL

reset Bayes 學錯的 eml

av_scanners

Email address 的 lookups 次序

容許 exe 檔通過

".docx" often incorrectly detected as ".exe"

Troubleshoot

Doc