banner

Introduction

A passwd entry is a database record that contains some basic information about a user. Traditionally this information included the user’s password, which is why a passwd entry is called a passwd entry, but these days the password itself is typically stored somewhere more secure.

Consequently, passwd entries can be used for listing or validating users but cannot be used for authenticating users.

Both Dovecot and Postfix must deal with this dichotomy.

This article:

In July 2025, Dovecot reorganised their online documentation to such an extent that there are no longer equivalents of the links that were provided in this article. Accordingly, all links to official Dovecot documentation have been deleted.

A common Dovecot+Postfix configuration with LDAP support

On networks with real Unix accounts for each mail user, a common Postfix+Dovecot configuration is:

  1. Dovecot validates and authenticates users via LDAP
  2. Dovecot offers an LDAP user authentication service to Postfix
  3. Postfix authenticates users by using the service provided by Dovecot
  4. Postfix validates users via LDAP (using OS-level LDAP services)

The first and last points need some clarification.

Firstly, Dovecot may need to look up the user's passwd entry or authenticate a user for several reasons:

We tell Dovecot to use LDAP as follows:

mandala# fgrep -B 1 -A 1 ldap /etc/dovecot/dovecot.conf
userdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}
--
passdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}

and we tell Dovecot the LDAP connection parameters as follows:

mandala# egrep -v '^ *(#|$)' /etc/dovecot/dovecot-ldap.conf
base = ou=Users,dc=pasta,dc=net
uris = ldaps://ldap.pasta.freemyip.com/
auth_bind = yes
auth_bind_userdn = uid=%u,ou=Users,dc=pasta,dc=net
tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt
...

With this information, Dovecot can establish a TCP connection to the LDAP server and make its query.

Additionally, we tell Dovecot how to hone its query and how to interpret the results:

...
user_filter = (&(objectClass=posixAccount)(uid=%n))
user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid
mandala#

Secondly, Postfix may need to look up the user’s passwd entry or authenticate a user for several reasons:

In the common Dovecot+Postfix configuration outlined earlier, Postfix delegates looking up a user’s password entry to either NSS or PAM and delegates authenticating a user to Dovecot.

The delegation to NSS or PAM are very similar, consisting of a series of functions calling other functions calling other functions with the last function connecting to an LDAP server, submitting a query, fetching results and passing them back up the call stack. I am going to describe the NSS route because I am more familiar with it and it is easier to follow.

The C standard libary contains a few functions for looking up passwd entries:

We would expect that the Postfix source code contains calls to some of these and we can confirm this by running:

mandala# dpkg -L postfix | xargs nm -D 2>/dev/null \
    sort -u | grep getpw
U getpwnam@GLIBC_2.2.5
U getpwnam_r@GLIBC_2.2.5
U getpwuid@GLIBC_2.2.5
U getpwuid_r@GLIBC_2.2.5
U getpwnam@GLIBC_2.2.5
U getpwuid@GLIBC_2.2.5
U getpwnam@GLIBC_2.2.5
U getpwnam@GLIBC_2.2.5
mandala#

That command is listing the contents of the postfix package - including directories, binaries, man pages, READMEs - and searching all of them for calls to functions containing getpw in their names. That search is obviously going to produce errors when it looks for calls inside directories, man pages and READMEs, so we discard any error messages. We are left with:

We will follow the route taken by getpwnam(3), but similar routes would apply to the other functions.

We have already established that Postfix calls getpwnam(3) and we can reasonably assume that it does this for the reasons described above. Note that “reasonably assume”; using nm to examine the functions that a program or library can call does not prove that the program or library actually calls a particular function, but it does strongly suggest that - in some circumstances - it does.

So then getpwnam(3) looks in /etc/nsswitch.conf to see where passwd entries could be stored and finds this:

mandala# grep passwd /etc/nsswitch.conf
passwd: files ldap
mandala#

So then getpwnam(3) checks if the user is mentioned in the local /etc/passwd file by delegating to this function:

mandala# nm -D /lib/x86_64-linux-gnu/libnss_files.so \
    | grep getpwnam
0000000000006770 T _nss_files_getpwnam_r@@GLIBC_PRIVATE
mandala#

and then - assuming the user it is looking up is not listed in /etc/passwd because it is listed in LDAP instead - it checks in LDAP by delegating to this function:

mandala# nm -D /lib/x86_64-linux-gnu/libnss_ldap.so.2 \
    | grep getpwnam
0000000000006900 T _nss_ldap_getpwnam_r@@EXPORTED
mandala#

nss_ldap_getpwnam_r(3) delegates to nslcd(8),the local LDAP name service daemon, which does LDAP queries on behalf of local processes:

mandala# ps -C nslcd
PID TTY TIME CMD
1075 ? 00:00:01 nslcd
mandala#

nss_ldap_getpwnam_r(3) probably communicates with nslcd through one of these sockets:

mandala# lsof -p 1075 -E | grep unix
nslcd 1075 nslcd 5u unix 0x0000000092a56bcc 0t0 18974 type=DGRAM ->INO=11198 229,systemd-j,6u 1,systemd,122u
nslcd 1075 nslcd 6u unix 0x00000000fb6e0ef7 0t0 18989 /var/run/nslcd/socket type=STREAM
mandala#

nslcd looks - or rather it looked when it was first started - in /etc/ldap/ldap.conf to get the LDAP connection parameters:

mandala# egrep -v '^ *($|#)' /etc/nslcd.conf
uid nslcd
gid nslcd
uri ldaps://ldap.pasta.freemyip.com/
base dc=pasta,dc=net
tls_cacertfile /etc/ssl/certs/ca-certificates.crt
mandala#

and connects to the LDAP server and forwards it the query.

The LDAP server replies to nslcd, which replies to nss_ldap_getpwnam_r(3), which replies to getpwnam(3), which replies to Postfix.

So what's wrong with that and how can we fix it?

Can it be simplified?

We will consider each of these options in the next sections.

Can Postfix and Dovecot be configured to access LDAP via NSS?

I would guess that making Postfix not delegate user authentication to Dovecot, by removing these lines from /etc/postfix/main.cf:

smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

would probably be enough to make it use NSS for user authentication instead.

However, making Dovecot use NSS is a bigger problem. The userdb documentation leads to the passwd authentication documentation, which suggests it is possible, by setting:

userdb {
    driver = passwd
    args = blocking=no
}

but the same page says that passwd cannot be used for passdb, because of the splitting out of the password into /etc/shadow.

There is the alternative nss driver, which is probably better (because probably the passwd driver goes straight to /etc/passwd, whereas nss will go to /etc/nsswitch.conf). It is documented here but that documentation says that a service parameter is required , like this:

userdb {
    driver = nss
    args = service=ldap
}

But we do not want to tell Dovecot to use LDAP; we want to tell it to use NSS (and for nss to look in /etc/nsswitch.conf and discover that I want it to use LDAP). I could not find any documention regarding what other args = server=whatever options were possible. If we omit args=service=ldap and duplicate the stanza for the passdb, i.e.:

userdb {
    driver = nss
}

passdb {
    driver = nss
}

then starting up mutt results in Dovecot complaining:

Dec 15 09:42:20 mandala dovecot: auth: Fatal: Unknown passdb driver 'nss'

The passdb page lists (rather obtusely) the possible passdb drivers. The equivalent of userdb/passwd seems to be passdb/shadow but the shadow authentication page says that can be problematic and that driver pam is preferred.

The PAM page makes for pessmistic reading but the file it refers to - /etc/pam.d/dovecot, which goes on to include /etc/pam.d/common-auth - actually looks quite suitable:

mandala# egrep -v '^(#|$)' /etc/pam.d/common-auth
auth [success=2 default=ignore] pam_unix.so nullok
auth [success=1 default=ignore] pam_ldap.so minimum_uid=1000 use_first_pass
auth requisite pam_deny.so
auth required pam_permit.so
mandala#

But rather inconsistently there is no pam driver for the userdb database. The closest seems to the nss driver. So let's try:

userdb {
    driver = nss
    #args = service=ldap
}

passdb {
    driver = pam
    args = failure_show_msg=yes
}

which causes:

Dec 15 10:02:39 mandala dovecot: auth: Fatal: Unknown userdb driver 'nss'

So NSS is not an option in Debian 11's Dovecot for either userdb or passdb.

What other userdb drivers would force Dovecot to delegate to the OS rather than going directly to LDAP? The userdb page does not list pam as a possibility but it does say:

The user and password databases … may be the same or they may be different depending on your needs.

If we try userdb/pam, then that fails with:

Dec 15 10:06:07 mandala dovecot: auth: Fatal: Unknown userdb driver 'pam'

So I conclude that both Postfix and Dovecot cannot be configured to access LDAP via NSS.

Can Postfix and Dovecot be configured to access LDAP directly?

We revert Dovecot to our original configuration:

userdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}

passdb {
    driver = ldap
    args = /etc/dovecot/dovecot-ldap.conf
}

with /etc/dovecot/dovecot-ldap.conf specifying how to connect to the LDAP server, hone queries and filter results, as discussed earlier.

Can we make Postfix talk directly to the LDAP server in the same way?

The postconf(5) man page does not mention accessing LDAP directly.

So I conclude that both Postfix and Dovecot cannot be configured to access LDAP directly.

A digression

While reading postconf(5) I saw this:

smtpd_relay_restrictions …

With Postfix versions before 2.10, the rules for relay permission and spam blocking were combined under smtpd_recipient_restrictions, resulting in error-prone configuration. As of Postfix 2.10, relay permission rules are preferably implemented with smtpd_relay_restrictions, so that a permissive spam blocking policy under smtpd_recipient_restrictions will no longer result in a permissive mail relay policy.

which looks like it might be useful!

Currently my smtp_recipient_restrictions are:

smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unauth_destination,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit

But this could be split this into:

smtpd_relay_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject

and:

smtpd_recipient_restrictions =
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    reject_unverified_recipient,
    check_policy_service inet:127.0.0.1:10023,
    permit

Note that the reject_unverified_recipient necessitates Postfix delegating to Dovecot and will result in a mail being rejected if the local part of the address does not match any known user (in LDAP).

Can Postfix be configured to delegate all LDAP communication to Dovecot?

I googled 'postfix validate local address dovecot' and that led here, which says:

With Dovecot 2.0 you can also use LMTP and the Postfix setting “reject_unverified_recipient” for dynamic address verification. It is really nice because Postfix does not need to query an external datasource (MySQL, LDAP…).

which sounds like what I want! I.e. stop Postfix from using (OS-level) LDAP and get it to use Dovecot.

Dovecot's page about LMTP says:

LMTP uses the same settings as LDA (see Common configuration), as specified in conf.d/15-lda.conf in example configuration. There is also a bit of extra configuration in conf.d/20-lmtp.conf.

I installed dovecot-lmtp package.

No settings are applied by default:

mandala# egrep -v '^ *(#|$)' /etc/dovecot/conf.d/15-lda.conf
protocol lda {
}
mandala# egrep -v '^ *(#|$)' /usr/share/dovecot/conf.d/20-lmtp.conf
protocol lmtp {
}
mandala#

I.e. no settings were applied by default.

I modified dovecot.conf:

protocols = " ... lmtp"

service lmtp {
    unix_listener /var/spool/postfix/private/dovecot-lmtp {
        group = postfix
        mode = 0600
        user = postfix
    }
}

and also in /etc/postfix/main.cf:

# This was the old setting
#mailbox_command = /usr/lib/dovecot/deliver
# This is the new setting
mailbox_transport = lmtp:unix:private/dovecot-lmtp

I restart Dovecot and Postfix.

My tests are as follows:

The test results were:

The digression revisited

By removing the two reject_rbl_client directives in the setting of smtpd_recipient_restrictions, I established that this mail that should be relayed is not being checked against smtpd_relay_restrictions, but rather it is checked against smtpd_recipient_restrictions, which is wrong: since (a) cercis is sending mail to alexis.huxley@mpcdf.mpg.de and (b) that domain is not in mydestination, then the relay rule should apply.

I found somebody asking for help with the same problem but no solution is offered.

Debian 11's postconf(5) suggests that that person and I are right to be puzzled:

smtpd_relay_restrictions (default: permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination)

Access restrictions for mail relay control that the Postfix SMTP server applies in the context of the RCPT TO command, before smtpd_recipient_restrictions. See SMTPD_ACCESS_README, section “Delayed evaluation of SMTP access restriction lists” for a discussion of evaluation context and time.

Note that “before”.

Googling 'smtpd_recipient_restrictions processed before smtp_relay_restrictions' sent me to this page, which includes:

smtpd_relay_before_recipient_restrictions (default: see “postconf -d” output)

Evaluate smtpd_relay_restrictions before smtpd_recipient_restrictions. Historically, smtpd_relay_restrictions was evaluated after smtpd_recipient_restrictions, contradicting documented behavior.

Background: the smtpd_relay_restrictions feature is primarily designed to enforce a mail relaying policy, while smtpd_recipient_restrictions is primarily designed to enforce spam blocking policy. Both are evaluated while replying to the RCPT TO command, and both support the same features.

This feature is available in Postfix 3.6 and later.

but:

mandala# postconf -d | grep smtpd_recipient_restrictions
mandala#

and indeed, my Postfix is too old to have this setting:

mandala# dpkg -l | grep postfix
ii postfix 3.5.6-1+b1 amd64 High-performance mail transport agent
mandala#

So the meaning of the two directives is:

Are there any other smtp_*_restrictions that might be helpful? Perhaps something like smtp_notrelay_restrictions?

mandala# man 5 postconf | grep '^smtpd_[^ ]*_restrictions'
smtpd_client_restrictions (default: empty)
smtpd_data_restrictions (default: empty)
smtpd_end_of_data_restrictions (default: empty)
smtpd_etrn_restrictions (default: empty)
smtpd_helo_restrictions (default: empty)
smtpd_recipient_restrictions (default: see postconf -d output)
smtpd_relay_restrictions (default: permit_mynetworks, permit_sasl_authenticated, defer_unauth_destination)
smtpd_sender_restrictions (default: empty)
mandala#

smtpd_client_restrictions would apply to google, not to a mail sent by a spammer to alexishuxley@gmail.com, so I think it is not useful.

smtpd_sender_restrictions would not help as senders are commonly faked.

How about I revert to using only smtpd_recipient_restrictions but now with the understanding that it's also for relayed mail?

# Restrictions for local *and* relayed mail.
smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unverified_recipient,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit

# No *additional* restrictions for relaying.
#smtpd_relay_restrictions =

I ran my tests:

It looks like lmtp needs to be told either to drop the @ sign and fully qualified domain name (FQDN) when it checks if the user is in LDAP, or it needs to be told what FQDNs are valid.

Googling 'dovecot lmtp drop domain' turned up a question on stackexchange, which suggests either to set auth_username_format in /etc/dovecot/conf.d/10-auth.conf:

auth_username_format = %Ln

or to set user_filter in /etc/dovecot/dovecot-ldap.conf to set:

user_filter = (&(objectClass=posixAccount)(uid=%n))

The auth_username_format's documentation says:

auth_username_format

Default: %Lu Values: String

Formatting applied to username before querying the auth database. You can use the standard variables here.

Examples:

%Lu - lowercases the username %n - drops the domain if one was supplied %n-AT-%d - changes the “@” symbol into “-AT-” before lookup

This translation is done after the changes specified with the auth_username_translation setting.

and the user_filter's documentation says:

user_filter

Default:

Values: String

Filter for user lookup (userdb lookup). See also LDAP Backend Configuration Below variables can be used.

Variable Long name Description %u %{user} username %n %{username} user part in user@domain, same as %u if there's no domain %d %{domain} domain part in user@domain, empty if user there's no domain

See Config Variables for full list

Example: user_filter = (&(objectClass=posixAccount)(uid=%u))

The former looks better because it takes effect slightly earlier. So I added the following to dovecot.conf:

auth_username_format = %Ln

and restarted Dovecot and re-ran my tests:

Since that did not work, I reverted that change to dovecot.conf.

It looks like lmtp is still looking up the wrong user. Perhaps auth_username_format does not apply when the username is coming from Postfix rather than from an IMAP client?

So then I tried the alternative. I changed the following in dovecot-ldap.conf:

#user_filter = uid=%u
user_filter = (&(objectClass=posixAccount)(uid=%n))

and restarted Dovecot and re-ran my tests:

Since that did not work, I reverted the change to dovecot-ldap.conf.

After some googling, I found a post on the Dovecot mailing list that described running doveadm -u user < _user_ > to test how Dovecot handles different addresses. So I commented out the setting of auth_username_format altogether, restarted Dovecot and ran:

mandala# doveadm user -u alexis@pasta.freemyip.com
userdb lookup: user alexis@pasta.freemyip.com doesn't exist
mandala#

This error was to be expected with the changes I just made.

So then I set:

auth_username_format = %Ln

restarted Dovecot and run the doveadm command and now it works:

mandala# doveadm user -u alexis@pasta.freemyip.com
userdb: alexis@pasta.freemyip.com
user : alexis
home : /home/alexis
uid : 1000
gid : 1000
mandala#

So although when I tested this setting of auth_username_format above, all my tests failed, I do now know that I am on the right track!

Perhaps private/dovecot-lmtp says alexis@pasta.freemyip.com not only in the message but also when it talks to the LDAP server?

As a stupid test, on my LDAP server orzo, I cloned the uid=alexis entry to uid=alexis@pasta.freemyip.com:

orzo# shelldap
~ > cd ou=Users
ou=Users,~ > ls
- uid=alexis
- uid=cercis
- uid=guest
- uid=repomaster
- uid=suzie
ou=Users,~ > cp uid=alexis uid=alexis@pasta.freemyip.com
Success
ou=Users,~ >

and then I retried (no need to restart Dovecot), but it made no difference: the test still failed.

Reading further in the same link I ran:

mandala# doveadm auth test alexis@pasta.freemyip.com
Password:
passdb: alexis@pasta.freemyip.com auth succeeded
extra fields:
user=alexis
original_user=alexis@pasta.freemyip.com
mandala#

At this point I felt quite stuck and I started composing an email to the Dovecot mailing list. However ...

Does this configuration necessitate virtual users?

... before I posted that it dawned on my that 'virtual' (a term bandied about in the Postfix documentation), could refer to users that do not have corresponding Unix accounts, which is the case when I stop nslcd. However, after more thought, I concluded:

  1. virtual was going to be much too complicated (there are so many different directives for main.cf that contain 'virtual'!)
  2. probably I would have to tell Postfix how to contact LDAP itself (or at OS level) and one of the points of this whole procedure was to get Postfix to always delegate interactions with LDAP to Dovecot
  3. telling Postfix to delegate to Dovecot only for user validation and user authentication is probably not sufficient (perhaps Postfix wants to examine a user's .forward file and so wants to know where ~user is).

So I think virtual users is not a viable solution.

Which part of Postfix is it that is complaining that it cannot find a user?

The last error message was:

Dec 16 09:40:17 mandala postfix/smtpd[1512]: NOQUEUE: reject: RCPT from b1962.mx.srv.dfn.de[194.95.234.160]: 550 5.1.1 <alexis@pasta.freemyip.com>: Recipient address rejected: User unknown in local recipient table; from=<alexis.huxley@mpcdf.mpg.de> to=<alexis@pasta.freemyip.com> proto=ESMTP helo=<b1962.mx.srv.dfn.de>

It's smtpd deciding that there is no local user! The recipient restriction that says to ask dovecot/lmtp did not reject the mail, so Dovecot understood that this is a local user (or a mail to be relayed, which would not cause it to reject it). Googling that message 'User unknown in local recipient table' turns up some Postfix documentation that says:

As of Postfix version 2.0, the Postfix SMTP server rejects mail for unknown recipients in local domains (domains that match $mydestination or the IP addresses in $inet_interfaces or $proxy_interfaces) with “User unknown in local recipient table”. This feature was optional with earlier Postfix versions.

So smtpd is validating the local user. With nslcd stopped it fails but it nslcd running it succeeds.

Reading further at that link:

The local_recipient_maps parameter specifies lookup tables with all names or addresses of local recipients. A recipient address is local when its domain matches $mydestination, $inet_interfaces or $proxy_interfaces. If a local username or address is not listed in $local_recipient_maps, then the Postfix SMTP server will reject the address with “User unknown in local recipient table”.

So this means that smtpd did not find 'alexis' in local_recipient_maps. So what is local_recipient_maps? I do not define it in main.cf and the default is:

mandala# postconf -d local_recipient_maps
local_recipient_maps = proxy:unix:passwd.byname $alias_maps
mandala#

So that explains it! getent passwd alexis fails when nslcd is not running and succeeds when it is. So what should I set local_recipient_maps to avoid it delegating to the OS?

postconf(5) says of local_recipient_maps:

If this parameter is non-empty (the default), then the Postfix SMTP server will reject mail for unknown local users.

To turn off local recipient checking in the Postfix SMTP server, specify “local_recipient_maps =” (i.e. empty).

So should I set local_recipient_maps = (i.e. without a value)? Well, yes, I think it is safe to do this: remember that the smtpd_recipient_restrictions already say that if an invalid local user is recipient then the mail is to be rejected, so I think it is safe to say here that we just accept all mails (that are in $mydestination). So I added:

local_recipient_maps =

and re-ran my test:

Okay, so the local transport agent is complaining now, presumably because it cannot establish some attribute of user alexis that it can establish when nslcd is running; the home directory of the user is the most probable.

But why is the local agent not handing the mail to Dovecot? I have in my main.cf:

#mailbox_command = /usr/lib/dovecot/deliver
mailbox_transport = lmtp:unix:private/dovecot-lmtp

(mailbox_command was my original one; mailbox_transport was what I added earlier in this procedure.)

Regarding mailbox_transport, postconf(5) says:

Optional message delivery transport that the local(8) delivery agent should use for mailbox delivery to all local recipients, whether or not they are found in the UNIX passwd database.

Whoah! Well, that sounds reasonable, right? But obviously the local transport agent is encountering a problem before it passes the mail on to Dovecot. Perhaps we can get Postfix to pass the mail on to Dovecot earlier , i.e. before the local transport agent does this home lookup (if that is what it is that causes the error) or perhaps we can get Postfix to use Dovecot as the transport!

I believe I need a local_transport setting (if it is possible to set it to one thing) or local_transports_map (if it is possible to set it to a list). Indeed local_transport exists and can I copy the value from mailbox_transport (and discard mailbox_transport altogether as that is only used by the Postfix local transport), i.e.:

#mailbox_command = /usr/lib/dovecot/deliver
#mailbox_transport = lmtp:unix:private/dovecot-lmtp
local_transport = lmtp:unix:private/dovecot-lmtp

I re-ran my tests:

The main question posed by this article is now addressed, but along the way some other questions came up.

Can recipient restriction ordering be improved?

smtpd_recipient_restrictions is set to:

smtpd_recipient_restrictions =
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_unverified_recipient,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit

which leads to a couple of questions:

Mails sent from my work email (mpcdf) to alexis2@pasta.freemip.com (not a valid address) should be rejected by smtpd because of the reject_unverified_recipient directive, but mails sent from cercis (a remote SMTP client that authenticates ) to the same address would be accepted by smtpd because the permit_sasl_authenticated directive is earlier in the list, but then later rejected because when Postfix passes the mail to dovecot/lmtp in its role as the local delivery agent, dovecot/lmtp will reject it, which presumably will trigger a bounce that could have been avoided, right?

Let's verify:

Dec 16 10:35:16 mandala postfix/smtpd[2565]: connect from b1962.mx.srv.dfn.de[194.95.234.160]
Dec 16 10:35:16 mandala postgrey[243]: action=greylist, reason=new, client_name=b1962.mx.srv.dfn.de, client_address=194.95.234.160/32, sender=alexis.huxley@mpcdf.mpg.de, recipient=alexis2@pasta.freemyip.com
Dec 16 10:35:16 mandala postfix/smtpd[2565]: NOQUEUE: reject: RCPT from b1962.mx.srv.dfn.de[194.95.234.160]: 450 4.1.1 <alexis2@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command); from=<alexis.huxley@mpcdf.mpg.de> to=<alexis2@pasta.freemyip.com> proto=ESMTP helo=<b1962.mx.srv.dfn.de>
Dec 16 10:35:16 mandala postfix/smtpd[2565]: disconnect from b1962.mx.srv.dfn.de[194.95.234.160] ehlo=2 starttls=1 mail=1 rcpt=0/1 data=0/1 rset=1 quit=1 commands=6/8

Okay, that was a bit unfortunate that postgrey greylisted it, but oddly lmtp did reject it, ah, ok, I get it! postgrey got its message logged first, but actually ran second. We know the latter because we can see the order of directives in the setting of smtpd_recipient_restrictions. Err … but in that case why was postgrey even called at all?

I checked what postconf(5) says about smtpd_recipient_restrictions and it does not even mention check_policy_service! I am sure that - perhaps in an earlier version - it was allowed. check_policy_service is mentioned in the section for smtpd_client_restrictions (that's client, not recipient). The man page says to also see the SMTPD_POLICY_README document. Let's do that, to see if there is any mention of sucj checks being done in parallel or for backward compatibility still allowed to be declared in the recipient restrictions but actually done during the client restriction phase (CONNECT-time, rather than RCTP TO-time?).

I did not find any reference in SMTPD_POLICY_README to what happened.

However, after a couple of tests, the postgrey program had got used to the ‘alexis2’ address and did not interfere, leaving it to lmtp to reject it:

Dec 16 11:00:27 mandala postfix/smtpd[2612]: connect from b1962.mx.srv.dfn.de[194.95.234.160]
Dec 16 11:00:28 mandala postgrey[243]: action=pass, reason=triplet found, client_name=b1962.mx.srv.dfn.de, client_address=194.95.234.160/32, sender=alexis.huxley@mpcdf.mpg.de, recipient=alexis2@pasta.freemyip.com
Dec 16 11:00:28 mandala postfix/smtpd[2612]: NOQUEUE: reject: RCPT from b1962.mx.srv.dfn.de[194.95.234.160]: 450 4.1.1 <alexis2@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 >alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command); from=<alexis.huxley@mpcdf.mpg.de> to=<alexis2@pasta.freemyip.com> proto=ESMTP helo=<b1962.mx.srv.dfn.de>
Dec 16 11:00:28 mandala postfix/smtpd[2612]: disconnect from b1962.mx.srv.dfn.de[194.95.234.160] ehlo=2 starttls=1 mail=1 rcpt=0/1 data=0/1 rset=1 quit=1 commands=6/8

Interestingly, I did not immediately see a bounce mail at MPCDF; just in case it does arrive it was subject 'test404'). So the right thing happened there.

But what about when I sent it from cercis, which will do SASL authentication and so be permitted due to smtpd_recipient_restrictions? Indeed:

Dec 16 11:04:36 mandala postfix/smtpd[2618]: connect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]
Dec 16 11:04:36 mandala postfix/smtpd[2618]: ED21A42CC: client=88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216], sasl_method=PLAIN, sasl_username=cercis
Dec 16 11:04:37 mandala postfix/cleanup[2622]: ED21A42CC: message-id=>20211216100435.F40A9769@cercis.freemyip.com>
Dec 16 11:04:37 mandala postfix/qmgr[2507]: ED21A42CC: from=<root@cercis.freemyip.com>, size=611, nrcpt=1 (queue active)
Dec 16 11:04:37 mandala postfix/smtpd[2618]: disconnect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
Dec 16 11:04:37 mandala dovecot: lmtp(2624): Connect from local
Dec 16 11:04:37 mandala postfix/lmtp[2623]: ED21A42CC: to=<alexis2@pasta.freemyip.com>, relay=mandala.pasta.net[private/dovecot-lmtp], delay=0.32, delays=0.19/0.01/0.07/0.05, dsn=5.1.1, status=bounced (host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command))
Dec 16 11:04:37 mandala dovecot: lmtp(2624): Disconnect from local: Client has quit the connection (state=READY)
Dec 16 11:04:37 mandala postfix/cleanup[2622]: 3F63F4587: message-id=<20211216100437.3F63F4587@mandala.pasta.net>
Dec 16 11:04:37 mandala postfix/bounce[2625]: ED21A42CC: sender non-delivery notification: 3F63F4587
Dec 16 11:04:37 mandala postfix/qmgr[2507]: 3F63F4587: from=<>, size=2762, nrcpt=1 (queue active)
Dec 16 11:04:37 mandala postfix/qmgr[2507]: ED21A42CC: removed
Dec 16 11:04:38 mandala postfix/smtp[2626]: 3F63F4587: to=<root@cercis.freemyip.com>, relay=smtp.gmail.com[173.194.76.109]:25, delay=1.7, delays=0.02/0.02/0.59/1.1, dsn=2.0.0, status=sent (250 2.0.0 OK 1639649078 l26sm4024502wms.15 - gsmtp)
Dec 16 11:04:38 mandala postfix/qmgr[2507]: 3F63F4587: removed

The mail was accepted for delivery and smtpd closed the connection with the client, so the client thinks it is going to be delivered. Only later does lmtp complain that it cannot save the mail into alexis2's INBOX and postfix/bounce sends a bounce to root@cercis.freemyip.com. Ok, it will not be able to delivery that, but that is smtp.gmail.com's problem to deal with.

But could we avoid that unnecessary bounce by re-ordering the restrictions so that reject_unverified_recipient appears before permit_sasl_authenticated? But does reject_unverified_recipient cause mails that are to be relayed to be rejected because they do not match local users?

Let's try:

smtpd_recipient_restrictions =
    reject_unverified_recipient,
    permit_sasl_authenticated,
    permit_mynetworks,
    reject_rbl_client zen.spamhaus.org,
    reject_rbl_client bl.spamcop.net,
    check_policy_service inet:127.0.0.1:10023,
    permit

cercis->alexis2@pasta.freemyip.com:

Dec 16 11:11:27 mandala postfix/smtpd[2811]: connect from unknown[141.98.10.220]
Dec 16 11:11:27 mandala postfix/smtpd[2811]: warning: unknown[141.98.10.220]: SASL LOGIN authentication failed: Invalid authentication mechanism
Dec 16 11:11:27 mandala postfix/smtpd[2811]: disconnect from unknown[141.98.10.220] ehlo=1 auth=0/1 quit=1 commands=2/3
Dec 16 11:12:19 mandala postfix/smtpd[2811]: connect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]
Dec 16 11:12:19 mandala postfix/smtpd[2811]: NOQUEUE: reject: RCPT from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]: 450 4.1.1 <alexis2@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command); from=<root@cercis.freemyip.com> to=<alexis2@pasta.freemyip.com> proto=ESMTP helo=<cercis.freemyip.com>
Dec 16 11:12:19 mandala postfix/smtpd[2811]: disconnect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216] ehlo=2 starttls=1 auth=1 mail=1 rcpt=0/1 data=0/1 rset=1 quit=1 commands=7/9

That looks good. Interestingly, cercis mail logs said:

Dec 16 11:12:18 cercis postfix/pickup[8528]: B52D876F: uid=0 from=<root>
Dec 16 11:12:18 cercis postfix/cleanup[8930]: B52D876F: message-id=<20211216101218.B52D876F@cercis.freemyip.com>
Dec 16 11:12:18 cercis postfix/qmgr[1130]: B52D876F: from=<root@cercis.freemyip.com>, size=387, nrcpt=1 (queue active)
Dec 16 11:12:19 cercis postfix/smtp[8932]: B52D876F: to=<alexis2@pasta.freemyip.com>, relay=mail.pasta.freemyip.com[188.192.9.35]:25, delay=1, delays=0.07/0.04/0.84/0.1, dsn=4.1.1, status=deferred (host mail.pasta.freemyip.com[188.192.9.35] said: 450 4.1.1 <alexis2@pasta.freemyip.com>: Recipient address rejected: unverified address: host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <alexis2@pasta.freemyip.com> User doesn't exist: alexis2@pasta.freemyip.com (in reply to RCPT TO command) (in reply to RCPT TO command))

Note the status=deferred! So the mail queue … err … yes, the mailq command shows the mail sitting on cercis waiting for another attempt.

Okay, so that test looks good now, but what about when cercis wants to relay mails through my mail server?

cercis->mpcdf:

Dec 16 11:18:08 mandala postfix/smtpd[2828]: connect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216]
Dec 16 11:18:08 mandala postfix/smtpd[2828]: 924B6420C: client=88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216], sasl_method=PLAIN, sasl_username=cercis
Dec 16 11:18:08 mandala postfix/cleanup[2835]: 924B6420C: message-id=<20211216101808.16E9376E@cercis.freemyip.com>
Dec 16 11:18:08 mandala postfix/qmgr[2809]: 924B6420C: from=<root@cercis.freemyip.com>, size=611, nrcpt=1 (queue active)
Dec 16 11:18:08 mandala postfix/smtpd[2828]: disconnect from 88-111-68-216.dynamic.dsl.as9105.com[88.111.68.216] ehlo=2 starttls=1 auth=1 mail=1 rcpt=1 data=1 quit=1 commands=8
Dec 16 11:18:10 mandala postfix/smtp[2838]: 924B6420C: to=<alexis.huxley@mpcdf.mpg.de>, relay=smtp.gmail.com[173.194.76.108]:25, delay=2.1, delays=0.07/0.03/0.64/1.3, dsn=2.0.0, status=sent (250 2.0.0 OK 1639649890 m36sm4207359wms.25 - gsmtp)
Dec 16 11:18:10 mandala postfix/qmgr[2809]: 924B6420C: removed

So the right thing happened: 'alexis.huxley' is not a local user but did not cause a rejection because reject_unverified_recipient is only checking when the domain matches something in $mydestination. Does postconf(5) confirm that is what reject_unverified_recipient does?

It is not clear because it says this is delegated to the verify(8) Postfix subsystem and I did not read that far. But what it did say was this:

The unverified_recipient_reject_code parameter specifies the numerical response code when an address is known to bounce (default: 450, change into 550 when you are confident that itis safe to do so).

Since Dovecot and Postfix have their own LDAP configuration, can I remove the OS's LDAP configuration?

I have deleted much of the original progress log stored here as it was very incoherent and complicated by use of PCMS. A summary is presented instead.

Without the following line in /etc/ldap/ldap.conf:

TLS_CACERT /etc/ssl/certs/ca-certificates.crt

mail does not work. But it looks like I can tell Dovecot directly about the certificate authorities instead; I added this to dovecot-ldap.conf:

tls_ca_cert_file = /etc/ssl/certs/ca-certificates.crt

and re-commented-out the line in /etc/ldap/ldap.conf, at which point there are no ldap processes or files outside of Dovecot.

Is it only the Postfix local delivery agent that understands aliases?

/etc/aliases specifies that mail to root should be sent to alexis.

I ran tests:

which worked!

main.cf contains:

alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases

which raises some questions:

postconf(5) does not really clarify this but does suggest that the local transport agent is still involved. I think I need simply to test it but removing things.

First I send a few mails from work to root@pasta.freemyip.com to get past the greylisting. Then by setting:

#alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliasesxxx

I was able to show that the newaliases command reads alias_database:

mandala# newaliases
postalias: fatal: open /etc/aliasesxxx: No such file or directory
mandala#

so we should set:

alias_database = hash:/etc/aliases

in order that the newaliases command works and that the aliases file is where we expect it to be. However:

mandala# postconf -d alias_database
alias_database = hash:/etc/aliases
mandala#

I.e. if that is what we are going to set it to then we do not need to set it explicitly at all. So we will comment alias_database out.

Regarding alias expansion, is it enough to set alias_database?

I had not realised it but I still had this entry:

virtual_alias_maps = hash:/etc/postfix/virtual

and:

mandala# cat /etc/postfix/virtual
root alexis@pasta.net
mandala#

So I think that that is why it worked!

When I commented out virtual_alias_maps and uncommented back in the alias_database and alias_maps then mail to root@pasta.freemyip.com failed as follows:

Dec 16 14:34:51 mandala postfix/smtpd[21012]: connect from b1962.mx.srv.dfn.de[194.95.234.160]
Dec 16 14:34:52 mandala postgrey[250]: action=pass, reason=triplet found, client_name=b1962.mx.srv.dfn.de, client_address=194.95.234.160/32, sender=alexis.huxley@mpcdf.mpg.de, recipient=root@pasta.freemyip.com
Dec 16 14:34:52 mandala postfix/smtpd[21012]: 337CD420D: client=b1962.mx.srv.dfn.de[194.95.234.160]
Dec 16 14:34:52 mandala postfix/cleanup[21017]: 337CD420D: message-id=<20211216133449.u4ckhgnkjhnmtlvt@damson.rzg.mpg.de>
Dec 16 14:34:52 mandala postfix/qmgr[21010]: 337CD420D: from=<alexis.huxley@mpcdf.mpg.de>, size=1393, nrcpt=1 (queue active)
Dec 16 14:34:52 mandala postfix/smtpd[21012]: disconnect from b1962.mx.srv.dfn.de[194.95.234.160] ehlo=2 starttls=1 mail=1 rcpt=1 data=1 quit=1 commands=7
Dec 16 14:34:52 mandala dovecot: lmtp(21019): Connect from local
Dec 16 14:34:52 mandala postfix/lmtp[21018]: 337CD420D: to=<root@pasta.freemyip.com>, relay=mandala.pasta.net[private/dovecot-lmtp], delay=0.37, delays=0.33/0.01/0.01/0.03, dsn=5.1.1, status=bounced (host mandala.pasta.net[private/dovecot-lmtp] said: 550 5.1.1 <root@pasta.freemyip.com> User doesn't exist: root@pasta.freemyip.com (in reply to RCPT TO command))
Dec 16 14:34:52 mandala dovecot: lmtp(21019): Disconnect from local: Client has quit the connection (state=READY)
Dec 16 14:34:52 mandala postfix/cleanup[21017]: 4896444DC: message-id=<20211216133452.4896444DC@mandala.pasta.net>
Dec 16 14:34:52 mandala postfix/qmgr[21010]: 4896444DC: from=<>, size=3529, nrcpt=1 (queue active)
Dec 16 14:34:52 mandala postfix/bounce[21020]: 337CD420D: sender non-delivery notification: 4896444DC
Dec 16 14:34:52 mandala postfix/qmgr[21010]: 337CD420D: removed
Dec 16 14:34:53 mandala postfix/smtp[21021]: 4896444DC: to=<alexis.huxley@mpcdf.mpg.de>, relay=smtp.gmail.com[173.194.76.108]:25, delay=1.3, delays=0.02/0.02/0.57/0.74, dsn=2.0.0, status=sent (250 2.0.0 OK 1639661693 n4sm5038645wrc.1 - gsmtp)
Dec 16 14:34:53 mandala postfix/qmgr[21010]: 4896444DC: removed

It is interesting that lmtp did not reject the mail for smtpd but it did reject it when it came to try to save it in the inbox, which, because smtpd had already hung up the connection, meant a bounce was triggered.

So now we know this:

So a configuration comes to mind:

alias_database = hash:/etc/aliases
alias_maps =
virt_alias_maps = hash:/etc/aliases

This way /etc/aliases becomes not only for local mail, newaliases knows to regenerate it and we do not create unnecessary extra maps.

It did not work: mail to root@pasta.freemyip.com was rejected. Changing virt_alias_maps back to /etc/postfix/virtual, which contained:

mandala# cat /etc/postfix/virtual
root alexis@pasta.net
mandala#

worked.

Hmm ... so either virt_alias_maps wants the maps to have entries of form user@domain (not just user) or it is unhappy about sharing its value with alias_database and alias_maps.

Let's set it back to hash:/etc/aliases but put fully qualified right hand sides in. That worked!

So now let's extend the test with this in aliases:

root: alexis@pasta.net
www-data: root
test: alexis.huxley@mpcdf.mpg.de

and try sending mails to those three addresses from outside:

So this configuration works:

virtual_alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
alias_maps = hash:/etc/aliases

But I think that alias_database should be redundant, so I set:

alias_database =

and I re-ran my tests:

Conclusions

Postfix can be made to delegate both validation and authentication to Dovecot!