path: root/fripost-docs.org

# -*- mode: org-mode; truncate-lines: nil -*-
#+TITLE: Systems documentation
#+AUTHOR: Fripost -- the Free E-mail Association
#+DESCRIPTION: Systems documentation for Fripost, the Free E-mail Association
#+KEYWORDS:
#+LANGUAGE:  en
#+OPTIONS:   H:3 num:t toc:t \n:nil @:t ::t |:t ^:t -:t f:t *:t <:t
#+OPTIONS:   TeX:t LaTeX:nil skip:nil d:nil todo:t pri:nil tags:not-in-toc
#+INFOJS_OPT: view:nil toc:nil ltoc:t mouse:underline buttons:0 path:http://orgmode.org/org-info.js
#+EXPORT_SELECT_TAGS: export
#+EXPORT_EXCLUDE_TAGS: noexport
#+LINK_UP:
#+LINK_HOME:
#+XSLT:
#+DRAWERS: HIDDEN STATE PROPERTIES CONTENT
#+STARTUP: indent

Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License,
Version 1.3 or any later version published by the Free Software
Foundation; with no Invariant Sections, no Front-Cover Texts and
no Back-Cover Texts.  A copy of the license is included in a
separate file called "COPYING".

This is the documentation of the server configuration used by the free e-mail
association, given here to provide a transparent system.

Debian GNU/Linux squeeze is the current target system. We might keep some notes
for lenny for some time yet since there might still be servers that have not
been upgraded.

The complete documentation is the actual configuration files on the servers.
This document intends to give a general idea of the setup and be of help if we
need to recreate a crashed server.  Also, if an administrator goes AWOL, it
should be easy to pick up where he left of.

The steps taken here will not necessarily give a perfect replica of our systems.
We are constantly (yes, constantly) working on improving the security and
reliability of our systems.  We do not think of security as a shoot and forget
sort of thing but instead as an ongoing effort.  Thus, while we strive to
document all configuration that we consider stable enough, the documentation may
sometimes lag behind.

We do not believe in security through obscurity. This means we are aiming
instead for a system that fulfills [[http://en.wikipedia.org/wiki/Kerckhoffs%27s_Principle][Kerckhoffs's Principle]]. However, some
information below might have been changed to inconvenience a potential
attacker. Beware and take according measures.

We welcome all criticism, suggestions for improvements, additions etc.  Please
send them to skangas@skangas.se.

* Basic Setup -- Checklist after having installed a new Debian GNU/Linux-server
** Basic installation instructions

- Use expert install to maximize fun.
- Preferably, only install the "Standard system utilities" and "SSH Server" tasks.
- Make sure to answer "yes" to shadow passwords and MD5.
- Do disable the root account.

** Install etckeeper

Install etckeeper immediately after install, to start tracking /etc.

** Uninstall a bunch of unnecessary packages

sudo aptitude remove --purge debian-faq dictionaries-common doc-debian \
doc-linux-text iamerican ibritish iswedish ispell laptop-detect nfs-common \
openbsd-inetd portmap tasksel tasksel-data w3m wbritish

** Packages to install
*** Administrative

sudo aptitude install emacs23-nox harden-servers logcheck molly-guard ntp \
ntpdate openssh-server rsync screen syslog-summary sudo unattended-upgrades

# If the system is on a dynamic IP (e.g. using DHCP):
sudo aptitude install resolvconf

# NB: harden-clients conflicts with telnet, which as we know is very handy
# during configuration.  Therefore, only optionally:
sudo aptitude install harden-clients

** Use GNU Emacs as the default editor

# NOTE: Emacs will be the default on all Fripost systems. If you prefer
# something else, use the EDITOR environment variable.

sudo update-alternatives --config editor

** Configure sudo

# If you disabled root account during installation, the default account is
# already in the sudo group.  Otherwise, follow these steps:

sudo adduser myuser sudo

sudo EDITOR="emacs" visudo

     %sudo ALL= (ALL) ALL

** Configure sshd

Make sure your private key is in ~/.ssh/authorized_keys2

:: /etc/ssh/sshd_config

    # Add relevant users here
    AllowUsers xx yy zz

    # Change these settings
    PermitRootLogin no
    PasswordAuthentication no
    X11Forwarding no

sudo /etc/init.d/ssh restart

# Without closing the current connection, try to connect to the server,
# verifying that you can still connect.

** Forward root email

:: /etc/aliases

    root: admin@fripost.org

** Configure logcheck

sudo aptitude install logcheck syslog-summary

:: /etc/logcheck/logcheck.conf

     INTRO=0
     SENDMAILTO="admin@fripost.org"

:: /etc/logcheck/ignore.d.server/local

# If the machine has a dynamic IP
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: FAILED:   updating [,._[:alnum:]-]+: Could not connect to dns.loopia.se/xdyndnsserver/xdyndns.php.$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: WARNING:  TIMEOUT: dns.loopia.se after 120 seconds$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: WARNING:  cannot connect to dns.loopia.se:80 socket: IO::Socket::INET: Bad hostname 'dns.loopia.se'$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: WARNING:  cannot connect to dns.loopia.se:80 socket: IO::Socket::INET: connect: Connection timed out$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: WARNING:  cannot connect to dns.loopia.se:443 socket: IO::Socket::SSL: SSL connect attempt failed because of handshake problemserror:00000000:lib\(0\):func\(0\):reason\(0\) IO::Socket::INET configuration failederror:00000000:lib\(0\):func\(0\):reason\(0\)$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: WARNING:  cannot connect to dns.loopia.se:443 socket: IO::Socket::SSL: SSL connect attempt failed with unknown errorerror:00000000:lib\(0\):func\(0\):reason\(0\) IO::Socket::INET configuration failederror:00000000:lib\(0\):func\(0\):reason\(0\)$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: WARNING:  cannot connect to dns.loopia.se:443 socket: IO::Socket::SSL: connect: Connection timed out IO::Socket::INET configuration failederror:00000000:lib\(0\):func\(0\):reason\(0\)$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: WARNING:  file /var/cache/ddclient/ddclient.cache, line [0-9]+: Invalid Value for keyword 'ip' = ''$
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ddclient\[[0-9]+\]: WARNING:  updating [._[:alnum:]-]+: nochg: No update required; unnecessary attempts to change to the current address are considered abusive$

# If the machine does LDAP virtual lookups
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ postfix/(smtpd|cleanup|trivial-rewrite|postmap)\[[0-9]+\]: warning: dict_ldap_open: /etc/postfix/ldap/ldap_virtual_alias_catchall_maps.cf: Fixed query_filter \(\&\(ObjectClass=virtualAliases\)\(mailLocalAddress=\)\(isActive=TRUE\)\) is probably useless$

# If the machine uses amavisd-new as a content filter
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ amavis\[[[:digit:]]+\]: \([-[:digit:]]+\) Passed [-+_[:alnum:]]+ {Relayed[[:alpha:]]+}, \S+(( LOCAL)? \[(IPv6:)?[[:xdigit:].:]{3,39}\]:[[:digit:]]+( \[(IPv6:)?[[:xdigit:].:]{3,39}\]){0,2})? <[^>]*> -> <[^>]*>(,<[^>]*>)*,( Message-ID: <[^>]+>( \((added by[^)]+|sfid-[_[:xdigit:]]+)\))?,)?( Resent-Message-ID: <[^>]+>,)? mail_id: [-+_[:alnum:]]+, Hits: (-?[.[:digit:]]*)+, size: [[:xdigit:]]+, queued_as: [[:xdigit:]]+( OK id=[-[:alnum:]]+)?,( dkim_(sd|new)=([._[:alnum:]-]+:[._[:alnum:]-]+,)+)? [[:digit:]]+ ms$

# If the machine hosts MX
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix/smtpd\[[[:digit:]]+\]: improper command pipelining after (EHLO|DATA) from [._[:alnum:]-]+\[[:[:xdigit:].]+\]$

# If the machine hosts a MSA
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ postfix/cleanup\[[0-9]+\]: [[:alnum:]]+: replace: header Received: from

# If the machine has several ethernet interfaces
^\w{3} [ :0-9]{11} [._[:alnum:]-]+ ntpd\[[0-9]+\]: [.0-9]{7,15} interface [.0-9]{7,15} -> [.0-9]{7,15}$

^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix/postfix-script\[[[:digit:]]+\]: refreshing the Postfix mail system$
^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix/master\[[[:digit:]]+\]: reload -- version

^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ ansible-[a-z]+: Invoked

** Configuring aptitude and friends

# We are going to automatically install many security updates using the package
# "unattended-upgrades".  Automated upgrades are in general not a very good
# idea, but "unattended-upgrades" takes steps to mitigate the problems with this
# approach.  Given the Debian security teams track record in recent years we
# believe the positives outweigh the negatives.
#
# For the situations when unattended-upgrades fails (e.g. when there are
# configuration changes), there is an e-mail sent to the administrator.
#

:: /etc/apt/apt.conf

    APT
    {
      // Configuration for /etc/cron.daily/apt
      Periodic
      {
         // Do "apt-get update" automatically every n-days (0=disable)
         Update-Package-Lists "1";
         // Do "apt-get autoclean" every n-days (0=disable)
         AutocleanInterval "1";
         // Do "apt-get upgrade --download-only" every n-days (0=disable)
         Download-Upgradeable-Packages "1";
         // Run the "unattended-upgrade" security upgrade script every n days
         Unattended-Upgrade "1";
      }
    };

    Aptitude
    {
      UI
      {
         Autoclean-After-Update:         true;
         Auto-Fix-Broken:                false;
         Keep-Recommends:                true;
         Recommends-Important:           true;
         Description-Visible-By-Default: false;
         HelpBar                         false;
         Menubar-Autohide                true;
         Purge-Unused:                   true;
         Prompt-On-Exit                  false;
      }
    }

# Using Debian squeeze:
:: /etc/apt/apt.conf.d/50unattended-upgrades

     Unattended-Upgrade::Mail "admin@fripost.org";

** Configure ddclient

:: /etc/ddclient.conf

### Not reproduced here due to containing sensitive information

:: /etc/default/ddclient

    run_daemon="true"

* Next Steps
** Configuring the backup solution
*** Bacula configuration
*** Simple rsync solution

General idea [[http://wikis.sun.com/display/BigAdmin/Using+rdist+rsync+with+sudo+for+remote+updating][from here]].  This is just a basic setup for now, will need to be
changed to rsnapshot or perhaps something even more sophisticated like bacula.

1. Install rsync
      - sudo aptitude install rsync
2. Create a key on the backup computer:
      - sudo mkdir /root/.ssh/backup_key
      - sudo ssh-keygen -N "" -b 4096 -f /root/.ssh/backup_key
      - cat /root/.ssh/backup_key.pub
3. Create a user on the computer that will be backed up
      - sudo adduser --disabled-password remupd
      - add the public key from above to ~remupd/.ssh/authorized_keys2
        prefix with: no-X11-forwarding,no-agent-forwarding,no-port-forwarding
      - sudo EDITOR="emacs" visudo
        Cmnd_Alias      RSYNCDIST=/usr/bin/rsync
        remupd	ALL=NOPASSWD:RSYNCDIST
4. Test the key from the backup computer:
      - ssh -i ~/.ssh/backup_key -l remupd example.com
5. Create a script on the backup computer to automatically backup
6. Add script to crontab

** Configuring the e-mail servers
*** Introduction
**** Overview

We will be using one main mail storage server, accessible by users via IMAP.
This server should be referred to as the main `IMAP server'. We will have two or
more mail gateways that will relay e-mail to the main server over secure
connections.  These are called `smarthosts'.

Credentials are managed by a LDAP server. For the users to be able to
authenticate to e.g., the IMAP server or the outgoing SMTP (via SASL), we will
use the so called "authenticate binds": services simply forward the login
information of the user to the LDAP server, that in turn hashes the password and
checks wheter it maches the stored copy; if it does, the LDAP server answers back
the query. See http://wanderingbarque.com/howtos/mailserver/big_picture.gif .
This way, if the IMAP or SMTP server is compromised, the attacker will NOT have
access to all credentials. Of course the LDAP server should only be listening to
the machines hosting these services and ideally, should not be directly facing the
internet.

**** Definitions

IMAP server = the main storage server

LDAP server = the server that stores users credentials and various other informations.

smarthost = the server receiving email from the internet (configured as MX)

outgoing SMTP = the Message Transfer Agent (MTA), that will relay emails originating from our network.

incoming SMTP = the Mail Submission Agent (MSA), that will receive emails from our users (via ESMTPSA).

*** Configuring an SSH tunnel between two hosts

# Definitions:
# originating host = the host that will be connecting
# destination host = the host that runs some service

# Begin by setting a few environment variables on both sides:

DEST_PORT="25"
ORIGIN_PORT="1917"
TUNNEL_USER="tunnel_service"    # e.g., tunnel_ldap or tunnel_smtp  (grep -P "\b${DEST_PORT}\b" /etc/services)
TUNNEL_KEY_FILE="/root/.ssh/${TUNNEL_USER}_key"
TUNNEL_HOME="/var/lib/inetd/${TUNNEL_USER}"

**** Prepare origin

1. Create a key on the originating host:

   sudo test -f $TUNNEL_KEY_FILE || sudo ssh-keygen -N "" -b 4096 -f $TUNNEL_KEY_FILE
   echo THE_PUBLIC_KEY="'$(sudo cat $TUNNEL_KEY_FILE.pub)'"

**** Prepare destination

2a. Install necessary software on the destination host:

   sudo aptitude install netcat-openbsd

2b. Create a new user on the destination host:

   sudo adduser --system --home="$TUNNEL_HOME" --shell=`type rbash|cut -d' ' -f3` \
                "$TUNNEL_USER"
   sudo chmod og-rwx "$TUNNEL_HOME"
   echo "exit" | sudo -u $TUNNEL_USER tee "$TUNNEL_HOME/.bash_profile"

   # Note: We need bash, so we can not change the shell to something else.

2c. Add $TUNNEL_USER to AllowUsers in /etc/ssh/sshd_config.

    sudo /etc/init.d/ssh restart

    # make sure the host is still reachable

2d. Add the public key from above to this user:

   THE_PUBLIC_KEY="ssh-rsa xxxxxxxxxxx" # from above

      sudo -u $TUNNEL_USER mkdir -m 700 -p $TUNNEL_HOME/.ssh
      ALLOWED_KEY="command=\"nc 127.0.0.1 $DEST_PORT\",no-X11-forwarding,no-agent-forwarding,no-port-forwarding $THE_PUBLIC_KEY"
      sudo grep -Fxq "$ALLOWED_KEY" $TUNNEL_HOME/.ssh/authorized_keys2 || \
                echo "$ALLOWED_KEY" | sudo -u $TUNNEL_USER tee -a $TUNNEL_HOME/.ssh/authorized_keys2

**** Set up the tunnel

3. Test the key on the originating host:

   sudo ssh -v -l $TUNNEL_USER -i $TUNNEL_KEY_FILE destination.example.com

    # Comment: You should be greeted by e.g.:
    # 220 mistral.fripost.org ESMTP Postfix (Debian/GNU)

4. Configure openbsd-inetd on the originating host:

   # Comment: We use inetd instead of ssh -L because, among other things, ssh
   #          -L tends to hang.

   sudo aptitude install openbsd-inetd

   :: /etc/inetd.conf

       127.0.0.1:$ORIGIN_PORT  stream  tcp     nowait  root    /usr/bin/ssh    -q -T -i $TUNNEL_KEY_FILE $TUNNEL_USER@destination.example.com

    sudo service openbsd-inetd restart

5. Whitelist the listening port on the origin host:

   :: /etc/rkhunter.conf

      INETD_ALLOWED_SVC=127.0.0.1:$ORIGIN_PORT

You should now be able to connect through the tunnel from the originating
host using something like:

telnet localhost $ORIGIN_PORT

*** Configuring the main LDAP server (provider)

On Debian Squeeze, OpenLDAP's configuration no longer uses `/etc/ldap/slapd.conf'
(by default, but may completely ignore it in the future), but the
`/etc/ldap/slapd.d' directory instead. Unfortunately most of the online
tutorials are describing methods using `/etc/ldap/slapd.conf'.

[Note: This has been written by a LDAP noob. It should probably be
rewritten/compressed in a couple of months. /Guilhem, 2012-04-03.]

**** Install packages

Here is a basic installation tutorial for Debian Squeeze:
http://www.rjsystems.nl/en/2100-d6-openldap-provider.php

sudo apt-get install slapd ldap-utils

If it does not prompt for your domain, admin password, etc., run
`dpkg-reconfigure -plow slapd'. Here is how we answer the questions:

Omit OpenLDAP server configuration?                             No
DNS domain name:                                                fripost.org
Organization name:                                              Fripost
Administrator password:                                         *********
Database backend to use:                                        HDB
Do you want the database to be removed when slapd is purged?    No
Move old database?                                              Yes
Allow LDAPv2 protocol?                                          No


We do not want to listen all the Internet: in `/etc/default/slapd', change
`SLAPD_SERVICES' accordingly. E.g., to only listen to (non SSL) localhost and
UNIX sockets, specify

SLAPD_SERVICES="ldap:///127.0.0.1:389 ldapi:///"

(This should be enough if the connection from the IMAP/SMTP services are
wrapped into SSH or SSL/TLS tunnels.)

(Note: Unless specified, connections through the sockets bind with the users
permissions, hence regular users may not be able to explore the directory.)

We can check the configuration with

  sudo ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -b "cn=config"

and modify the directory using a .ldif file with

  sudo ldapmodify -Y EXTERNAL -H ldapi:/// -f "<file.ldif>"

(granted writing rights, see below)

**** Performance considerations

References:
- https://wiki.zimbra.com/wiki/OpenLDAP_Performance_Tuning_5.0
- http://www.openldap.org/doc/admin24/tuning.html


1. On single- and dual-core systems, change the maximum number of
threads to 8. (The default, 16, is fine for 4- and 8-core systems.)

  :: ldapmodify -Y EXTERNAL -H ldapi:///

    dn: cn=config
    changetype: modify
    add: olcThreads
    olcThreads: 8


2. It may be a good idea to modify DB_CONFIG, depending on the output
of

  db4.8_stat -m -h /var/lib/ldap/ | head -16

(For optimal performance, the Requested pages found in the cache
should be above 95%, and the pages forced from the cache should be 0.)

and

  db4.8_stat -m -h /var/lib/ldap/ | head -16

(For optimal performance, usage should be within 85% of the configured
values.)

**** Database configuration

  :: mkdir -m 0700 /var/lib/ldap/mailHosting && chown openldap:openldap /var/lib/ldap/mailHosting

We may want to remove the existing (default, empty) database on a fresh installation:

:HIDDEN:
slapcat -n1 # Ensure it is empty
/etc/init.d/slapd stop
mv /var/lib/ldap/DB_CONFIG /var/lib/ldap/mailHosting
find /var/lib/ldap/ -maxdepth 1 -type f -delete
rm -f '/etc/ldap/slapd.d/cn=config/olcDatabase={1}hdb.ldif'
:END:

And define ours:

  :: ldapadd -Y EXTERNAL -H ldapi:///

    dn: olcDatabase=hdb,cn=config
    objectClass: olcDatabaseConfig
    objectClass: olcHdbConfig
    olcDbDirectory: /var/lib/ldap/mailHosting
    olcSuffix: o=mailHosting,dc=fripost,dc=org
    olcLastMod: TRUE
    olcDbCheckpoint: 512 30
    # Require LDAPv3 protocol and authentication prior to directory
    # operations.
    olcRequires: LDAPv3 authc
    # We don't want to give "canCreate{Alias,ML}" write access to alias/ml
    # attributes.
    olcAddContentAcl: FALSE
    # The root user has all rights on the whole database (when SASL-binding
    # on a UNIX socket).
    olcRootDN: gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth

**** Fripost's schema

TODO: upgrade

We base our schema on qmail's (http://dhits.nl/download/qmail.new.schema) and
Jamm's (http://jamm.sourceforge.net/howto/html/implementation.html).

  o=mailHosting, dc=fripost, dc=org
    |- ou=managers
    |   |- cn=admin1
    |   |   userPassword: xxxxxx
    |   `- cn=admin2
    |
    |- ou=services
    |   `- cn=SMTP
    |       userPassword: xxxxxx
    |
    `- ou=virtual
        |- dc=fripost.org
        |   isActive: TRUE
        |   |- mailTarget=user1@fripost.org
        |   |    mailLocalAddress: user1-alias
        |   |    isActive: TRUE
        |   |- uid=user1
        |   |    userPassword: xxxxxx
        |   |    isActive: TRUE
        |   |
        |   `- uid=user2
        |
        `- dc=example.org
             owner: uid=user1,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
             isActive: TRUE
             `- mailTarget=user1@fripost.org
             |    mailLocalAddress: user1
             |    isActive: TRUE
             |
             `- mailTarget=user1-alias@fripost.org



  :: /etc/ldap/fripost/fripost.ldif

    dn: cn=mail.fripost.org,cn=schema,cn=config
    objectClass: olcSchemaConfig
    cn: mail.fripost.org
    olcAttributeTypes: ( 1.3.6.1.4.1.7914.1.2.1.1 NAME 'quota'
        DESC 'The quota on a mailbox e.g., "50MB".'
        EQUALITY caseExactMatch
        SYNTAX 1.3.6.1.4.1.1466.115.121.1.44 SINGLE-VALUE )
    olcAttributetypes: ( 1.3.6.1.4.1.7914.1.2.1.2 NAME 'isActive'
        DESC 'Is the leaf active?'
        EQUALITY booleanMatch
        SYNTAX 1.3.6.1.4.1.1466.115.121.1.7 SINGLE-VALUE )
    olcAttributeTypes: ( 1.3.6.1.4.1.7914.1.2.1.3 NAME 'mailTarget'
        DESC 'The target of e-mail virtual aliases.'
        EQUALITY caseExactIA5Match
        SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 SINGLE-VALUE )
    olcObjectclasses: ( 1.3.6.1.4.1.12461.1.2.1 NAME 'virtualDomain'
        SUP top STRUCTURAL
        DESC 'Virtual Domains.'
        MUST ( dc $ isActive )
        MAY ( owner $ description ) )
    olcObjectclasses: ( 1.3.6.1.4.1.12461.1.2.2 NAME 'virtualAliases'
        SUP top STRUCTURAL
        DESC 'Virtual Aliases.'
        MUST ( mailTarget $ isActive )
        MAY ( mailLocalAddress ) )
    olcObjectclasses: ( 1.3.6.1.4.1.12461.1.2.3 NAME 'virtualMailbox'
        SUP top STRUCTURAL
        DESC 'Virtual Mailboxes.'
        MUST ( uid $ userPassword $ isActive )
        MAY ( gn $ sn $ quota ) )


Note: For the meaning of the sequences of digits above, grep the output of
`sudo ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -b "cn=config"', or read
http://www.openldap.org/doc/admin24/schema.html#Attribute%20Type%20Specification .
(For instance, 1.3.6.1.4.1.1466.115.121.1.26 is a IA5String, meaning unicode is
allowed.)

We can now add it to the schema directory:

    sudo ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/ldap/fripost/fripost.ldif

(A [dirty] way to delete the schema is to remove the coresponding file in
`/etc/ldap/slapd.d/cn=config/cn=schema/' and to restart slapd. Ensure that nothing
[databases, ACL, indices,...] are making use of the schema, of course.)


Note: If the LDIF files our schema depends on are not in loaded (in `/etc/ldap/slapd.d/cn=config/cn=schema/'),
you may have to do it yourself. A dirty way is to create a file `/tmp/upgrade.conf' with the
following:

    include         /etc/ldap/schema/core.schema
    include         /etc/ldap/schema/cosine.schema
    include         /etc/ldap/schema/nis.schema
    include         /etc/ldap/schema/misc.schema

and a directory `/tmp/upgrade', then to run `slaptest -f /tmp/upgrade.conf -F /tmp/upgrade'.
It creates a bunch of LDIF files that you need to clean (cf. https://help.ubuntu.com/10.04/serverguide/C/samba-ldap.html)
and add with `sudo ldapadd -Y EXTERNAL -H ldapi:/// -f <file.ldif>'.
[TODO: that's just ugly. Find a better way.]

***** Add custom indexes

TODO: upgrade

The default indexes below are not enough for our purpose, since e.g.,
Postfix will heavily be looking for e.g., the `uid' attribute.

 :: sudo ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -b cn=config "(olcDatabase={1}hdb)" olcDbIndex
    [...]
    olcDbIndex: objectClass eq

 :: /etc/ldap/fripost/index.ldif

    dn: olcDatabase={1}hdb,cn=config
    changetype: modify
    # Needed for the replicates.
    add: olcDbIndex
    olcDbIndex: entryUUID eq
    -
    delete: olcDbIndex
    olcDbIndex: objectClass eq
    -
    add: olcDbIndex
    olcDbIndex: objectClass pres,eq
    -
    add: olcDbIndex
    olcDbIndex: cn eq
    -
    add: olcDbIndex
    olcDbIndex: ou eq
    -
    add: olcDbIndex
    olcDbIndex: dc eq,sub
    -
    add: olcDbIndex
    olcDbIndex: uid eq,sub
    -
    add: olcDbIndex
    olcDbIndex: mailTarget,mailLocalAddress eq
    -
    add: olcDbIndex
    olcDbIndex: isActive eq
    -
    add: olcDbIndex
    olcDbIndex: owner eq

TODO: After having amended the schema as specified above, we'll also need a
`sub' index on aliases:

    add: olcDbIndex
    olcDbIndex: mailAliasGoto,mailAliasFrom eq,sub


sudo ldapmodify -Y EXTERNAL -H ldapi:/// -f /etc/ldap/fripost/index.ldif


 :: ldapsearch -Y EXTERNAL -H ldapi:/// -b cn=config "(olcDatabase={1}hdb)"
    [...]
    olcDbIndex: entryUUID eq
    olcDbIndex: objectClass pres,eq
    olcDbIndex: cn eq
    olcDbIndex: ou eq
    olcDbIndex: dc eq,sub
    olcDbIndex: uid eq,sub
    olcDbIndex: mailTarget,mailLocalAddress eq
    olcDbIndex: isActive eq
    olcDbIndex: owner eq


Note: We can add indexes on a populated database, but then we need to reindex the directory:

    sudo /etc/init.d/slapd stop
    sudo -u openldap slapindex -b 'o=mailHosting,dc=fripost,dc=org'
    sudo /etc/init.d/slapd start

***** Restrict the access

TODO: upgrade

The default ACL is not restrictive enough for our purpose: We don't want
the services to see passwords, and the users should see/edit only the
relevant bits.

Note: The ACLs are evaluated in order, hence the more specific rules should come
first.
We are using the so-called "Sets" to let the users manage their domain themselves.
See section 8.5 "Sets - Granting rights based on relationships" in LDAP's manual
http://www.openldap.org/doc/admin24/access-control.html for details.

 :: sudo ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -b cn=config "(olcDatabase={1}hdb)" olcAccess
    [...]
    olcAccess: {0}to attrs=userPassword,shadowLastChange by self write by anonymous auth by dn="cn=admin,dc=fripost,dc=org" write by * none
    olcAccess: {1}to dn.base="" by * read
    olcAccess: {2}to * by self write by dn="cn=admin,dc=fripost,dc=org" write by * read
    [...]


 :: /etc/ldap/fripost/acl.ldif

    dn: olcDatabase={1}hdb,cn=config
    changetype: modify
    # Service passwords are only writable (hence readable) by the admins.
    # Anonymous services are only allowed to bind.
    add: olcAccess
    olcAccess: {0}to dn.one="ou=services,o=mailHosting,dc=fripost,dc=org"
            attrs=userPassword
        by self read
        by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write
        by anonymous auth
    -
    # User passwords are only writable (hence readable) by the admins and the
    # user him/herself. Anonymous users are only allowed to bind.
    add: olcAccess
    olcAccess: {1}to dn.children="o=mailHosting,dc=fripost,dc=org"
            attrs=userPassword
        by self write
        by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write
        by anonymous auth
    -
    # User names are only writable (hence readable) by the admins and the user
    # him/herself.
    add: olcAccess
    olcAccess: {2}to dn.children="o=mailHosting,dc=fripost,dc=org" attrs=gn,sn
        by self write
        by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write
    -
    # Users are allowed to manage (create/delete/toggle activation) the
    # the domains they own.
    add: olcAccess
    olcAccess: {3}to dn.regex="(.+,)?(dc=[^,]+,ou=virtual,o=mailHosting,dc=fripost,dc=org)$"
        by set.expand="[$2]/owner & user" write
        by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write
        by * break
    -
    # Admins have writing rights on the branch. Authenticated users can read
    # their entry. The SMTP and SASLauthd servervices can read entries on the
    # branch (but not the passwords). Others can only search.
    add: olcAccess
    olcAccess: {4}to dn.subtree="o=mailHosting,dc=fripost,dc=org"
        by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write
        by self read
        by dn.exact="cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org" read
        by dn.exact="cn=SASLauth,ou=services,o=mailHosting,dc=fripost,dc=org" read
        by * search

ldapmodify -Y EXTERNAL -H ldapi:/// -f /etc/ldap/fripost/acl.ldif


 :: ldapsearch -Y EXTERNAL -H ldapi:/// -b cn=config "(olcDatabase={1}hdb)"
    [...]
    olcAccess: {0}to dn.one="ou=services,o=mailHosting,dc=fripost,dc=org" attrs=userPassword by self read by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write   by anonymous auth
    olcAccess: {1}to dn.children="o=mailHosting,dc=fripost,dc=org" attrs=userPassword by self write by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write by anonymous auth
    olcAccess: {2}to dn.children="o=mailHosting,dc=fripost,dc=org" attrs=gn,sn by self write by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write
    olcAccess: {3}to dn.regex="(.+,)?(dc=[^,]+,ou=virtual,o=mailHosting,dc=fripost,dc=org)$" by set.expand="[$2]/owner & user" write by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write by * break
    olcAccess: {4}to dn.subtree="o=mailHosting,dc=fripost,dc=org" by dn.one="ou=managers,o=mailHosting,dc=fripost,dc=org" write by self read by dn.exact="cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org" read by dn.exact="cn=SASLauth,ou=services,o=mailHosting,dc=fripost,dc=org" read by * search
    olcAccess: {5}to attrs=userPassword,shadowLastChange by self write by anonymous auth by dn="cn=admin,dc=fripost,dc=org" write by * none
    olcAccess: {6}to dn.base="" by * read
    olcAccess: {7}to * by self write by dn="cn=admin,dc=fripost,dc=org" write by * read
    [...]

Note: Users are allowed to manage their domain, but an admin is needed to add a domain to the
directry. A possibility to avoid that with a web-form is to send a mail to the postmaster@example.org
(or even to the mail that appears in the WHOIS) with a confirmation hash. That would simply require
a new ACL with writable [ou=virtual,...]/children, and [dc=...,ou=virtual,...]/entry. (And probably a
"semi-admin" with only these rights.)

**** Create the base entries

 :: ldapadd -xWD cn=admin,dc=fripost,dc=org

    dn: o=mailHosting,dc=fripost,dc=org
    objectClass: organization
    description: Mail hosting

    dn: ou=virtual,o=mailHosting,dc=fripost,dc=org
    objectClass: organizationalUnit
    description: Virtual Hosting

    dn: ou=managers,o=mailHosting,dc=fripost,dc=org
    objectClass: organizationalUnit
    description: Postmasters

    dn: ou=services,o=mailHosting,dc=fripost,dc=org
    objectClass: organizationalUnit
    description: E-mail services


To delete an entry (add `-r' to delete the whole sub-tree):
    ldapdelete -xWD cn=admin,dc=fripost,dc=org 'dc=example.org,ou=virtual,o=mailHosting,dc=fripost,dc=org'

**** Populate the directory

 :: ldapadd -xWD cn=admin,dc=fripost,dc=org

    dn: cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org
    objectClass: simpleSecurityObject
    objectClass: organizationalRole
    userPassword: {SSHA}xxxxxx

    dn: cn=admin1,ou=managers,o=mailHosting,dc=fripost,dc=org
    objectClass: simpleSecurityObject
    objectClass: organizationalRole
    userPassword: {SSHA}xxxxxx

    dn: dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    objectClass: virtualDomain
    isActive: TRUE

    dn: uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    objectClass: virtualMailbox
    gn: First Name
    sn: Last Name
    userPassword: {SSHA}xxxxxx
    isActive: TRUE

    dn: dc=example.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    objectClass: virtualDomain
    owner: uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    isActive: TRUE

    dn: mailTarget=user-alias@fripost.org,dc=example.org,ou=virtual,o=mailHosting,dc=fripost,  dc=org
    objectClass: inetLocalMailRecipient
    objectClass: virtualAliases
    isActive: TRUE
    mailLocalAddress: user
    mailLocalAddress: user-alias

    dn: uid=user2,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    objectClass: virtualMailbox
    gn: First Name
    sn: Last Name
    userPassword: {SSHA}xxxxxx
    isActive: FALSE

    dn: mailTarget=user@fripost.org,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    objectClass: inetLocalMailRecipient
    objectClass: virtualAliases
    mailLocalAddress: user-alias
    isActive: TRUE


Note: This should obviously be wrapped in a script; `ldapadd' reads the standard
input, so there's no need to write on disk. The salted SHA-1 can be created with
e.g., `slappasswd -h "{SSHA}"'.

**** Check the ACLs

TODO: upgrade

`slapacl' is an helpful tool to debugs the ACLS. For instance, to check what are
the rights of user@fripost.org on the domain example.org, we can run:
  sudo slapacl -b 'dc=example.org,ou=virtual,o=mailHosting,dc=fripost,dc=org' -D 'uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org'

We can also check ACLs with concrete examples:

ldapwhoami -xWD "uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org"

should return the whole dn:

"uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org"

***** Admin

`slpacat' (run as root) dumps everything in the subtree, including the (hashed)
passwords. So should

    ldapsearch -xWD "cn=admin,dc=fripost,dc=org" -b 'ou=virtual,o=mailHosting,dc=fripost,dc=org'

and

    ldapsearch -xWD "cn=admin1,ou=managers,o=mailHosting,dc=fripost,dc=org" -b 'ou=virtual,o=mailHosting,dc=fripost,dc=org'

***** Anonymous user

`ldapsearch -x -b "ou=virtual,o=mailHosting,dc=fripost,dc=org"' should exit
with return status 0, but shouldn't print anything.

***** Services

ldapsearch -xWD "cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org" -b 'ou=virtual,o=mailHosting,dc=fripost,dc=org'

should not disclose the passwords.

***** Self

ldapsearch -xWD "uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org" -b 'ou=virtual,o=mailHosting,dc=fripost,dc=org'

should return all the information for this very user, but not e.g., the password of the other users.


The user should be able to change his/her password, and aliases in his/her own domain:

 :: ldapmodify -xWD "uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org"

    dn: uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    changetype: modify
    replace: userPassword
    userPassword: xxxxxx

    dn: mailTarget=user@fripost.org,dc=example.org,ou=domain,o=mailHosting,dc=fripost,dc=org
    changetype: modify
    add: mailLocalAddress
    mailLocalAddress: user-alias2@example.org


[Note: Still that should be wrapped up in a script, and there is no need to write on
disk since the data is read from the standard input.]
[Note: If the task is merely to change the password, there is also `ldappasswd'.]

We now ensure that the leaf has been updated:

 :: slapcat -s "uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org"
    [...]
    userPassword:: aG9w
    entryCSN: 20120404215647.957317Z#000000#000#000000
    modifiersName: uid=user,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    modifyTimestamp: 20120404215647Z

On other modifications, for instance of `maildir', `ldapmodify'
should refuse with `Insufficient access (50)'.

**** Using SyncProv, to define our main server as the provider

Our main LDAP server (provider) will sent periodic updates of the directory to
other servers (consumers, running on the MX:s).

References:
- http://www.openldap.org/doc/admin24/replication.html#Syncrepl
- http://www.zytrax.com/books/ldap/ch7/#ol-syncrepl-rap
- man 5 slapo-syncprov

  :: ldapmodify -Y EXTERNAL -H ldapi:///

    dn: cn=module{0}, cn=config
    changetype: modify
    add: olcModuleLoad
    olcModuleLoad: syncprov.la

  :: ldapadd -Y EXTERNAL -H ldapi:///

    dn: olcOverlay=syncprov,olcDatabase={1}hdb,cn=config
    objectClass: olcOverlayConfig
    objectClass: olcSyncProvConfig
    olcOverlay: syncprov
    # contextCSN saved to database every 50 updates or 5
    # minutes
    olcSpCheckpoint: 50 5
    syncprov-reloadhint: TRUE

*** Configuring the main IMAP server
**** Install packages

sudo aptitude install postfix postfix-ldap

**** /etc/postfix/main.cf

TODO: add file contents

    [...]
    virtual_mailbox_domains = ldap:$config_directory/ldap/ldap_virtual_mailbox_domains.cf
    virtual_mailbox_maps    = ldap:$config_directory/ldap/ldap_virtual_mailbox_maps.cf
    virtual_alias_maps      = ldap:$config_directory/ldap/ldap_virtual_alias_maps.cf
                              ldap:$config_directory/ldap/ldap_virtual_alias_catchall_maps.cf
    [...]

***** Virtual lookups

Ideally, Postfix would send the request to a UNIX socket (hence on the MX:s,
OpenLdap would not listen to the network).  Note that the socket has to be in
Postfix's chroot jail; To specifify that, change `SLAPD_SERVICES' in
`/etc/default/slapd' to

SLAPD_SERVICES="ldapi://%2Fvar%2Fspool%2Fpostfix%2Fvar%2Frun%2Fldapi/????x-mod=0777"

Note that in the configuration files below, the `server_host' is relative to
Postfix's jail, hence one should drop the prefix "/var/spool/postfix".  But to
test the files with `postmap' one has to put back the prefix (or chroot first).

TODO: Postfix 2.7 does not support SASL binds. Hence one cannot SASL-bind to
the socket with the EXTERNAL mechanism, which leads to a flood of warnings
"connection_read(XX): no connection!" in the syslog. One can also reproduce the
warning with
    ldapsearch -H 'ldapi://%2Fvar%2Fspool%2Fpostfix%2Fvar%2Frun%2Fldapi/' -x -WD 'cn=guilhem,ou=managers,o=mailHosting,dc=fripost,dc=org' -b 'o=mailHosting,dc=fripost.org,dc=org'
instead of the proper
    ldapsearch -H 'ldapi://%2Fvar%2Fspool%2Fpostfix%2Fvar%2Frun%2Fldapi/' -Y EXTERNAL -WD 'cn=guilhem,ou=managers,o=mailHosting,dc=fripost,dc=org' -b 'o=mailHosting,dc=fripost.org,dc=org'
(The first one performs a simple bind and does not unbind properly, while
the second one is safe and performs a SASL bind with the EXTERNAL mechanism.)

See also:
- http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=643970
- http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=660223
- http://www.openldap.org/lists/openldap-software/200811/msg00078.html

TODO: For the time being, we stick to simple binds on 127.0.0.1:389, but when
"wheezy" will be the next Debian stable, we will do the following instead [Not
tested]

    bind = sasl
    sasl_mechs = EXTERNAL

This will be both more secure (SASL vs. simple bind) and efficient (UNIX socket
vs. inet).

:: /etc/postfix/ldap/ldap_virtual_mailbox_domains.cf

    #server_host = ldapi://%2Fvar%2Frun%2Fldapi/
    server_host = ldap://127.0.0.1:389/
    version = 3
    search_base = dc=%s,ou=virtual,o=mailHosting,dc=fripost,dc=org
    scope = base
    bind = yes
    bind_dn = cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org
    bind_pw = xxxxxx
    query_filter = (&(ObjectClass=virtualDomain)(dc=%s)(isActive=TRUE))
    result_attribute = dc

Test it:
    postmap -q fripost.org ldap:/etc/postfix/ldap/ldap_virtual_domains_maps.cf || echo 'failed!'
    postmap -q example.org ldap:/etc/postfix/ldap/ldap_virtual_domains_maps.cf || echo 'failed!'
    postmap -q fake.org ldap:/etc/postfix/ldap/ldap_virtual_domains_maps.cf || echo 'failed!'


:: /etc/postfix/ldap/ldap_virtual_mailbox_maps.cf

    #server_host = ldapi://%2Fvar%2Frun%2Fldapi/
    server_host = ldap://127.0.0.1:389/
    version = 3
    version = 3
    search_base = uid=%u,dc=%d,ou=virtual,o=mailHosting,dc=fripost,dc=org
    scope = base
    bind = yes
    bind_dn = cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org
    bind_pw = xxxxxx
    query_filter = (&(ObjectClass=virtualMailbox)(uid=%u)(isActive=TRUE))
    result_attribute = uid

Test it:
    postmap -q user@fripost.org ldap:/etc/postfix/ldap/ldap_virtual_mailbox_maps.cf || echo 'failed!'
    postmap -q fake@fake.org ldap:/etc/postfix/ldap/ldap_virtual_mailbox_maps.cf || echo 'failed!'


:: /etc/postfix/ldap/ldap_virtual_alias_maps.cf

    #server_host = ldapi://%2Fvar%2Frun%2Fldapi/
    server_host = ldap://127.0.0.1:389/
    version = 3
    search_base = dc=%d,ou=virtual,o=mailHosting,dc=fripost,dc=org
    scope = one
    bind = yes
    bind_dn = cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org
    bind_pw = xxxxxx
    query_filter = (&(ObjectClass=virtualAliases)(mailLocalAddress=%u)(isActive=TRUE))
    result_attribute = mailTarget

:: /etc/postfix/ldap/ldap_virtual_alias_catchall_maps.cf

    #server_host = ldapi://%2Fvar%2Frun%2Fldapi/
    server_host = ldap://127.0.0.1:389/
    version = 3
    search_base = dc=%d,ou=virtual,o=mailHosting,dc=fripost,dc=org
    scope = one
    bind = yes
    bind_dn = cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org
    bind_pw = xxxxxx
    query_filter = (&(ObjectClass=virtualAliases)(mailLocalAddress=)(isActive=TRUE))
    result_attribute = mailTarget

Test it:
    postmap -q user-alias@fripost.org ldap:/etc/postfix/ldap/ldap_virtual_alias_maps.cf ldap:/etc/postfix/ldap/ldap_virtual_alias_catchall_maps.cf
    postmap -q user@example.org ldap:/etc/postfix/ldap/ldap_virtual_alias_maps.cf ldap:/etc/postfix/ldap/ldap_virtual_alias_catchall_maps.cf
    postmap -q fake@example.org ldap:/etc/postfix/ldap/ldap_virtual_alias_maps.cf ldap:/etc/postfix/ldap/ldap_virtual_alias_catchall_maps.cf

Note: There is no way to get rid of the warning `Fixed query_filter
[...] is probably useless'.  It is harmless in our case, since the
search base is precise enough. However, we add a logcheck exception not
to be flooded.
TODO: That will change with the new schema.

**** Setting up the MDA

# squeeze has dovecot-1.2. upgrade notes:
# - we might want to upgrade to their sieve (instead of cmusieve)
# - we want to add the -s flag to deliver in master.cf

***** Installing

sudo aptitude install dovecot-imapd

***** Configuring

:: /etc/dovecot/dovecot.conf

    protocol lda {
      # Address to use when sending rejection mails.
      postmaster_address = postmaster@fripost.org

      # Hostname to use in various parts of sent mails, eg. in Message-Id.
      # Default is the system's real hostname.
      hostname = imap.fripost.org

      # Support for dynamically loadable plugins. mail_plugins is a space separated
      # list of plugins to load.
      #mail_plugins =
      #mail_plugin_dir = /usr/lib/dovecot/modules/lda

      # Binary to use for sending mails.
      sendmail_path = /usr/lib/sendmail

      # UNIX socket path to master authentication server to find users.
      auth_socket_path = /var/run/dovecot/auth-master

      # Enabling Sieve plugin for server-side mail filtering
      mail_plugins = cmusieve
    }

    [...]

      ## dovecot-lda specific settings
      ##
      socket listen {
        master {
          path = /var/run/dovecot/auth-master
          mode = 0600
          user = xxx # User running Dovecot LDA
          #group = mail # Or alternatively mode 0660 + LDA user in this group
        }
      }

:: /etc/postfix/master.cf

    dovecot   unix  -       n       n       -       -       pipe
      flags=DRhu user=xxx:xxx argv=/usr/lib/dovecot/deliver -f ${sender} -d ${recipient} -n


:: /etc/postfix/main.cf

    virtual_transport = dovecot
    dovecot_destination_recipient_limit = 1

http://wiki.dovecot.org/LDA/Postfix
http://www.tehinterweb.co.uk/roundcube/#pisieverules

**** Test delivery

sudo mkdir -p /home/mail/virtual/fripost.org/

 :: ldapadd -xWD cn=admin,dc=fripost,dc=org

    dn: dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    objectClass: virtualDomain
    isActive: TRUE

    dn: uid=example,dc=fripost.org,ou=virtual,o=mailHosting,dc=fripost,dc=org
    objectClass: virtualMailbox
    userPassword: test666
    isActive: TRUE


sudo /etc/init.d/postfix restart

echo "test at `date`" | mail -s "test" exempel@fripost.org

**** Configuring Dovecot

sudo aptitude install dovecot-imapd

:: /etc/dovecot/dovecot.conf

# Note: These settings are already in the file but commented out or set to other
#       values.

:HIDDEN:
protocols = imaps
protocol imap {
	ssl_listen = *:993
}
disable_plaintext_auth = yes
mail_location = maildir:/home/mail/virtual/%d/%u/Maildir

# Set this to something that works for the Maildirs
first_valid_uid = XXX
first_valid_gid = XXX

# Allow clients to be fancy if they want to
mechanisms = plain cram-md5

#passdb pam <--- comment this stuff out

# uncomment this stuff
passdb ldap {
    args = /etc/dovecot/dovecot-ldap.conf
}

#userdb passwd  <--- comment this stuff out

# uncomment this stuff
userdb static {
    args = uid=115 gid=8 home=/home/mail/virtual/%d/%n/ allow_all_users=yes
}
# We are not making use of the User Database (to ensure that Dovecot's `deliver' checks
# that the recipient exists) here, since `deliver' should only be called by Postfix which
# takes care of that (cf. `ldap_virtual_mailbox_maps.cf'). Hence the `allow_all_users=yes'
# above.

# Do not needlessly run as root
user = nobody
:END:

***** Use LDAP authenticate binds, and LDAP user queries.

Instead of making a LDAP query to fetch the (hashed) passwords, which implies
to expose all (hashed) credentials to Dovecot, an other approach is to forward
the login information to our LDAP server, that will match it against the hashed
copy contained in its database. This way if your IMAP server is compromised,
the attacker will not have access to all the e-mails and user credentials.

Documentation:
http://wiki2.dovecot.org/HowTo/DovecotOpenLdap
http://wiki2.dovecot.org/AuthDatabase/LDAP/AuthBinds

Debian provides a squeleton configuration in /usr/share/dovecot/dovecot-ldap.conf .
Copy this file in /etc/dovecot, and chmod 600 it.

:: /etc/dovecot/dovecot-ldap.conf

:HIDDEN:
hosts = localhost
ldap_version = 3
auth_bind = yes
auth_bind_userdn = uid=%n,dc=%d,ou=virtual,o=mailHosting,dc=fripost,dc=org
base = uid=%n,dc=%d,ou=virtual,o=mailHosting,dc=fripost,dc=org
deref = never
scope = base
pass_attrs = uid=user
pass_filter = (&(objectClass=virtualMailbox)(uid=%n)(isActive=TRUE))
:END:

sudo /etc/init.d/dovecot restart

# Provided there is a user, you should now be able to login using any IMAP
# client.
#
#   $~ openssl s_client -connect localhost:993
#   [...]
#   * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.

**** Making sure the services are not started at boot [might not be needed]
sudo update-rc.d -n dovecot stop 2 3 4 5 .
sudo update-rc.d -n postfix stop 2 3 4 5 .

*** Configuring an Anti-SPAM on the Mail Delivery Agent (MDA)
We will use Amavisd-new v2.7 (earlier version do not support per-user bayes database).
The WhiteList and the Bayes filter are stored in a MySQL database, managed
directly by SpamAssassin. Viruses are detected by ClamAV.
The use of Amavis allows us to have a per-user configuration (for instance, everyone
can opt-in/-out the Virus scaning or fine-tune the sensibility of the Anti-SPAM).

**** Install packages
  apt-get install clamav clamav-daemon clamav-freshclam
  apt-get install mysql-server spamassassin razor spamc
  apt-get install -t squeeze-backports amavisd-new

The user 'clamav' needs to be added to the group 'amavis':
  adduser clamav amavis

We need to remove '/etc/cron.daily/amavisd-new', now subsumed by '/etc/cron.d/amavisd-new':
  rm /etc/cron.daily/amavisd-new

**** Install the MySQL database

  sudo mysql -p
  mysql> CREATE DATABASE spamassassin CHARSET utf8 COLLATE utf8_unicode_ci;
  mysql> use spamassassin;

A SQL script to create tables to store the WhiteList and Bayes filter can be found
in '/usr/share/doc/spamassassin/sql/awl_mysql.sql', and
'/usr/share/doc/spamassassin/sql/bayes_mysql.sql', however it does not work well with
the unicode character set. We use a custom script instead:

CREATE TABLE awl (
  username  VARCHAR(100)   CHARACTER SET latin1 NOT NULL DEFAULT '',
  email     VARBINARY(255)                      NOT NULL DEFAULT '',
  ip        VARCHAR(40)    CHARACTER SET latin1 NOT NULL DEFAULT '',
  count     INT(11)                             NOT NULL DEFAULT 0,
  totscore  FLOAT                               NOT NULL DEFAULT 0,
  signedby  VARCHAR(255)   CHARACTER SET latin1 NOT NULL DEFAULT '',
  PRIMARY KEY (username,email,signedby,ip)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE bayes_global_vars (
  variable VARCHAR(30)  CHARACTER SET latin1 NOT NULL DEFAULT '',
  value    VARCHAR(200) CHARACTER SET latin1 NOT NULL DEFAULT '',
  PRIMARY KEY (variable)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
INSERT INTO bayes_global_vars VALUES ('VERSION','3');

CREATE TABLE bayes_vars (
  id                 INT(11)                           NOT NULL AUTO_INCREMENT,
  username           VARCHAR(200) CHARACTER SET latin1 NOT NULL DEFAULT '',
  spam_count         INT(11)                           NOT NULL DEFAULT 0,
  ham_count          INT(11)                           NOT NULL DEFAULT 0,
  token_count        INT(11)                           NOT NULL DEFAULT 0,
  last_expire        INT(11)                           NOT NULL DEFAULT 0,
  last_atime_delta   INT(11)                           NOT NULL DEFAULT 0,
  last_expire_reduce INT(11)                           NOT NULL DEFAULT 0,
  oldest_token_age   INT(11)                           NOT NULL DEFAULT 2147483647,
  newest_token_age   INT(11)                           NOT NULL DEFAULT 0,
  PRIMARY KEY (id),
  UNIQUE KEY bayes_vars_idx1 (username)
) ENGINE=InnoDB AUTO_INCREMENT=16 DEFAULT CHARSET=utf8;

CREATE TABLE bayes_expire (
  id      INT(11),
  runtime INT(11) NOT NULL DEFAULT 0,
  KEY bayes_expire_idx1 (id),
  FOREIGN KEY (id) REFERENCES bayes_vars(id) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE bayes_seen (
  id    INT(11),
  msgid VARCHAR(200) CHARACTER SET latin1 NOT NULL DEFAULT '',
  flag  CHAR(1)      CHARACTER SET latin1 NOT NULL DEFAULT '',
  FOREIGN KEY (id) REFERENCES bayes_vars(id) ON DELETE CASCADE ON UPDATE CASCADE,
  PRIMARY KEY (id,msgid)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

CREATE TABLE bayes_token (
  id         INT(11),
  token      BINARY(5) NOT NULL DEFAULT '',
  spam_count INT(11)   NOT NULL DEFAULT 0,
  ham_count  INT(11)   NOT NULL DEFAULT 0,
  atime      INT(11)   NOT NULL DEFAULT 0,
  FOREIGN KEY (id) REFERENCES bayes_vars(id) ON DELETE CASCADE ON UPDATE CASCADE,
  PRIMARY KEY (id,token),
  KEY bayes_token_idx1 (id,atime)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;


Note that the 'InnoDB' engine needs to be allowed by the MySQL configuration. We
can now create our users and grant minimal rights:

mysql> create user 'sa_awl'@'localhost' IDENTIFIED BY '...';
mysql> create user 'sa_bayes'@'localhost' IDENTIFIED BY '...';
mysql> GRANT SELECT, INSERT, UPDATE, DELETE ON spamassassin.awl               TO 'sa_awl'@'localhost';
mysql> GRANT SELECT, INSERT,         DELETE ON spamassassin.bayes_seen        TO 'sa_bayes'@'localhost';
mysql> GRANT SELECT, INSERT, UPDATE, DELETE ON spamassassin.bayes_token       TO 'sa_bayes'@'localhost';
mysql> GRANT SELECT                         ON spamassassin.bayes_global_vars TO 'sa_bayes'@'localhost';
mysql> GRANT SELECT, INSERT, UPDATE, DELETE ON spamassassin.bayes_vars        TO 'sa_bayes'@'localhost';
mysql> GRANT SELECT, INSERT,         DELETE ON spamassassin.bayes_expire      TO 'sa_bayes'@'localhost';

We're now ready to configure SpamAssassin:
  touch /etc/spamassassin/sql.cf
  chown root:amavis /etc/spamassassin/sql.cf
  chmod 0644 /etc/spamassassin/sql.cf

  :: /etc/spamassassin/sql.cf
    bayes_store_module          Mail::SpamAssassin::BayesStore::MySQL
    bayes_sql_dsn               DBI:mysql:spamassassin:localhost
    bayes_sql_username          sa_bayes
    bayes_sql_password          ...

    auto_whitelist_factory      Mail::SpamAssassin::SQLBasedAddrList
    user_awl_dsn                DBI:mysql:spamassassin:localhost
    user_awl_sql_username       sa_awl
    user_awl_sql_password       ...


**** Configure Amavis

  :: /etc/amavis/conf.d/05-domain_id
    $mydomain = "fripost.org";
    @local_domains_acl = ( ".$mydomain" );
    @local_domains_maps = ( ".$mydomain" );

  :: /etc/amavis/conf.d/50-user
    @sa_username_maps = (
      new_RE ( [ qr'^([^@]+@.*)'i => '${1}' ] ),
      'amavis'
    );

# TODO: use LDAP lookups for per-user configuration
# http://www.ijs.si/software/amavisd/LDAP.schema

*** Configuring a new smarthost to relay e-mail to the main IMAP server
**** Overview

We relay mail from our smarthosts to the main IMAP server.

This is to avoid having a single point of failure and to separate concerns. The
IMAP server then only needs to deal with authenticated clients and the
smarthosts.

**** Prerequisites

Before this can work we must make sure that:
- SyncProv is configured on our main LDAP server, and
- There is an SSH tunnel for the smtp.

If they are both setup, we can configure postfix on the smarthost to relay
emails through the tunnel.

**** Configuring a partial replicate of the LDAP directory

Partial, since we will restrict the replication to what is needed for Postfix
virtual lookups. In particular, the passwords will not be replicated.

Refererences:
- http://www.openldap.org/doc/admin24/replication.html#Syncrepl
- http://www.zytrax.com/books/ldap/ch7/#ol-syncrepl-rap

In the rest of this section, we assume there is a tunnel from the provider
LDAP server to the consumer (i.e., ldap://127.0.0.1:3890 on the MX actually
speaks to the the main provider).

***** Installation

Cf. installation on the main LDAP server.  (We also need to install Fripost's
schema and indexes, but not the ACLs.)

***** Using SyncRepl

  :: ldapmodify -Y EXTERNAL -H ldapi:///

    dn: olcDatabase={1}hdb,cn=config
    changetype: modify
    replace: olcSyncRepl
    # Increase the rid for the different consumers
    olcSyncRepl: rid=000
    provider=ldap://127.0.0.1:3890
    bindmethod=simple
    binddn="cn=SMTP,ou=services,o=mailHosting,dc=fripost,dc=org"
    credentials="xxxxxx"
    type=refreshAndPersist
    retry="5 5 300 +"
    searchbase="ou=virtual,o=mailHosting,dc=fripost,dc=org"
    filter="(&(|(objectClass=FripostVirtualDomain)(objectClass=FripostVirtualMailbox)(objectClass=FripostVirtualAlias)(objectClass=FripostVirtualML))(fripostIsStatusActive=TRUE))"
    attrs="fripostIsStatusActive,fripostMaildrop,fvd,fvu,fva,fvml,fripostMLCommand,fripostMLManager"
    scope=sub
    schemachecking=off

(Since in our case we have several slaves, we increment the rid accross the
different MX:s.)

***** Restrict the access

Ideally, we would use the following single ACL

  :: ldapmodify -Y EXTERNAL -H ldapi:///

    dn: olcDatabase={1}hdb,cn=config
    changetype: modify
    replace: olcAccess
    olcAccess: to dn.subtree="ou=virtual,o=mailHosting,dc=fripost,dc=dev"
        by dn.exact="gidNumber=121+uidNumber=113,cn=peercred,cn=external,cn=auth" =rsd

Were 113 and 121 are the uid and gid of the "postfix" user, respectively (which
you can find by typing `id postfix').  This would give =rsd access to the
"postfix" user on the directory, when SASL-binding on a UNIX socket. However
SASL binding have only been introduced with Postfix 2.8 [TODO: after upgrade,
change that], so we will use anonymous access for the time being. It isn't very
satisfactory, but since the replicate stripped out the irrelevant and critical
bits it should be fine for now.

Ensure that authentication is not required, and that no ACL is defined (the
default is then to allow read access to anyone).

  :: sudo ldapsearch -LLL -Y EXTERNAL -H ldapi:/// -b "cn=config" "(olcDatabase={1}hdb)" olcRequires olcAccess
    [...]
    olcRequires: LDAPv3

**** Configuration files

TODO: add the necessary configuration files

*** Configuring the outgoing SMTP (MTA)
**** Install packages

sudo aptitude install postfix

:: /etc/postfix/main.cf

smtp_bind_address               = 88.80.16.139
smtp_bind_address6              = 2A00:16B0:242:13F::1
[...]
smtp_tls_security_level         = may
smtp_tls_note_starttls_offer    = yes

(Note: Ideally, the IPv4 and IPv6 address above should resolve to our
hostname, namely `smtp.fripost.org' here.)

We don't want to force the SMTP client to use encrypted connection
regardless, as some servers may not support it :-/

**** Relay emails from trusted hosts

  :: /etc/postfix/main.cf

    relay_clientcerts = hash:$config_directory/relay_clientcerts
    [...]
    # TODO: should be "secure" on 25
    smtpd_tls_security_level        = may
    # TODO: proper certs
    smtpd_tls_cert_file             = /etc/ssl/certs/ssl-cert-snakeoil.pem
    smtpd_tls_key_file              = /etc/ssl/private/ssl-cert-snakeoil.key
    smtpd_tls_CApath                = /etc/ssl/certs/
    smtpd_tls_session_cache_database= btree:${data_directory}/smtpd_tls_session_cache
    smtpd_tls_received_header       = yes
    smtpd_tls_ask_ccert             = yes
    smtpd_tls_session_cache_timeout = 3600s
    smtpd_tls_fingerprint_digest    = sha1
    smtpd_tls_eecdh_grade           = strong
    [...]
    smtpd_recipient_restrictions =
        [...]
        permit_mynetworks
        permit_tls_clientcerts
        [...]

/etc/postfix/relay_clientcerts lists (SHA-1) fingerprints and hostnames
of our trusted hosts. Fingerprints can be obtained with

  openssl x509 -fingerprint -sha1 -noout -in /path/to/pubkey.pem

    :: /etc/postfix/relay_clientcerts
    E0:3C:E7:05:2D:2E:99:7B:EF:A1:D0:5A:A7:79:2C:6D:0B:66:FD:17 luxemburg
    [...]

Do not forget do update this file if the you change the hostname or certificate of the
trusted hosts! And to hash it afterwards:

  postmap /etc/postfix/relay_clientcerts


We need to force the trusted hosts (our MX:s and the webmail) to use TLS when talking to
the mailhub. For instance on mx1.fripost.org,

  :: /etc/postfix/main.cf
    [...]
    smtp_tls_security_level = may
    smtp_tls_policy_maps    = hash:$config_directory/tls_policy
    smtp_tls_cert_file      = /path/to/pubkey.pem
    smtp_tls_key_file       = /path/to/privkey.key
    [...]

  :: /etc/postfix/tls_policy
    smtp:[smtp.fripost.org]:25 secure ciphers=high

(Note: The `secure' TLS policy will not accept self-signed certificates, or
certificates which CN doesn't match!)

We don't want to force the SMTP client to use encrypted connection
regardless, as some servers may not support it :-/

*** Configuring the Mail Submission Agent (MSA)

We offer a SMTP relay for authenticated users (via SASL). Currently the MSA and
MTA are hosted on the same machine (gnu).

Firewall: The MSA needs 587/TCP in, and the MTA 25/TCP both in and out.

**** Install packages

sudo apt-get install sasl2-bin libsasl2-modules

(Scrictly speaking sasl2-bin is not necessary, but it offers some programs to
test our installation.)

In the rest of this section, we assume there is a tunnel from the master
LDAP server to the machine that hosts SASLauthd (i.e., ldap://127.0.0.1:3890 on
this machine actually speaks to the master).

**** Fixing CApath

As of Debian Squeeze, Postfix doesn't copy the content of 'smtp_tls_CApath' and
'smtpd_tls_CApath' in the chroot jail. This leads to a flood of "Untrusted
connections" since Postfix doesn't have any root CA to trust.
To do it by hand, copy the files (don't forget the symlink targets) under
'/var/spool/postfix/etc/ssl/certs' and c_rehash this last directory.

The issue is fixed under Postfix 2.8. For the time being, a script to automatize
the above process can be found in the fripost-admin repository.

  $~ sudo postfix-fixcerts.sh

**** Configure SASLauthd

 :: /etc/default/saslauthd
    [...]
    START=yes
    MECHANISMS=ldap
    OPTIONS="-c -m /var/spool/postfix/var/run/saslauthd -O /etc/saslauthd.conf"
    [...]

(Note: The socket has to be in Postfix's chroot jail.)

 :: /etc/saslauthd.conf

    ldap_servers: ldap://127.0.0.1:3890/
    ldap_version: 3
    ldap_bind_dn: cn=SASLauth,ou=services,o=mailHosting,dc=fripost,dc=org
    ldap_bind_pw: xxxxxx
    ldap_auth_method: bind
    ldap_search_base: uid=%U,dc=%d,ou=virtual,o=mailHosting,dc=fripost,dc=org
    ldap_filter: (&(objectClass=virtualMailbox)(uid=%U)(isActive=TRUE))
    ldap_scope: base

We need to bind to `cn=SASLauth,...' here, because SASLauthd performs the
search before binding to the user (unlike Dovecot). Hence it needs to have read
access on the user's entry (except his/her password, of course). TODO: maybe
=sd access is enough actually?

After restarting saslauthd (`/etc/init.d/saslauthd restart'), we can test the
authentication:

    sudo testsaslauthd -f /var/spool/postfix/var/run/saslauthd/mux -u user@fripost.org -p password

(The password cannot be prompted, so you may want to create a dummy user.)

**** Configure Postfix

If everything goes through, it is now time to modify Postfix's main.cf:
(Documentation: http://www.postfix.org/SASL_README.htm)

 :: /etc/postfix/main.cf
    [...]
    smtpd_sasl_auth_enable          = no
    smtpd_sasl_authenticated_header = yes
    smtpd_sasl_local_domain         = fripost.org
    smtpd_sasl_security_options     = noanonymous, noplaintext
    smtpd_sasl_tls_security_options = noanonymous
    broken_sasl_auth_clients        = no
    smtpd_sasl_type                 = cyrus
    smtpd_sasl_path                 = smtpd
    [...]
    smtpd_recipient_restrictions =
            [...]
            permit_sasl_authenticated
            reject_unauth_destination
    [...]

  :: /etc/postfix/sasl/smtpd.conf
    pwcheck_method: saslauthd
    mech_list: PLAIN LOGIN

Finally, we can add the submission service to our master.cf, with customized policy:

 :: /etc/postfix/master.cf

    smtp      inet  n       -       -       -       -       smtpd
    submission inet n       -       -       -       -       smtpd
      -o smtpd_tls_security_level=encrypt
      -o smtpd_sasl_auth_enable=yes
      -o smtpd_client_restrictions=permit_sasl_authenticated,reject
      -o smtpd_helo_restrictions=reject_invalid_helo_hostname
    [...]

(We don't reject soon-to-be authenticated clients for having a non-FQDN hostame.)

Postfix needs to be added to the `sasl' group to talk to SASLauthd:

  $~ adduser postfix sasl

We now have to restart Postfix: `/etc/init.d/postfix restart'. (Maybe `postfix reload'
is enough actually.)

**** Test it

[Note: if you test it from localhost, you have to set smtpd_sasl_exceptions_networks
first.]

First, we ensured that encrypted conections are required.

  :: telnet localhost 25
     [...]
     250-ETRN
     250-STARTTLS
     250-ENHANCEDSTATUSCODES
     250-8BITMIME
     250 DSN

What the user type is here emphasized and prefixed with a `*'

  :: openssl s_client -connect localhost:25 -starttls smtp -CApath /etc/ssl/
     [...]
         Verify return code: 0 (ok)
     ---
     250 DSN
  *  EHLO localhost
     [...]
     250-ETRN
     250-AUTH LOGIN PLAIN
     250-ENHANCEDSTATUSCODES
     250-8BITMIME
     250 DSN
  *  AUTH PLAIN AHVzZXJAZnJpcG9zdC5vcmcAdXNlcg==
     235 2.7.0 Authentication successful
  *  mail from:<user@fripost.org>
     250 2.1.0 Ok
  *  rcpt to:<user@fripost.org>
     250 2.1.5 Ok
  *  DATA
     354 End data with <CR><LF>.<CR><LF>
  *  Subject: test
  *  \o/
  *  .
     250 2.0.0 Ok: queued as 3D7767B4BD

Where "AHVzZXJAZnJpcG9zdC5vcmcAdXNlcg==" is a base-64 encoding of the user's,
credentials, in our case login "user@fripost.org" and password "user", which
can be obtained by the command

    echo -ne '\000user@fripost.org\000user' | openssl base64

or slightly better (does not write password in the ~/.bash_history)

    read U PW; echo -ne "\000$U\000$PW" | openssl base64

**** Anonymize the senders

***** Overview

Reference: https://we.riseup.net/debian/mail#getting-your-postfix-anonymized

If RoudCube automatically anonymize the sender (by simply shortening the
trace), it's not the case (by default) for SquirrelMail, or when clients
connect via ESMTP/ESMTPS/ESMTPA/ESMTPSA. Here are a couple of traces we want
to obfuscate, to prevent the recicipient and/or the intermediate SMTP relays
to track the sender.

    Received: from localhost (smtp.fripost.org [127.0.0.1])
            by fripost.org (Postfix) with ESMTP id C9DAB841F4
            for <recipient@example.org>; Thu, 22 Mar 2012 16:27:56 +0100 (CET)
    Received: from fripost.org ([127.0.0.1])
            by localhost (smtp.fripost.org [127.0.0.1]) (amavisd-new, port 10024)
            with ESMTP id 8onAXWOvImDh for <recipient@example.org>;
            Thu, 22 Mar 2012 16:27:56 +0100 (CET)
    Received: from webmail.fripost.org (localhost [IPv6:::1])
            by fripost.org (Postfix) with ESMTP id 3ADAB8243D
            for <recipient@example.org>; Thu, 22 Mar 2012 16:27:56 +0100 (CET)
    Received: from 192.168.1.5
            (SquirrelMail authenticated user username)
            by webmail.fripost.org with HTTP;
            Thu, 22 Mar 2012 16:27:56 +0100

    Received: from localhost (smtp.fripost.org [127.0.0.1])
            by fripost.org (Postfix) with ESMTP id 2D1098243D
            for <recipient@example.org>; Thu, 22 Mar 2012 16:36:36 +0100 (CET)
    Received: from fripost.org ([127.0.0.1])
            by localhost (smtp.fripost.org [127.0.0.1]) (amavisd-new, port 10024)
            with ESMTP id Hr2J-eRTN0jI for <recipient@example.org>;
            Thu, 22 Mar 2012 16:36:35 +0100 (CET)
    Received: from client.example.org (client.example.org [192.168.1.1])
            (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
            (Client CN "client.example.org", Issuer "example.org" (not verified))
            by machine.org (Postfix) with ESMTPS id DA22981B95
            for <recipient@example.org>; Thu, 22 Mar 2012 16:36:35 +0100 (CET)
    Received: (nullmailer pid 5057 invoked by uid 0);
            Thu, 22 Mar 2012 15:36:34 -0000

    Received: from localhost (smtp.fripost.org [127.0.0.1])
            by fripost.org (Postfix) with ESMTP id DBAFE816BB
            for <recipient@example.org>; Thu, 22 Mar 2012 14:48:01 +0100 (CET)
    Received: from fripost.org ([127.0.0.1])
            by localhost (smtp.fripost.org [127.0.0.1]) (amavisd-new, port 10024)
            with ESMTP id Upen4QhYpKf4 for <recipient@example.org>;
            Thu, 22 Mar 2012 14:48:01 +0100 (CET)
    Received: from client.example.org (client.example.org [192.168.1.5])
            (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
            (Client CN "", Issuer "" (not verified))
            (Authenticated sender: username)
            by smtp.fripost.org (Postfix) with ESMTPSA id 40284804F5
            for <recipient@example.org>; Thu, 22 Mar 2012 14:48:01 +0100 (CET)
    Received: by client.example.org (Postfix, from userid 1000)
            id 1D24F41747; Thu, 22 Mar 2012 14:48:00 +0100 (CET)

(The first one was sent using a SquirrelMail; The second using ESMTPS;
And the third using ESMTPSA).
If we are to hide the sender, we could simply clean the trace (like
RoundCube does) when the mail leaves the server. However, some aggressive
mailfilters may reject the mail since the trace is incomplete (if RoundCube
hides the history I guess it doesnt' happen that often, but who knows...).

Another option would be to clean the trace and to simply add a fake field
to pretend that the mail is sent from localhost by the user nobody:
    Received: by fripost.org (Postfix, from userid 65535)
            id 2C537816BB; Thu, 22 Mar 2012 14:08:45 +0100 (CET)
This possible by adding "smtp_header_checks = regexp:$config_directory/smtp_header_checks"
in the Postfix's main.cf, with a suitable file "smtp_header_check" in the Postfix
configuration directory.

Yet another option is not to hide the trace, but rather forge it to
pretend that the ESMTP/... connections are all coming from localhost.
This way we are not hiding the fact that a client has logged in using a
valid certificate, and in case of an SMTP relay, the early part of the
trace (before it entered our Postfix sever) remains unchanged. For
example, the early part of the third trace would become:

    Received: from [127.0.0.1] (localhost [127.0.0.1])
            (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
            (Authenticated sender: username)
            by smtp.fripost.org (Postfix) with ESMTPSA id 40284804F5
            for <recipient@example.org>; Thu, 22 Mar 2012 14:48:01 +0100 (CET)
    Received: by client.example.org (Postfix, from userid 1000)
            id 1D24F41747; Thu, 22 Mar 2012 14:48:00 +0100 (CET)

(the other field remaining unchanged). This is also made possible by
smtp_header_checks. In that case, the corresponding file would contain
the following rexep, forging the header by pretending that the client
has EHLO'ed from localhost:

    /^Received:\s+from\s+([._[:alnum:]-]+\s+\([._[:alnum:]-]+\s+\[[[:xdigit:].:]{3,39}\]\))(\s+\(using\s+(TLSv1|SSLv[23])\s+with\s+cipher\s+\S+\s+\([\/0-9]+\s+bits\)\)\s+).*(\(Authenticated sender:\s+[^)]+\)\s+).*(by\s+smtp\.fripost\.org\s+\([^)]+\)\s+with\s+E?SMTPS?A?\s+id\s+[[:xdigit:]]+.*)/
        REPLACE Received: from [127.0.0.1] (localhost [127.0.0.1])${2}${4}${5}

You can try out the regexp using "postmap -h -q - regex:/etc/postfix/smtp_header_checks < email"
(where `email' may also be a bunch of traces).

We also forge the certificate the client send during the TLS/SSL
handshake, since its CN and Issuer may help to track him/her down.

DISCLAIMER: The regexp probably needs tests (especially for multiple hops,
in case of relaying SMTPs). Also, note that the hostname of the client has
NOT been obfuscated in the above trace (and that will break the relaying path
if the client has a routable hostname that doesn't point to the SMTP server!).
However, this line has been added by the client itself, so it's his/her
responsability to masquerade it I suppose.

***** Install packages

  apt-get install postfix-pcre

***** Configure postfix

  :: /etc/postfix/smtp_anonymize_sender.pcre

    /^Received:\s+from\s+([._[:alnum:]-]+\s+\([._[:alnum:]-]+\s+\[[[:xdigit:].:]{3,39}\]\))(\s+\(using\s+(TLSv1|SSLv[23])\s+with\s+cipher\s+\S+\s+\([\/0-9]+\s+bits\)\)\s+).*(\(Authenticated sender:\s+[^)]+\)\s+).*(by\s+smtp\.fripost\.org\s+\([^)]+\)\s+with\s+E?SMTPS?A?\s+id\s+[[:xdigit:]]+.*)/
        REPLACE Received: from [127.0.0.1] (localhost [127.0.0.1])${2}${4}${5}

    /^X-Originating-IP:/    IGNORE


  :: /etc/postfix/master.cf
    submission inet n       -       -       -       -       smtpd
        [...]
        -o cleanup_service_name=cleanup2
    [...]
    cleanup2   unix  n       -       -       -       0       cleanup
        -o header_checks=pcre:$config_directory/smtpd_anonymize_sender.pcre

By default, postfix logs the header checks (or rewrites in our case).
For privacy reasons, we drop these when they entery syslog:

  :: /etc/rsyslog.conf

    # Do not log our Postfix's header rewrites, that are meant to
    # anonymize senders.
    :msg, ereregex, "^ [[:alnum:]]+: replace: header Received: from" ~
    [...]
    auth,authpriv.*                 /var/log/auth.log
    *.*;auth,authpriv.none          -/var/log/syslog
    [...]

** Configuring the webserver

sudo apt-get install apache2

sudo a2enmod ssl rewrite

:: /etc/apache2/ports.conf

    <IfModule mod_ssl.c>
      NameVirtualHost *:443
    </IfModule>

:: /etc/apache2/conf.d/security

    ServerTokens Prod

*** Roundcube

**** Installing roundcube

# Add the backports repository first, to make sure we're running a somewhat more
# current version than the one currently in stable.

:: /etc/apt/sources.list

    deb http://backports.debian.org/debian-backports squeeze-backports main

sudo apt-get install roundcube

:: /etc/php5/apache2/php.ini

    log_errors = Off
    post_max_size = 25M
    upload_max_filesize = 25M
    tmp_dir = FIXME

:: /etc/roundcube/main.inc.php  ## checked for roundcube 0.5.4+dfsg-1~bpo60+1

    # Use caching
    $rcmail_config['enable_caching'] = TRUE;

    # fripost.org specific
    $rcmail_config['force_https'] = TRUE;
    $rcmail_config['default_host'] = 'ssl://imap.fripost.org';
    $rcmail_config['imap_auth_type'] = 'plain';
    $rcmail_config['username_domain'] = 'fripost.org';

    $rcmail_config['smtp_server'] = 'localhost';
    $rcmail_config['smtp_port'] = 25;

    # use IP for extra paranoia
    $rcmail_config['ip_check'] = true;

    # Locale settings
    $rcmail_config['language'] = 'sv_SE';
    $rcmail_config['date_long'] = 'Y-m-d.Y H:i';

    $rcmail_config['product_name'] = 'Fripost';

    # IMAP Folders (I guess these were changed for compatibility with SquirrelMail)
    $rcmail_config['drafts_mbox'] = 'INBOX.Drafts';
    $rcmail_config['junk_mbox'] = 'INBOX.Junk';
    $rcmail_config['sent_mbox'] = 'INBOX.Sent';
    $rcmail_config['default_imap_folders'] = array('INBOX', 'INBOX.Drafts', 'INBOX.Sent', 'INBOX.Junk', 'Trash');
    $rcmail_config['create_default_folders'] = TRUE;

    $rcmail_config['smtp_log'] = false;
    $rcmail_config['http_received_header'] = false;
    $rcmail_config['http_received_header_encrypt'] = false;

    # timezone
    $rcmail_config['timezone'] = 'CET';

    # compose html formatted messages by default
    $rcmail_config['htmleditor'] = TRUE;

:: /etc/roundcube/htaccess

    php_value	upload_max_filesize	25M
    php_value	post_max_size		30M

**** Installing custom logo

wget https://fripost.org/images/logo2011_webmail.png
LOGO="logo2011_webmail.png"
sudo mv /var/lib/roundcube/skins/default/images/roundcube_logo.png /var/lib/roundcube/skins/default/images/roundcube_logo2.png
sudo mv $LOGO /var/lib/roundcube/skins/default/images/roundcube_logo.png
sudo chmod 0644 /var/lib/roundcube/skins/default/images/roundcube_logo.png

**** Adding a link to https://fripost.org

in

:: /usr/share/roundcube/skins/default/templates/login.html

make

    <roundcube:object name="logo" src="/images/roundcube_logo.png" id="logo" border="0" style="margin:0 11px" />

into an anchor element:

    <a href="https://fripost.org"><roundcube:object name="logo" src="/images/roundcube_logo.png" id="logo" border="0" style="margin:0 11px" /></a>

**** Adding a custom message on login page

Before this

: <roundcube:object name="preloader" images="

in

:: /usr/share/roundcube/skins/default/templates/login.html

    <div style="margin: 20px;"/>

    <div style="max-width: 45em; margin: 0px auto; border: dotted 3px red; padding:1em;">

    <h3>Important message</h3>

    <p align="left"><strong>Mon Feb 13 12:55:30 CET 2012</strong> </p>
    <p>
    Lorem ipsum dolor sit amet, consectetur adipiscing
    elit. Pellentesque molestie, velit vel tristique iaculis, massa diam viverra
    arcu, sit amet pellentesque dui enim vitae ipsum.</p>

    <p>J. Random Hacker</p>

    </div>

**** Allow the users to change their password

We neet to install a plugin http://trac.roundcube.net/browser/trunk/roundcubemail/plugins/password ,
which you can find in:

  :: apt-get install roundcube-plugins

Depends on PHP's LDAP library:

  :: apt-get install php5-ldap

In the rest of this section, we assume there is a tunnel from the master
LDAP server to the machine that hosts the webmail (i.e., ldap://127.0.0.1:3890
on this machine actually speaks to the master).

It is important to bind to localhost only, since with simple binds
passwords are sent clear over the network.

  :: /etc/roundcube/plugins/password/config.inc.php

    $rcmail_config['password_driver']             = 'ldap_simple';
    $rcmail_config['password_confirm_current']    = true;
    $rcmail_config['password_minimum_length']     = 12;
    $rcmail_config['password_require_nonalpha']   = false;
    $rcmail_config['password_log']                = false;
    $rcmail_config['password_ldap_host']          = '127.0.0.1';
    $rcmail_config['password_ldap_port']          = '3890';
    $rcmail_config['password_ldap_starttls']      = false;
    $rcmail_config['password_ldap_version']       = '3';
    $rcmail_config['password_ldap_basedn']        = 'ou=virtual,o=mailHosting,dc=fripost,dc=org'
    $rcmail_config['password_ldap_method']        = 'user';
    $rcmail_config['password_ldap_userDN_mask']   = 'uid=%name,dc=%domain,ou=virtual,o=mailHosting,dc=fripost,dc=org';
    $rcmail_config['password_ldap_encodage']      = 'ssha';
    $rcmail_config['password_ldap_pwattr']        = 'userPassword';
    $rcmail_config['password_ldap_force_replace'] = true;

*** ikiwiki

- sudo apt-get install ikiwiki

- Add separate ikiwiki user

[[http://rtime.felk.cvut.cz/~sojka/blog/using-ikiwiki-with-gitolite/][Link: Integration with ikiwiki]]

*** gitolite and gitweb

# Note: incomplete steps

sudo apt-get install gitolite

sudo dpkg-reconfigure gitolite

  :: /var/lib/gitolite/.gitolite.rc

    $REPO_UMASK = 0027;       # gets you 'rwxr-x---'

# Add the repositories/users to gitolite
# This is mostly self-explanatory, but begin on your local workstation:

git clone gitolite@githost:gitolite-admin

cd gitolite-admin

... make edits

git push

# Push all repositories

cd myrepo

git push --all gitolite@githost:myrepo

git push --tags gitolite@githost:myrepo

# Add the gitweb user to gitolite

sudo apt-get install gitweb

sudo usermod -a -G gitolite www-data

sudo /etc/init.d/apache2 stop

sudo /etc/init.d/apache2 start

# Add repositories to gitweb

sudo ln -s /var/lib/gitolite/repositories/myrepo.git /var/cache/git/myrepo.git

... etc

# Make sure one can checkout the repository via http

[[http://www.kernel.org/pub/software/scm/git/docs/howto/setup-git-server-over-http.txt][Git docs]]

sudo su gitolite

cd /var/lib/gitolite/repositories/myrepo.git
git update-server-info
mv hooks/post-update.sample hooks/post-update

  :: /etc/apache2/sites-available/default

    AliasMatch ^/pub(/.*\.git)(/.*)? /var/cache/git$1$2

  :: /usr/share/gitweb/indextext.html

    För att klona ett av dessa träd, installera <a href="http:///">git</a> och kör:

    <blockquote><code>git clone http://git.fripost.org/pub/</code> + projektets sökväg</blockquote>

    <p>
    För mer information om <a href="http://www.kernel.org/pub/software/scm/git/">git</a>, se en
    <a href="http://git.or.cz/">överblick</a>, en
    <a href="http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html">tutorial</a>
     eller
    <a href="http://www.kernel.org/pub/software/scm/git/docs">manualsidorna</a>.
    </p>

# Add a description of a repository for gitweb

echo "Mötesprotokoll" > fripost-meetings.git/description

** Configuring the list managers

Right now, the list managers are hosted on our outgoing SMTP (and Mail
Submission Agent), namely GNU. However, incoming email that is to be delivered
to a list, as for regular email, is handled by the MX:s since we do not relay a
whole domain for lists.

*** Configuring the MTA on the MX:s

Postfix does not support virtual transport out of the box. Virtual lists need
to be forwarded to a local alias first (replacing the '@' by '#', hence '#' needs to
be forbidden in list names), that can in turn be piped into a command
or transported elswere.

This is not optimal since a domain owner/postmaster cannot control all the lists under
his/her domain for instance. But it is the only solution we found so far.

  :: /etc/postfix/main.cf
     virtual_alias_maps = ..., ldap:$config_directory/ldap/virtual_alias_lists.cf
     mailbox_transport_maps = ldap:$config_directory/ldap/transport_lists.cf

  :: /etc/postfix/ldap/virtual_alias_maps.cf
     test-list@fripost.org                   test-list#fripost.org
     test-list-admin@fripost.org             test-list-admin#fripost.org
     test-list-bounces@fripost.org           test-list-bounces#fripost.org
     test-list-confirm@fripost.org           test-list-confirm#fripost.org
     test-list-join@fripost.org              test-list-join#fripost.org
     test-list-leave@fripost.org             test-list-leave#fripost.org
     test-list-owner@fripost.org             test-list-owner#fripost.org
     test-list-request@fripost.org           test-list-request#fripost.org
     test-list-subscribe@fripost.org         test-list-subscribe#fripost.org
     test-list-unsubscribe@fripost.org       test-list-unsubscribe#fripost.org

     test-schleuder@fripost.org              test-schleuder#fripost.org
     test-schleuder-bounces@fripost.org      test-schleuder-bounces#fripost.org
     test-schleuder-sendkey@fripost.org      test-schleuder-sendkey#fripost.org
     TODO: give the LDAP configuration

  :: /etc/postfix/ldap/transport_lists.cf
     test-list#fripost.org              smtp:[127.0.0.1]:2345
     test-list-admin#fripost.org        smtp:[127.0.0.1]:2345
     test-list-bounces#fripost.org      smtp:[127.0.0.1]:2345
     test-list-confirm#fripost.org      smtp:[127.0.0.1]:2345
     test-list-join#fripost.org         smtp:[127.0.0.1]:2345
     test-list-leave#fripost.org        smtp:[127.0.0.1]:2345
     test-list-owner#fripost.org        smtp:[127.0.0.1]:2345
     test-list-request#fripost.org      smtp:[127.0.0.1]:2345
     test-list-subscribe#fripost.org    smtp:[127.0.0.1]:2345
     test-list-unsubscribe#fripost.org  smtp:[127.0.0.1]:2345

     test-schleuder#fripost.org         smtp:[127.0.0.1]:2345
     test-schleuder-bounces#fripost.org smtp:[127.0.0.1]:2345
     test-schleuder-sendkey#fripost.org smtp:[127.0.0.1]:2345
     TODO: give the LDAP configuration

Note: in 'virtual_alias_maps', 'virtual_alias_lists.cf' should come before the
catchalls to be effective.


So every email that is to be delivered to a list manager is dropped into
127.0.0.1:2345 using the SMTP protocol.

*** Configuring the MTA on the machine hosting the list managers

In the rest of this section, we assume there is a tunnel from each MX (port 2345)
to the machine hosting the lists managers (port 2345).

Since this machine is currently also hosting the outgoing SMTP and the Mail
Submission Agent, we cannot the whole Postfix server to lists. Instead, we create a
new Postfix instance for this purpose. (We need to because we need custom
'virtual_alias_maps' that cannot be specified for a particular SMTP server only.)

**** Installation

sudo apt-get install postfix postfix-pcre postfix-cdb

**** Creating a new postfix instance

Reference: http://www.postfix.org/MULTI_INSTANCE_README.html

  sudo postmulti -e init
  sudo postmulti -I postfix-lists -G mta -e create
  sudo ln -s ../postfix/dynamicmaps.cf /etc/postfix-lists/


/etc/postfix/main.cf should be modified with

  :: /etc/postfix/main.cf
    ...
    multi_instance_wrapper = ${command_directory}/postmulti -p --
    multi_instance_enable = yes
    multi_instance_directories = /etc/postfix-lists
    ...

  :: /etc/postfix-lists/main.cf
    master_service_disable =
    queue_directory = /var/spool/postfix-lists
    mail_owner = postfix
    multi_instance_group = mta
    multi_instance_name = postfix-lists
    multi_instance_enable = yes

    readme_directory = no
    data_directory = /var/lib/postfix-lists

    smtpd_banner = $myhostname ESMTP $mail_name (Debian/GNU)
    myorigin     = /etc/mailname
    myhostname   = lists.fripost.org

    mydestination    = $myhostname
    mynetworks       = 127.0.0.0/8 [::FFFF:127.0.0.0]/104 [::1]/128
    inet_interfaces  = loopback-only
    inet_protocols   = all

    default_database_type = cdb

    recipient_delimiter  = +
    alias_database       =
    alias_maps           =
    local_recipient_maps = $transport_maps

    virtual_mailbox_domains = pcre:$config_directory/virtual_domains.pcre
    virtual_alias_maps      = pcre:$config_directory/virtual_aliases.pcre
    virtual_mailbox_maps    =

    virtual_transport = error:5.1.1 Virtual transport unavailable
    default_transport = smtp:[127.0.0.1]

    relay_domains                         = $myhostname
    transport_maps                        = cdb:$config_directory/transport_mailman
                                            cdb:$config_directory/transport_schleuder
    mailman_destination_recipient_limit   = 1
    schleuder_destination_recipient_limit = 1

  :: /etc/postfix-lists/master.cf
    2345      inet  n       -       -       -       -       smtpd
    ...
    mailman   unix  -       n       n       -       -       pipe
        flags=FR user=list:list argv=/usr/lib/mailman/bin/postfix-to-mailman.py 127.0.0.1 ${user}
        # TODO: put ${nexthop} back (it's lists.fripost.org)
    schleuder  unix  -       n       n       -       -       pipe
        flags=FR user=schleuder:schleuder argv=/usr/local/bin/postfix-to-schleuder.sh ${user}

(Don't forget to remove the other 'inet' services in the /etc/postfix-lists/master.cf)

Note: you need to to append the configuration directory to Postfix commands to talk to this
instance, for instance:
- sudo postfix -c /etc/postfix-lists reload     # reload (without -c, it reloads both the slave and the master instances)
- sudo postmap -c /etc/postfix-lists /etc/postfix-lists/transport_mailman   # postmap
- sudo postfix -c /etc/postfix-lists flush   # flush the mail queue
- mailq -C /etc/postfix-lists   # dump the mail queue
- ...

  :: /etc/postfix-lists/virtual_domains.pcre
    # Accept all domains that are not our destination.
    # (Only the MX's destinations are required, but...)
    !/^lists\.fripost\.org$/    OK

  :: /etc/postfix-lists/virtual_aliases.pcre
    # Keep the local part, but replace the local part by our relay domain.
    /^([^@]+)@/   ${1}@lists.fripost.org

  :: /etc/postfix-lists/transport_mailman
    test-mailman#fripost.org@lists.fripost.org                 mailman:
    test-mailman-admin#fripost.org@lists.fripost.org           mailman:
    test-mailman-bounces#fripost.org@lists.fripost.org         mailman:
    test-mailman-confirm#fripost.org@lists.fripost.org         mailman:
    test-mailman-join#fripost.org@lists.fripost.org            mailman:
    test-mailman-leave#fripost.org@lists.fripost.org           mailman:
    test-mailman-owner#fripost.org@lists.fripost.org           mailman:
    test-mailman-request#fripost.org@lists.fripost.org         mailman:
    test-mailman-subscribe#fripost.org@lists.fripost.org       mailman:
    test-mailman-unsubscribe#fripost.org@lists.fripost.org     mailman:

  :: /etc/postfix-lists/transport_schleuder
    test-schleuder#fripost.org@lists.fripost.org            schleuder:
    test-schleuder-bounces#fripost.org@lists.fripost.org    schleuder:
    test-schleuder-sendkey#fripost.org@lists.fripost.org    schleuder:

Note: we could use LDAP lookups in transport as well, but it is not easy for
list commands, and we have write access to the disk when adding a new list
anyway. Also, searching in a CDB table is much more efficent.

After modifying /etc/postfix-lists/transport_mailman, type

  sudo postmap -c /etc/postfix-lists /etc/postfix-lists/transport_mailman

to produce a CDB table. It is not necessary to reload Postfix after that, but
you may have to wait one minute or two for Postfix to reload the file in memory.
If you are in a hurry, type

  sudo postfix -c /etc/postfix-lists reload

to reload this instance only.


Finally, we need a new set of rules for logcheck:

  :: /etc/logcheck/ignore.d.server/postfix-lists
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/smtpd\[[[:digit:]]+\]: (dis)?connect from [^[:space:]]+$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/smtpd\[[[:digit:]]+\]: [[:alnum:]]+: client=[._[:alnum:]-]+\[[[:xdigit:].:]{3,39}\]$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/cleanup\[[[:digit:]]+\]: [[:alnum:]]+: (resent-|)message-id=<?[^>]+>?( \(added by [^[:space:]]+\))?$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/qmgr\[[[:digit:]]+\]: [[:alnum:]]+: from=<[^[:space:]]*>, size=[[:digit:]]+, nrcpt=[[:digit:]]+ \(queue active\)$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/pipe\[[[:digit:]]+\]: [[:upper:][:digit:]]+: to=<[^[:space:]]+>,( orig_to=<[^[:space:]]+>,)* relay=(mailman|schleuder|mklist-mailman|mklist-schleuder), delay=[.[:digit:]]+(, delays=([.[:digit:]]+/){3}[.[:digit:]]+)?(, dsn=2(\.[[:digit:]]+){2})?, status=sent \(delivered via \2 service\)$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/local\[[[:digit:]]+\]: [[:upper:][:digit:]]+: to=<[^[:space:]]+>,( orig_to=<[^[:space:]]+>,)? relay=local, delay=[[:digit:].]+(, delays=([.[:digit:]]+/){3}[.[:digit:]]+)?(, dsn=[45](\.[[:digit:]]+){2})?, status=(deferred|bounced) \(.+\)$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/n?qmgr\[[[:digit:]]+\]: [[:alnum:]]+: from=<.*>, status=expired, returned to sender$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/n?qmgr\[[[:digit:]]+\]: [[:alnum:]]+: message-id=(<?[^[:space:]]+>?)?( \(added by [^[:space:]]+\))?$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/n?qmgr\[[[:digit:]]+\]: [[:alnum:]]+: removed$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/n?qmgr\[[[:digit:]]+\]: [[:alnum:]]+: skipped, still being delivered$
    ^\w{3} [ :[:digit:]]{11} [._[:alnum:]-]+ postfix-lists/trivial-rewrite\[[[:digit:]]+\]: table cdb:/etc/postfix-lists/(:?mailman|schleuder)/transport\([-,|_[:alnum:]]+\) has changed -- restarting$

(We could use the whole /etc/logcheck/ignore.d.server/postfix, but it's better to
stick to the smallest rule set.)

*** GNU Mailman

**** Installation

  :: sudo apt-get install mailman

As of Debian 6.0 (Squeeze), saldy only mailman2 is available, and we need to apply
third party patches for virtual domains to work. Hopefully GNU Mailman 3 will be
available with Wheezy: it has native support for virtual domains, a LMTP server, a
much nicer interface and design...

References:
- http://wiki.list.org/pages/viewpage.action?pageId=4030604
- http://mail.python.org/pipermail/mailman-users/2010-January/068571.html
- for Mailman 3: http://wiki.list.org/display/DEV/Mailman+3.0


  cd $HOME && wget http://www.msapiro.net/mm/2.1.13-1_vhost.patch
  cd /var/lib/mailman
  sudo patch -p1 < $HOME/2.1.13-1_vhost.patch

Two hunks fail due to Debian specific patches, but it's merely line numbers that changed:
1 out of 1 hunk FAILED -- saving rejects to file Mailman/Defaults.py.in.rej
1 out of 1 hunk FAILED -- saving rejects to file Mailman/HTMLFormatter.py.rej

In 'Defaults.py', the DEFAULT_MSG_FOOTER should be kept to
  ...
  %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

as we want the a fully qualified list here. But we need to fix 'HTMLFormatter.py', and apply
other custom patches:

  cd /var/lib/mailman && sudo patch -p1 < $HOME/mailman.patch
  sudo find -L /var/lib/mailman -type f -a \( -name '*.orig' -o -name '*.rej' \) -delete

Grab our patch in the fripost-admin repository. Our patch makes the following modifications
 - Keep fully qualified lists in URLs (HTMLFormatter.py, MailList.py, Decorate.py)
 - Keep fully qualified lists in archive links (Archiver.py)
 - Make Postfix strips out the list command and add the domain part (postfix-to-mailman.py)

We now need to pin mailman to avoid our patches being overwritten by an apt-get upgrade.

  :: /etc/apt/preferences
    Explanation: We applied custom patches to many files created by this package.
    Package: mailman
    Pin: version *
    Pin-Priority: -30000

**** Configuration

  :: /etc/mailman/mm_cfg.py
    DEFAULT_URL_PATTERN = 'http://%s/cgi-bin/mailman/'
    PRIVATE_ARCHIVE_URL = '/cgi-bin/mailman/private'
    IMAGE_LOGOS         = '/images/mailman/'
    DEFAULT_EMAIL_HOST = 'lists.fripost.org'
    DEFAULT_URL_HOST   = 'smtp.fripost.org' # TODO: change that to lists.fripost.org once the A record is changed
    MTA = None
    DEB_LISTMASTER = 'listmaster@lists.fripost.org'
    ACCEPTABLE_LISTNAME_CHARACTERS = '[-_.=a-z0-9@]'
    PUBLIC_ARCHIVE_URL = 'http://%(hostname)s/pipermail/%(listname)s/'
    DEFAULT_MSG_FOOTER = """_______________________________________________
    %(real_name)s mailing list
    %(real_name)s@%(host_name)s
    %(web_page_url)slistinfo%(cgiext)s/%(real_name)s@%(host_name)s"""
    DEFAULT_CHARSET = 'UTF-8'
    add_language('en', 'English', 'utf-8')
    add_language('sv', 'Swedish', 'utf-8')

TODO: https; use a better URL scheme for the two list managers, perhaps something like
https://lists.fripost.org/mailman/ and https://lists.fripost.org/schleuder/ .

TODO: what URL format shall we choose (cf. DEFAULT_MSG_FOOTER)?
%(real_name)s@%(host_name)s vs %(host_name)s/%(host_name)s
(Right now it is the first choice.)


A first list 'mailman' is required:

  sudo -u list ./bin/newlist -q -u smtp.fripost.org mailman listmaster@fripost.org xxxxxxxxxxxxxxxx

The daemon can now be started:

  sudo /etc/init.d/mailman start


To create a list:

  sudo -u list ./bin/newlist -q -u smtp.fripost.org test-mailman@fripost.org user@fripost.org xxxxxxxxxxxxxxxx

TODO: switch to '-u lists.fripost.org' when the DEFAULT_URL_HOST is updated.
Note: in our case it is *not* the proper way to create lists, since the LDAP directory has to be updated first.

To delete a list:

  sudo rm -rf /var/lib/mailman/lists/domain/listname

And the archives:

  sudo rm -rf /var/lib/mailman/archives/{public,private}/domain/listname

**** Web server configuration (nginx + fcgiwrap)

    sudo apt-get install nginx fcgiwrap

In our case the archives under /pipermail/ do not have the right forwat, a quick &
dirty fix is to use a RewriteRule:

  :: /etc/nginx/sites-enabled/lists
    # Default list manager
    location = / {
        rewrite ^ /mailman/listinfo permanent;
    }

    # Mailman
    location ^~ /mailman/ {
        location ^~ /mailman/create { return 403; }
        rewrite ^/mailman/?$  /mailman/listinfo permanent;
        root /usr/lib/cgi-bin;
        gzip off;
        include fastcgi/params;
#        include fastcgi/ssl; #TODO
        fastcgi_pass unix:/var/run/fcgiwrap.socket;
    }
    location ^~ /images/mailman/ {
        alias /var/lib/mailman/icons/;
    }
    location ^~ /pipermail/ {
        rewrite ^/pipermail/([^@/]+)@([^@/]+)/?(.*)$ /pipermail/$2/$1/$3 last;
        index index.html;
        alias /var/lib/mailman/archives/public/;
    }

For the web interface to be available, one needs to start *both* fcgiwrap and nginx:
  sudo /etc/init.d/fcgiwrap start
  sudo /etc/init.d/nginx start

Note: We forbid access to '/create' as it is not a proper way to create lists in our
setting (one has to update the LDAP directory first).
We have a rewrite rule for '/pipermail/' as internally, archives are stored hierarchically
under the list's domain.

Note: when creating a new list with '-u lists.example.org', it is not visible under
"http://smtp.fripost.org/cgi-bin/mailman/listinfo", but one can access it under
"http://smtp.fripost.org/cgi-bin/mailman/listinfo/listname@lists.example.org". (TODO:
check that). As usual the list owner can make the list invisible, though.

*** Schleuder

Documentation: https://schleuder2.nadir.org/

**** Installation

  sudo apt-get install schleuder
  sudo chown 'schleuder:schleuder' /etc/schleuder/lists/
  sudo chmod 0700 /etc/schleuder/lists/

**** Patches

We apply custom patches to:
 - strip the list commands out of the local part of the list name (schleuder)
 - allow lists not to have any members (schleuder-newlist,list.rb), which avoid doing
to much on our panel.

  sudo patch -p1 < $HOME/schleuder.patch
(Grab our patch in the fripost-admin repository.)


We now need to pin schleuder to avoid our patches being overwritten by an update.

  :: /etc/apt/preferences
    Explanation: We applied custom patches to /usr/bin/schleuder, /usr/bin/schleuder-newlist,
    Explanation: and /usr/lib/ruby/1.8/schleuder/list.rb
    Package: schleuder
    Pin: version *
    Pin-Priority: -30000

**** Configuration

  :: /etc/schleuder/schleuder.conf
    smtp_host: smtp.fripost.org
    log_rotate_keep: 3
    superadminaddr: listmaster@fripost.org

A first list can be created with:
(Note: it can take a very long while, up to 20min. Isn't it weird by the way?)

  sudo -u schleuder schleuder-newlist test-schleuder@fripost.org -email test-schleuder@fripost.org -realname test-schleuder -nointeractive -adminaddress user@fripost.org -initmember user@fripost.org -initmemberkey /tmp/user.pub

Note: in our case it is *not* the proper way to create lists, since the LDAP directory has to be updated first.

To add members, from the command line:

  sudo -u schleuder gpg --homedir /var/schleuder/lists/test-schleuder@fripost.org --import < /tmp/user2.pub
  # sudo -u schleuder bash
  echo '- email: user2@fripost.org' >> /var/schleuder/lists/foobar/members.conf

To delete a list:

  sudo rm -rf /var/lib/schleuder/listname/ /var/log/schleuder/listname.log /etc/schleuder/lists/listname.conf

**** Web server configuration

We will use webschleuder [http://webschleuder.nadir.org/]. Sadly, it is shipped
in a precompiled Debian Package. Instruction for the installation:
http://webschleuder.nadir.org/documentation/installation.html

  sudo apt-get install rubygems libactiverecord-ruby librack-ruby camping
  sudo gem install bcrypt-ruby
  sudo mkdir -m 0755 /opt/webschleuder
  sudo chown nobody /opt/webschleuder
  cd /opt
  sudo -u nobody git clone git://git.immerda.ch/webschleuder.git webschleuder
  sudo chmod +x /opt/webschleuder/contrib/enable_webschleuder.rb


Grab our patch in the fripost-admin repository. It fixes camping's version
(Squeeze's version is too old: 2.0), and allows password to be given from
STDIN, to avoid leaking them through the proccess table.

  cd /opt/webschleuder
  sudo patch -p1 < $HOME/webschleuder.patch
  sudo find -L /opt/webschleuder -type f -a \( -name '*.orig' -o -name '*.rej' \) -delete
  sudo chown -R 'nobody:schleuder' /opt/webschleuder

Configuration:

  sudo cp webschleuder/ext/webschleuder.conf /etc/schleuder/
  sudo chown 'schleuder:schleuder' /etc/schleuder/webschleuder.conf
  sudo chmod 0600 /etc/schleuder/webschleuder.conf
  :: /etc/schleuder/webschleuder.conf
    session_secret: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    web_authtimeout: 900


If you want the list to be available from the web interface, you have to create
a file called 'web.conf' in the list's config directory, containing the bcrypt-ed
password for the web interface. It can be done by hand, or using

  echo xxxxxxxxxxxx | sudo -u schleuder ruby /opt/webschleuder/contrib/enable_webschleuder.rb test-schleuder@fripost.org

A test server (for testing/debugging/development purposes only) can be started with

  sudo -u schleuder camping webschleuder/webschleuder.rb

***** Configuring nginx + thin

  sudo apt-get install nginx thin
  sudo mkdir /var/run/webschleuder/
  sudo chown 'schleuder:schleuder' /var/run/webschleuder/

  :: /etc/thin/webschleuder.yml
    ---
    rackup: config.ru
    pid: /var/run/webschleuder/pid
    timeout: 30
    log: /var/log/schleuder/webschleuder.log
    max_conns: 1024
    require: []

    max_persistent_conns: 512
    environment: production
    user: schleuder
    group: schleuder
    servers: 1
    daemonize: true
    chdir: /opt/webschleuder
    socket: /var/run/webschleuder/socket

  :: /etc/nginx/sites-enabled/lists
    # Webschleuder
    location ^~ /schleuder/ {
        gzip off;
        proxy_set_header X-Real-IP  $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_redirect   off;
        proxy_pass http://unix:/var/run/webschleuder/socket.0;
    }
    location ^~ /static/ {
        alias /opt/webschleuder/static/;
    }

For the web interface to be available, one needs to start *both* thin and nginx:
  sudo /etc/init.d/thin start
  sudo /etc/init.d/nginx start

*** Creating new lists

The task is somewhat more tricky than for domains / aliases / mailbox creation as
the list manager has to be asked to create the list *on disk*.

The solution we chose is to create a special address 'mklist@fripost.org', for which each
email will be piped by Postfix into a list creation script run with the right user
(list or schleuder, depending on the extension: mklist+mailman@fripost.org or
mklist+schleuder@fripost.org.)

Of course, we don't want to give everyone the permission to create lists, hence the script:
 1/ ensures the email is signed with a known *and authorized* GPG key,
 2/ ensures the list that is to be created is in the "pending" state,
 3/ asks the list manager to create the list (for schleuder, also create the web.conf),
 4/ updates and hashes the lookup tables, and
 5/ adds the list commands to the LDAP directory, and removes the "pending" state.

The script can be found in the "fripost-panel" repository. See INSTALL and README for
further insformations.

** Logging
*** Overview
We want to limit how much we log for privacy reasons. At the same time we want
to be able to debug technical problems and detect intrusions.

For the webmail, we only log messages of priority warn or higher.
*** Configuration

  :: /etc/rsyslog.conf

    *.*;auth,authpriv.none;mail.err	-/var/log/syslog

# NOTE: /var/log/mail.{err,warn} can be kept at the default
# values since they do not contain any sensitive information.
  :: /etc/logrotate.d/rsyslog

    /var/log/mail.log
    /var/log/mail.info
    {
    	rotate 3
    	daily
    	missingok
    	ifempty
    	compress
    	delaycompress
    	sharedscripts
    	postrotate
    		invoke-rc.d rsyslog reload > /dev/null
    	endscript
    }
    /var/log/mail.warn
    /var/log/mail.err
    [...]

  :: /etc/logrotate.d/roundcube-core

    /var/log/roundcube/errors
    /var/log/roundcube/sendmail
    /var/log/roundcube/userlogins
    {
        create 0640 www-data adm
        compress
        missingok
        notifempty
        rotate 3
        daily
    }

  :: /etc/logrotate.d/apache2

    /var/log/apache2/*.log {
        daily
        missingok
        rotate 3
        compress
        delaycompress
        notifempty
        create 640 root adm
        sharedscripts
        postrotate
            /etc/init.d/apache2 reload > /dev/null
        endscript
    }

** Necessary stuff to fix for security
*** Bacula for backups
Also has tripwire-like capabilities.
*** OSSEC
*** Firewall rules
TODO: Add nice rules.
** Ideas for improved security
*** Monitoring

* Hardening
** Overview

The [[http://www.debian.org/doc/manuals/securing-debian-howto/][Securing Debian Manual]] is the definitive reference for Debian security.

These are just some quick notes for easy access to the administrators.
** ntp
# Let's be overly paranoid... ;-)

:: /etc/ntp.conf

-restrict -4 default kod notrap nomodify nopeer noquery
-restrict -6 default kod notrap nomodify nopeer noquery
+restrict default ignore
+restrict -6 default ignore

** rkhunter

sudo aptitude install rkhunter

sudo rkhunter -c --nomow --rwo

:: /etc/rkhunter.conf

    MAIL-ON-WARNING=admin@fripost.org

    ALLOWHIDDENDIR=/dev/.udev
    ALLOWHIDDENDIR=/dev/.initramfs
    ALLOWHIDDENDIR=/etc/.git

    ALLOWHIDDENFILE=/etc/.gitignore
    ALLOWHIDDENFILE=/etc/.etckeeper

    # something like: (adapt port as needed)
    INETD_ALLOWED_SVC=127.0.0.1:2000

    # in case whitelisting is needed, use something like:
    # (whitespace important)
    APP_WHITELIST=" openssl:0.9.8g sshd:4.7p1 "

:: /etc/default/rkhunter

    REPORT_EMAIL="admin@fripost.org"
    NICE="19"

# testing:

sudo rkhunter -c --nomow --rwo



* NEED TO KNOW FOR SERVER ADMINS

** Procedure for restarting mistral (the VPS)

1. There is one password which has to be provided at boot. This is given to our
   VPS host provider via some insecure means of communication.

2. When the server is booted, this password is changed.

3. The partition on /home/mail is then mounted. A separate password is provided
   for this.

4. Once the partition is mounted, dovecot and postfix may be started.

** Document your changes

The latest version of this document is always available at:

    git clone http://git.fripost.org/pub/fripost-docs.git

To get commit access, contact admin@fripost.org with your request.

** Use etckeeper

We keep /etc in a git repository using the tool etckeeper. This makes it
possible to use standard git commands in /etc, e.g. `git log'. `etckeeper' has
the benefit of keeping track of file permissions, which git by itself will not.

Every time you make changes to any files in /etc, you are encouraged to commit
them using a descriptive commit message.

$ etckeeper commit "postfix: relay messages to remote hosts via smtp"

If you do not commit your changes, they will be automatically committed.  This
is not ideal, since this means other administrators might have to guess as to
why changes were being made and by whom.  Please try to avoid putting your
co-administrators in this uncomfortable position.

** Use Cluster SSH

This pretty much sums it up:

"ClusterSSH controls a number of xterm windows via a single graphical console
window to allow commands to be interactively run on multiple servers over an ssh
connection."

** Use fripost-tools

We have written some tools to make administration tasks easier. They can be
found at:

    git clone http://git.fripost.org/pub/fripost-tools.git