Skip to content

Latest commit

 

History

History
705 lines (553 loc) · 25.5 KB

INSTALL.md

File metadata and controls

705 lines (553 loc) · 25.5 KB

caesonia (beta)

OpenBSD Email Service - Installation Guide

Preface

While there are many ways to install, on different hosts, this guide will focus on Kernel-based Virtual Machine (KVM), a popular offer for Virtual Private Server (VPS).

Hosting

When searching for a hosting company, some useful keywords are "KVM" and "VPS", with "unmanaged" or "self-managed" type of service.

Popular forums:

Minimum system requirements:

  • 512MB RAM
  • 20GB SSD/HDD

The host must be able to mount a recent OpenBSD ISO image. Some hosting companies do not advertise OpenBSD in their offering, but will mount cdXX.iso or installXX.iso when asked. It may be a good idea to open a ticket with them, and verify if mounting a custom ISO is supported.

n.b. now is a good time to make sure the prerequisites are met. The substitutions below depend on DNS records, otherwise customize.

OpenBSD Installation

A response file is used to provide answers to the installation questions, as well as an autopartitioning template for disklabel. Edit and upload these files to a web server, or use default as presented.

At the (I)nstall, (U)pgrade, (S)hell prompt, pick "shell"

Next, aquire an IP address:

dhclient vio0

Download and edit the response file:

cd /tmp && ftp https://raw.githubusercontent.com/vedetta-com/caesonia/master/src/var/www/htdocs/mercury.example.com/install.conf

Install OpenBSD:

install -af /tmp/install.conf

After reboot, syspatch, and mail, configure doas.conf:

doas tmux

n.b.: if syspatch installed a kernel patch:

shutdown -r now

Verify if egress IP matches DNS record:

ping -vc1 \
	$(dig +short $(hostname | sed "s/$(hostname -s).//") mx | \
	awk -vhostname="$(hostname)" '{if ($2 == hostname".") print $2;}')

ping6 -vc1 \
	$(dig +short $(hostname | sed "s/$(hostname -s).//") mx | \
	awk -vhostname="$(hostname)" '{if ($2 == hostname".") print $2;}')

Update hostname.if and reset:

sh /etc/netstart $(ifconfig egress | while IFS=: read a b; do printf "%s" "$a" && break; done;)

Create Makefile.local to customize the email service, and continue with the steps from README.md for a guided installation and configuration.

Otherwise, please follow the instructions below on how to manually install and configure the email service.

OpenBSD Configuration

If the console entry in the ttys(5) file does not contain the "secure" flag, then init will require that the superuser password be entered before the system will start a single-user shell.

sed -i '/^console/s/ secure//' /etc/ttys

Install packages:

pkg_add dovecot dovecot-pigeonhole dkimproxy rspamd opensmtpd-extras gnupg-2.2.12

n.b.: dovecot package comes with instructions for self-signed certificates, which are not used in this guide:

pkg_info -M dovecot

n.b.: python 2.7 is used by devel/glib2. If desired, the package contains instructions to set as default system python:

pkg_info -M python

Services:

rcctl enable httpd dkimproxy_out rspamd dovecot
rcctl disable check_quotas sndiod

Dovecot Virtual Users are mapped to system user "vmail":

useradd -m -u 2000 -g =uid -c "Virtual Mail" -d /var/vmail -s /sbin/nologin vmail

With backup MX, Dovecot Replication needs a user to dsync:

useradd -m -u 2001 -g =uid -c "Dsync Replication" -d /home/dsync -s /bin/sh dsync

dsync SSH limited to one "command" restricted in doas.conf to match "dsync_remote_cmd":

su - dsync
ssh-keygen
echo "command=\"doas -u vmail \${SSH_ORIGINAL_COMMAND#*}\" $(cat ~/.ssh/id_rsa.pub)" | \
	ssh [email protected] "cat >> /home/dsync/.ssh/authorized_keys"
exit

Update /home/dsync, on primary and backup MX:

chown -R root:dsync /home/dsync
chmod 750 /home/dsync/.ssh
chmod 640 /home/dsync/.ssh/{authorized_keys,id_rsa.pub}
chmod 400 /home/dsync/.ssh/id_rsa
chown dsync /home/dsync/.ssh/id_rsa

Update /root/.ssh/known_hosts:

ssh -4 -i/home/dsync/.ssh/id_rsa -ldsync hermes.example.com "exit"
ssh -6 -i/home/dsync/.ssh/id_rsa -ldsync hermes.example.com "exit"

Email Service Configuration

Download a recent release:

cd ~ && ftp https://github.com/vedetta-com/caesonia/archive/v6.5.0-beta.tar.gz
tar -C ~ -xzf ~/v6.5.0-beta.tar.gz

n.b.: to use Git or SVN:

pkg_add git

Update default values in the local copy:

cd src/

Backup MX role may be enabled in etc/mail/smtpd.conf and depends on DNS record.

n.b.: Backup MX instructions may be skipped, if not applicable.

Update interface name:

grep -r vio0 .
find . -type f -exec sed -i "s|vio0|$(ifconfig egress | awk 'NR == 1{print $1;}' | sed 's/://')|g" {} +

Primary domain name (from example.com to domainname):

grep -r example.com .
find . -type f -exec sed -i "s|example.com|$(hostname | sed "s/$(hostname -s).//")|g" {} +

Virtual domain name (from example.net to example.org):

grep -r example.net .
find . -type f -exec sed -i 's|example.net|example.org|g' {} +

Primary's hostname (from mercury to hostname -s):

grep -r mercury .
find . -type f -exec sed -i "s|mercury|$(hostname -s)|g" {} +

Backup's hostname (from hermes to DNS record):

grep -r hermes .
find . -type f -exec sed -i "s|hermes|$(dig +short $(hostname | sed "s/$(hostname -s).//") mx | awk -vhostname="$(hostname)" '{if ($2 != hostname".") print $2;}')|g" {} +

Update the allowed mail relays source table src/etc/mail/relays to add the primary and backup MX IPs:

cd ../
echo "# primary's IP" > src/etc/mail/relays
dig +short mercury.example.com a >> src/etc/mail/relays
dig +short mercury.example.com aaaa >> src/etc/mail/relays
echo "# backup's IP" >> src/etc/mail/relays
dig +short hermes.example.com a >> src/etc/mail/relays
dig +short hermes.example.com aaaa >> src/etc/mail/relays

Update wheel user name "puffy":

sed -i "s|puffy|$USER|g" \
	src/etc/mail/aliases \
	src/etc/mail/passwd \
	src/etc/mail/virtual

n.b.: Without backup MX, remove configuration for user "dsync":

sed -i 's/dsync\ //g' src/etc/pf.conf

n.b.: Without backup MX, remove configuration for MTA-STS:

sed -i '/hermes/d' src/var/www/mta-sts/mta-sts.txt

n.b.: Select the "backup" dispatcher in smtpd.conf for Backup MX role: action "mda" # "backup"

Update virtual users credentials table src/etc/mail/passwd using smtpctl encrypt:

smtpctl encrypt
> secret
> $2b$...encrypted...passphrase...
vi src/etc/mail/passwd
> [email protected]:$2b$...encrypted...passphrase...::::::

n.b.: user quota limit can be overriden from src/etc/mail/passwd:

[email protected]:$2b$...encrypted...passphrase...::::::userdb_quota_rule=*:storage=7G

Review virtual domains aliasing table src/etc/mail/virtual.

Email Service Installation

After review:

install -o root -g wheel -m 0640 -b src/etc/acme-client.conf /etc/
install -o root -g wheel -m 0644 -b src/etc/daily.local /etc/
install -o root -g wheel -m 0644 -b src/etc/changelist.local /etc/
install -o root -g wheel -m 0640 -b src/etc/dhclient.conf /etc/
install -o root -g wheel -m 0640 -b src/etc/dkimproxy_out.conf /etc/
install -o root -g wheel -m 0640 -b src/etc/doas.conf /etc/
install -o root -g wheel -m 0640 -b src/etc/httpd.conf* /etc/
install -o root -g wheel -m 0644 -b src/etc/login.conf /etc/
install -o root -g wheel -m 0644 -b src/etc/newsyslog.conf /etc/
install -o root -g wheel -m 0600 -b src/etc/pf.conf* /etc/
install -o root -g wheel -m 0644 -b src/etc/sysctl.conf /etc/

install -o root -g wheel -m 0755 -d src/etc/dovecot/conf.d /etc/dovecot/conf.d
install -o root -g wheel -m 0644 -b src/etc/dovecot/local.conf /etc/dovecot/
install -o root -g wheel -m 0644 -b src/etc/dovecot/dovecot-trash.conf.ext /etc/dovecot/
install -o root -g wheel -m 0644 -b src/etc/dovecot/conf.d/* /etc/dovecot/conf.d/

install -o root -g wheel -m 0644 -b src/etc/mail/aliases /etc/mail/
install -o root -g _smtpd -m 0640 -b src/etc/mail/blacklist /etc/mail/
install -o root -g wheel -m 0644 -b src/etc/mail/mailname /etc/mail/
install -o _dovecot -g _smtpd -m 0640 -b src/etc/mail/passwd /etc/mail/
install -o root -g wheel -m 0644 -b src/etc/mail/relays /etc/mail/
install -o root -g wheel -m 0644 -b src/etc/mail/smtpd.conf /etc/mail/
install -o root -g wheel -m 0644 -b src/etc/mail/vdomains /etc/mail/
install -o root -g _smtpd -m 0640 -b src/etc/mail/virtual /etc/mail/
install -o root -g _smtpd -m 0640 -b src/etc/mail/whitelist /etc/mail/

install -o root -g wheel -m 0600 -b src/etc/mtree/special.local /etc/mtree/

install -o root -g wheel -m 0755 -d src/etc/rspamd/local.d /etc/rspamd/local.d
install -o root -g wheel -m 0755 -d src/etc/rspamd/override.d /etc/rspamd/override.d
install -o root -g wheel -m 0644 -b src/etc/rspamd/local.d/* /etc/rspamd/local.d/
install -o root -g wheel -m 0644 -b src/etc/rspamd/override.d/* /etc/rspamd/override.d/

mkdir -m 700 /var/crash/rspamd

install -o root -g wheel -m 0644 -b src/etc/ssh/sshd_banner /etc/ssh/
install -o root -g wheel -m 0644 -b src/etc/ssh/sshd_config /etc/ssh/

install -o root -g wheel -m 0644 -b src/root/.ssh/config /root/.ssh/

install -o root -g wheel -m 0755 -d src/etc/ssl/acme /etc/ssl/acme
install -o root -g wheel -m 0700 -d src/etc/ssl/acme/private /etc/ssl/acme/private

install -o root -g vmail -m 0550 -b src/usr/local/bin/dovecot-lda.sh /usr/local/bin/
install -o root -g vmail -m 0550 -b src/usr/local/bin/learn_*.sh /usr/local/bin/
install -o root -g bin -m 0500 -b src/usr/local/bin/rmchangelist.sh /usr/local/bin/
install -o root -g vmail -m 0550 -b src/usr/local/bin/quota-warning.sh /usr/local/bin/
install -o root -g vmail -m 0550 -b src/usr/local/bin/wks-server.sh /usr/local/bin/

install -o root -g crontab -m 0640 -b src/var/cron/cron.allow /var/cron/
crontab -u root src/var/cron/tabs/root

mkdir -p -m 750 /var/dovecot/{imapsieve,sieve,sieve-pipe}
chgrp vmail /var/dovecot/{imapsieve,sieve,sieve-pipe}

install -o root -g vmail -m 0750 -d src/var/dovecot/imapsieve/after /var/dovecot/imapsieve/after
install -o root -g vmail -m 0750 -d src/var/dovecot/imapsieve/before /var/dovecot/imapsieve/before
install -o root -g vmail -m 0640 -b src/var/dovecot/imapsieve/before/report-ham.sieve /var/dovecot/imapsieve/before/
install -o root -g vmail -m 0640 -b src/var/dovecot/imapsieve/before/report-spam.sieve /var/dovecot/imapsieve/before/

install -o root -g vmail -m 0750 -d src/var/dovecot/sieve/after /var/dovecot/sieve/after
install -o root -g vmail -m 0750 -d src/var/dovecot/sieve/before /var/dovecot/sieve/before
install -o root -g vmail -m 0640 -b src/var/dovecot/sieve/before/spamtest.sieve /var/dovecot/sieve/before/
install -o root -g vmail -m 0640 -b src/var/dovecot/sieve/before/00-wks.sieve /var/dovecot/sieve/before/

install -o root -g wheel -m 0644 -b src/var/unbound/etc/unbound.conf /var/unbound/etc/

install -o root -g daemon -m 0755 -d src/var/www/htdocs/mercury.example.com /var/www/htdocs/$(hostname)
install -o root -g daemon -m 0644 -b src/var/www/htdocs/mercury.example.com/index.html /var/www/htdocs/$(hostname)/

install -o root -g daemon -m 0755 -d src/var/www/htdocs/mercury.example.com/mail /var/www/htdocs/$(hostname)/mail
install -o root -g daemon -m 0644 -b src/var/www/htdocs/mercury.example.com/mail/config-v1.1.xml /var/www/htdocs/$(hostname)/mail/

install -o root -g daemon -m 0755 -d src/var/www/openpgpkey /var/www/openpgpkey
install -o root -g daemon -m 0644 -b src/var/www/openpgpkey/policy /var/www/openpgpkey/
install -o root -g daemon -m 0644 -b src/var/www/openpgpkey/submission-address /var/www/openpgpkey/

install -o vmail -g daemon -m 0755 -d src/var/www/openpgpkey/hu /var/www/openpgpkey/hu

install -o root -g daemon -m 0755 -d src/var/www/mta-sts
install -o root -g daemon -m 0644 -b src/var/www/mta-sts/mta-sts.txt

install -o root -g wheel -m 0755 -d src/var/lib /var/lib
install -o root -g wheel -m 0755 -d src/var/lib/gnupg /var/lib/gnupg
install -o root -g wheel -m 2750 -d src/var/lib/gnupg/wks /var/lib/gnupg/wks

DNS resolver

Unbound DNS validating resolver from root nameservers, with fallback:

rcctl enable unbound
rcctl restart unbound
cp -p /etc/resolv.conf /etc/resolv.conf.old
cp src/etc/resolv.conf /etc/

Sieve

Compile sieve scripts:

sievec /var/dovecot/imapsieve/before/report-ham.sieve
sievec /var/dovecot/imapsieve/before/report-spam.sieve
sievec /var/dovecot/sieve/before/00-wks.sieve
sievec /var/dovecot/sieve/before/spamtest.sieve

Replication

Master/Master replication, on primary and backup MX:

mv /etc/dovecot/conf.d/90-replication.conf.optional /etc/dovecot/conf.d/90-replication.conf

Backup local files

Install "changelist.local":

cp -p /etc/changelist /etc/changelist-6.5
cat /etc/changelist.local >> /etc/changelist

n.b.: Uninstall "changelist.local":

sed -i '/changelist.local/,$d' /etc/changelist

n.b.: Remove from "/var/backups":

/usr/local/bin/rmchangelist.sh

Let's Encrypt

Start httpd:

rcctl start httpd

Initialize a new account and domain key:

acme-client -vAD $(hostname)
rcctl reload httpd

OCSP response:

ocspcheck -vNo /etc/ssl/acme/$(hostname).ocsp.resp.der \
	/etc/ssl/acme/$(hostname).fullchain.pem
rcctl reload httpd

OpenPGP Web Key Service (WKS)

An important aspect of using OpenPGP is trusting the (public) key. Off-channel key exchange is not always practical, OpenPGP DANE protocol lacks confidentially, and HKPS' a mess. OpenPGP proposed a new protocol to automate and build trust in the process of exchanging public keys.

Web Key Service has two main functions for our Email Service:

  1. Allow all users to locate and retreive public keys by email address using HTTPS
  2. Allow local user's email client to automatically publish and revoke public keys

Self-hosting has the advantage of full authority on the user mail addresses for their domain name. By design, only one WKS can exist for a domain name. Furthermore, only local users can make requests to WKS Submission Address, and replies to local users only. Moreover, the service automatically verifies the sender is in possesion of the secret key, before publishing their public key. Self-hosting the public key server finally makes OpenPGP oportunistic encryption user friendly.

To get started, a GnuPG 2.1 safe configuration is provided: gpg.conf

n.b.: temp WKS installation patch for doas.conf

echo "permit nopass root as vmail cmd env" >> /etc/doas.conf

Web Key Service maintains a Web Key Directory (WKD) which needs the following configuration for each virtual domain:

mkdir -m 755 /var/lib/gnupg/wks/example.com
chown -R vmail:vmail /var/lib/gnupg/wks

cd /var/lib/gnupg/wks/example.com

ln -sf /var/www/openpgpkey/hu .
chown -h root:vmail hu

ln -s /var/www/openpgpkey/submission-address .
chown -h root:vmail submission-address

doas -u vmail \
	env -i HOME=/var/vmail \
	gpg-wks-server --list-domains

Web Key Service uses a Submission Address, which needs the following configuration:

Add virtual password for the Submission Address:

smtpctl encrypt
> secret
> $2b$...encrypted...passphrase...
vi /etc/mail/passwd
> [email protected]:$2b$...encrypted...passphrase...::::::

Create the submission key:

doas -u vmail /usr/local/bin/gpg2 --batch --passphrase "" \
	--quick-gen-key [email protected]

Verify:

doas -u vmail \
	env -i HOME=/var/vmail \
	gpg2 -K --with-fingerprint

List the z-Base-32 encoded SHA-1 hash of the mail address' local-part (i.e. key-submission):

doas -u vmail \
	env -i HOME=/var/vmail \
	gpg2 --with-wkd-hash -K [email protected]
> [email protected]

Publish the key, using the hash of the string "key-submission" (i.e. 54f6ry7x1qqtpor16txw5gdmdbbh6a73):

doas -u vmail /usr/local/bin/gpg2 \
	-o /var/lib/gnupg/wks/example.com/hu/54f6ry7x1qqtpor16txw5gdmdbbh6a73 \
		--export-options export-minimal --export [email protected]

n.b.: To delete this key:

gpg2 --delete-secret-key "[email protected]"
gpg2 --delete-key "[email protected]"

n.b.: (!) revert temp WKS installation patch for doas.conf

sed -i '/permit nopass root as vmail cmd env$/ d' /etc/doas.conf

n.b.: Enigmail/Thunderbird, Kmail and Mutt (perhaps other MUA) support the Web Key Service. Once published, a communication partner's MUA automatically downloads the public key with the following gpg.conf directive:

#expert
#no-emit-version
#interactive
auto-key-retrieve
auto-key-locate		local,wkd

The key can be manually retreived too:

gpg2 --auto-key-locate clear,wkd --locate-keys [email protected]

To simply check a key:

$(gpgconf --list-dirs libexecdir)/gpg-wks-client -v --check [email protected]

Or a hex listing:

gpg-connect-agent --dirmngr --hex 'wkd_get [email protected]' /bye

n.b: If the local-part of an email address exists alike for multiple domains (e.g. puffy@example.com and puffy@example.net), the hash of the (local-part) string is identical and each key publication overwrites the same web key. This is desired behavior when the same key is used for both uid. To have different keys, the workaround is using a subaddress (i.e. +tag) to create the uid (e.g. [email protected]) for the key, and go through the process of key submission and confirmation using the MUA interface with the tagged email address (e.g. [email protected]).

Following Bernhard's recommendation to support WKD implementations without SRV lookup (e.g. Mailvelope, Enigmail), the apex domain (i.e. example.com) must have A and AAAA records, and its http server must return codes 301 or 302 to send a Location: header for redirection to https://wkd.example.com:

server "example.com" {
	listen on "*" tls port https
...
	# OpenPGP Web Key Directory
	location "/.well-known/openpgpkey/*" {
		block return 302 "https://wkd.example.com$REQUEST_URI"
	}
...
}

n.b.: assuming DKIM keys are set.

Restart

Restart the email service:

pfctl -f /etc/pf.conf
rcctl restart sshd httpd dkimproxy_out rspamd dovecot smtpd

Logs

/var/log/messages
/var/log/daemon
/var/log/maillog
/var/log/rspamd/rspamd.log
/var/www/logs/access.log
/var/www/logs/error.log

Client Configuration

n.b.: MUA auto-configuration via Autoconfiguration and SRV Records for Locating Email Services

  • IMAP server: mercury.example.com (or hermes.example.com)

    • Security: TLS
    • Port: 993
    • Username: [email protected]
    • Password: ********
    • Autodetect IMAP namespace ☑️
    • Use compression ☑️
    • Poll when connecting for push ☑️
  • SMTP server: mercury.example.com (or hermes.example.com)

    • Security: TLS
    • Port: 465
    • Require sign-in ☑️
    • Username: [email protected]
    • Authentication: Normal password
    • Password: ********
  • SMTP server: mercury.example.com (or hermes.example.com)

    • Security: STARTTLS
    • Port: 587
    • Require sign-in ☑️
    • Username: [email protected]
    • Authentication: Normal password
    • Password: ********

Administration

Suppose the address "[email protected]" needs to be hosted, with a "johndoe" alias.

n.b.: Assuming DNS Prerequisites for Virtual Domains are met

Add virtual domain:

echo "example.ca" >> /etc/mail/vdomains

Add DKIM signature:

sed -i '/^domain/s/$/,example.ca/' /etc/dkimproxy_out.conf

Whitelist local sender:

echo "@example.ca" >> /etc/mail/whitelist

Add virtual alias:

echo "[email protected] \t\[email protected]" >> /etc/mail/virtual

Add virtual user:

echo "[email protected] \t\tvmail" >> /etc/mail/virtual

Add virtual password:

smtpctl encrypt
> secret
> $2b$...encrypted...passphrase...
vi /etc/mail/passwd
> [email protected]:$2b$...encrypted...passphrase...::::::

WiP:

  • Add to httpd.conf
  • Add to acme-client.conf
  • Add to WKD
  • Submit GPG key
  • Remove user

Reload:

rcctl restart dkimproxy_out
rcctl reload dovecot
smtpctl update table virtuals
smtpctl update table vdomains
smtpctl update table passwd
smtpctl update table whitelist

Suppose the foreign address "[email protected]" behaves badly.

Blacklist external sender:

echo "[email protected]" >> /etc/mail/blacklist
smtpctl update table blacklist

or blacklist everybody "@example.meh" for bad behavior:

echo "@example.meh" >> /etc/mail/blacklist
smtpctl update table blacklist

Suppose "example.meh" is a lost cause:

Gather relevant bad subdomains:

dig +short example.meh mx

for each bad subdomain, add its IP (A and AAAA record) to pf:

echo $IP >> /etc/pf.conf.table.ban

and reload the pf table:

pfctl -t ban -T replace -f /etc/pf.conf.table.ban

Microsoft Network

When sending emails to Microsoft network from a new IP, the following error 550 may occur:

Mar 22 17:56:37 mercury smtpd[45037]: 8f4864084ecc48f4 mta event=delivery evpid=7077717e797a776e from=<[email protected]> to=<[email protected]> rcpt=<-> source="203.0.113.1" relay="104.47.38.33 (104.47.38.33)" delay=1s result="PermFail" stat="550 5.7.1 Unfortunately, messages from [203.0.113.1] weren't sent. Please contact your Internet service provider since part of their network is on our block list (AS3150). You can also refer your provider to http://mail.live.com/mail/troubleshooting.aspx#errors. [BL2NAM02FT031.eop-nam02.prod.protection.outlook.com]"

To add a new IPv4 to Microsoft's reputation based greylist, manual intervention from postmasters is required:

  1. Delist IP from other blocking lists: http://multirbl.valli.org/

  2. Add IP to Microsoft's greylist:

  3. Check Inbox/ for an auto-reply email, followed by a response:

    • conditionally mitigated, meaning the IP has been added to the greylist
    • or the IP is not eligible for delisting, meaning
      1. Politely reply, asking the reason why the IP is not eligible for delisting
      2. A human will delist the IP in a few hours

Microsoft recommends postmasters to join Smart Network Data Service for monitoring the new IP's reputation score, and the associated Junk Mail Reporting Program (n.b. requires Microsoft account)

What's next

Add your own Sieve scripts in /var/vmail/example.com/puffy/sieve, then:

cd /var/vmail/example.com/puffy/
ln -s sieve/script.sieve .dovecot.sieve
sievec .dovecot.sieve

To disable filtering based on 3rd party blocking lists:

mv /etc/rspamd/local.d/rbl.conf.optional \
   /etc/rspamd/local.d/rbl.conf

sed -i '|enabled|s|^#||' /etc/rspamd/local.d/surbl.conf

mv /etc/rspamd/override.d/emails.conf.optional \
   /etc/rspamd/override.d/emails.conf
vi /etc/rspamd/override.d/bad_emails.list
rcctl reload rspamd