Setup Postfix Mail Relay on Ubuntu VPS: Complete Guide

Understanding Mail Relay Configuration for VPS Email

Mail relay configuration on Ubuntu VPS allows your server to send emails through an external SMTP provider rather than directly to recipient mail servers. This approach improves email deliverability, reduces the chance of your IP being blacklisted, and provides better authentication options for business email.

Many hosting customers struggle with email delivery when sending directly from their VPS. Major email providers like Gmail, Outlook, and Yahoo often reject emails from unestablished IP addresses. A properly configured mail relay solves this problem by routing your outbound mail through trusted SMTP services.

This tutorial walks you through setting up Postfix mail relay on Ubuntu VPS, covering authentication, encryption, and testing procedures. You'll learn to configure both commercial SMTP services and internal relay scenarios.

Prerequisites and Initial System Setup

Before configuring Postfix mail relay, verify your Ubuntu VPS meets these requirements:

• Ubuntu 20.04 or later with sudo access
• Postfix installed (we'll cover installation if needed)
• Valid domain name with proper DNS records
• SMTP credentials from your chosen relay provider

Check if Postfix is already installed:

systemctl status postfix

If Postfix isn't installed, install it now:

sudo apt update
sudo apt install postfix mailutils

During installation, select "Internet Site" and enter your server's fully qualified domain name. We'll adjust these settings in the relay configuration.

Verify your hostname is properly configured:

hostname -f

The output should match your domain name. If not, update it using:

sudo hostnamectl set-hostname mail.yourdomain.com

Configuring Postfix for SMTP Relay Authentication

Now we'll configure Postfix to use your SMTP relay service. Start by backing up the existing configuration:

sudo cp /etc/postfix/main.cf /etc/postfix/main.cf.backup

Edit the main Postfix configuration file:

sudo nano /etc/postfix/main.cf

Add or modify these settings for relay configuration:

# Relay host configuration
relayhost = [smtp.example.com]:587

# Enable SASL authentication
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous

# TLS encryption settings
smtp_tls_security_level = encrypt
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache

Replace `smtp.example.com` with your actual SMTP server address. Common relay services use these settings:

• Gmail: smtp.gmail.com:587
• SendGrid: smtp.sendgrid.net:587
• Mailgun: smtp.mailgun.org:587
• Amazon SES: email-smtp.us-east-1.amazonaws.com:587

For enhanced security on managed VPS hosting, you can also configure these additional security options:

# Additional security settings
smtp_tls_note_starttls_offer = yes
smtp_tls_protocols = !SSLv2, !SSLv3
smtp_tls_mandatory_protocols = !SSLv2, !SSLv3

Creating SMTP Authentication Credentials File

Create the SASL password file containing your SMTP credentials:

sudo nano /etc/postfix/sasl_passwd

Add your relay server and credentials in this format:

[smtp.example.com]:587 username:password

For Gmail, use your full email address and app password. For other services, use the provided SMTP username and password.

Secure the credentials file:

sudo chmod 600 /etc/postfix/sasl_passwd
sudo chown root:root /etc/postfix/sasl_passwd

Generate the password database:

sudo postmap /etc/postfix/sasl_passwd

This creates `/etc/postfix/sasl_passwd.db` which Postfix will use for authentication.

Verify the database was created correctly:

ls -la /etc/postfix/sasl_passwd*

You should see both the original file and the `.db` file with restricted permissions.

Advanced Relay Configuration Options

For production environments, configure additional relay options to handle various scenarios:

sudo nano /etc/postfix/main.cf

Add these advanced settings:

# Sender-dependent relay routing
sender_dependent_relayhost_maps = hash:/etc/postfix/sender_relay

# Relay restrictions
smtp_sender_dependent_authentication = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd

# Fallback relay configuration
smtp_fallback_relay = [backup-smtp.example.com]:587

# Connection limits
default_destination_concurrency_limit = 10
smtp_destination_concurrency_limit = 5

Configure sender-dependent routing if you need different relay servers for different domains:

sudo nano /etc/postfix/sender_relay

Add routing rules:

@domain1.com [smtp.provider1.com]:587
@domain2.com [smtp.provider2.com]:587
root@yourdomain.com [smtp.gmail.com]:587

Generate the sender relay database:

sudo postmap /etc/postfix/sender_relay

This configuration is particularly useful for agencies managing multiple client domains on the same VPS.

Testing Mail Relay Configuration

Before putting the relay into production, test the configuration thoroughly:

sudo systemctl restart postfix
sudo systemctl status postfix

Check for any configuration errors in the logs:

sudo journalctl -u postfix -n 20

Test email sending using the mail command:

echo "Test email via relay" | mail -s "Test Subject" recipient@example.com

Monitor the mail log to verify relay authentication:

sudo tail -f /var/log/mail.log

Look for successful authentication messages like:

postfix/smtp[12345]: 4B2F4567890: SASL authentication succeeded

If authentication fails, common issues include:

• Incorrect credentials in sasl_passwd file
• App passwords not enabled for Gmail
• Firewall blocking outbound SMTP connections
• Incorrect relay host or port configuration

For detailed testing of email authentication, refer to our guide on Setup Postfix SPF Authentication on Ubuntu VPS: Complete Guide.

Monitoring and Troubleshooting Mail Relay Issues

Set up monitoring to track relay performance and catch issues early:

sudo nano /etc/logrotate.d/postfix-relay

Add log rotation configuration:

/var/log/postfix-relay.log {
    daily
    rotate 7
    compress
    delaycompress
    missingok
    notifempty
    postrotate
        /bin/kill -HUP `cat /var/run/rsyslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

Configure Postfix to log relay-specific information:

sudo nano /etc/postfix/main.cf

Add debugging options for troubleshooting:

# Debug logging for relay issues
smtp_tls_loglevel = 1
smtp_sasl_auth_soft_bounce = no
notify_classes = bounce, delay, policy, protocol, resource, software

Common troubleshooting commands:

# Check queue status
sudo postqueue -p

# Flush mail queue
sudo postfix flush

# Test configuration syntax
sudo postfix check

# View detailed connection logs
sudo grep "relay" /var/log/mail.log

For email queue management best practices, see our tutorial on Setup Postfix Email Queue Management on Ubuntu VPS: Complete Guide.

Security Hardening for Mail Relay Setup

Secure your mail relay configuration against common threats:

sudo nano /etc/postfix/main.cf

Add security restrictions:

# Prevent open relay abuse
smtpd_relay_restrictions = permit_mynetworks, reject_unauth_destination

# Rate limiting
smtpd_client_connection_count_limit = 10
smtpd_client_connection_rate_limit = 30

# Authentication required for submission
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot

Configure fail2ban to protect against brute force attacks:

sudo apt install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

Edit the fail2ban configuration:

sudo nano /etc/fail2ban/jail.local

Enable Postfix protection:

[postfix-sasl]
enabled = true
port = smtp,submission
logpath = /var/log/mail.log
maxretry = 3
bantime = 600

Restart fail2ban:

sudo systemctl restart fail2ban
sudo systemctl enable fail2ban

For comprehensive email security, also implement DMARC policies using our Setup Postfix DMARC Policy on Ubuntu VPS: Complete Email Authentication guide.

Frequently Asked Questions

How do I verify my mail relay is working correctly?

Send a test email and check `/var/log/mail.log` for successful SASL authentication and relay delivery. You should see entries showing connection to your relay server and successful message handoff. Use `postqueue -p` to check for stuck messages in the queue.

Can I use multiple SMTP relay providers simultaneously?

Yes, configure sender-dependent relay routing using `sender_dependent_relayhost_maps`. This allows different domains or users to route through different relay providers based on the sender address, useful for agencies managing multiple client accounts.

What happens if my relay provider is temporarily unavailable?

Configure `smtp_fallback_relay` to specify a backup relay server. Postfix will attempt the fallback relay if the primary relay is unreachable. You can also adjust `maximal_queue_lifetime` to control how long messages stay in queue during outages.

How do I troubleshoot SASL authentication failures?

Check that your credentials in `/etc/postfix/sasl_passwd` are correct and that you've run `postmap` after changes. For Gmail, ensure you're using an app password, not your regular password. Enable `smtp_tls_loglevel = 1` for detailed TLS negotiation logs.

Should I configure both submission and relay on the same server?

For production environments, it's recommended to separate user submission (port 587) from relay functionality. Configure submission with proper authentication while using relay for outbound delivery. This provides better security isolation and easier troubleshooting.