Skip to main content

Aggressive pf(4) configuration for SSH protection

Tim Chase

If some unknown person came up to your house and started trying to open each window, jiggling each door handle, and punching in random codes on your garage-door opener, you could safely assume that they sought trouble and that you should stop them. Similarly, if some unknown computer on the internet started probing at ports on your server where you offer no services, it should set off alarm-bells and you should prevent this miscreant from interacting with your server. Especially SSH access. You know what services your machine provides. If anybody attempts to connect to some other service on your machine, in all likelihood they don't have your best interests in mind.

In this post, we configure pf on OpenBSD to catch inappropriate connection-attempts and block the offender.

  1. Assumptions
  2. Setup
  3. Goals
  4. Move the SSH port
    1. Change the listening port
    2. Change the default connecting port
  5. Creating a minefield
  6. Rate-limiting connections
  7. Limiting by country
    1. Download GeoIP data
    2. Automating GeoIP downloading
    3. Create a GeoIP-specific user
      1. Modify permissions on GeoIP data directory
      2. Allow access to restart pf
      3. Create a script to download & install GeoIP info
      4. Scheduling the download
  8. Final file
  9. Conclusion

Assumptions

This post assumes a few things about you:

OS installed that can run pf
While I wrote this article using pf on OpenBSD, many of the aspects should translate to FreeBSD even though FreeBSD's version of pf represents an older snapshot from when their codebases diverged. Particularly,
  • I think the abstract egress interface-group may not exist so you'd have to specify the interface directly (either inline or as a macro)
  • Where I use doas, you either have to install sudo or doas or else you must su - to become root for many of these commands.
  • in the _geoip script, you would need to replace the use of ftp with either fetch, wget, or curl
Administrative rights
This should go without saying, but if you don't administer the machine, you can't modify the firewall configuration.
Basic CLI
In this case, we have minimal work to do at the command-line but it helps to know your way around. OpenBSD has a template /etc/examples/pf.conf you might want to copy as a starting-point; you might find it easier to cd into /etc directory; or you might use tmux and ssh to access the server. Note that editing /etc/pf.conf can leave your server inaccessible (like cutting off the tree-branch while sitting on it), so you'll want to make sure that you have console access — locally, via a serial connection/KVM, or VNC connection.
Basic networking knowledge
The basic networking ideas of UDP, TCP, ports, services, etc. If you need to get up to speed here you might want a resource like Michael W. Lucas's Networking for System Administrators, O'Reilly's TCP/IP Network Administration, or some other such book.
Edit text-files
This focuses primarily on modifying your /etc/pf.conf which you can edit with your favorite $EDITOR, whether nano, emacs/mg, vi/vim/neovim, ed, cat, a magnetized needle and a steady hand, or butterflies. But this guide assumes that you can edit text files and save them.

Setup

For purposes here, I'll use OpenBSD's doas command to elevate privileges but you can use sudo if you have it installed on FreeBSD or execute these commands under a su - to root if you like to live dangerously. This assumes that doas (or sudo) has a configuration that lets you run the commands in question.

This also assumes starting with an empty (or mostly-empty if you did doas cp /etc/examples/pf.conf /etc/ ) pf.conf file. If you already have an existing configuration, you'll need to work out how these changes integrate with your existing setup.

Goals

This post covers several tricks to protect the actual SSH port.

  1. change port on which SSH listens
  2. flag any IP addresses that touch our server's minefield and prevent them from accessing SSH
  3. if anybody actually finds the SSH port and hammers on it, ban them too
  4. limit the country/countries from which we can SSH

Change the sshd port number

For purposes of this post, I configure sshd to listen on port 2345 but you should choose a different one that suits you. This doesn't give much in the way of security, but it reduces some of the noise in your log-files and makes the minefield work by making it unknown and surrounded by trip-ports.

Change the listening port

Edit your /etc/ssh/sshd_config file and specify the Port by adding/editing the line to read 

Port 2345
/etc/ssh/sshd_config
Save the file and restart sshd 
doas rcctl restart sshd
Note: If you make this configuration change while SSHed into the server, you might cut off your access. As mentioned, make sure that you have console, serial, or VNC access in case you need to recover from making an error here.

Change the default connecting port

On my local machine, I find it painful to remember how to specify the port to various ssh commands. Some take -p $PORTNUM or -P $PORTNUM, while others take -o Port=$PORTNUM and yet others expect it as an environment variable. To simplify this, edit (or create) your ~/.ssh/config to specify the port-number there 

host example
    HostName example.com
    Port 2345
~/.ssh/config
Now ssh, scp, sftp, rsync, etc, should all default to the right port-number without you needing to specify it explicitly.

Creating a minefield

Here we configure a minefield of ports such that if anybody tries to connect on any of these fake ports, they get added to a table of troublemakers. For the minefield, I default to 1024:9999 but if you want a broader range, check the output of 

sysctl net.inet.ip.port{first,last}
which gives a larger swath. If you extend it below 1024, make sure you don't interfere with other services that you might actually want to provide. 

##########
# macros #
##########
minefield="1024:9999"
ssh_alternate_port=2345

##########
# tables #
##########
table <troublemakers> persist

#########
# rules #
#########
block quick proto tcp from <troublemakers> \
  to (egress) port $ssh_alternate_port

## these are just randomly probing

# port 22 is a dead-giveaway
# because we know we moved our SSH server elsewhere
# Also, we have no telnet or SMB
# so anyone poking there
# is up to no good
pass in on egress proto tcp \
  from any \
  to (egress) port { \
    telnet, \
    ssh, \
    netbios-ns, \
    netbios-ssn, \
    microsoft-ds \
  } \
  synproxy state \
  tag trouble

# tag stuff in the range $minefield as trouble
pass in on egress proto tcp from any to (egress) port $minefield \
  synproxy state \
  tag trouble

# unless it's to our hole-in-one
pass in proto tcp from any to (egress) port $ssh_alternate_port tag good

# if we're still trouble, add to the troublemakers
pass proto tcp from any to (egress) port $ssh_alternate_port \
  tagged trouble \
  synproxy state \
  (max-src-conn 1, max-src-conn-rate 1/10, \
  overload <troublemakers> flush global \
  )
/etc/pf.conf

We only specify TCP because malicious actors could spoof UDP source-addresses, blocking innocent folks.

Feel free to add other services to the list of services you don't provide (I have telnet, SSH, and SMB in my list) if you notice strangers knocking elsewhere.

We tag packets as "trouble", and then tag the one port we know as "good". If a packet still has the "trouble" tag, we use some trickery (involving max-src-conn & max-src-conn-rate to limit them to one connection and to one attempt every 10 seconds) to add them to the troublemakers table, banning all future SSH access for them. The flush global keyword kills any other existing connections this offending visitor currently has.

We also need the synproxy state which allows pf to handle the connection to ports that don't actually have services backing them. (Aside: I think that all ports that don't have a real service behind them need the synproxy state but it might turn out that only the overload item needs it.)

With all that in place, any attempt to connect to a port in our minefield (or select services we know we don't run) results in the remote IP address getting added to the troublemakers table and subsequently blocked from our SSH port.

Rate-limiting connections

If an attacker does manage to guess our actual port 2345 on the first try, they will likely try to hammer on it. So we want to rate-limit them: 

##########
# Tables #
##########
table <bruteforce> persist

#########
# Rules #
#########
block quick proto tcp \
  from <bruteforce> \
  to (egress) port $ssh_alternate_port

# these made it through to the SSH port but abusing it
pass proto tcp from any to (egress) \
  port {$ssh_alternate_port} \
  flags S/SA keep state \
  (max-src-conn 5, max-src-conn-rate 5/5, \
  overload <bruteforce> flush global \
  )
/etc/pf.conf
This limits to 5 concurrent connections (max-src-conn 5) and 5 connection-attempts during a 10-second window (max-src-conn-rate 5/10).

Limiting by country

To begin with, I know that I will almost certainly never connect to SSH from outside the United States. IPDeny.com provides regularly-updated lists of IPv4 & IPv6 ranges for each country that I can use as a rough determination of country-of-origin. Yes, this can get thrown off by VPNs but it cuts off a surprising number of attackers.

Download GeoIP data

First, create the folder where pf will find the GeoIP tables. Inside that, create a tmp/ directory so we can download there, compare the results with what we already have, and move the new data atop the old data atomically: 

DEST="/usr/local/share/geoip"
doas mkdir -p "$DEST/tmp"
doas chmod 755 "$DEST"
doas chmod 750 "$DEST/tmp"

For now, we will manually download the IPv4 and IPv6 GeoIP blocks: 

DEST="/usr/local/share/geoip"
V4URL="https://www.ipdeny.com/ipblocks/data/aggregated/us-aggregated.zone"
V6URL="https://www.ipdeny.com/ipv6/ipaddresses/aggregated/us-aggregated.zone"
V4DEST="$DEST/ipv4_us-aggregated.zone"
V6DEST="$DEST/ipv6_us-aggregated.zone"
ftp -o "$V4DEST" "$V4URL"
ftp -o "$V6DEST" "$V6URL"
Now with these files, we can point pf at them and prohibit access from non-US visitors: 
##########
# Tables #
##########
table <domesticv4> \
  persist file "/usr/local/share/geoip/ipv4_us-aggregated.zone"
table <domesticv6> \
  persist file "/usr/local/share/geoip/ipv6_us-aggregated.zone"

#########
# Rules #
#########
# non-domestic IPv4/IPv6 connections simply not allowed to touch SSH
block quick inet proto tcp \
  from ! <domesticv4> \
  to (egress) port $ssh_alternate_port
block quick inet6 proto tcp \
  from ! <domesticv6> \
  to (egress) port $ssh_alternate_port
/etc/pf.conf
With this in place, any visitors from a non-US IP address simply cannot access the alternate SSH port.

Automating GeoIP downloading

Occasionally the GeoIP ranges change so you may want to download them automatically and have pf pick them up automatically. For this we'll create a stand-alone _geoip user, modify the permissions of the GeoIP data directory, set up a script to automatically download GeoIP data, compare it to existing data, and (if it changed) replace the old data and restart pf.

Create a _geoip user

Here we use the adduser command to create a new user. I put this user in the "daemon" login-class, specify an empty password, and then disable password logins: 

doas adduser _geoip
Use option ``-silent'' if you don't want to see all warnings and questions.

Reading /etc/shells
Check /etc/master.passwd
Check /etc/group

Ok, let's go.
Don't worry about mistakes. There will be a chance later to correct any input.
Enter username []: _geoip
Enter full name []: GeoIP downloader
Enter shell csh git-shell ksh nologin sh [ksh]:
Uid [1002]:
Login group _geoip [_geoip]:
Login group is ``_geoip''. Invite _geoip into other groups: guest no 
[no]:
Login class authpf bgpd daemon default pbuild staff unbound
[default]: daemon
Enter password []:
Disable password logins for the user? (y/n) [n]: y

Name:  _geoip
Password:    ****
Fullname:    GeoIP downloader
Uid:   1002
Gid:   1002 (_geoip)
Groups:      _geoip
Login Class: daemon
HOME:  /home/_geoip
Shell:       /bin/ksh

You could specify the shell as nologin but then have to do some work-arounds to get a shell to create the script and set up the cron job. After configuring the setup below, you can go back and use chsh to change the shell to nologin for added lockdown.

Modify permissions on GeoIP data directory

Now we need to make sure that the _geoip user can write to the directories 

doas chown -R _geoip:_geoip "$DEST"

Allow the _geoip user access to restart pf

In order for the _geoip user to instruct pf to reload its configuration and pick up the new GeoIP files, we need to add a line in our /etc/doas.conf file: 

permit nopass _geoip cmd /sbin/pfctl args -f /etc/pf.conf
/etc/doas.conf
This allows the _geoip user to run only pfctl as specified by full path, and only with the arguments -f /etc/pf.conf to minimize things. If you want to lock it down even further you can limit the _geoip user to only reload those two tables with something like this (untested): 
permit nopass _geoip \
  cmd /sbin/pfctl \
  args -t domesticv4 -T replace -f "/usr/local/share/geoip/ipv4_us-aggregated.zone"
permit nopass _geoip \
  cmd /sbin/pfctl \
  args -t domesticv6 -T replace -f "/usr/local/share/geoip/ipv6_us-aggregated.zone"
/etc/doas.conf
If you choose to go this route, make sure that you update the script below to use 
/sbin/pfctl -t domesticv4 -T replace -f "/usr/local/share/geoip/ipv4_us-aggregated.zone"
/sbin/pfctl -t domesticv6 -T replace -f "/usr/local/share/geoip/ipv6_us-aggregated.zone"
~_geoip/bin/update_zone.sh
instead.

Create a script to download & install GeoIP info

I switch to the _geoip user and create the script to use 

doas su - _geoip
mkdir -p ~/bin
touch ~/bin/update_zone.sh
chmod +x ~/bin/update_zone.sh
I then used my $EDITOR to populate that ~/bin/update_zone.sh file with the following script: 
#!/bin/sh
FTPOPTS="-MV"
DEST="/usr/local/share/geoip"
V4URL="https://www.ipdeny.com/ipblocks/data/aggregated/us-aggregated.zone"
V6URL="https://www.ipdeny.com/ipv6/ipaddresses/aggregated/us-aggregated.zone"
V4TMP="$DEST/tmp/ipv4_us-aggregated.zone"
V4DEST="$DEST/ipv4_us-aggregated.zone"
V6TMP="$DEST/tmp/ipv6_us-aggregated.zone"
V6DEST="$DEST/ipv6_us-aggregated.zone"

CHANGED=false
# fetch them to the temp location
# and if successful and they differ,
# replace the original
ftp $FTPOPTS -o $V4TMP "$V4URL" && ! cmp -s "$V4TMP" "$V4DEST" && mv "$V4TMP" "$V4DEST" && CHANGED=true
ftp $FTPOPTS -o $V6TMP "$V6URL" && ! cmp -s "$V6TMP" "$V6DEST" && mv "$V6TMP" "$V6DEST" && CHANGED=true

# if it changed, reload pf.conf
$CHANGED && (echo "Loading new domestic zones for SSH blocking" ; doas /sbin/pfctl -f /etc/pf.conf )
~_geoip/bin/update_zone.sh" id="script

Scheduling the download

On OpenBSD 6.7 and later cron allows the use of the ~ operator to indicate a randomly chosen time to help distribute the running of tasks without everything happening at one time. So after issuing su - _geoip to become the _geoip user, I used 

crontab -e
to edit the crontab and add an entry to run the script: 
# min, hr, day-of-mon, mon, day-of-wk
~ 1~3 * * 2 $HOME/bin/install_us_zone.sh
crontab -e
This picks some random time between 1:00am and 3:59am on Tuesday ("2") to run our script. If you use another cron or an older version on OpenBSD, choose an appropriate time. The IPDeny terms-and-conditions offer very liberal usage limits so you could download them more frequently. However, I find that the data doesn't change too frequently, so weekly works for me.

Final file

Putting it all together my final file looks something like this (including the initial entries from /etc/examples/pf.conf) 

##########
# Macros #
##########
# see also `sysctl net.inet.ip.port{first,last}`
minefield="1024:9999"
ssh_alternate_port=2345


##########
# Tables #
##########
table <bruteforce> persist
table <troublemakers> persist
table <domesticv4> persist \
  file "/usr/local/share/geoip/ipv4_us-aggregated.zone"
table <domesticv6> persist \
  file "/usr/local/share/geoip/ipv6_us-aggregated.zone"


###########
# Options #
###########
set skip on lo


#########
# Rules #
#########
block return    # block stateless traffic
pass      # establish keep-state

# block brute-forcers
block quick proto tcp from <bruteforce> \
  to (egress) port $ssh_alternate_port
block quick proto tcp from <troublemakers> \
  to (egress) port $ssh_alternate_port

# non-domestic v4/v6 connections simply not allowed to touch SSH
block quick inet proto tcp \
  from ! <domesticv4> \
  to (egress) port $ssh_alternate_port
block quick inet6 proto tcp \
  from ! <domesticv6> \
  to (egress) port $ssh_alternate_port

# Port build user does not need network
block return out log proto {tcp udp} user _pbuild

# these made it through to the SSH port but abusing it
pass proto tcp from any to (egress) \
  port {$ssh_alternate_port} \
  flags S/SA keep state \
  (max-src-conn 5, max-src-conn-rate 5/5, \
  overload <bruteforce> flush global \
  )

## these are just randomly probing

# port 22 is a dead-giveaway
# because we know we moved our SSH server elsewhere
# Also, we have no telnet or SMB
# so anyone poking there
# is up to no good
pass in on egress proto tcp \
  from any \
  to (egress) port { \
    telnet, \
    ssh, \
    netbios-ns, \
    netbios-ssn, \
    microsoft-ds \
  } \
  synproxy state \
  tag trouble

# tag stuff in the range $minefield as trouble
pass in on egress proto tcp from any to (egress) port $minefield \
  synproxy state \
  tag trouble

# unless it's to our hole-in-one
pass in proto tcp from any to (egress) port $ssh_alternate_port tag good

# if we're still trouble, add to the troublemakers
pass proto tcp from any to (egress) port $ssh_alternate_port \
  tagged trouble \
  synproxy state \
  (max-src-conn 1, max-src-conn-rate 1/10, \
  overload <troublemakers> flush global \
  )

pass proto tcp from any to any port {http https} \
  keep state (max-src-conn 10, max-src-conn-rate 10/3)
/etc/pf.conf

Conclusion

With all this in place, the SSH port now has protection from non-domestic IP addresses, folks probing around randomly to ports they have no business connecting to, and folks who actually find the port but bang against it too hard. From here, one might also implement fail2ban, blacklistd, or sshguard to monitor your sshd logs, watching for failed logins and then banning based on repeated login failures.