Caddy is the relatively new kid on the block in web servers offering a simple configuration for common setups that would normally take multiple lines in nginx for the same configuration. I wanted to convert my personal VPS from nginx to caddy for all of my sites.

The Current Setup

• VPS with web sites running under a non-privileged user.
• Nginx listening on ports 80 and 433. Each site is defined in /etc/nginx/snippets/ and symlinked from /etc/nginx/sites-available/ to /etc/nginx/sites-enabed/.
• Certbot installed to handle SSL certificates through using webroot verification.
• All sites and apps live under /app/.
• Rails webapps - which includes this blog under two top-level domains and five under subdomains. The puma webservers for the webapps put sockets in /app/socks/ which nginx reverse proxies to.
• One static site under a different domain.
• An instance of The Lounge RSS client under a subdomain using a reverse_proxy.
• Two static sites under a subdomain that are behind basic auth and are not using SSL.

Install Caddy

Using the official installation docs, I copy/pasted the commands to install the stable release.

$ sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https curl
$ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
$ curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
$ sudo apt update
$ sudo apt install caddy

All this time, nginx is still running and taking up ports 80 and 443 which caddy also wants to use. So the systemd service included in the install will be in a failed state. This is fine for now.

$ systemctl status caddy
× caddy.service - Caddy
     Loaded: loaded (/usr/lib/systemd/system/caddy.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Fri 2024-09-27 22:13:22 UTC; 19s ago
       Docs: https://caddyserver.com/docs/
   Main PID: 4022 (code=exited, status=1/FAILURE)
     Status: "loading new config: http app module: start: listening on :80: listen tcp :80: bind: address already in u>
        CPU: 66ms

For now, I want to configure my apps then shutdown nginx and start caddy to let it take over.

Configuring Caddy

Caddy's configuration file (singular) lives at /etc/caddy/Caddyfile. The default file contains some example code which I removed as it's not needed.

Each site or webapp lives in a server block. The first part of the block (outside of the { } is the domain(s). It's optional to include the http/https. Caddy will assume you want to support SSL by default. Inside the block is where you define where your site lives, other settings, and matchers. Unlike nginx which could require tens of lines to configure everything about a site including SSL, custom headers, etc, Caddy makes a lot of assumptions and configuration for you.

Configuring Rails Apps

All six Rails apps are configured the same. The domains/subdomains used, location of the public directory, and the location of the socket is what changes.

Here's the complete block for configuring this blog:

t27duck.com, www.t27duck.com, t27duck.xyz, www.t27duck.xyz {
  root * /apps/t27duck/current/public
  encode zstd gzip
  file_server

  @static {
    file
    path *.ico *.css *.js *.gif *.webp *.avif *.jpg *.jpeg *.png *.svg *.woff *.woff2 *.map *.swf *.flv
  }
  header @static Cache-Control "public, max-age=5184000"

  @notStatic {
    not file
  }
  reverse_proxy @notStatic unix//apps/socks/puma.t27duck.sock
}

• The root directive points to the public root of the website. In this case, we're all requests are relative to the location of the public directory of the Rails app.
• The encode directive implements compression for what's returned from the request.
• The file_server directive allows for file system access to everything in the root of the site. This allows things such as CSS and JavaScript files which physically live in the public directory to be served.
• @static defines a matcher. For any static file requests (the "file" part) in which the path ends in any of the listed file extensions, a cache-control header is included in the response.
• @noStatic defines another matcher. In this case I'm defining "@notStatic" as anything that is not a physical file on disc.
• Lastly, reverse_proxy directs all requests that match "@notStatic" (meaning a request that does not match a file living in the public directory) to the socket for Rails to process.

The order of the declarations in the block does not matter. Caddy has a set precedence for each directive. For example, reverse_proxy takes precidence over file_server. This is why we have to instruct revese_proxy to match @not_file to allow physical files to be served via file_server and skip the proxy.

This block is repeated for each Rails app with applicable file paths and domains changed.

Configure The Lounge Installation

Per The Lounge's documentation, all that is needed for Caddy is to specify the subdomain I want and a single reverse_proxy directive pointing to the running installation.

irc.t27duck.xyz {
  reverse_proxy http://127.0.0.1:9000
}

Configure static sites

For the basic static sites, a root and file_server directory is all that is required.

otherdomain.com, www.otherdomain.com {
  root * /apps/staticsite1
  file_server
}

Configuring http-only + basic auth Static Sites

Lastly, we need to configure sites that I do not want SSL over and "lock" (because this is really not secure) them behind a basic auth prompt.

By default, caddy assumes you want to enforce SSL for your sites. For cases where you just want http access, you can include the leading "http" for your domain list.

Basic auth is handled with the basic_auth directive. Its block takes the username and the hashed password. A hashed password can be generated via the caddy hash-password command.

$ caddy hash-password -p mypassword

The output of the above command can be used in the basic_auth directive.

The rest of the server block is identical to the other static sites.

http://subdomain.t27duck.com {
  basic_auth {
    theusername $2a$14$6abcdefg....
  }
  root * /apps/staticsite2
  file_server
}

Configuring SSL

Caddy transparently handles SSL certificate configuration and renewals. It requires a proper A/AAA DNS setting for each domain. There's no need for certbot or configuring challenge-answer checks via a .well-known directory.

Bonus: Block php requests

There are times when bots and scanners hit my sites thinking it's possibly a Wordpress blog resulting in false positive exception alerts. The following matcher and declaration can send any request ending in .php back as a 404:

@blocked {
  path *.php
}
respond @blocked 404

Switching over to caddy

With the configuration done, nginx can be shut down (freeing up ports 80 and 443) and caddy can start up.

$ sudo systemctl stop nginx
$ sudo systemctl start caddy

Checking the status of caddy shows the service running as expected.

$ systemctl status caddy
● caddy.service - Caddy
     Loaded: loaded (/usr/lib/systemd/system/caddy.service; enabled; preset: enabled)
     Active: active (running) since Fri 2024-09-27 22:50:33 UTC; 11s ago
       Docs: https://caddyserver.com/docs/
   Main PID: 5461 (caddy)
      Tasks: 9 (limit: 9444)
     Memory: 14.5M (peak: 15.3M)
        CPU: 291ms
     CGroup: /system.slice/caddy.service
             └─5461 /usr/bin/caddy run --environ --config /etc/caddy/Caddyfile

Cleaning Up and Removing nginx.

To complete the change over, once all sites are verified to be working as expected, existing nginx configs can be backed up to the home directory. After that, all packages and files related to both nginx and certbot can be safely removed. Caddy's SSL management automatically created new certificates to serve requests.

$ tar -czf ~/nginxconf.tar.gz /etc/nginx/snippets /etc/nginx/sites-available /etc/nginx/.htpasswd
$ sudo apt-get purge nginx nginx-common
$ sudo apt-get purge certbot
$ sudo apt-get autoremove

The End

The end result of all this is I'm now running an new modern web server using 70-80% less configuration than before. This makes managing and adding additional sites quicker and easier over time.

Documentation Referenced and Other Helpful Links

Caddy Installation: https://caddyserver.com/docs/install#debian-ubuntu-raspbian
Sending static files to file_server and not reverse_proxy: https://caddy.community/t/v2-serve-rails-application-with-static-assets/9006
Auto HTTPS: https://caddyserver.com/docs/automatic-https
Basic auth: https://caddyserver.com/docs/caddyfile/directives/basic_auth
Blocking access based on request extension: https://caddy.community/t/how-to-block-file-and-directory-access-in-caddy-v2/8010
Rewrite rules: https://caddy.community/t/caddy-v2-help-with-rewrite-rule-using-value-contained-in-the-query-string/8524
Caddy and system tmp: https://caddy.community/t/reverse-proxy-wiht-unix-socket-not-working/23223