Deploying over FTP: from copying files by hand to releases with rollback

The developer edits code locally and uploads the changed files with an FTP client straight into the directory the site runs from. Then they sign in to the server over SSH and, as needed, run composer install and migrations by hand. Small fixes are sometimes made directly on the server.

The FTP credentials are saved in the FTP client on two computers. FTP sends them across the network unencrypted.

In the middle of the season, the connection dropped while a larger change was being uploaded. The server was left with half the new files and half the old ones. The booking page ended in a fatal PHP error until the developer finished the upload. The Nginx log showed that every booking attempt in that window was affected.

A few days later a second fault surfaced: one file had been forgotten in the upload. The error appeared on only one path through the booking flow, and a customer was the one to find it.

Going back to the previous version was not possible. Nobody knew exactly what was on the server, because some fixes had been made there and were not in Git.

Deployment was not atomic. Files were replaced one by one while the site served requests, so during the upload, and after it broke off, a mix of two versions was running.

There was no single built and tested release. The server got whatever the developer selected, without tests and without any check that everything was there.

The server had drifted from the repository, so the repository was not a reliable source of truth and there was nothing to go back to.

Alongside that, a security problem: FTP sends the password unencrypted, and the FTP client keeps it on disk. Whoever obtains it can change the code of the live site.

The server's contents were pulled into a separate branch. git diff showed files edited directly on the server that were missing from Git. They were merged into the main branch so nothing was lost.

The Nginx and PHP-FPM logs showed how long the outage lasted and which paths it hit.

The FTP server configuration confirmed that it did not require an encrypted connection.

Everything is in Git, and the server changes only through a deployment. FTP is off and its password changed.

GitHub Actions on every merge to the main branch: composer install --no-dev --optimize-autoloader, the front end build, php artisan test and packaging of the finished release. If the tests fail, nothing is deployed.

The package reaches the server over SSH, with a key that belongs to CI alone and a dedicated user with rights only to the application directory. Each release is unpacked into its own directory and the site switches to it in a single step.

#!/usr/bin/env bash
set -euo pipefail

APP=/srv/booking
REL="$APP/releases/$(date -u +%Y%m%d%H%M%S)"

mkdir -p "$REL"
tar -xzf /tmp/release.tar.gz -C "$REL"      # built and tested in CI
ln -s "$APP/shared/.env" "$REL/.env"
rm -rf "$REL/storage" && ln -s "$APP/shared/storage" "$REL/storage"

cd "$REL"
php artisan migrate --force
php artisan config:cache
php artisan route:cache

# atomic switch: rename(2) replaces the link in one step, ln -sfn does not
ln -s "$REL" "$APP/current.tmp"
mv -T "$APP/current.tmp" "$APP/current"

sudo systemctl reload php8.3-fpm            # drops OPcache entries of the old release
curl -fsS --retry 3 https://rezervacie.example.sk/up > /dev/null

# keep the last five releases for rollback
ls -1d "$APP"/releases/* | head -n -5 | xargs -r rm -rf

Nginx has to resolve the symlink on every request. Otherwise PHP would keep using the old release's paths.

root /srv/booking/current/public;

location ~ \.php$ {
    include fastcgi_params;
    # resolve the symlink per request, so PHP never mixes two releases
    fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
    fastcgi_param DOCUMENT_ROOT $realpath_root;
    fastcgi_pass unix:/run/php/php8.3-fpm.sock;
}

Rollback is the same final step with the previous directory, plus a PHP-FPM reload: seconds. The deploying user may run only that one reload through sudo, nothing else.

Migrations are written so that the previous version of the code still works with the new schema: columns are added first and old ones removed only in a later deployment. Otherwise rolling back the code would not help.

On the same filesystem, mv -T calls rename, which replaces the current link in one step. Every request sees either the whole old release or the whole new one, never a mix. ln -sfn would delete the link and then create a new one, leaving a brief moment without it.

$realpath_root in Nginx and the PHP-FPM reload make sure that after the switch PHP reads the new release's files and OPcache does not hold on to the old paths.

The release is built and tested in CI, so what reaches the server is always a whole that passed the tests, not a selection of files.

Old releases stay on disk, so rollback builds and downloads nothing. It just switches the link.

An SSH key belonging to CI alone replaces an FTP password that travelled the network unencrypted.

What it does not address: switching the link does not roll back a database migration. That is why migrations are written to be backward compatible.