An AWS access key in the Git history

The application runs on a VPS at another provider, so it cannot be given an IAM role the way a server inside AWS can, and it signs in to S3 with the long-lived access key of the IAM user app-uploads. That user had the managed policy AmazonS3FullAccess, meaning full access to every bucket in the account, including the one holding database backups.

While the repository was being handed to a new external developer, the full history was scanned with gitleaks (gitleaks git -v .). It found an AWS access key in an .env file committed by mistake more than a year earlier and deleted by another commit two days later. Deleting it from the file did not remove it from history. Anyone with a clone had it on disk.

The key was still active and production was using it.

Who could have it: current and former developers, two freelancers, their laptops and any service with access to the repository. With it one could read, overwrite and delete customer files and database backups.

Assumption: the repository was never public. Had it been, one would have to assume a stranger already has the key, because public repositories are scanned by bots looking for exactly this.

The secret was in a file that was not in .gitignore, and nothing checked commits before they were pushed. Git is designed to keep history. Deleting a file in a later commit removes nothing from the earlier ones.

A second cause widened the impact: the application needed to upload and read attachments in one bucket, but the key had permissions over all of S3. Leaking one key therefore also meant access to the backups.

The IAM console shows, for each key, when, in which region and for which service it was last used. The last use matched the production application.

CloudTrail keeps management events, such as listing buckets or changing permissions, for 90 days without any setup, and its event history can be filtered by access key. There were no calls from IP addresses other than the production server.

Reads and writes of individual S3 objects are data events, which CloudTrail does not record by default, and they were not enabled. It therefore cannot be shown that nobody downloaded files. The incident record says exactly that, not that no leak occurred.

1. Rotation without downtime: create a new key, deploy it to the server, confirm the application works, deactivate the old key and delete it after a few days without errors. A deactivated key can be switched back on if something breaks; a deleted one cannot.

2. The new key belongs to a user whose policy allows only what the application does: upload and read an object under one prefix of one bucket. It has no access to the backups at all.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:PutObject", "s3:GetObject"],
      "Resource": "arn:aws:s3:::rezervacie-prilohy/uploads/*"
    }
  ]
}

3. Secrets out of the repository: .env is in .gitignore and only an .env.example with empty values stays in the repository. On the server the file is readable only by the user the application runs as. Deployment secrets are in GitHub Actions secrets.

4. Checks before and after push: gitleaks runs as a pre-commit hook for developers and again in CI on every pull request, where it cannot be skipped by disabling the hook. The old, already invalid key is listed by its finding fingerprint in .gitleaksignore, or the history scan in CI would fail forever.

5. History: after rotation the key in the history is invalid and therefore harmless. Rewriting history (with git filter-repo, for example) is possible, but everyone would have to re-clone, and copies that already exist would not be affected. The company left the history alone and relies on rotation.

Alternatives considered: running in AWS, the application would need no long-lived key at all and would receive short-lived credentials through an IAM role. Outside AWS there is IAM Roles Anywhere, but it requires your own certificate authority, which was needlessly complex for this team. Blocking secrets at push time (push protection) for private repositories is a paid GitHub feature.

Rotation makes the leaked key worthless: AWS rejects a deactivated or deleted key wherever it is stored. That is why rotation is the main measure and rewriting history would be cosmetic.

The narrower policy limits the impact of any future leak. A key that may only s3:PutObject and s3:GetObject under one prefix cannot list buckets, delete anything or read the backups.

gitleaks looks for the patterns of known secret types in changes. The hook catches a mistake before it leaves the machine; CI catches the case where someone does not have the hook or bypasses it.

What it does not address: the key on the server is still long-lived, and whoever controls the server gets it. Detection is pattern based, so a secret in an unusual format can slip through.