2022-09-10

Preventing git leaks

I track most of my code and prose via git and I store it on GitHub in public repos. I do it because I get reliable storage for free that I can access from any computer. And some of the repos might be helpful to others. Also it engages my hubris :-). It’s working nicely but sometimes I get a bad feeling when I push stuff. I’m worried that I might leak some sensitive information like passwords, API keys or tokens.

NOTE: don’t assume that committing secrets is ok when the repo is private. The files or their contents get copied and the platforms holding the repos get compromised.

Spotting secrets in new commits

The obvious solution is to think twice before committing and pushing data. But there are also some helpful tools, like gitleaks. It basically finds and reports secrets in the files you are about to commit. I want to run it whenever I commit something in any of my repos. These are the steps to make that happen:

  1. Install gitleaks.
  2. Add this to your ~/.gitconfig:
[core]
    hooksPath = ~/.git-global-hooks
  1. Create ~/.git-global-hooks/pre-commit:
#!/bin/bash
# Detect secrets in a git repo using https://github.com/zricethezav/gitleaks

if [[ $SKIP == "gitleaks" ]]; then
    echo "skipping gitleaks checks ..."
    exit 0
fi

set -xe

# Check uncommitted changes (parsing output of 'git diff') that had been 'git add'ed.
gitleaks protect --no-banner --staged

If, for some good reason, you want to skip gitleaks when committing: SKIP=gitleaks git commit -m "commit message"

You can also make gitleaks ignore a secret either by #gitleaks:allow inline comment next to the secret or by adding finding’s fingerprint to the .gitleaksignore file at the root of your repo. See docs for details.

Existing commits

The steps above will prevent you from committing secrets from now on. But you should also check existing commits because you might have committed a secret in the past. You can either do it on each commit by adding these lines to ~/.git-global-hooks/pre-commit:

# Check existing commits (parsing output of 'git log -p').
gitleaks detect --no-banner

But on bigger repos this might take several seconds every time you commit. To avoid this you can check all your historical commits in all your repos once. I used gh and runp to do it:

export GHORG=jreisinger # CHANGE ME
mkdir /tmp/$GHORG && cd /tmp/$GHORG
# clone all my (1000) repos in parallel
gh repo list $GHORG --source --limit 1000 | cut -f 1 | runp -p 'gh repo clone'
# check existing commits in all repos in parallel
ls | runp -p 'gitleaks detect --no-banner -s'

If runp exits with 0, all is good. Otherwise scroll up to review the output. To check a repo for committed leaks:

cd <repo>
gitleaks detect --no-banner -v

Gitleaks cheatsheet

# show secrets I'm about to commit (use --staged in pre-commit hook)
gitleaks protect -v

# show secrets that have been committed in the past
gitleaks detect -v

# show secrets no matter whether they are tracked by git
gitleaks detect --no-git -v
No comments: