Completely Removing Secrets from Git History
What I Learned
Sensitive information (API keys, private keys, configuration files, etc.) that has been accidentally committed continues to remain in Git’s history even with normal deletion. By using git filter-repo, you can rewrite Git history and completely remove sensitive information. However, removing from history does not make already leaked credentials safe, so you must always revoke and reissue the relevant credentials.
Details
Background
Previously, methods using git filter-branch were introduced, but warnings are now issued in the official documentation, and when creating new procedures, the use of git filter-repo is recommended.
Tools Used
gitgit filter-repo
Precautions Before Work
- This procedure is a destructive operation that rewrites history
- When implementing in shared repositories, notify stakeholders in advance
- After work, request all users to re-clone
Case-by-Case Procedures
Case 1: Completely Remove a Specific File from History
1. Mirror Clone
1git clone --mirror REMOTE_URL REPO_NAME.git
2cd REPO_NAME.git
Mirror cloning retrieves all branches, tags, and references. To accurately reflect the results of rewriting history to the remote, this procedure uses mirror cloning.
2. Remove Sensitive Files from History
1git filter-repo \
2 --path SENSITIVE_PATH_1 \
3 --path SENSITIVE_PATH_2 \
4 --invert-paths
SENSITIVE_PATH_n: repository-relative path of files to remove from history (multiple can be specified)
3. Reflect to Remote
1git push --force --all
2git push --force --tags
Case 2: Remove/Replace Sensitive Strings (API Keys, etc.) from History
1. Mirror Clone
1git clone --mirror REMOTE_URL REPO_NAME.git
2cd REPO_NAME.git
Normal cloning doesn’t include some references, so use mirror cloning for history modification work.
2. Create Replacement Rules File
1cat > replace-rules.txt << 'EOF'
2regex:SECRET_PATTERN_1==>***REDACTED***
3regex:SECRET_PATTERN_2==>***REDACTED***
4EOF
Replacement rule format:
[search mode]:[pattern to search]==>[replacement string]
- Specify
regexetc. for search mode- When using regular expressions, verify carefully to avoid unintended replacements
3. Rewrite History
1git filter-repo --replace-text replace-rules.txt
4. Reflect to Remote
1git push --force --all
2git push --force --tags
Case 3: Process Multiple Sensitive Files and Strings Simultaneously
1git clone --mirror REMOTE_URL REPO_NAME.git
2cd REPO_NAME.git
3
4git filter-repo \
5 --path SENSITIVE_PATH_1 \
6 --path SENSITIVE_PATH_2 \
7 --invert-paths \
8 --replace-text replace-rules.txt
9
10git push --force --all
11git push --force --tags
What You Must Do After Work
- Notify stakeholders of the impact scope
- Have all local repositories re-cloned
- Revoke and reissue all credentials that may have been leaked
Important Points
- Always invalidate and reissue sensitive information even after removing it from history
git push --forcedestroys history, so notify the team beforehand- Don’t include sensitive information in repositories; manage them with
.envfiles etc. - Always add
.envand files containing secrets to.gitignore - Enable GitHub Secret Scanning for early detection of sensitive information