How to Scan GitHub Repository for Credentials? [8 Tools]

Friend,

As a fellow tech geek and data security enthusiast, I want to guide you on an important topic – how to scan your GitHub repositories for exposed credentials using some amazing tools.

GitHub hosts millions of repositories containing precious source code. But as human beings, we make mistakes. Developers can accidentally commit passwords, API keys, SSH keys and other secrets into GitHub repositories.

According to a recent survey by Red Hat, over 80% of developers have leaked credentials on GitHub in the past. Another report by GitGuardian found over 100,000 secrets leaked on public GitHub in 2021 alone!

Exposed secrets can lead to serious security incidents and data breaches. Attackers are actively scanning GitHub using automated tools, looking for secrets they can abuse.

So how do you secure your GitHub repositories? The answer lies in these powerful open source tools that can automatically scan your code for exposed secrets.

Secret Scanning by GitHub

GitHub recently launched their own secret scanning feature. This scans every commit and pull request in your public and private repositories for any secrets.

Once a secret is found, GitHub sends alerts to repo admins. They also notify the concerned third party service like AWS, Azure, Google Cloud. This allows fast remediation of the exposed credential.

The secret scanning feature is automatically enabled for all public repositories. For private repos, you need to explicitly enable it under security settings.

GitHub can detect secrets using pattern matching as well as entropy checks. Their patterns database is continuously updated and currently has 1400+ secret formats from platforms like AWS, Google Cloud, Azure, Slack, Stripe and many more.

The service is free for public repos. But for private repos, you need a GitHub Enterprise plan.

Overall, GitHub‘s secret scanning provides a baseline level of protection. But for more rigorous scanning, you need specialized tools.

Git Secrets

Git secrets is an open source tool from AWS Labs that scans git repositories for secrets.

It hooks into the git workflow. Before you commit any changes, git secrets scans the changes to detect any exposed secrets. If a blacklist pattern is matched, it blocks the commit.

At AWS, git secrets helped reduce credential leaks by 50% from thousands of repositories. It‘s a must have for any security conscious organization.

Git secrets relies on regex based pattern matching to detect secrets. The patterns database covers AWS keys, Slack tokens, Google OAuth keys, AWS SSH keys and more.

You can also add custom regex patterns tailored to your environment, like database passwords or internal API keys.

The tool also supports "allowlists" – regex patterns for false positive matches that should be excluded from scanning. This avoids false alarms when committing legitimate content.

Overall, git secrets provides rigorous scanning while being easy to integrate into developer workflows. I highly recommend using it to lock down your GitHub repositories.

Repo Supervisor

Repo supervisor is an interesting open source tool from Auth0 that focuses on secrets detection for GitHub repositories.

It scans repository code as well as committed files for exposed secrets. Repo supervisor also analyzes GitHub pull requests to detect secrets before they are merged.

The tool provides great integration with the GitHub ecosystem:

It can be installed as a GitHub action for automated scans.
You can also set it up as a webhook to scan on pull request events.
It checks out code files from the repo to scan contents.
The scan results are published as GitHub actions logs or pull request comments.

Repo supervisor relies on 900+ regex patterns to detect secrets like API keys, cloud service keys and internal credentials. The patterns are sourced from public databases like GitGuardian patterns.

Although repo supervisor focuses on GitHub integration, it can also be used for general secrets scanning via its command line interface.

Overall, repo supervisor brings seamless secrets detection to your GitHub repositories and PRs. The integration makes it super easy to set up for developers.

TruffleHog

TruffleHog is a veteran in the secrets detection space. This open source tool has been around since 2017.

TruffleHog scans git repositories to detect high entropy strings that could be secrets. It also employs regex checks and keyword searches.

The major value proposition of TruffleHog is its ability to deeply scan a repository‘s entire commit history. It can detect secrets that were committed earlier but deleted later on.

TruffleHog provides integrations for popular code hosts like GitHub and Bitbucket. You can run scans on entire organizations to hunt for exposed secrets.

Here are some powerful features:

Supports entropy based checks to detect random secret strings.
Searches commit logs and commit diffs to find hidden secrets.
Can dump scan findings in JSON format for integration with other tools.
Comes with hundreds of regex patterns for common secrets.
Integrates with GitHub Actions, CircleCI and other CI/CD pipelines.

TruffleHog is a great choice if you need to deeply analyze your repository history for any leaked secrets. Its entropy checks provide an extra layer of protection compared to just regex scanning.

Git Hound

Git Hound is a unique secrets detection tool that leverages GitHub‘s native code search capabilities.

It uses advanced pattern matching and entropy checks to detect secrets in GitHub code. But the special sauce is its use of GitHub code search API to hunt down publicly exposed secrets.

Code search allows GitHound to scan thousands of public GitHub repositories for secrets matching your patterns. It does not need direct access to repositories, unlike other tools.

GitHound also deeply scans commit histories and diffs to uncover deleted secrets. Special detectors are included for SSH keys and AWS console login links.

Here are some notable features:

Scans public GitHub via code search API for exposed secrets.
Deep scanning of repository histories using diff analysis.
Detects and removes false positives using ML and stats analysis.
Supports secrets matching via regex, keywords and high entropy.
Built-in detector for AWS login links.
Dumps findings in JSON format.

Overall, GitHound brings unique GitHub-centric scanning capabilities for exposed secrets. The code search integration makes it incredibly powerful for public GitHub audits.

Gitleaks

Gitleaks is another long-running open source secrets detection tool for git repositories.

It is designed for developer-friendly usage. Gitleaks can be installed as a pre-commit hook to trigger scanning before a commit. This workflow integration detects secrets early before they are persisted.

Gitleaks shines with its huge catalog of detection rules sourced from public databases. It contains 1400+ regex patterns to recognize secrets for various services like AWS, Databricks, GitHub, GitLab, Google Cloud and many more.

The tool also does recursive scans to detect secrets in the entire repository codebase, not just the changed files. Here are some notable features:

Easy to install pre-commit hook for early secrets detection.
Huge database of 1400+ regex patterns to detect secrets.
Scans entire repo recursively instead of just changed files.
Integrates with CI systems like GitHub Actions.
Supports whitelisting certain secrets via regex.

Gitleaks provides comprehensive scanning capabilities powered by its vast detection patterns database. The pre-commit integration provides a simple way to bake it into your development workflows.

Repo Security Scanner

The Repo Security Scanner is an open source CLI tool from UK Home Office for scanning secrets in git repositories.

It searches code files for exposed credentials based on pattern matching and entropy checks.

The tool recursively scans the entire repository codebase. It also deeply analyzes commit histories and commit diffs to uncover previously deleted secrets.

Some interesting features:

Recurses through code tree to find secrets instead of scanning commits.
Analyzes commit histories and diffs for exposed secrets.
Checks for high entropy strings as secret indicators.
Comes with 200+ regex patterns for common secrets.
Integrates with CI systems like Travis CI.
Supported on major git repository hosting services.
Can ignore certain file types or paths during scan.

Although not as feature rich as commercial tools, Repo Security Scanner provides a simple open source option for basic secrets hygiene. The entropy checks add additional detection capabilities beyond just regex matching.

Git Guardian

Finally, I want to discuss a managed solution – GitGuardian.

https://www.youtube.com/watch?v=FBvVQZoCqeA

While the previous tools require manual setup, GitGuardian provides secrets detection as a fully managed service.

It continuously monitor your public and private GitHub repositories (500+ supported) for any exposed credentials. This covers secrets present in code, commit messages, issues, pull requests and any other GitHub context.

Leak detection is based on regex pattern matching powered by their vast proprietary patterns database. The patterns are continuously updated to cover new secret formats.

Once a potential secret is found, you get alerts on email, Slack or MS Teams. This allows quick remediation. GitGuardian also helps you safely revoke the exposed secrets.

Notable features:

Continuous secrets monitoring for 500+ GitHub integrations.
Huge proprietary patterns database maintained by large team.
Alerts on email, Slack and Microsoft Teams.
Automatic secret revocation process.
Scans code, commits, PRs, issues, comments.
Support for public repos, private repos and enterprise installations.
Freemium plan available with 5000 scans/month.

GitGuardian provides automated secrets monitoring for your GitHub landscape via a SaaS platform. While the open source tools covered earlier provide more customization, GitGuardian offers convenience.

So in summary, I have equipped you with a great arsenal of secrets detection tools for securing your GitHub repositories. I hope you found this guide useful! Do let me know if you have any other questions.

Stay safe out there and keep hacking 🙂