Git Bisect and Debugging History¶
Overview¶
Production worked in v1.4.0 but fails in v1.5.0 — which of 200 commits broke it? Manual search is impractical. git bisect performs binary search through history, halving the candidate set with each test. Combined with automated test scripts, bisect finds regression commits in logarithmic time.
This is Tutorial 15 in Module 5: Recovery & Debugging of the REBASH Academy Git series.
Prerequisites¶
Learning Objectives¶
By the end of this tutorial, you will be able to:
- Start and run manual git bisect sessions
- Automate bisect with test scripts (
git bisect run) - Mark commits good/bad/skip appropriately
- Bisect across merge commits and tags
- Visualize bisect progress and results
- Integrate bisect into CI debugging workflows
- Reset cleanly after bisect with
git bisect reset
Bisect Process Diagram¶
flowchart TB
START["Start bisect<br/>bad=HEAD good=v1.4.0"]
MID[Checkout middle commit]
TEST{Test passes?}
GOOD["Mark good<br/>search upper half"]
BAD["Mark bad<br/>search lower half"]
FOUND[First bad commit found]
START --> MID --> TEST
TEST -->|yes| GOOD --> MID
TEST -->|no| BAD --> MID
GOOD --> FOUND
BAD --> FOUND```
## Theory
### Binary Search Concept
Given known **good** commit (works) and **bad** commit (broken), bisect checks out midpoint and asks: good or bad? Each answer eliminates half the remaining commits.
For N commits, ~log₂(N) steps — 1024 commits need ~10 tests.
### Starting Bisect
```bash
git bisect start
git bisect bad # current HEAD is broken
git bisect good v1.4.0 # tag known good Git checks out middle commit. You test manually:
Repeat until first bad commit identified.
Automated Bisect¶
Script exit codes:
- 0 — good
- 1–124, 126–127 — bad
- 125 — skip (untestable commit)
Automation enables overnight bisect on large histories.
Skip Commits¶
Non-buildable commits (missing dependency, broken CI):
Too many skips may fail to find exact commit — bisect warns.
Bisect and Merges¶
Bisect walks first-parent chain by default on merge-heavy repos. Use --first-parent explicitly when appropriate:
Finding Good/Bad Boundaries¶
| Reference | Example |
|---|---|
| Tag | v1.4.0 |
| Commit SHA | abc1234 |
| Relative | HEAD~50 |
| Branch tip | origin/main |
Document good/bad boundaries in incident tickets.
After Bisect¶
Always reset — bisect leaves you in detached HEAD.
DevOps Use Cases¶
- Terraform plan regression — which commit changed module source?
- Docker build failure — which Dockerfile layer change broke build?
- Flaky test introduction — bisect with probabilistic test (harder)
- Performance regression — script comparing response time threshold
git bisect log and replay¶
Share bisect session with teammates.
Limitations¶
- Requires reproducible good/bad test
- Shallow clones may lack commits — unshallow first
- Flaky tests produce wrong results
- Submodule repos need extra care
Hands-on Lab¶
Step 1 – Create history with intentional bug¶
Command:
mkdir -p /tmp/git-bisect-lab && cd /tmp/git-bisect-lab
git init -b main
good_test() { test "$(cat counter.txt 2>/dev/null)" -lt 5; }
for i in 1 2 3 4 5 6 7 8; do
echo "$i" > counter.txt
git add counter.txt
if [ "$i" -le 4 ]; then
git commit -m "feat: increment to $i (good)"
else
git commit -m "feat: increment to $i (bad region)"
fi
done
git log --oneline
Explanation: Commits 5-8 are "bad" when threshold is < 5.
Step 2 – Manual bisect¶
Command:
git bisect start
git bisect bad HEAD
git bisect good HEAD~7
# Test current counter
VAL=$(cat counter.txt)
if [ "$VAL" -lt 5 ]; then git bisect good; else git bisect bad; fi
# Repeat until found — or use run script below
git bisect reset
Step 3 – Automated bisect¶
Command:
cat > test.sh << 'EOF'
#!/usr/bin/env bash
val=$(cat counter.txt)
test "$val" -lt 5
EOF
chmod +x test.sh
git bisect start HEAD HEAD~7
git bisect run ./test.sh
git bisect reset
Expected: First bad commit is when counter reaches 5.
Step 4 – Inspect culprit commit¶
Command:
Step 5 – Clean up¶
Command:
Commands & Code¶
| Command | Description | Example |
|---|---|---|
git bisect start | Begin session | git bisect start bad good |
git bisect good | Mark current good | After successful test |
git bisect bad | Mark current bad | After failed test |
git bisect skip | Skip untestable | Broken build commit |
git bisect run script | Automated bisect | git bisect run ./test.sh |
git bisect reset | End and restore | Always run when done |
git bisect log | Export session | Share with team |
Terraform plan bisect test¶
#!/usr/bin/env bash
# tf-plan-bisect.sh — exit 0 if plan succeeds without errors
set -euo pipefail
terraform init -backend=false -input=false >/dev/null 2>&1
terraform validate
terraform plan -detailed-exitcode -input=false >/dev/null 2>&1
rc=$?
# 0 = no changes (good), 2 = changes (good), 1 = error (bad)
test "$rc" -ne 1
Common Mistakes¶
Forgetting git bisect reset
Leaves repo in detached HEAD — confusing subsequent work.
Wrong good/bad boundaries
If good is actually bad, bisect returns wrong commit. Verify boundaries first.
Bisect with flaky tests
Random failures mislead binary search. Stabilize test or run multiple times.
Shallow clone missing commits
git fetch --unshallow before bisect across long history.
Best Practices¶
Automate with git bisect run
Manual bisect error-prone on >20 steps. Script the test.
Use tags for release boundaries
git bisect good v1.4.0 clearer than SHA in incident docs.
Document bisect result in postmortem
Link first-bad commit to fix PR and root cause.
Add bisect helper to Makefile
make bisect-good-release standardizes test command for team.
Troubleshooting¶
| Issue | Cause | Solution |
|---|---|---|
| Bisect cannot find commit | Too many skips | Reduce skips; widen good/bad range |
| Wrong commit identified | Flaky or wrong test | Fix test; re-run bisect |
fatal: Needed a single revision | Invalid ref | Verify tag/SHA exists |
| Bisect stuck in loop | Inconsistent good/bad | Reset; verify test logic |
| Script always exits 125 | Misconfigured skip | Fix script exit codes |
| No commits to bisect | Good and bad adjacent | Already found; inspect good..bad |
Summary¶
- git bisect binary-searches history between known good and bad commits
- Manual mode: mark good/bad after each test; automated: git bisect run
- Exit code 125 skips untestable commits; 0 good, non-zero bad for scripts
- Always git bisect reset when finished
- Essential for regression hunting in IaC, builds, and integration tests
Interview Questions¶
- How does git bisect work algorithmically?
- What exit codes should a bisect test script use?
- When would you use git bisect skip?
- How many steps to bisect 1000 commits approximately?
- What is git bisect run?
- Why must you run git bisect reset after finishing?
- What are limitations of bisect with flaky tests?
- How do tags help bisect in release debugging?
- How does shallow clone affect bisect?
- Give a DevOps scenario where bisect saves significant time.
Sample Answers (Questions 1 and 10)
Q1 — Algorithm: Bisect maintains a range of commits between known good and bad boundaries. It checks out the midpoint commit. If test passes, the bad commit must be later — shrink range to upper half. If fails, bad is in lower half. Repeat until the first bad commit is isolated. Complexity O(log n).
Q10 — DevOps scenario: After weekly deploy, Terraform plan suddenly wants to destroy production RDS. Good state is last week's tag v2.3.0; bad is current main. Bisect with terraform plan exit code script finds the exact commit that changed module version or removed lifecycle block — hours of manual log review reduced to ~10 automated tests.
Related Tutorials¶
- Cherry-pick and Reflog (previous)
- Git Hooks and Automation (next — Module 6)
- Viewing History and Diffs
- Undoing Changes — Reset, Revert, and Stash
- Git – Category Overview