How do I properly git stash/pop in pre-commit hooks to get a clean working tree for tests?

There is—but let's get there in a slightly roundabout fashion. (Also, see warning below: there's a bug in the stash code which I thought was very rare, but apparently more people are running into.)

git stash save (the default action for git stash) makes a commit that has at least two parents (see this answer to a more basic question about stashes). The stash commit is the work-tree state, and the second parent commit stash^2 is the index-state at the time of the stash.

After the stash is made (and assuming no -p option), the script—git stash is a shell script—uses git reset --hard to clean out the changes.

When you use --keep-index, the script does not change the saved stash in any way. Instead, after the git reset --hard operation, the script uses an extra git read-tree --reset -u to wipe out the work-directory changes, replacing them with the "index" part of the stash.

In other words, it's almost like doing:

git reset --hard stash^2

except that git reset would also move the branch—not at all what you want, hence the read-tree method instead.

This is where your code comes back in. You now # Run tests on the contents of the index commit.

Assuming all goes well, I presume you want to get the index back into the state it had when you did the git stash, and get the work-tree back into its state as well.

With git stash apply or git stash pop, the way to do that is to use --index (not --keep-index, that's just for stash-creation time, to tell the stash script "whack on the work directory").

Just using --index will still fail though, because --keep-index re-applied the index changes to the work directory. So you must first get rid of all of those changes ... and to do that, you simply need to (re)run git reset --hard, just like the stash script itself did earlier. (Probably you also want -q.)

So, this gives as the last # Restore changes step:

# Restore changes
git reset --hard -q
git stash pop --index -q

(I'd separate them out as:

git stash apply --index -q && git stash drop -q

myself, just for clarity, but the pop will do the same thing).

As noted in a comment below, the final git stash pop --index -q complains a bit (or, worse, restores an old stash) if the initial git stash save step finds no changes to save. You should therefore protect the "restore" step with a test to see if the "save" step actually stashed anything.

The initial git stash --keep-index -q simply exits quietly (with status 0) when it does nothing, so we need to handle two cases: no stash exists either before or after the save; and, some stash existed before the save, and the save did nothing so the old existing stash is still the top of the stash stack.

I think the simplest method is to use git rev-parse to find out what refs/stash names, if anything. So we should have the script read something more like this:

#! /bin/sh
# script to run tests on what is to be committed

# First, stash index and work dir, keeping only the
# to-be-committed changes in the working directory.
old_stash=$(git rev-parse -q --verify refs/stash)
git stash save -q --keep-index
new_stash=$(git rev-parse -q --verify refs/stash)

# If there were no changes (e.g., `--amend` or `--allow-empty`)
# then nothing was stashed, and we should skip everything,
# including the tests themselves.  (Presumably the tests passed
# on the previous commit, so there is no need to re-run them.)
if [ "$old_stash" = "$new_stash" ]; then
    echo "pre-commit script: no changes to test"
    sleep 1 # XXX hack, editor may erase message
    exit 0
fi

# Run tests
status=...

# Restore changes
git reset --hard -q && git stash apply --index -q && git stash drop -q

# Exit with status from test-run: nonzero prevents commit
exit $status

warning: small bug in git stash

There's a minor bug in the way git stash writes its "stash bag". The index-state stash is correct, but suppose you do something like this:

cp foo.txt /tmp/save                    # save original version
sed -i '' -e '1s/^/inserted/' foo.txt   # insert a change
git add foo.txt                         # record it in the index
cp /tmp/save foo.txt                    # then undo the change

When you run git stash save after this, the index-commit (refs/stash^2) has the inserted text in foo.txt. The work-tree commit (refs/stash) should have the version of foo.txt without the extra inserted stuff. If you look at it, though, you'll see it has the wrong (index-modified) version.

The script above uses --keep-index to get the working tree set up as the index was, which is all perfectly fine and does the right thing for running the tests. After running the tests, it uses git reset --hard to go back to the HEAD commit state (which is still perfectly fine) ... and then it uses git stash apply --index to restore the index (which works) and the work directory.

This is where it goes wrong. The index is (correctly) restored from the stash index commit, but the work-directory is restored from the stash work-directory commit. This work-directory commit has the version of foo.txt that's in the index. In other words, that last step—cp /tmp/save foo.txt—that undid the change, has been un-un-done!

(The bug in the stash script occurs because the script compares the work-tree state against the HEAD commit in order to compute the set of files to record in the special temporary index before making the special work-dir commit part of the stash-bag. Since foo.txt is unchanged with respect to HEAD, it fails to git add it to the special temporary index. The special work-tree commit is then made with the index-commit's version of foo.txt. The fix is very simple but no one has put it into official git [yet?].

Not that I want to encourage people to modify their versions of git, but here's the fix.)