Handling En Passant, Castling Rights, and the 50-Move Rule Correctly

I said the next post would be about search.

It isn't, and the reason is that two of the four bugs in the perft post were really the same bug wearing different hats. I didn't see it until I sat down to write about alpha-beta and realised I'd fixed the symptoms without ever writing down the disease.

Table of Contents

• The state that isn't on the board
• En passant
• Castling rights
• The 50-move rule
• The rules I stopped resenting
• Frequently asked questions

The State That Isn't On The Board

Here's the disease. En passant, castling rights and the halfmove clock are the only parts of a chess position that don't live on the board. Everything else is piece placement, and piece placement I'd had handled since the day I got bitboards working. These three are side-channel state. They get set by one move, consumed or destroyed by another, and they have to be restored exactly when you unmake.

Get any of it slightly wrong and your engine still plays chess. It just plays chess that's illegal about once every ten thousand positions.

Nobody's blog post skips these rules because they're hard to understand. They skip them because there's nothing interesting to say about the happy path, and all the difficulty is in edge cases you only ever meet when a node count disagrees with a published one.

En Passant

The rule is four sentences long. My implementation of it has been wrong in three separate ways.

A double push is not a legal capture

My first mental model was that the en passant square meant "an en passant capture is available." It doesn't. It means "a pawn moved two squares last ply, and if an enemy pawn happens to be sitting on the right square, it may capture there." Those are different claims, and conflating them is what leads to the Zobrist bug at the end of this section.

So the board stores the target square, the square the pushing pawn skipped over rather than the one it landed on, and nothing else:

// Set during makeMove, before the side to move flips.
if (movedPiece == PAWN && Math.abs(to - from) == 16) {
    epSquare = (from + to) >>> 1;      // the square jumped over
} else {
    epSquare = NO_SQUARE;              // 64, deliberately out of range
}

The (from + to) >>> 1 is just the midpoint of the two squares in 0–63 indexing, which for a two-rank push is exactly the skipped square. Cute, and it saved me a branch on colour.

Clear it every move. Every single move

That else branch is the whole post in miniature. My original version didn't have one. I set epSquare after a double push and cleared it inside the en passant capture handler, on the reasonable-sounding logic that the square gets used up when someone captures there.

Nobody captures there most of the time. The square just sat, stale, for the rest of the game.

The symptom was a phantom capture: a pawn arriving on the fifth rank two or three moves later could capture en passant onto a square whose pawn had long since moved on. Perft on position 3 came back high, and my first assumption was that capture generation was broken, when the generator was fine and the state it was reading from was a lie.

If you write down one thing from this post: the en passant square is set by exactly one kind of move and cleared by every other kind. It isn't an event flag. It's a property of the current position that happens to be null almost always.

The FEN field engines disagree about

Standard FEN says the fourth field holds the en passant target square whenever a double push just happened, whether or not a capture is possible. So after 1.e4 you get e3, even though Black has no pawn within reach of it.

Plenty of engines don't do that. They emit - unless an en passant capture is actually available to the side to move, because that's the version that makes position hashing correct. Both conventions are in the wild, and the practical consequence is that a FEN pasted from one engine into another may not round-trip.

For perft this matters more than it sounds like it should, because several standard test positions carry an en passant field. My parser takes whatever the FEN gives it and trusts it. My FEN writer uses the strict convention. That's inconsistent, I know it's inconsistent, and it hasn't bitten me yet.

Two pawns leave the rank at once

This is the one that cost me an afternoon, and it's worth restating properly, because in the perft post I fixed it without explaining why every normal legality check misses it.

Black king on a4, black pawn on c4, white rook on h4. All on rank 4. White plays d2–d4, and c4xd3 e.p. looks completely legal. The c4 pawn isn't pinned, because on its own it isn't blocking anything.

Except the capture removes two pawns from rank 4. The capturing pawn leaves c4, and the captured pawn is lifted from d4. Two squares open on one line, in one move. En passant is the only move in chess that does this, which is why pin detection can't see it. Pin detection asks whether this piece, on this square, is blocking a line to my king. Neither pawn is individually a blocker. Together they were.

Two ways out. The blunt one:

boolean isLegal(int move) {
    make(move);
    int ksq = Long.numberOfTrailingZeros(pieces[us][KING]);
    boolean ok = !isAttacked(ksq, them);
    unmake(move);
    return ok;
}

And the targeted one, which only costs anything on the rare move that needs it:

// Only for EP captures. Rebuild occupancy as it will be AFTER the capture,
// then ask whether a rook or queen now sees the king along the rank.
long afterEp = occupied
             ^ bit(from) ^ bit(capturedPawnSq) | bit(to);

long rankSliders = (pieces[them][ROOK] | pieces[them][QUEEN])
                 & RANK_MASK[rankOf(kingSq)];

if (rankSliders != 0 && (rookAttacks(kingSq, afterEp) & rankSliders) != 0) {
    return false;   // discovered check along the rank
}

The line doing the work is occupied ^ bit(from) ^ bit(capturedPawnSq) | bit(to): clear both departing pawns, set the arrival square, then query sliding attacks against that hypothetical occupancy using the magic lookups. You never mutate the real board. The rank filter up front means the attack lookup only runs when there's genuinely an enemy rook or queen on the king's rank, which is close to never.

I use the blunt version. It's slower and it's incapable of being subtly wrong, and I'd rather revisit that trade when a profiler tells me to than debug this class of bug twice.

Hash the file, and only when it matters

Zobrist keys need the en passant state, or two positions that differ only in capture availability collapse to the same key. The standard trick is to hash the file rather than the square, since the rank is implied by the side to move. Eight keys instead of sixty-four.

The part that took me embarrassingly long: hash it only when an en passant capture actually exists.

// After the EP square has been set for this position.
if (epSquare != NO_SQUARE
        && (pawnAttacks(them, epSquare) & pieces[us][PAWN]) != 0) {
    key ^= ZOBRIST_EP_FILE[fileOf(epSquare)];
}

pawnAttacks(them, epSquare) & pieces[us][PAWN] asks whether any of my pawns could reach the target square. It's the cheap way of testing "is there a pawn positioned to make this capture" without generating a single move.

Skip that guard and here's what happens. You play 1.e4 and hash the e-file. Later the same position recurs by transposition with no double push behind it, and the keys differ. Threefold repetition never fires. The engine walks into repetitions it should be avoiding and misses draws it should be claiming, and nothing anywhere reports an error, because a hash that's merely different is indistinguishable from a position that's merely new. It's the quietest bug in this whole post.

Castling Rights

Four bits, one array

Four booleans work. A 4-bit mask works better, and not for the memory:

static final int WK = 1, WQ = 2, BK = 4, BQ = 8;
int castlingRights;        // 0b1111 at startpos

Packing means the whole thing saves and restores as one int, XORs into the Zobrist key as one lookup on a 16-entry table, and, the real payoff, updates with a single &= instead of a nest of conditionals.

Rights are lost, never regained

There's no move in chess that restores a castling right. That sounds like a simplification and it's actually a trap, because it means you cannot recompute rights from the board. A position with rooks on a1 and h1 and a king on e1 tells you nothing about whether that king has already taken a walk and come home.

So rights go on the undo stack, alongside everything else that can't be derived:

record Undo(int move, int captured, int castlingRights,
            int epSquare, int halfmoveClock, long key) {}

private final Undo[] stack = new Undo[MAX_PLY];

The captured piece is in there for the same reason. Nothing in the resulting position tells you what used to be standing on the destination square.

The rook that got captured

Here was my rights update, and it's the version almost everyone writes first:

if (movedPiece == KING) rights &= ~(us == WHITE ? (WK | WQ) : (BK | BQ));
if (movedPiece == ROOK) rights &= ~rightForSquare(from);

Both lines are correct. Together they're incomplete, because they only ever consider the piece that moved, and a rook standing on h8 can lose Black the kingside right without moving at all. It gets captured. It never moves, it stops existing. Nothing above notices, and Black castles kingside afterwards with a rook that isn't there.

Kiwipete found this immediately. Depth 3 came back at 97,905 against an expected 97,862: high, meaning I was generating moves that don't exist.

The fix is a 64-entry table:

static final int[] CASTLE_MASK = new int[64];
static {
    Arrays.fill(CASTLE_MASK, 0b1111);
    CASTLE_MASK[E1] = ~(WK | WQ) & 0b1111;
    CASTLE_MASK[A1] = ~WQ & 0b1111;
    CASTLE_MASK[H1] = ~WK & 0b1111;
    CASTLE_MASK[E8] = ~(BK | BQ) & 0b1111;
    CASTLE_MASK[A8] = ~BQ & 0b1111;
    CASTLE_MASK[H8] = ~BK & 0b1111;
}

// In makeMove — one line, no branches, handles every case.
castlingRights &= CASTLE_MASK[from] & CASTLE_MASK[to];

Every square is 0b1111 except the six that matter. Applying it to from covers the king moving and the rook moving. Applying it to to covers a rook being captured on its home square, and for free it covers the case where the capturing piece was itself a rook leaving its own corner. One line, no conditionals, and it took me an hour of perft divides to earn.

The legality conditions, precisely

Getting rights right is half of it. The other half is the move itself, and this is where I overcorrected and pushed Kiwipete depth 4 down to 4,085,571 against 4,085,603. Low, meaning I'd started rejecting castles that are legal.

The conditions, and they are three genuinely different things:

1. The king is not currently in check.
2. Every square the king passes through and lands on is unattacked. Kingside that's e1, f1, g1; queenside it's e1, d1, c1.
3. Every square between the king and the rook is empty. Kingside f1 and g1; queenside b1, c1 and d1.

The trap is b1 and b8. They must be empty, because the rook slides across them. They don't have to be safe, because the king never sets foot there. I'd merged the two square sets into one constant out of tidiness, and quietly made queenside castling illegal whenever an enemy piece happened to eye b1.

static final long WQ_EMPTY = bit(B1) | bit(C1) | bit(D1);
static final int[] WQ_SAFE = { E1, D1, C1 };          // no B1

Two constants, because they answer two questions.

Zobrist handling is the boring correct thing. XOR the old rights key out before the update and the new one in after, rather than trying to XOR individual bits as they fall:

key ^= ZOBRIST_CASTLE[castlingRights];       // XOR out the old rights
castlingRights &= CASTLE_MASK[from] & CASTLE_MASK[to];
key ^= ZOBRIST_CASTLE[castlingRights];       // XOR in the new ones

All of the above assumes standard chess. Chess960 throws it out: the rook can start anywhere on the back rank, so rights are stored per-file, "between king and rook" stops being a fixed constant, and castling becomes a king-and-rook swap that can land the king on a square the rook currently occupies. My engine doesn't support it and I've made peace with that.

The 50-Move Rule

100 plies, not 50 moves

The halfmove clock increments on every ply and resets to zero on any pawn move or any capture. That's the entire mechanism:

if (movedPiece == PAWN || captured != NONE) {
    halfmoveClock = 0;
} else {
    halfmoveClock++;
}

A "move" in the rule's sense is White and Black each playing one, so 50 moves is 100 plies. I knew that and still wrote >= 50 the first time, because the field is called the halfmove clock and my fingers typed what the rule is called rather than what it counts. My engine declared draws at exactly half the right point, in endgames that were still perfectly winnable.

Perft did not catch this and never could. Perft counts legal moves, and the 50-move rule doesn't make any move illegal, it makes a position drawn. That's why this one survived long after the move generator was clean. It surfaced in self-play, when the engine kept agreeing to draws in positions it was winning.

Claimable, automatic, and what to actually do

FIDE makes the 50-move draw claimable: at 100 plies either player may claim it, and if nobody claims, play continues. At 75 moves, 150 plies, the arbiter ends it whether anyone asks or not.

An engine playing under a GUI should just treat 100 as a draw and score it accordingly. You're always willing to claim a draw you're entitled to, and if you're winning you'll have reset the clock long before you get there.

The exception matters, though. Checkmate delivered on the hundredth ply is checkmate. The draw only becomes claimable if the position isn't already over, so the draw test has to come after you know there's at least one legal reply:

if (halfmoveClock >= 100) {
    if (!inCheck(us) || hasLegalMove()) return DRAW_SCORE;
    // in check with no legal move — it's mate, fall through
}

The short-circuit is doing real work here. hasLegalMove() is expensive, and it only runs when we're in check in a position that has already hit the limit, which is somewhere between rare and never.

One rule helps the other

This is the part I actually enjoyed. A move that resets the halfmove clock is irreversible. Pawns don't march backwards and captured pieces don't come back. So no position from before a reset can ever repeat after it.

Which means repetition detection doesn't need to scan the whole game history. It needs to scan back halfmoveClock plies and stop:

boolean isRepetition() {
    int end = Math.max(0, ply - halfmoveClock);
    for (int i = ply - 4; i >= end; i -= 2) {   // same side to move only
        if (history[i] == key) return true;
    }
    return false;
}

Stepping by 2 skips the positions with the opposite side to move, which can't be repetitions of the current one. Starting at ply - 4 skips the two most recent, since the soonest a position can recur is four plies later. And ply - halfmoveClock is the hard floor, because everything older sits behind an irreversible move and provably can't match. In a middlegame with heavy trading the loop usually runs zero times.

Two rules that both looked like bookkeeping tax, and one of them turns out to bound the other.

Draw scores and contempt

In search, both draws return the same thing, and the interesting question is what that thing should be. Zero says a draw is exactly as good as an equal position. Most engines shade it, giving a draw a small negative score so the engine prefers keeping play alive over repeating against an opponent it thinks it can outplay. That offset is contempt, and it's usually a few centipawns. Mine is a small one I've never properly tuned, which I mention mainly so nobody takes it as a recommendation.

The Rules I Stopped Resenting

For a long time I thought of these three as chores, the paperwork you fill in around the interesting parts like sliding attacks and search. That was backwards. They're the only rules in chess whose state exists nowhere on the board, which makes them the only ones your make and unmake can silently corrupt, and the only ones where a bug survives contact with real games because it fires too rarely to notice.

Perft found two of them for me in an afternoon. It couldn't find the third, and that's the honest lesson here: perft is a complete test of legality and nothing more. Anything that changes what a position is worth rather than which moves are legal has to be tested somewhere else.

If you've hit one of these differently to me, especially if you've got a clean way to fold the en passant rank case into the pin mask rather than bolting it on afterwards, leave a comment. I'd like to stop making my legality check pay for the rarest move in chess.

Frequently Asked Questions

Why must the en passant square be cleared on every move?

Because it's a property of the position, not an event flag. One kind of move sets it and every other kind has to clear it. Clearing it only inside the capture handler leaves it stale, because most of the time nobody captures there, and a pawn arriving on the fifth rank several moves later will happily capture onto a square whose pawn is long gone.

Why does en passant expose the king when neither pawn is pinned?

It's the only move that takes two pieces off the same rank at once. If both pawns were together shielding the king from a rook or queen along that rank, the capture opens the line. Pin detection asks about one piece on one square, and neither pawn was a blocker on its own, so it never sees it. Make the move and test the king square, or rebuild the post-capture occupancy and run a targeted rank query.

Should I hash the en passant file when no capture is possible?

No. Guard it on an enemy pawn actually being in position to capture. Hash it unconditionally and the same position reached by two different routes gets two different keys, threefold repetition silently stops firing, and nothing tells you, because a different hash looks exactly like a new position.

Why does capturing a rook change castling rights?

Because the rook loses the right without moving. It gets taken on its home square and simply stops existing, so update code that only inspects the moving piece misses it entirely. A 64-entry mask array applied as rights &= mask[from] & mask[to] handles king moves, rook moves and rook captures in one branchless line.

Does b1 have to be unattacked for queenside castling?

Empty, yes. Safe, no. The rook slides across b1 so it has to be vacant, but the king travels e1 to c1 and never lands there, so only e1, d1 and c1 need to be unattacked. Merge those two square sets and you'll reject legal castles, which shows up in perft as a count that's too low.

Is the 50-move rule 50 or 100?

100 plies, since a move is both sides playing one. Test against 50 and your engine calls draws at half the correct point. The FIDE rule is claimable rather than automatic at that count and automatic at 75 moves, but an engine under a GUI should just score 100 plies as a draw, with the exception that mate on the hundredth ply is still mate.

Next time, actually search. Minimax, alpha-beta, and the first attempt at pruning that made the engine slower than counting the nodes had been.