Living reference
The words,
pinned down.
Short, canonical meanings for terms the course uses repeatedly. When ordinary ML usage is ambiguous, this project’s meaning wins.
Data and evaluation
- Training example #
- An input context paired with one or more target tokens whose prediction error contributes to the loss. It is not automatically the same thing as a whole chess game — one game usually generates many examples.
- Split unit #
- The smallest group assigned wholly to one data split. In our current evaluation it is a complete chess game, so overlapping positions from one game cannot cross from training into validation.
- Data leakage #
- Information crosses into training or model selection that would not be available in the claimed evaluation scenario, making measured performance misleadingly optimistic. Splitting overlapping positions from one game across training and validation is one form.
- Training split #
- The games whose targets may update model parameters.
- Validation split #
- Held-out games used repeatedly to choose models and hyperparameters. They never update parameters directly.
- Test split #
- Held-out games used only for infrequent final evaluation after development decisions are made. Consulting it repeatedly to pick models quietly turns it into a second validation set.
- Generalization #
- Performing well on examples the model never trained on, because it learned reusable patterns rather than the training data itself. The opposite failure — fitting the training data without the pattern — is memorization.
- Baseline #
- A deliberately simple reference model measured under the same protocol as its challengers. An improvement claim means nothing except relative to a trustworthy baseline.
Model mechanics
- Token #
- One discrete symbol from the vocabulary, represented inside the model by an integer ID. A chess move may require several tokens.
- Vocabulary #
- The frozen, ordered set of all tokens the model can receive or predict. Its order defines the token IDs.
- Parameter #
- A scalar value learned by gradient descent and stored in the submitted model. Under the proposed tournament rule, all unique trainable parameters count, including embeddings.
- Logit #
- One raw model score for a possible next token, before conversion into probabilities.
- Legal-token mask #
- An inference rule that assigns zero probability to tokens that cannot continue any legal move. It constrains output; it does not prove the model learned chess legality.
Experimental practice