Course Pass 4: Sequence models

RNN/LSTM/GRU LMs, seq2seq with attention, beam search in the CLI. About 4 weeks at 10 to 12 hours a week.

Part of: the course. Needs: Pass 3.

Gate: MS-P4 = MS-L3 + MS-L4.

Stages, chapter links, and check columns are in path.tsv. Use practice/bin/ol learn course-p04-p04-sequence-models to see progress.

Stages

Generated from path.tsv. Track progress locally with practice/bin/ol learn.

Stage	Read	Done when
1	LLN, CLT, confidence intervals, bootstrap LLN, CLT, confidence intervals, bootstrap	Your implementation of M07.4 passes its course checks.
2	Solve set: LLN/CLT, confidence intervals Solve set: LLN/CLT, confidence intervals	You complete the S-M07c solve set and its check passes.
3	Vanilla RNN with manual BPTT and truncation Vanilla RNN with manual BPTT and truncation	Your implementation of L3.1 passes its course checks.
4	LSTM (torch gate order) LSTM (torch gate order)	Your implementation of L3.2 passes its course checks.
5	GRU (torch gate order) GRU (torch gate order)	Your implementation of L3.3 passes its course checks.
6	Bidirectional RNN with length-aware reversal Bidirectional RNN with length-aware reversal	Your implementation of L3.4 passes its course checks.
7	RNN language model with stateful TBPTT RNN language model with stateful TBPTT	Your implementation of L3.6 passes its course checks.
8	Bahdanau additive attention Bahdanau additive attention	Your implementation of L4.2 passes its course checks.
9	Luong attention and input feeding Luong attention and input feeding	Your implementation of L4.3 passes its course checks.
10	Encoder-decoder with teacher forcing Encoder-decoder with teacher forcing	Your implementation of L4.1 passes its course checks.
11	Beam search, generic over a step function Beam search, generic over a step function	Your implementation of L4.4 passes its course checks.
12	Sequence metrics: exact match, BLEU, chrF Sequence metrics: exact match, BLEU, chrF	Your implementation of L4.5 passes its course checks.
13	Mutation testing in depth: equivalent mutants, semantic mutants from pitfalls, reading survivors Mutation testing in depth: equivalent mutants, semantic mutants from pitfalls, reading survivors	Your implementation of craft.07 passes its course checks.
14	Milestone L3 Pass 4 milestones	The MS-L3 gate passes for your system.
15	Milestone L4 Pass 4 milestones	The MS-L4 gate passes for your system.
16	Milestone P4 Pass 4 milestones	The MS-P4 gate passes for your system.
17	ELMo: biLM, ScalarMix, linear probes ELMo: biLM, ScalarMix, linear probes	Your implementation of L3.5 passes its course checks.