Contents
  1. Learning across experiences
    1. Extend a model over time
    2. Specify the learning sequence
    3. Separate acquisition from interference
      1. Separate the changed variables
    4. Origins of stability and rehearsal
  2. Measuring accumulated capability
    1. Read capabilities across checkpoints
      1. Choose the reference
  3. Preserving useful learning
    1. Rehearse selected experience
      1. Availability, exposure, and influence
      2. When original examples are unavailable
    2. Constrain parameter movement
      1. A local compromise
      2. Carry protection across stages
    3. Constrain earlier behavior
      1. Choose what deserves preservation
    4. Separate trainable capacity
  4. Deciding when to learn
    1. Separate triggers from permission to train
    2. Balance cadence and learning intensity
  5. Evidence over time
    1. Respect information arrival
      1. Keep historical and current views
    2. Compare complete learning policies
      1. Measure the ability to keep learning
  6. Release and continued learning
    1. Preserve the learning state
      1. Match the saved boundary
    2. Promote only supported improvements
    3. Recover before learning again
  7. Check understanding
  8. Open questions
  9. Selected talks
  10. References
  11. Talk library
← All topics

Continual learning

Continual learning aims to acquire new capabilities from successive experience while preserving useful earlier behavior. This chapter focuses on systems that update model parameters: learning from today's examples can improve today's work while damaging yesterday's capabilities. Building a useful learner therefore requires choosing which experience enters each update, measuring what improves and what deteriorates, and preserving enough state to recover and continue learning.

I. Learning across experiences

Extend a model over time

Continual learning develops capabilities from sequential experience rather than assuming that all training data remain available together. Here, the focus is successive parameter updates. Parameters are the model's learned numerical state; a checkpoint saves a particular model state. Each update inherits both useful capabilities and obligations to preserve behavior that still matters.

Continuing from a capable checkpoint can reuse what earlier training established instead of rebuilding it from initialization. The mechanics of an individual update belong to Pretraining and Midtraining and Post-training and Alignment. This chapter adds the obligations created by repetition: preserving earlier capabilities, choosing what history to retain, and controlling what each successor inherits.

Experience can also persist outside the model. Agent Memory stores information that later requests can supply to an unchanged model. This can improve behavior without demonstrating retention through parameter updates. The initial Continual Learning Bench (CL-Bench), for example, evaluated retained conversations, shortened conversation histories, retrieved memories, and notepads—not weight-updating methods. The distinction matters when interpreting a reported learning gain: what state was allowed to change?

A concrete parameter-learning setting is TiC-CLIP, an ICLR 2024 study of image–text models updated with successive yearly data batches. Its checkpoints are assessed on earlier and later periods as well as static benchmarks. That arrangement makes the central obligation visible: learning from the next period should not be judged only on that period. TiC-CLIP paper

Specify the learning sequence

A task specifies an input-to-output problem. A distribution describes the mix of cases encountered. Before choosing a method, state what changes between experiences and what information the model receives at evaluation. The three incremental-learning settings distinguish these assumptions.

SettingWhat changesInference obligation
Task-incrementalSuccessive tasksTask identity is supplied or clear; select the appropriate solution.
Domain-incrementalInput conditions; the output problem persistsSolve the same problem under changed conditions.
Class-incrementalAvailable output categories expandDistinguish all encountered classes, including across episodes.

Learning cats versus dogs, then cows versus horses, does not alone establish the ability to distinguish cats from cows. A supplied task identifier removes that cross-episode choice; a unified classifier must make it.

Online learning updates from arriving examples or small batches; it does not simply mean serving requests over a network. Continual learning can instead use periodic batches. Nor must each experience announce a new task. In the CLOC photographic-geolocation study, the model learns to predict where photographs were taken from images arriving in chronological order. Conditions change naturally over time, without explicit task boundaries telling the learner when to switch strategies.

Complete the specification with historical-data access, recurring conditions, and feedback timing. A label is the target or outcome used for supervision; it may arrive well after its input. An evaluation that supplies it immediately gives the learner information it would not yet possess. Also distinguish a changed population from a changed label definition or broken input format, as developed in Maintain fitness through change.

Separate acquisition from interference

Acquisition is improvement on a newly targeted capability; retention is preservation of earlier useful behavior. Both can depend on the same parameters. Training changes those parameters to reduce a loss, a numerical penalty for prediction errors. A change that reduces loss on new cases can increase it on earlier ones. Catastrophic forgetting names severe deterioration following subsequent learning, not every decline observed later. Specialization can interfere with retention develops this shared-parameter mechanism.

Stability protects established behavior; plasticity permits new acquisition. The stability–plasticity dilemma is that suppressing change can protect yesterday's solution while preventing today's learning. Because each stage changes the starting point for the next, the order of experiences and the accumulated changes matter—not just the size of the latest update.

Separate the changed variables

To investigate deterioration, cross checkpoint identity with assessment population. The following design contains no measured results. Keep preprocessing, target meanings, graders, and execution conditions fixed wherever a comparison claims to isolate model change.

Fixed historical casesCurrent-population cases
Earlier checkpointHistorical referenceEarlier model under current conditions
Updated checkpointRetention comparisonCurrent-capability comparison

Compare downward within a column to examine a checkpoint change on matched cases. Compare across a row to examine one checkpoint under different populations. Neither comparison alone explains the cause. If an answer became obsolete because a policy changed, preserving that answer is not retention of useful capability; the target definition needs a new version. Paired comparisons establish the model-change comparison, while data maintenance establishes what changed in the work.

Retention includes behavior beyond task accuracy. Qi and colleagues' October 2023 study found worse harmful-response assessments after benign fine-tuning of GPT-3.5 Turbo and Llama-2-7b-Chat under tested recipes. These were historical, single-adaptation experiments, not a demonstration of cumulative forgetting. They nevertheless show why refusal behavior needs its own assessment rather than being inferred from utility gains. Study; behavioral dimensions.

Origins of stability and rehearsal

The problem predates language models. Earlier research asked whether learning systems could establish representations, acquire exceptions, and continue learning without repeatedly receiving everything together. Three contributions clarify why preservation and rehearsal remain central.

Stability, interference, and rehearsal

  1. 1987Adaptive Resonance TheoryStable categories with continuing adaptation.Sources & context
  2. 1989Catastrophic InterferenceLater learning disrupts still-valid facts.Sources & context
  3. 1995Complementary Learning SystemsFast acquisition and slower interleaved integration.Sources & context

    Contributors: James McClelland, Bruce McNaughton, and Randall O’Reilly

Three complementary contributions, not successive replacements. Events are spaced for reading, not elapsed time.

ART matches inputs to learned categories and uses reset-and-search when a match is inadequate. This protects established categories from inappropriate recoding within that architecture; it is not a guarantee for gradient-trained language models.

The arithmetic network first learned addition facts involving one, then facts involving two. The second phase severely disrupted earlier answers although they remained valid. Retention tests made no weight updates, exposing interference in shared connections rather than a need to revise obsolete facts.

Reinstating earlier experience lets integration encounter old and new material together. The biological interpretation is a proposed account, not a claim that engineered replay reproduces human memory.

These are complementary problem formulations, not successive replacements. An architecture can protect established computations, a training objective can discourage interference, and replay can change which experience participates in learning. Before choosing among them, the sequence needs measurements that reveal what is acquired and what is lost.

II. Measuring accumulated capability

Read capabilities across checkpoints

To see whether capabilities accumulate, test each checkpoint on earlier capabilities as well as the newest one, using held-out cases that were not used for training. Following Gradient Episodic Memory (GEM), let Ri,jR_{i,j} denote performance on capability jj after learning stage ii. Each row is a checkpoint; each column is a fixed assessment. The initial row records what the model could already do before the sequence began.

Illustrative accuracy percentages on fixed, separate test sets. Stage 1 targets A, stage 2 targets B, and stage 3 targets C.
CheckpointCapability ACapability BCapability C
Initial50%40%30%
After stage 180%45%Not measured
After stage 285%75%35%
After stage 370%78%80%

Read the table within columns. Stage 2 acquires 30 percentage points on B: 75 minus 45. It also improves A by 5 points. Stage 3 acquires 45 points on C while A loses 15 points from its preceding score and B gains 3. The three-column average rises from 65% to 76%, concealing A's regression. The missing C measurement after stage 1 is not zero and cannot support a stage-1 transfer calculation.

Choose the reference

Acquisitionj=Rj,j−Rj−1,j,Backward changei,j=Ri,j−Rj,j(i>j).\text{Acquisition}_j=R_{j,j}-R_{j-1,j},\qquad \text{Backward change}_{i,j}=R_{i,j}-R_{j,j}\quad(i>j). Here stage jj first targets capability jj. Backward transfer is later learning's effect relative to that post-acquisition reference.

Riemannian Walk instead measures forgetting from the best earlier held-out score. In the example, A has lost 15 points from its maximum of 85%, but only 10 from its acquisition score of 80%. These references answer different questions. An average across tasks should therefore identify its reference rule and preserve important per-capability results.

Forward transfer measures performance before direct training on a later task relative to a baseline. GEM uses random initialization; a pretrained starting checkpoint requires an explicitly adapted baseline. Faster learning after exposure is a different measurement.

Capability columns need not all be accuracy benchmarks. In Samuel Denton's workflow-adaptation experiment, submission-tool use, passing the task's tests, and satisfying both were measured separately. An agent could learn to submit without learning to solve, or solve without submitting. That separation also belongs in a continual learner's retention suite, alongside instruction following and consequential safety behaviors. Use matched-work comparisons for uncertainty and paired analysis.

III. Preserving useful learning

Rehearse selected experience

Replay trains again on selected earlier experience alongside new examples. A replay buffer is the bounded training collection retained for that purpose. Including older examples gives an update pressure to remain useful on them rather than optimizing only the arriving cases.

The 1995 complementary-learning-systems simulation makes the mechanism concrete. Focused learning of penguins as birds that swim but cannot fly disrupted other concepts. Interleaving those facts with earlier material reduced interference but slowed acquisition. Total exposure differed, so this was not a matched-compute comparison. The lesson is about competing learning pressure, not a universal replay ratio.

Availability, exposure, and influence

Buffer management contains three separate decisions. Admission and eviction determine what remains available. Sampling determines which retained examples actually enter an update. Loss weighting determines their relative influence once selected. Replay implementations can configure current-data and memory batch sizes separately. The collection's size therefore does not specify its training exposure.

Three controls on historical influence

Example

Retention, sampling, and weighting act at different boundaries.

Earlier examplesEligible historical arrivals.
Admission and evictionChoose which examples remain available.
Replay bufferSelected historical training records.
Unrepresented populationNo retained example from this population.
Replay sampleExamples actually selected for this update.
New examplesCurrent eligible training records.
Weighted training objectiveSelected examples contribute weighted losses; their gradients also depend on the examples and current model.
Earlier examples enter a bounded collection before sampling and loss weighting. Neither sampling more frequently nor increasing loss weight can recover examples absent from the buffer.
Read the diagram as text
  • Earlier examples. Eligible historical arrivals.
  • Admission and eviction. Choose which examples remain available.
  • Replay buffer. Selected historical training records.
  • Unrepresented population. No retained example from this population.
  • Replay sample. Examples actually selected for this update.
  • New examples. Current eligible training records.
  • Weighted training objective. Selected examples contribute weighted losses; their gradients also depend on the examples and current model.
  • Earlier examples → Admission and eviction: data: historical candidates.
  • Admission and eviction → Replay buffer: data: retained records.
  • Admission and eviction → Unrepresented population: selection outcome: no coverage.
  • Replay buffer → Replay sample: data: sampler-selected records.
  • Replay sample → Weighted training objective: data: replay terms with weights.
  • New examples → Weighted training objective: data: new terms with weights.
Selection policyWhat it prioritizesWhat it can miss
Reservoir samplingFor capacity M after n arrivals, each example has inclusion probability M/n when n ≥ M.Uniform inclusion does not guarantee every class survives.
Class-balanced allocationReserve approximately equal capacity for encountered classes.Rare variations within each class can still disappear.
Recent-example retentionEvict older examples to emphasize recent conditions.Recurring older conditions can lose representation.

iCaRL, by Rebuffi, Kolesnikov, Sperl, and Lampert, combines replay with output preservation while adding image classes. Its selected examples, or exemplars, receive roughly K/tK/t slots per class for capacity KK and tt classes. Within each class, examples are ordered so that retained prefixes approximate the class's average learned feature vector. As more classes arrive, eviction removes each list's tail. Images are stored rather than fixed vectors because training changes how the model represents them. The complete method also classifies using exemplar means, so its results cannot be attributed to replay alone. Paper

Treat dilution as an explicit design consequence: fixed capacity divided among more capabilities leaves fewer examples for each. Then inspect exposure separately. A retained rare case offers little protection if it is almost never sampled; repeatedly sampling it does not create broader coverage. Tiny buffers did improve generalization in studied settings, so repetition is not automatically overfitting—but its effect must be measured. Tiny episodic memories study

Check whether an older target remains valid and permitted before replaying it. A policy revision can turn previously correct supervision into a stale target. Keep record identity and eligibility attached to retained examples, following dataset lineage. Replay should preserve useful capabilities, not mechanically reinforce every historical answer.

When original examples are unavailable

Generative replay replaces stored examples with generated inputs and reference targets. In the 2017 Deep Generative Replay study, a previous generator produced inputs and its paired solver supplied targets for subsequent training. Reproducing historical output frequencies on meaningless noise inputs did not preserve digit discrimination. Input coverage mattered, and effectiveness depended on generator quality. Generated examples are therefore neither complete historical coverage nor automatically private; Synthetic Data develops their correlated-error limits.

Constrain parameter movement

Retention regularization adds a penalty intended to discourage harmful changes. Kirkpatrick and colleagues' Elastic Weight Consolidation (EWC), introduced in a 2016 preprint, anchors parameters to an earlier solution and weights departures by estimated importance. Its diagonal Fisher estimate assigns each parameter an importance value based on local sensitivity of predictions on earlier-task data. “Diagonal” means it treats parameters separately, omitting their interactions. Importance estimates guide resistance to change; they do not identify which facts a parameter stores. EWC paper

L(θ)=Lnew(θ)+λ2∑kFk(θk−θk∗)2.L(\theta)=L_{\mathrm{new}}(\theta)+\frac{\lambda}{2}\sum_k F_k(\theta_k-\theta_k^*)^2. LnewL_{\mathrm{new}} is the new-task loss, θ\theta the candidate parameters, θ∗\theta^* the reference, FkF_k estimated importance, and λ\lambda preservation strength.

Unlike shrinkage toward zero, this penalty protects a learned reference. Larger importance resists movement more strongly. The local diagonal approximation omits parameter interactions; it does not guarantee behavioral retention.

A local compromise

Consider an illustrative two-parameter example centered on the reference. Let displacements be xx and yy, with importance 1 and 4 and λ=1\lambda=1. The preservation penalty is P=(x2+4y2)/2P=(x^2+4y^2)/2. Moving 0.5 along xx costs 0.125; the same distance along yy costs 0.5. Equal movement need not mean equal resistance.

Lnew=12[(x−1)2+(y−1)2].L_{\mathrm{new}}=\tfrac12[(x-1)^2+(y-1)^2]. Without preservation, the minimum is (1,1)(1,1). Adding PP gives stationary conditions 2x−1=02x-1=0 and 5y−1=05y-1=0, so the minimum becomes (0.5,0.2)(0.5,0.2).

The compromise moves less in the more strongly protected direction. These are objective minima, not a simulated optimizer trajectory. In particular, the preservation gradient is zero exactly at the reference, so it would be wrong to depict the first gradient step there as already deflected by that penalty.

Importance changes the compromise

Example

The more strongly protected direction permits less displacement at the combined minimum.

Equal movement, unequal penalty

Importance is 1 along x and 4 along y.

-1.3-0.60.10.81.5-1.3-0.60.10.81.5Parameter displacement x (dimensionless)Parameter displacement y (dimensionless)P = 0.5x displacementy displacementReferenceP = 0.125P = 0.5Reference
  • 1. P = 0.5
  • 2. x displacement
  • 3. y displacement
  • 4. Reference
Read coordinates and regions as data

X: -1.3–1.5 dimensionless; Y: -1.3–1.5 dimensionless, increasing up. Equal scale on both axes.

P = 0.5 (polyline)

(1, 0); (0.99518, 0.049009); (0.98079, 0.097545); (0.95694, 0.14514); (0.92388, 0.19134); (0.88192, 0.2357); (0.83147, 0.27779); (0.77301, 0.3172); (0.70711, 0.35355); (0.63439, 0.38651); (0.55557, 0.41573); (0.4714, 0.44096); (0.38268, 0.46194); (0.29028, 0.47847); (0.19509, 0.49039); (0.098017, 0.49759); (0, 0.5); (-0.098017, 0.49759); (-0.19509, 0.49039); (-0.29028, 0.47847); (-0.38268, 0.46194); (-0.4714, 0.44096); (-0.55557, 0.41573); (-0.63439, 0.38651); (-0.70711, 0.35355); (-0.77301, 0.3172); (-0.83147, 0.27779); (-0.88192, 0.2357); (-0.92388, 0.19134); (-0.95694, 0.14514); (-0.98079, 0.097545); (-0.99518, 0.049009); (-1, 0); (-0.99518, -0.049009); (-0.98079, -0.097545); (-0.95694, -0.14514); (-0.92388, -0.19134); (-0.88192, -0.2357); (-0.83147, -0.27779); (-0.77301, -0.3172); (-0.70711, -0.35355); (-0.63439, -0.38651); (-0.55557, -0.41573); (-0.4714, -0.44096); (-0.38268, -0.46194); (-0.29028, -0.47847); (-0.19509, -0.49039); (-0.098017, -0.49759); (0, -0.5); (0.098017, -0.49759); (0.19509, -0.49039); (0.29028, -0.47847); (0.38268, -0.46194); (0.4714, -0.44096); (0.55557, -0.41573); (0.63439, -0.38651); (0.70711, -0.35355); (0.77301, -0.3172); (0.83147, -0.27779); (0.88192, -0.2357); (0.92388, -0.19134); (0.95694, -0.14514); (0.98079, -0.097545); (0.99518, -0.049009); (1, 0)

x displacement (polyline)

(0, 0); (0.5, 0)

y displacement (polyline)

(0, 0); (0, 0.5)

Reference (points)

(0, 0)

P = 0.125: (0.58, -0.14)

P = 0.5: (0.08, 0.63)

Reference: (-0.08, -0.17)

Combined-objective minimum

The preservation penalty is unchanged; the new-task loss prefers (1,1).

-1.3-0.60.10.81.5-1.3-0.60.10.81.5Parameter displacement x (dimensionless)Parameter displacement y (dimensionless)P = 0.5ReferenceNew-task minimumCombined minimumNew loss: (1, 1)Combined: (0.5, 0.2)Reference
  • 1. P = 0.5
  • 2. Reference
  • 3. New-task minimum
  • 4. Combined minimum
Read coordinates and regions as data

X: -1.3–1.5 dimensionless; Y: -1.3–1.5 dimensionless, increasing up. Equal scale on both axes.

P = 0.5 (polyline)

(1, 0); (0.99518, 0.049009); (0.98079, 0.097545); (0.95694, 0.14514); (0.92388, 0.19134); (0.88192, 0.2357); (0.83147, 0.27779); (0.77301, 0.3172); (0.70711, 0.35355); (0.63439, 0.38651); (0.55557, 0.41573); (0.4714, 0.44096); (0.38268, 0.46194); (0.29028, 0.47847); (0.19509, 0.49039); (0.098017, 0.49759); (0, 0.5); (-0.098017, 0.49759); (-0.19509, 0.49039); (-0.29028, 0.47847); (-0.38268, 0.46194); (-0.4714, 0.44096); (-0.55557, 0.41573); (-0.63439, 0.38651); (-0.70711, 0.35355); (-0.77301, 0.3172); (-0.83147, 0.27779); (-0.88192, 0.2357); (-0.92388, 0.19134); (-0.95694, 0.14514); (-0.98079, 0.097545); (-0.99518, 0.049009); (-1, 0); (-0.99518, -0.049009); (-0.98079, -0.097545); (-0.95694, -0.14514); (-0.92388, -0.19134); (-0.88192, -0.2357); (-0.83147, -0.27779); (-0.77301, -0.3172); (-0.70711, -0.35355); (-0.63439, -0.38651); (-0.55557, -0.41573); (-0.4714, -0.44096); (-0.38268, -0.46194); (-0.29028, -0.47847); (-0.19509, -0.49039); (-0.098017, -0.49759); (0, -0.5); (0.098017, -0.49759); (0.19509, -0.49039); (0.29028, -0.47847); (0.38268, -0.46194); (0.4714, -0.44096); (0.55557, -0.41573); (0.63439, -0.38651); (0.70711, -0.35355); (0.77301, -0.3172); (0.83147, -0.27779); (0.88192, -0.2357); (0.92388, -0.19134); (0.95694, -0.14514); (0.98079, -0.097545); (0.99518, -0.049009); (1, 0)

Reference (points)

(0, 0)

New-task minimum (points)

(1, 1)

Combined minimum (points)

(0.5, 0.2)

New loss: (1, 1): (1, 1.15)

Combined: (0.5, 0.2): (0.6, 0.4)

Reference: (-0.08, -0.17)

In this two-parameter example, the P=0.5 contour is narrower along y: importance 4 resists that displacement more strongly than importance 1 along x. The combined objective therefore prefers (0.5, 0.2).

Carry protection across stages

Successive stages also require a reference-update rule. Huszár's analysis of quadratic EWC penalties explains how penalties centered at successive learned optima can double-count earlier evidence. Under its recursive approximation, one penalty centered at the latest solution can carry accumulated importance. Specifying only “use EWC” therefore leaves an important multi-stage choice unresolved.

Progress & Compress, introduced by Schwarz and colleagues at ICML 2018, separates acquisition from consolidation. An active network learns new skills; a fixed-size knowledge-base network then learns to reproduce its behavior while online EWC protects earlier learning. Online EWC retains one reference parameter vector and updates importance as Fnew=γFprevious+FcurrentF_{\mathrm{new}}=\gamma F_{\mathrm{previous}}+F_{\mathrm{current}}, where γ\gamma discounts older importance. Discounting limits accumulated resistance, but strong preservation still impaired acquisition in some Atari experiments. Fixed-size retained state is not unlimited learning capacity. Paper

Constrain earlier behavior

Behavioral regularization constrains predictions rather than parameter movement. It uses distillation—training a student against a teacher's outputs—to preserve earlier behavior during adaptation. See Distillation for the general mechanism.

Li and Hoiem's Learning without Forgetting (2016) caches old-task probabilities on new-task images, then combines agreement with those targets and new-label supervision. Parameters can change without running the reference at every update. It avoids old training images, not data altogether. Paper

Input coverage limits protection. Agreement on new-task images need not preserve behavior in absent old-input regions; the authors identify distribution differences as a limitation. Increasing the agreement weight favors preservation over acquisition.

Choose what deserves preservation

Matching a reference and solving the task are separate outcomes. A distillation study explicitly separated predictive agreement from independently labeled accuracy; in its self-distillation experiments, more agreement could accompany lower accuracy. A reference's mistakes can be reproduced faithfully. Does Knowledge Distillation Really Work?

Replay can supply historical inputs on which agreement matters: iCaRL combines historical exemplars with pre-update predictions. Conversely, if an old answer is obsolete, preserving it conflicts with the intended correction. The preservation population and target validity must be chosen deliberately.

The teacher is also continuing state. Avalanche's versioned LearningWithoutForgetting implementation copies the model after an experience to become the next reference and records encountered classes. Rejecting candidate weights without considering that reference can leave the next update protecting the wrong behavior. This is a recovery implication of the lifecycle, not an automatic rollback feature.

Separate trainable capacity

Parameter isolation assigns new learning to separate capacity. Rusu and colleagues' Progressive Neural Networks (June 2016) freeze earlier task-specific computation paths, called columns, and train a new column using their features through lateral connections. Storage grows, and inference requires selecting the appropriate task column. Paper

The preservation condition is precise: the same unchanged computation under unchanged input and execution conditions retains its behavior. It does not establish that a selector will choose that path correctly or that separate paths form a unified classifier. This is why the task-identity assumption matters.

Schematic Progressive Neural Networks architecture: the earlier computation stays fixed, lateral features enter later new-column computations, and each task retains its output. Added capacity increases storage; task identity must still be supplied.

Low-Rank Adaptation, or LoRA, freezes a base matrix and trains a smaller added update. Freezing the base does not freeze the adapted system's outputs. The study LoRA Learns Less and Forgets Less measured specialization separately from retained commonsense and science performance; tradeoffs varied with domain, duration, and configuration. A small trainable component is a restriction on change, not a retention certificate.

These mechanisms can be combined, but they protect different things.
MechanismPreservation pressureContinuing dependency
ReplayFit selected earlier experienceExamples, membership, sampling, and weights
Parameter penaltyRemain near an important reference solutionReference parameters and importance estimates
Output constraintAgree with reference predictions on selected inputsInputs, targets or teacher, and agreement weight
IsolationLeave earlier execution paths unchangedRetained paths and a valid selection rule

IV. Deciding when to learn

Separate triggers from permission to train

An update trigger is a rule that starts investigation or a candidate-training process. Schedules, new-data availability, deteriorating performance, and changed statistics are common triggers. Google's continuous-training guidance separates these from data validation and candidate-model validation. A signal to begin work is not evidence that the resulting checkpoint should be deployed.

A trigger should name the evidence needed before committing to a parameter update.
TriggerWhat it establishesNext evidence needed
Calendar intervalA scheduled review is due.Enough eligible, useful new supervision has arrived.
Data-volume thresholdA chosen amount of data accumulated.Its labels, coverage, and relevance justify learning.
Performance declineMeasured behavior worsened.The assessment is comparable and the defect calls for learned change.
Changed data statisticsThe monitored distribution changed.The change affects required capability and supplies a learnable signal.

ADWIN, by Albert Bifet and Ricard Gavaldà, provides a concrete change detector. Published in the SDM 2007 proceedings, it compares averages within an adaptive observation window and discards older observations when subwindow differences indicate change. It can monitor prediction errors without fixing one historical window length. Its theoretical setup assumes independent, bounded observations, and its false-alarm bound is per time step—not a lifetime promise. An alarm neither identifies the cause nor proves that neural retraining will help. Paper

Diagnosis determines the repair layer. A stale-policy answer may require refreshed information; a missed escalation may require a workflow gate. Soheil Feizi's repair-layer discussion recommends the smallest durable change at the layer that explains the failure. Training is appropriate only when the intended learned behavior—not merely a tool, input, or execution rule—needs to change.

Use data-change diagnosis to distinguish outages, representation defects, target revisions, and population shifts. Use context maintenance when current information is missing from the request. For an actual learning intervention, record the target capability, trustworthy supervision, retained behaviors, resource limits, and owner authorized to approve the candidate. Delayed labels may postpone that decision even while unlabeled inputs accumulate.

Balance cadence and learning intensity

Cadence controls when updates begin. Learning intensity describes the work and exposure within each update: optimization steps, learning rates, replay frequency, and stopping conditions. These are independent controls. Ten short updates can repeatedly train on the same retained examples and accumulate substantial parameter change. The within-run mechanisms are developed in Training schedules.

Small-batch updates can incorporate available supervision promptly; periodic batches wait for more data before training. Broader consolidation revisits a wider mixture. None removes feedback delay or computation time. A real-time continual-learning protocol keeps predicting with the latest available model while training proceeds; slower methods consequently miss more potential training batches under that protocol's scheduling rule. Skipping batches is a design choice, not an inherent requirement. Real-time evaluation study

Consider an illustrative eight-hour interval. Feedback for an observation arrives after two hours, training ends at hour four, and candidate validation finishes at hour five. The earlier checkpoint supplies predictions until then. Starting training sooner would help only if valid supervision and resources were already available; finishing training would still not itself authorize promotion.

Optimization choices can change the acquisition–retention balance. In controlled language-model continuation experiments, learning-rate rewarming—raising the rate again for continuation—accelerated adaptation but increased forgetting. Replay comparisons held continuation compute constant. The experiments covered particular English and German shifts and model sizes; they do not prescribe a universal restart schedule. Continuation study

CLOC likewise found different schedule rankings for different goals: cosine decay gave the strongest final historical performance but the weakest online fit among compared schedules. A larger replay buffer could also hinder adaptation. These are reasons to measure both arriving-case performance and retained capability, not to infer either from the other. CLOC study

Compare schedules on the same arrival stream, with the same label-availability rules, and disclose their training work. Record distinct examples separately from repeated exposures. Stop an update according to acquisition and retention criteria, then assess how that stopping rule behaves across many updates. The best interval is a property of the workload and learning policy, not of the word “continual.”

V. Evidence over time

Respect information arrival

Prequential evaluation, also called test-then-train, records a prediction before learning from that observation. MOA distinguishes this from periodic held-out testing. The stream score combines predictions made by successive model states; it does not remeasure the current checkpoint on earlier cases. A rising stream score can therefore coexist with historical forgetting.

With delayed outcomes, preserve the original prediction until its label arrives. River's progressive evaluator separates prediction and learning events using an explicit delay. The later label scores that recorded prediction and may supervise a later update; it cannot retroactively become information available to the earlier model.

Keep historical and current views

Keep two assessment views. Fixed historical suites compare checkpoints on stable capabilities. Fresh chronological cohorts measure performance under later conditions. Version their cases, labels, graders, and execution settings. If a target definition changes, preserve the old assessment's meaning rather than silently substituting new labels and calling the difference forgetting.

Two assessments, different questions

Score the original prediction; assess candidate retention separatelyCheckpoint A predicts on observation x before outcome y arrives. Recorded prediction p keeps A's identity. Later y scores that same p and can enter training only after eligibility checks. Training begins from A and produces candidate B, without authorizing promotion. In a separate lane, the same protected historical suite evaluates A and B without training.Stream · event order, not elapsed durationObserve xCheckpoint A predictspredictionRecord p + A’s identityKeep this predictionScoring input: saved pScore p against yA’s historical predictionLater outcome y arrivesScoring input: yRetained input xAvailable target yCheck (x,y) eligibilityOnly eligible supervision updates AEligible supervisionTrain from ACandidate BNot promotedStarting parameters: Ay never changes the recorded p. Completing training does not authorize serving B.Protected assessment · no parameter updatesSame historical suiteFixed cases and targetsReserved from trainingAssess checkpoint AAssess candidate BCompare retentionMatched conditionsDashed arrows: held-out assessment. These cases do not feed the training lane.
The stream score evaluates the prediction A recorded before y arrived. The protected historical suite compares A and candidate B on the same cases. Training completion does not authorize promotion; event order carries no duration scale.

Test-before-train protects one boundary, not every boundary. Cases inspected to trigger an update or select its replay policy have already informed development. They remain valuable regression tests, but repeated selection against them does not provide an independent final assessment. Reserve later or otherwise protected cases and preserve related-record separation, following Protect the independent assessment.

Report outcome availability beside performance. A recent interval with unresolved labels is incomplete, not successful. If only escalated cases receive review, the observed labels describe a selected population. Incomplete feedback explains that distinction. Show uncertainty and coverage by meaningful time window and capability slice, especially when conditions recur and an aggregate would hide a temporary failure.

Compare complete learning policies

A learning policy specifies what history is retained, how it is sampled, what changes are constrained, when updates run, and how candidates are selected. Compare complete policies from matched starting checkpoints and stream access. Report training work, historical-data access, auxiliary storage, tuning effort, and adaptation delay separately from capability outcomes.

This is an experiment-design table, not a ranking or a table of measured outcomes.
PolicyDistinct obligationResource or access condition
Unchanged checkpointEstablish whether updating helps at all.No additional parameter-training work.
New-data-only continuationMeasure acquisition and historical regressions.Current eligible data; stated update budget.
Continuation with replayTest whether retained coverage changes the tradeoff.Buffer capacity, sampling, and replay exposure.
Replay plus a retention constraintEstablish benefit beyond replay alone.Importance estimates or reference targets; added compute.
Broader historical retrainingProvide a reference under broader access.Potentially more data and compute; not a guaranteed upper bound.

TiC-CLIP compares continuation with replay, new-data-only continuation, equal-budget restarting, and more expensive pooled-data training. Replay improved historical and static-task performance over new-data-only continuation in reported comparisons. Bounded buffers remained competitive with full-history retention under tested budgets. Its period-specific accuracies are not GEM-style transfer differences, and unequal-budget references must remain labeled as such.

Measure the ability to keep learning

Retention alone can reward a learner that barely changes. Loss of plasticity is deterioration in the ability to acquire subsequent tasks. Dohare and colleagues' August 2024 Nature study evaluated long sequences of two-category image-classification tasks under fixed per-task training budgets. Some networks learned well initially but later approached or fell below a linear baseline. Their continual-backpropagation method selectively reinitialized low-utility units—resetting selected internal components so they could learn afresh—and maintained acquisition over the tested sequences. The protocol announced task boundaries and reset each task's output head, the part producing its category predictions. It chiefly demonstrated sustained acquisition, not joint long-horizon retention for language models.

Test enough stages to expose cumulative effects, including relevant orderings and returns to earlier conditions. An unchanged checkpoint is a useful baseline but fails a requirement to acquire genuinely new capability. A highly plastic learner can fail retention requirements. Select among policies that meet the explicit obligations; then compare their resource use and delay. Missing measurements remain unavailable rather than being converted into favorable scores.

VI. Release and continued learning

Preserve the learning state

Inference and continued learning need different state. Weights plus compatible architecture and preprocessing may be enough to produce predictions. A resumable checkpoint also identifies how optimization continues. Continual learning adds dependencies that determine what the next update will rehearse and protect.

Preserve the components the selected method actually uses; this is a design inventory, not a universal checkpoint format.
StateWhy it affects continuation
Parent and accepted data intervalIdentify the starting model and newly admitted experience.
Replay membership and selection stateDetermine which earlier examples can influence future updates.
Teacher, cached targets, or importance estimatesDetermine which reference behavior or parameter changes are protected.
Optimizer, schedule, and random stateDetermine subsequent numerical updates and sampling.
Data progress and completed-update boundaryDistinguish consumed input from completed optimization.
Evaluation versions and decisionsIdentify what behavior was assessed and accepted.

Match the saved boundary

Checkpoint APIs have specific recovery boundaries. Avalanche's checkpoint implementation serializes a learning strategy, random-number manager, and completed-experience counter. Its documented save operation runs outside training and evaluation loops. Excluding attributes changes what can be restored. This supports experience-boundary recovery, not arbitrary mid-update transactions.

TorchData's StatefulDataLoader separately records progress through inputs. Its default state tracks yielded batches, and custom samplers or datasets can supply their own state. Worker state is aggregated, but distributed-rank coordination is not supplied by that mechanism. A yielded batch is not necessarily a completed optimizer update; the application must coordinate those boundaries.

Lineage records artifact–execution dependencies. ML Metadata distinguishes datasets and models from executions that consume and produce them. Use that model to trace a candidate to its parent, inputs, and retained learning state, extending dataset lineage. A recorded dataset dependency does not prove every record was consumed, and provenance alone performs no recovery.

Keep the served checkpoint separate from the experimental learner. When rejecting or branching an update, either restore a compatible set of continuing state or deliberately define a new starting policy. Do not accidentally pair old weights with a new teacher or unrelated replay snapshot. Even logical restoration is not bitwise reproducibility: PyTorch does not guarantee identical execution across releases, platforms, or CPU and GPU implementations.

Promote only supported improvements

Candidate acceptance should connect directly to the capability matrix: improvement on intended work, acceptable changes on retained capabilities, and compatible operation. Feizi's verifiable continual-learning proposal combines an executable failure case, measured before-and-after improvement, and regression tests. Passing those tests supports preservation within their coverage, not a promise that nothing can break.

Include retention during candidate development rather than selecting solely for the newest task and inspecting regressions only afterward. That makes preservation part of the search criterion. It does not remove the need for independent assessment, nor does it supply an algorithm that can check an indefinitely growing history at constant cost.

Specify thresholds, coverage requirements, and decision owners before inspecting candidate results.
DecisionRequired claim
Accept a training candidateThe target improvement meets its criterion and required retained slices remain acceptable.
Begin bounded exposureOffline gates and compatibility checks pass; live uncertainty is explicitly identified.
Promote more broadlyThe bounded live results satisfy prespecified quality and operating requirements.
Reject or withdrawA required criterion fails or evidence is insufficient for continued exposure.

Shadow evaluation assesses a candidate alongside the incumbent while the incumbent still supplies production outputs. A canary allows a bounded population to receive candidate outputs. Their evidence differs: shadow results cannot establish the consequences of acting on the candidate. Keep side effects isolated and use the experimental guidance in Choose the live experiment. AWS's deployment guidance likewise separates validation, promotion criteria, and switching between versioned models.

GPT-4o's April 2025 sycophancy incident shows the importance of a missing behavioral check. OpenAI's May 2 retrospective reports favorable offline and small-scale preference results before a release that excessively agreed with users. Dedicated sycophancy deployment evaluations were missing. The preliminary explanation implicated interacting training changes, including an added reward from user feedback. A prompt mitigation preceded restoration of the previous model. This was a release regression and serving recovery, not demonstrated cumulative forgetting. Incident account

Recover before learning again

Serving rollback changes which checkpoint supplies predictions. Learning recovery establishes which inputs and state may produce its successor. Those are separate responsibilities: a serving pointer does not encompass the learner's buffer, teacher, optimizer, or data cursor.

The separation follows from concrete state mutations. ReplayPlugin updates its stored experience after training, while a previous-experience teacher can also advance. Returning to earlier weights does not reverse those changes. Recovery therefore needs an explicit decision about auxiliary state, not just the weight file.

First contain exposure and pause affected learning or promotion. Trace the rejected candidate's inputs and descendants. Then choose between restoring a compatible earlier learning bundle and deliberately rebuilding one from an appropriate checkpoint and eligible data. Reassess the restored or rebuilt learner before it can produce an accepted successor. This is an engineering recovery design; the component libraries do not collectively provide an automatic transaction implementing it.

Serving rollback changes the selected version. Learning recovery separately restores a compatible bundle or deliberately rebuilds state, then reassesses it. This engineering design does not promise an atomic rollback across libraries or bitwise reproduction.

An older served version must still be compatible with current application state. Restoring it changes future execution but does not reverse messages, database writes, or other completed effects. Those require the separate reconciliation or compensation described in Restore software without erasing effects.

Removing a bad example from replay prevents that route of future exposure; it does not establish that already-trained parameters no longer reflect it. Cao and Yang's 2015 machine-unlearning paper distinguishes source deletion from reversing learned influence. The response to that stronger requirement belongs to Address learned influence, not to a claim that buffer deletion undoes training.

Post-release outcomes become candidate learning inputs, not automatic ground truth. A data flywheel collects interactions and feedback, curates them, and trains and evaluates candidates; that loop alone supplies no forgetting protection. NVIDIA's data-flywheel discussion illustrates the operational cycle. Close it with the same disciplines used at the start: trustworthy supervision, retained-capability requirements, identifiable learning state, and permission to reject the next update.

Open questions

  1. Bounded memory must protect rare, recurring capabilities while remaining useful for new conditions. Uniform sampling, class balance, and recency each prioritize different coverage. Progress would show sustained acquisition and slice-level retention under fixed storage and training budgets, not merely a favorable final average.

  2. Preserving earlier behavior and preserving the ability to learn are distinct objectives. Methods that maintain acquisition over long sequences still need joint tests of historical retention, especially without announced task boundaries. Progress would establish both properties under the same continuing protocol.

  3. Realistic personalization evaluation needs feedback that reflects changing human preferences rather than only a simulator's regularities. This matters because success in easily verified environments may not transfer to long-term user adaptation. Progress would connect simulated learning gains to independently observed user outcomes.

  4. Growing retention suites make repeated candidate selection increasingly expensive. Selective testing must avoid systematically omitting the very capabilities an update threatens. Progress would quantify missed-regression risk alongside saved evaluation work across long learning sequences.

  5. Recovery must coordinate model state with the replay, references, and data progress that shape future updates. Component checkpoints provide only particular boundaries. Progress would demonstrate failure-tested restoration of a compatible learner and verify its next update, rather than demonstrating only that old weights can serve again.

Follow the curated reading path through the speakers and demonstrations behind this entry.

19 min

AI Engineer World's Fair 2026 · 2026

Bringing Continual Learning into Enterprises

Samuel Denton

Cited in this entry

A concrete workflow-adaptation example separates submission behavior, task correctness, and their intersection; the distillation discussion also distinguishes historical traces from current-policy steps.

Watch talk
20 min

AI Engineer World's Fair 2026 · 2026

Taking Reinforcement Learning Cross Datacenter

Nan Jiang

Cited in this entry

Extends checkpoint identity into asynchronous collection: requests identify acceptable policy versions, and responses identify the version actually used. This exposes stale experience without prescribing an acceptable lag.

Watch talk

Explore more talks

The rest of the library, beyond the curated path. Cited talks support this entry; reviewed transcripts were processed in full. Metadata candidates have not been reviewed as sources or verified as topic members.

25 matching talks

TalkSpeakerEventYear
Will BrownAI Engineer World's Fair 20262026
Jack MorrisAI Engineer Code 20252025
Vibhu SapraAI Engineer World's Fair 20252025
Sylendran ArunagiriAI Engineer World's Fair 20252025
Daniel ChalefAI Engineer World's Fair 20262026
Abi AryanAI Engineer Summit 20232023
Vibhor KumarAI Engineer World's Fair 20242024
Sara HookerAI Engineer World's Fair 20262026
Vivek TrivedyAI Engineer World's Fair 20262026
Yu SuAI Engineer World's Fair 20262026
Raymond FengAI Engineer World's Fair 20262026
Ronak MaldeAI Engineer World's Fair 20262026
Matt PocockAI Engineer Code 2025Unknown
Charlie Guo, Gabriel ChuaAI Engineer World's Fair 20262026
Marc KlingenAI Engineer World's Fair 20262026
Greg PstruchaAI Engineer World's Fair 20262026
Jake BroekhuizenAI Engineer World's Fair 20262026
Dru KnoxAI Engineer World's Fair 20262026
William LyonEvent unknownUnknown
Ang LiAI Engineer World's Fair 20262026
Zach LloydEvent unknownUnknown
Shubhankar SrivastavaAI Engineer World's Fair 20262026
Rafal WilinskiAI Engineer World's Fair 20262026
Nacho MartinezEvent unknownUnknown
Matthew RagoneAI Engineer NYC 2026 Online Talks2026

References

Coverage and source review
Processed transcripts
11 processed in full · 4 in the curated path
Automated source review
Passed
Metadata candidates
18 unreviewed; not verified topic membership
Corpus version
1bd8e407b26a07b33815594e1b2db5f41827119a2b3cb6fbf240f9fc571fc767

Automated review checks source support; it is not publication approval.

A synthesis of selected conference talks and technical references. Citations link to the source material; they do not imply that every talk on this subject is included.

  1. Three types of incremental learning

    https://www.nature.com/articles/s42256-022-00568-3

  2. Simple and Scalable Strategies to Continually Pre-train Large Language Models

    https://arxiv.org/html/2403.08763

  3. Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments

    https://arxiv.org/html/2606.05661v1

  4. TiC-CLIP: Continual Training of CLIP Models

    https://proceedings.iclr.cc/paper_files/paper/2024/file/47eb71e8e778dc9def7780baa7b35687-Paper-Conference.pdf

  5. Real-Time Evaluation in Online Continual Learning: A New Hope

    https://arxiv.org/pdf/2302.01047

  6. Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data

    https://arxiv.org/html/2108.09020

  7. progressive_val_score — River

    https://riverml.xyz/dev/api/evaluate/progressive-val-score/

  8. Gradient Episodic Memory for Continual Learning

    https://arxiv.org/pdf/1706.08840

  9. A Massively Parallel Architecture for a Self-Organizing Neural Pattern Recognition Machine

    https://sites.bu.edu/steveg/files/2016/06/CarGro1987CVGIP.pdf

  10. Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem

    https://www.andywills.info/hbab/mccloskeycohen.pdf

  11. Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

    https://arxiv.org/pdf/2310.03693

  12. Why There Are Complementary Learning Systems in the Hippocampus and Neocortex

    https://stanford.edu/~jlmcc/papers/McCMcNaughtonOReilly95.pdf

  13. Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence

    https://arxiv.org/pdf/1801.10112

  14. Gradient Episodic Memory for Continual Learning

    https://arxiv.org/pdf/1706.08840

  15. Bringing Continual Learning into Enterprises

    https://www.youtube.com/watch?v=ZTA0GwpAUak&t=681

  16. On Tiny Episodic Memories in Continual Learning

    https://arxiv.org/pdf/1902.10486

  17. Avalanche v0.3.0 ReplayPlugin — source documentation

    https://avalanche-api.continualai.org/en/v0.3.0/_modules/avalanche/training/plugins/replay.html

  18. iCaRL: Incremental Classifier and Representation Learning

    https://arxiv.org/pdf/1611.07725

  19. Stuffing Context is not Memory, Updating Weights is

    https://www.youtube.com/watch?v=Jty4s9-Jb78&t=3492

  20. Continual Learning with Deep Generative Replay

    https://proceedings.neurips.cc/paper_files/paper/2017/file/0efbe98067c6c73dba1250d2beaa81f9-Paper.pdf

  21. Overcoming catastrophic forgetting in neural networks

    https://arxiv.org/html/1612.00796

  22. On Quadratic Penalties in Elastic Weight Consolidation

    https://arxiv.org/pdf/1712.03847

  23. Progress & Compress: A scalable framework for continual learning

    https://proceedings.mlr.press/v80/schwarz18a/schwarz18a.pdf

  24. Learning without Forgetting

    https://arxiv.org/pdf/1606.09282

  25. Does Knowledge Distillation Really Work?

    https://arxiv.org/html/2106.05945

  26. Avalanche v0.3.0 LearningWithoutForgetting

    https://avalanche-api.continualai.org/en/v0.3.0/generated/avalanche.training.LearningWithoutForgetting.html

  27. Progressive Neural Networks

    https://arxiv.org/pdf/1606.04671

  28. LoRA Learns Less and Forgets Less

    https://arxiv.org/html/2405.09673v2

  29. MLOps: Continuous Delivery and Automation Pipelines in Machine Learning

    https://docs.cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning

  30. Learning from Time-Changing Data with Adaptive Windowing

    https://www.cs.upc.edu/~Gavalda/papers/adwin06.pdf

  31. Continual Learning for AI Agents: From Failures to Durable Improvements - Soheil Feizi, RELAI

    https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=763

  32. Command Line — MOA

    https://moa.cms.waikato.ac.nz/details/classification/command-line/

  33. Loss of plasticity in deep continual learning

    https://www.nature.com/articles/s41586-024-07711-7

  34. avalanche.checkpointing.checkpoint — source documentation

    https://avalanche-api.continualai.org/en/latest/_modules/avalanche/checkpointing/checkpoint.html

  35. Stateful DataLoader — TorchData

    https://meta-pytorch.org/data/main/torchdata.stateful_dataloader.html

  36. ML Metadata — TensorFlow Extended

    https://www.tensorflow.org/tfx/guide/mlmd

  37. PyTorch 2.9: Reproducibility

    https://docs.pytorch.org/docs/2.9/notes/randomness.html

  38. Continual Learning for AI Agents: From Failures to Durable Improvements - Soheil Feizi, RELAI

    https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=661

  39. Continual Learning for AI Agents: From Failures to Durable Improvements - Soheil Feizi, RELAI

    https://www.youtube.com/watch?v=2IxD9OB3XuQ&t=800

  40. Continuous deployment — AWS Prescriptive Guidance

    https://docs.aws.amazon.com/prescriptive-guidance/latest/mlops-checklist/continuous-deployment.html

  41. Expanding on what we missed with sycophancy

    https://openai.com/index/expanding-on-sycophancy/

  42. Towards Making Systems Forget with Machine Unlearning

    https://www.yinzhicao.org/unlearning/UnlearningOakland15.pdf

  43. Effective AI Agents Need Data Flywheels, Not The Next Biggest LLM – Sylendran Arunagiri, NVIDIA

    https://www.youtube.com/watch?v=6lTxD_oUjXQ&t=175

  44. Connectionist Models of Recognition Memory: Constraints Imposed by Learning and Forgetting Functions

    https://condor.depaul.edu/dallbrit/extra/hon207/readings/ratcliff-1990-connectionist-models-of-recognition-memory.pdf

  45. Dive into Deep Learning: environment and distribution shift

    https://en.d2l.ai/chapter_linear-classification/environment-and-distribution-shift.html

  46. Beyond Static Intelligence: Evaluating Continual Learning

    https://www.youtube.com/watch?v=iqloyWCGYQQ&t=1170

  47. Bringing Continual Learning into Enterprises

    https://www.youtube.com/watch?v=ZTA0GwpAUak&t=401

  48. Beyond Static Intelligence: Evaluating Continual Learning

    https://www.youtube.com/watch?v=iqloyWCGYQQ&t=376

  49. Beyond Static Intelligence: Evaluating Continual Learning

    https://www.youtube.com/watch?v=iqloyWCGYQQ&t=494

  50. Taking Reinforcement Learning Cross Datacenter

    https://www.youtube.com/watch?v=maRzp4kImJ4&t=916