The fastai library structures its training process around the
Learner class, whose object binds together a PyTorch model, a dataset, an optimizer, and a loss function; the entire learner object then will allow us to launch training.
basic_train defines this
Learner class, along with the wrapper around the PyTorch optimizer that the library uses. It defines the basic training loop that is used each time you call the
fit method (or one of its variants) in fastai. This training loop is very bare-bones and has very few lines of codes; you can customize it by supplying an optional
Callback argument to the
callback defines the
Callback class and the
CallbackHandler class that is responsible for the communication between the training loop and the
Callback's methods. The
CallbackHandler maintains a state dictionary able to provide each
Callback object all the information of the training loop it belongs to, putting any imaginable tweaks of the training loop within your reach.
callbacks implements each predefined
Callback class of the fastai library in a separate module. Some modules deal with scheduling the hyperparameters, like
callback.general_sched. Others allow special kinds of training like
callbacks.fp16 (mixed precision) and
callbacks.hooks are useful to save some internal data generated in the training loop.
train then uses these callbacks to implement useful helper functions. Lastly,
metrics contains all the functions and classes you might want to use to evaluate your training results; simpler metrics are implemented as functions while more complicated ones as subclasses of
Callback. For more details on implementing metrics as
Callback, please refer to creating your own metrics.
Walk-through of key functionalities¶
We'll do a quick overview of the key pieces of fastai's training modules. See the separate module docs for details on each.
from fastai.vision import * path = untar_data(URLs.MNIST_SAMPLE) data = ImageDataBunch.from_folder(path)
URLs.MNIST_SAMPLE is a small subset of the classic MNIST dataset containing the images of just 3's and 7's for the purpose of demo and documentation here. Common
datasets can be downloaded with
untar_data - which we will use to create an
model = simple_cnn((3,16,16,2))
learn = Learner(data, model)
To see how our training is going, we can request that it reports various kinds of
metrics after each epoch. You can pass it to the constructor, or set it later. Note that metrics are always calculated on the validation set.
You can use
callbacks to modify training in almost any way you can imagine. For instance, we've provided a callback to implement Leslie Smith's 1cycle training method.
cb = OneCycleScheduler(learn, lr_max=0.01) learn.fit(1, callbacks=cb)
Recorder callback is automatically added for you, and you can use it to see what happened in your training, e.g.:
Many of the callbacks can be used more easily by taking advantage of the
Learner extensions in
train. For instance, instead of creating OneCycleScheduler manually as above, you can simply call
Note that if you're training a model for one of our supported applications, there's a lot of help available to you in the application modules:
For instance, let's use
vision) to quickly fine-tune a pre-trained Imagenet model for MNIST (not a very practical approach, of course, since MNIST is handwriting and our model is pre-trained on photos!).
learn = cnn_learner(data, models.resnet18, metrics=accuracy) learn.fit_one_cycle(1)