Callbacks that saves the tracked metrics during training

CSV Logger

class CSVLogger[source][test]

CSVLogger(learn:Learner, filename:str='history', append:bool=False) :: LearnerCallback

Tests found for CSVLogger:

  • pytest -sv tests/test_callbacks_csv_logger.py::test_logger [source]

To run tests please refer to this guide.

A LearnerCallback that saves history of metrics while training learn into CSV filename.

First let's show an example of use, with a training on the usual MNIST dataset.

path = untar_data(URLs.MNIST_TINY)
data = ImageDataBunch.from_folder(path)
learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy, error_rate], callback_fns=[CSVLogger])

learn.fit(3)
Total time: 00:02

epoch train_loss valid_loss accuracy error_rate time
0 0.643073 0.536124 0.931330 0.068670 00:00
1 0.523130 0.246320 0.952790 0.047210 00:00
2 0.402764 0.147998 0.938484 0.061516 00:00

Training details have been saved in 'history.csv'.

Note that it only saves float/int metrics, so time currently is not saved. This could be saved but requires changing the recording - you can submit a PR fixing that.

learn.path.ls()

[PosixPath('/home/stas/.fastai/data/mnist_tiny/models'),
 PosixPath('/home/stas/.fastai/data/mnist_tiny/labels.csv'),
 PosixPath('/home/stas/.fastai/data/mnist_tiny/cleaned.csv'),
 PosixPath('/home/stas/.fastai/data/mnist_tiny/train'),
 PosixPath('/home/stas/.fastai/data/mnist_tiny/test'),
 PosixPath('/home/stas/.fastai/data/mnist_tiny/valid'),
 PosixPath('/home/stas/.fastai/data/mnist_tiny/export.pkl'),
 PosixPath('/home/stas/.fastai/data/mnist_tiny/history.csv')]

Note that, as with all LearnerCallback, you can access the object as an attribute of learn after it has been created. Here it's learn.csv_logger.

read_logged_file[source][test]

read_logged_file()

Tests found for read_logged_file:

Some other tests where read_logged_file is used:

  • pytest -sv tests/test_callbacks_csv_logger.py::test_logger [source]

To run tests please refer to this guide.

Read the content of saved file

learn.csv_logger.read_logged_file()
epoch train_loss valid_loss accuracy error_rate
0 0 0.643073 0.536124 0.931330 0.068670
1 1 0.523130 0.246320 0.952790 0.047210
2 2 0.402764 0.147998 0.938484 0.061516

Optionally you can set append=True to log results of consequent stages of training.

# don't forget to remove the old file
if learn.csv_logger.path.exists(): os.remove(learn.csv_logger.path)

learn = Learner(data, simple_cnn((3, 16, 16, 2)), metrics=[accuracy, error_rate],
                callback_fns=[partial(CSVLogger, append=True)])

# stage-1
learn.fit(3)
Total time: 00:02

epoch train_loss valid_loss accuracy error_rate time
0 0.604549 0.493241 0.766810 0.233190 00:00
1 0.486367 0.226189 0.948498 0.051502 00:00
2 0.377847 0.127142 0.958512 0.041488 00:00 
# stage-2
learn.fit(3)
Total time: 00:02

epoch train_loss valid_loss accuracy error_rate time
0 0.189441 0.118532 0.954220 0.045780 00:00
1 0.177432 0.110913 0.965665 0.034335 00:00
2 0.168626 0.097646 0.968526 0.031474 00:00 
learn.csv_logger.read_logged_file()
epoch train_loss valid_loss accuracy error_rate
0 0 0.604549 0.493241 0.766810 0.233190
1 1 0.486367 0.226189 0.948498 0.051502
2 2 0.377847 0.127142 0.958512 0.041488
3 epoch train_loss valid_loss accuracy error_rate
4 0 0.189441 0.118532 0.954220 0.045780
5 1 0.177432 0.110913 0.965665 0.034335
6 2 0.168626 0.097646 0.968526 0.031474

Calback methods

You don't call these yourself - they're called by fastai's Callback system automatically to enable the class's functionality.

on_train_begin[source][test]

on_train_begin(**kwargs:Any)

No tests found for on_train_begin. To contribute a test please refer to this guide and this discussion.

Prepare file with metric names.

on_epoch_end[source][test]

on_epoch_end(epoch:int, smooth_loss:Tensor, last_metrics:MetricsList, **kwargs:Any) → bool

No tests found for on_epoch_end. To contribute a test please refer to this guide and this discussion.

Add a line with epoch number, smooth_loss and last_metrics.

on_train_end[source][test]

on_train_end(**kwargs:Any)

No tests found for on_train_end. To contribute a test please refer to this guide and this discussion.

Close the file.