LRScheduler

class paddle.callbacks.LRScheduler ( by_step=True, by_epoch=False )

LRScheduler 是一个学习率回调函数。

参数：

  • by_step (bool，可选) - 是否每个step都更新学习率。默认值：True。

  • by_epoch (bool，可选) - 是否每个epoch都更新学习率。默认值：False。

代码示例：

  1. import paddle
  2. import paddle.vision.transforms as T
  3. from paddle.static import InputSpec
  5. inputs = [InputSpec([-1, 1, 28, 28], 'float32', 'image')]
  6. labels = [InputSpec([None, 1], 'int64', 'label')]
  8. transform = T.Compose([
  9.     T.Transpose(),
  10.     T.Normalize([127.5], [127.5])
  11. ])
  12. train_dataset = paddle.vision.datasets.MNIST(mode='train', transform=transform)
  14. lenet = paddle.vision.LeNet()
  15. model = paddle.Model(lenet,
  16.     inputs, labels)
  18. base_lr = 1e-3
  19. boundaries = [5, 8]
  20. wamup_steps = 4
  22. def make_optimizer(parameters=None):
  23.     momentum = 0.9
  24.     weight_decay = 5e-4
  25.     values = [base_lr * (0.1**i) for i in range(len(boundaries) + 1)]
  26.     learning_rate = paddle.optimizer.lr.PiecewiseDecay(
  27.         boundaries=boundaries, values=values)
  28.     learning_rate = paddle.optimizer.lr.LinearWarmup(
  29.         learning_rate=learning_rate,
  30.         warmup_steps=wamup_steps,
  31.         start_lr=base_lr / 5.,
  32.         end_lr=base_lr,
  33.         verbose=True)
  34.     optimizer = paddle.optimizer.Momentum(
  35.         learning_rate=learning_rate,
  36.         weight_decay=weight_decay,
  37.         momentum=momentum,
  38.         parameters=parameters)
  39.     return optimizer
  41. optim = make_optimizer(parameters=lenet.parameters())
  42. model.prepare(optimizer=optim,
  43.             loss=paddle.nn.CrossEntropyLoss(),
  44.             metrics=paddle.metric.Accuracy())
  46. # if LRScheduler callback not set, an instance LRScheduler update by step
  47. # will be created auto.
  48. model.fit(train_dataset, batch_size=64)
  50. # create a learning rate scheduler update by epoch
  51. callback = paddle.callbacks.LRScheduler(by_step=False, by_epoch=True)
  52. model.fit(train_dataset, batch_size=64, callbacks=callback)