快速开始序贯（Sequential）模型

序贯模型是多个网络层的线性堆叠，也就是“一条路走到黑”。

可以通过向Sequential模型传递一个layer的list来构造该模型：

  1. from keras.models import Sequential
  2. from keras.layers import Dense, Activation
  4. model = Sequential([
  5. Dense(32, units=784),
  6. Activation('relu'),
  7. Dense(10),
  8. Activation('softmax'),
  9. ])

也可以通过.add()方法一个个的将layer加入模型中：

  1. model = Sequential()
  2. model.add(Dense(32, input_shape=(784,)))
  3. model.add(Activation('relu'))

指定输入数据的shape

模型需要知道输入数据的shape，因此，Sequential的第一层需要接受一个关于输入数据shape的参数，后面的各个层则可以自动的推导出中间数据的shape，因此不需要为每个层都指定这个参数。有几种方法来为第一层指定输入数据的shape

  • 传递一个input_shape的关键字参数给第一层，input_shape是一个tuple类型的数据，其中也可以填入None，如果填入None则表示此位置可能是任何正整数。数据的batch大小不应包含在其中。

  • 有些2D层，如Dense，支持通过指定其输入维度input_dim来隐含的指定输入数据shape,是一个Int类型的数据。一些3D的时域层支持通过参数input_dim和input_length来指定输入shape。

  • 如果你需要为输入指定一个固定大小的batch_size（常用于stateful RNN网络），可以传递batch_size参数到一个层中，例如你想指定输入张量的batch大小是32，数据shape是（6，8），则你需要传递batch_size=32和input_shape=(6,8)。

  1. model = Sequential()
  2. model.add(Dense(32, input_dim=784))
  1. model = Sequential()
  2. model.add(Dense(32, input_shape=(784,)))

编译

在训练模型之前，我们需要通过compile来对学习过程进行配置。compile接收三个参数：

  • 优化器optimizer：该参数可指定为已预定义的优化器名，如rmsprop、adagrad，或一个Optimizer类的对象，详情见optimizers

  • 损失函数loss：该参数为模型试图最小化的目标函数，它可为预定义的损失函数名，如categorical_crossentropy、mse，也可以为一个损失函数。详情见losses

  • 指标列表metrics：对分类问题，我们一般将该列表设置为metrics=['accuracy']。指标可以是一个预定义指标的名字,也可以是一个用户定制的函数.指标函数应该返回单个张量,或一个完成metric_name - > metric_value映射的字典.请参考性能评估

  1. # For a multi-class classification problem
  2. model.compile(optimizer='rmsprop',
  3.               loss='categorical_crossentropy',
  4.               metrics=['accuracy'])
  6. # For a binary classification problem
  7. model.compile(optimizer='rmsprop',
  8.               loss='binary_crossentropy',
  9.               metrics=['accuracy'])
  11. # For a mean squared error regression problem
  12. model.compile(optimizer='rmsprop',
  13.               loss='mse')
  15. # For custom metrics
  16. import keras.backend as K
  18. def mean_pred(y_true, y_pred):
  19.     return K.mean(y_pred)
  21. model.compile(optimizer='rmsprop',
  22.               loss='binary_crossentropy',
  23.               metrics=['accuracy', mean_pred])

训练

Keras以Numpy数组作为输入数据和标签的数据类型。训练模型一般使用fit函数，该函数的详情见这里。下面是一些例子。

  1. # For a single-input model with 2 classes (binary classification):
  3. model = Sequential()
  4. model.add(Dense(32, activation='relu', input_dim=100))
  5. model.add(Dense(1, activation='sigmoid'))
  6. model.compile(optimizer='rmsprop',
  7.               loss='binary_crossentropy',
  8.               metrics=['accuracy'])
  10. # Generate dummy data
  11. import numpy as np
  12. data = np.random.random((1000, 100))
  13. labels = np.random.randint(2, size=(1000, 1))
  15. # Train the model, iterating on the data in batches of 32 samples
  16. model.fit(data, labels, epochs=10, batch_size=32)
  1. # For a single-input model with 10 classes (categorical classification):
  3. model = Sequential()
  4. model.add(Dense(32, activation='relu', input_dim=100))
  5. model.add(Dense(10, activation='softmax'))
  6. model.compile(optimizer='rmsprop',
  7.               loss='categorical_crossentropy',
  8.               metrics=['accuracy'])
  10. # Generate dummy data
  11. import numpy as np
  12. data = np.random.random((1000, 100))
  13. labels = np.random.randint(10, size=(1000, 1))
  15. # Convert labels to categorical one-hot encoding
  16. one_hot_labels = keras.utils.to_categorical(labels, num_classes=10)
  18. # Train the model, iterating on the data in batches of 32 samples
  19. model.fit(data, one_hot_labels, epochs=10, batch_size=32)

例子

这里是一些帮助你开始的例子

在Keras代码包的examples文件夹中，你将找到使用真实数据的示例模型：

  • CIFAR10 小图片分类：使用CNN和实时数据提升
  • IMDB 电影评论观点分类：使用LSTM处理成序列的词语
  • Reuters（路透社）新闻主题分类：使用多层感知器（MLP）
  • MNIST手写数字识别：使用多层感知器和CNN
  • 字符级文本生成：使用LSTM…

基于多层感知器的softmax多分类：

  1. from keras.models import Sequential
  2. from keras.layers import Dense, Dropout, Activation
  3. from keras.optimizers import SGD
  5. # Generate dummy data
  6. import numpy as np
  7. x_train = np.random.random((1000, 20))
  8. y_train = keras.utils.to_categorical(np.random.randint(10, size=(1000, 1)), num_classes=10)
  9. x_test = np.random.random((100, 20))
  10. y_test = keras.utils.to_categorical(np.random.randint(10, size=(100, 1)), num_classes=10)
  12. model = Sequential()
  13. # Dense(64) is a fully-connected layer with 64 hidden units.
  14. # in the first layer, you must specify the expected input data shape:
  15. # here, 20-dimensional vectors.
  16. model.add(Dense(64, activation='relu', input_dim=20))
  17. model.add(Dropout(0.5))
  18. model.add(Dense(64, activation='relu'))
  19. model.add(Dropout(0.5))
  20. model.add(Dense(10, activation='softmax'))
  22. sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
  23. model.compile(loss='categorical_crossentropy',
  24.               optimizer=sgd,
  25.               metrics=['accuracy'])
  27. model.fit(x_train, y_train,
  28.           epochs=20,
  29.           batch_size=128)
  30. score = model.evaluate(x_test, y_test, batch_size=128)

MLP的二分类：

  1. import numpy as np
  2. from keras.models import Sequential
  3. from keras.layers import Dense, Dropout
  5. # Generate dummy data
  6. x_train = np.random.random((1000, 20))
  7. y_train = np.random.randint(2, size=(1000, 1))
  8. x_test = np.random.random((100, 20))
  9. y_test = np.random.randint(2, size=(100, 1))
  11. model = Sequential()
  12. model.add(Dense(64, input_dim=20, activation='relu'))
  13. model.add(Dropout(0.5))
  14. model.add(Dense(64, activation='relu'))
  15. model.add(Dropout(0.5))
  16. model.add(Dense(1, activation='sigmoid'))
  18. model.compile(loss='binary_crossentropy',
  19.               optimizer='rmsprop',
  20.               metrics=['accuracy'])
  21. model.fit(x_train, y_train,
  22.           epochs=20,
  23.           batch_size=128)
  24. score = model.evaluate(x_test, y_test, batch_size=128)

类似VGG的卷积神经网络：

  1. import numpy as np
  2. import keras
  3. from keras.models import Sequential
  4. from keras.layers import Dense, Dropout, Flatten
  5. from keras.layers import Conv2D, MaxPooling2D
  6. from keras.optimizers import SGD
  8. # Generate dummy data
  9. x_train = np.random.random((100, 100, 100, 3))
  10. y_train = keras.utils.to_categorical(np.random.randint(10, size=(100, 1)), num_classes=10)
  11. x_test = np.random.random((20, 100, 100, 3))
  12. y_test = keras.utils.to_categorical(np.random.randint(10, size=(20, 1)), num_classes=10)
  14. model = Sequential()
  15. # input: 100x100 images with 3 channels -> (100, 100, 3) tensors.
  16. # this applies 32 convolution filters of size 3x3 each.
  17. model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(100, 100, 3)))
  18. model.add(Conv2D(32, (3, 3), activation='relu'))
  19. model.add(MaxPooling2D(pool_size=(2, 2)))
  20. model.add(Dropout(0.25))
  22. model.add(Conv2D(64, (3, 3), activation='relu'))
  23. model.add(Conv2D(64, (3, 3), activation='relu'))
  24. model.add(MaxPooling2D(pool_size=(2, 2)))
  25. model.add(Dropout(0.25))
  27. model.add(Flatten())
  28. model.add(Dense(256, activation='relu'))
  29. model.add(Dropout(0.5))
  30. model.add(Dense(10, activation='softmax'))
  32. sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
  33. model.compile(loss='categorical_crossentropy', optimizer=sgd)
  35. model.fit(x_train, y_train, batch_size=32, epochs=10)
  36. score = model.evaluate(x_test, y_test, batch_size=32)

使用LSTM的序列分类

  1. from keras.models import Sequential
  2. from keras.layers import Dense, Dropout
  3. from keras.layers import Embedding
  4. from keras.layers import LSTM
  6. model = Sequential()
  7. model.add(Embedding(max_features, output_dim=256))
  8. model.add(LSTM(128))
  9. model.add(Dropout(0.5))
  10. model.add(Dense(1, activation='sigmoid'))
  12. model.compile(loss='binary_crossentropy',
  13.               optimizer='rmsprop',
  14.               metrics=['accuracy'])
  16. model.fit(x_train, y_train, batch_size=16, epochs=10)
  17. score = model.evaluate(x_test, y_test, batch_size=16)

使用1D卷积的序列分类

  1. from keras.models import Sequential
  2. from keras.layers import Dense, Dropout
  3. from keras.layers import Embedding
  4. from keras.layers import Conv1D, GlobalAveragePooling1D, MaxPooling1D
  6. model = Sequential()
  7. model.add(Conv1D(64, 3, activation='relu', input_shape=(seq_length, 100)))
  8. model.add(Conv1D(64, 3, activation='relu'))
  9. model.add(MaxPooling1D(3))
  10. model.add(Conv1D(128, 3, activation='relu'))
  11. model.add(Conv1D(128, 3, activation='relu'))
  12. model.add(GlobalAveragePooling1D())
  13. model.add(Dropout(0.5))
  14. model.add(Dense(1, activation='sigmoid'))
  16. model.compile(loss='binary_crossentropy',
  17.               optimizer='rmsprop',
  18.               metrics=['accuracy'])
  20. model.fit(x_train, y_train, batch_size=16, epochs=10)
  21. score = model.evaluate(x_test, y_test, batch_size=16)

用于序列分类的栈式LSTM

在该模型中，我们将三个LSTM堆叠在一起，是该模型能够学习更高层次的时域特征表示。

开始的两层LSTM返回其全部输出序列，而第三层LSTM只返回其输出序列的最后一步结果，从而其时域维度降低（即将输入序列转换为单个向量）

regular_stacked_lstm

  1. from keras.models import Sequential
  2. from keras.layers import LSTM, Dense
  3. import numpy as np
  5. data_dim = 16
  6. timesteps = 8
  7. num_classes = 10
  9. # expected input data shape: (batch_size, timesteps, data_dim)
  10. model = Sequential()
  11. model.add(LSTM(32, return_sequences=True,
  12.                input_shape=(timesteps, data_dim)))  # returns a sequence of vectors of dimension 32
  13. model.add(LSTM(32, return_sequences=True))  # returns a sequence of vectors of dimension 32
  14. model.add(LSTM(32))  # return a single vector of dimension 32
  15. model.add(Dense(10, activation='softmax'))
  17. model.compile(loss='categorical_crossentropy',
  18.               optimizer='rmsprop',
  19.               metrics=['accuracy'])
  21. # Generate dummy training data
  22. x_train = np.random.random((1000, timesteps, data_dim))
  23. y_train = np.random.random((1000, num_classes))
  25. # Generate dummy validation data
  26. x_val = np.random.random((100, timesteps, data_dim))
  27. y_val = np.random.random((100, num_classes))
  29. model.fit(x_train, y_train,
  30.           batch_size=64, epochs=5,
  31.           validation_data=(x_val, y_val))

采用stateful LSTM的相同模型

stateful LSTM的特点是，在处理过一个batch的训练数据后，其内部状态（记忆）会被作为下一个batch的训练数据的初始状态。状态LSTM使得我们可以在合理的计算复杂度内处理较长序列

请FAQ中关于stateful LSTM的部分获取更多信息

  1. from keras.models import Sequential
  2. from keras.layers import LSTM, Dense
  3. import numpy as np
  5. data_dim = 16
  6. timesteps = 8
  7. num_classes = 10
  8. batch_size = 32
  10. # Expected input batch shape: (batch_size, timesteps, data_dim)
  11. # Note that we have to provide the full batch_input_shape since the network is stateful.
  12. # the sample of index i in batch k is the follow-up for the sample i in batch k-1.
  13. model = Sequential()
  14. model.add(LSTM(32, return_sequences=True, stateful=True,
  15.                batch_input_shape=(batch_size, timesteps, data_dim)))
  16. model.add(LSTM(32, return_sequences=True, stateful=True))
  17. model.add(LSTM(32, stateful=True))
  18. model.add(Dense(10, activation='softmax'))
  20. model.compile(loss='categorical_crossentropy',
  21.               optimizer='rmsprop',
  22.               metrics=['accuracy'])
  24. # Generate dummy training data
  25. x_train = np.random.random((batch_size * 10, timesteps, data_dim))
  26. y_train = np.random.random((batch_size * 10, num_classes))
  28. # Generate dummy validation data
  29. x_val = np.random.random((batch_size * 3, timesteps, data_dim))
  30. y_val = np.random.random((batch_size * 3, num_classes))
  32. model.fit(x_train, y_train,
  33.           batch_size=batch_size, epochs=5, shuffle=False,
  34.           validation_data=(x_val, y_val))