Caffe（Convolutional Architecture for Fast Embedding）是一种清晰且高效的深度学习框架，具有易上手、速度快、模块化、开放性和社区好等优势。

版本说明

Pycaffe 组件内核是 Caffe 1.0 版本。Pycaffe 组件中使用的 Python 版本和支持的第三方模块信息如下：

  • Python 2.7.12
  • SciPy 0.17.0
  • NumPy 1.11.0

如果您需要使用其他第三方的 lib，可使用 pip 在代码内安装，示例如下：

  1. import pip
  2. pip.main(['install', "package_name"])

操作步骤

  • 添加组件从左侧菜单栏中，选择【组件】>【深度学习】列表下的【PyCaffe】节点，将其拖拽至画布中。
  • 配置参数
    • 脚本及依赖包文件上传：将任务脚本上传至程序脚本框。如果需要依赖文件，则压缩为 zip 文件后通过 依赖包文件 框上传。4.2.4. PyCaffe组件 - 图1
    • 程序依赖：指定位于 COS 中的用户依赖文件路径，指定内容将被拷贝到程序脚本同一级目录下。支持目录或者文件依赖，若指定多个文件则以英文逗号分隔 。
    • 程序参数：指定运行任务脚本的参数。4.2.4. PyCaffe组件 - 图2
  • 配置资源在【资源参数】列表框中配置任务的资源参数。4.2.4. PyCaffe组件 - 图3
  • 运行单击【保存】并运行工作流。
  • 查看 PyCaffe 控制台和日志在 PyCaffe 节点上单击右键，可查看任务状态和详细日志。 详细日志如下：

4.2.4. PyCaffe组件 - 图4

stdout.log 为全部日志，stderr.log 为错误日志。

示例

以下代码展示了在 PyCaffe 框架中，构建一个卷积神经网络（Lenet）的方法。

  1. import os
  2. import sys
  3. import shlex
  4. import subprocess
  5. caffe_root="/caffe"
  6. os.chdir(caffe_root)
  7. sys.path.insert(0, os.path.join(caffe_root,"python"))
  8. import caffe
  10. # transform data into lmdb form
  11. os.system('bash %s/create_mnist.sh' % 'cos_path_to_shell_scripts')
  13. save_path = '/cos_person/cos_path_to_output_model'
  14. train_net_path = '%s/lenet_train.prototxt' % save_path
  15. test_net_path = '%s/lenet_test.prototxt' % save_path
  16. deploy_net_path = '%s/lenet_deploy.prototxt' % save_path
  17. solver_file= '%s/lenet_solver.prototxt' % save_path
  19. lmdb_data_path = 'path_to_lenet_lmdb' # better not in cos for now
  20. train_lmdb_path = '%s/mnist_train_lmdb' % lmdb_data_path
  21. test_lmdb_path = '%s/mnist_test_lmdb' % lmdb_data_path
  23. lenet_snapshot_prefix = '%s/lenet' % save_path
  25. from caffe import layers as L, params as P
  27. def lenet(lmdb, batch_size):
  28.     # our version of LeNet: a series of linear and simple nonlinear transformations
  29.     n = caffe.NetSpec()
  31.     n.data, n.label = L.Data(batch_size=batch_size, backend=P.Data.LMDB, source=lmdb,
  32.                              transform_param=dict(scale=1./255), ntop=2)
  34.     n.conv1 = L.Convolution(n.data, kernel_size=5, num_output=20, weight_filler=dict(type='xavier'))
  35.     n.pool1 = L.Pooling(n.conv1, kernel_size=2, stride=2, pool=P.Pooling.MAX)
  36.     n.conv2 = L.Convolution(n.pool1, kernel_size=5, num_output=50, weight_filler=dict(type='xavier'))
  37.     n.pool2 = L.Pooling(n.conv2, kernel_size=2, stride=2, pool=P.Pooling.MAX)
  38.     n.fc1 =   L.InnerProduct(n.pool2, num_output=500, weight_filler=dict(type='xavier'))
  39.     n.relu1 = L.ReLU(n.fc1, in_place=True)
  40.     n.score = L.InnerProduct(n.relu1, num_output=10, weight_filler=dict(type='xavier'))
  41.     n.loss =  L.SoftmaxWithLoss(n.score, n.label)
  43.     return n.to_proto()
  45. with open(train_net_path, 'w') as f:
  46.     f.write(str(lenet(train_lmdb_path, 64)))    
  47. with open(test_net_path, 'w') as f:
  48.     f.write(str(lenet(test_lmdb_path, 100)))
  50. deploy_net = lenet(test_lmdb_path, 100)
  51. del deploy_net.layer[0]
  52. with open(deploy_net_path, 'w') as f:
  53.     f.write(str(deploy_net))    
  55. from caffe.proto import caffe_pb2
  56. s = caffe_pb2.SolverParameter()
  58. # Set a seed for reproducible experiments:
  59. # this controls for randomization in training.
  60. s.random_seed = 0xCAFFE
  62. # Specify locations of the train and (maybe) test networks.
  63. s.train_net = train_net_path
  64. s.test_net.append(test_net_path)
  65. s.test_interval = 500  # Test after every 500 training iterations.
  66. s.test_iter.append(100) # Test on 100 batches each time we test.
  68. s.max_iter = 10000     # no. of times to update the net (training iterations)
  70. # EDIT HERE to try different solvers
  71. # solver types include "SGD", "Adam", and "Nesterov" among others.
  72. s.type = "SGD"
  74. # Set the initial learning rate for SGD.
  75. s.base_lr = 0.01  # EDIT HERE to try different learning rates
  76. # Set momentum to accelerate learning by
  77. # taking weighted average of current and previous updates.
  78. s.momentum = 0.9
  79. # Set weight decay to regularize and prevent overfitting
  80. s.weight_decay = 5e-4
  82. # Set `lr_policy` to define how the learning rate changes during training.
  83. # This is the same policy as our default LeNet.
  84. s.lr_policy = 'inv'
  85. s.gamma = 0.0001
  86. s.power = 0.75
  87. # EDIT HERE to try the fixed rate (and compare with adaptive solvers)
  88. # `fixed` is the simplest policy that keeps the learning rate constant.
  89. # s.lr_policy = 'fixed'
  91. # Display the current training loss and accuracy every 1000 iterations.
  92. s.display = 1000
  94. # Snapshots are files used to store networks we've trained.
  95. # We'll snapshot every 5K iterations -- twice during training.
  96. s.snapshot = 5000
  97. s.snapshot_prefix = lenet_snapshot_prefix
  99. # Train on the GPU
  100. s.solver_mode = caffe_pb2.SolverParameter.GPU
  102. # Write the solver to a file.
  103. with open(solver_file, 'w') as f:
  104.     f.write(str(s))
  106. # Train
  107. caffe.set_device(0)
  108. caffe.set_mode_gpu()
  109. solver = caffe.SGDSolver(solver_file)
  110. solver.solve()