Clean up

原文：https://docs.gitlab.com/ee/raketasks/cleanup.html

Clean up

GitLab 提供 Rake 任务来清理 GitLab 实例.

Remove unreferenced LFS files

在 GitLab 12.10 中引入 .

危险：不要在 GitLab 升级后的 12 小时内运行此程序. 这是为了确保所有后台迁移均已完成，否则可能导致数据丢失.

当您从存储库的历史记录中删除 LFS 文件时，它们将变成孤立文件并继续占用磁盘空间. 使用此 Rake 任务，您可以从数据库中删除无效的引用，这将允许对 LFS 文件进行垃圾回收.

例如：

  1. # omnibus-gitlab
  2. sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_PATH="gitlab-org/gitlab-foss"
  4. # installation from source
  5. bundle exec rake gitlab:cleanup:orphan_lfs_file_references RAILS_ENV=production PROJECT_PATH="gitlab-org/gitlab-foss"

您也可以使用PROJECT_ID而不是PROJECT_PATH指定项目.

例如：

  1. $ sudo gitlab-rake gitlab:cleanup:orphan_lfs_file_references PROJECT_PATH="gitlab-org/gitlab-foss"
  2. I, [2019-12-13T16:35:31.764962 #82356]  INFO -- :  Looking for orphan LFS files for project GitLab Org / GitLab Foss
  3. I, [2019-12-13T16:35:31.923659 #82356]  INFO -- :  Removed invalid references: 12

默认情况下，此任务不会删除任何内容，但会显示它可以删除多少个文件引用. 如果您确实要删除引用，请运行DRY_RUN=false的命令. 您还可以使用LIMIT={number}参数来限制已删除引用的数量.

请注意，此 Rake 任务仅删除对 LFS 文件的引用. 未引用的 LFS 文件将在以后（每天一次）进行垃圾收集. 如果您需要立即进行垃圾回收，请运行以下描述的rake gitlab:cleanup:orphan_lfs_files .

Remove unreferenced LFS files immediately

在 GitLab 12.10 中引入 .

每天都会删除未引用的 LFS 文件，但是如果需要，您可以立即删除它们. 例如：

  1. # omnibus-gitlab
  2. sudo gitlab-rake gitlab:cleanup:orphan_lfs_files
  4. # installation from source
  5. bundle exec rake gitlab:cleanup:orphan_lfs_files

输出示例：

  1. $ sudo gitlab-rake gitlab:cleanup:orphan_lfs_files
  2. I, [2020-01-08T20:51:17.148765 #43765]  INFO -- : Removed unreferenced LFS files: 12

Clean up project upload files

如果项目上传文件在 GitLab 数据库中不存在，请清理它们.

Clean up project upload files from filesystem

在 GitLab 11.2 中引入 .

如果本地项目上传文件在 GitLab 数据库中不存在，请清理它们. 如果该任务可以找到其项目，则该任务将尝试修复该文件，否则，它将文件移至丢失并找到的目录.

  1. # omnibus-gitlab
  2. sudo gitlab-rake gitlab:cleanup:project_uploads
  4. # installation from source
  5. bundle exec rake gitlab:cleanup:project_uploads RAILS_ENV=production

输出示例：

  1. $ sudo gitlab-rake gitlab:cleanup:project_uploads
  3. I, [2018-07-27T12:08:27.671559 #89817]  INFO -- : Looking for orphaned project uploads to clean up. Dry run...
  4. D, [2018-07-27T12:08:28.293568 #89817] DEBUG -- : Processing batch of 500 project upload file paths, starting with /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out
  5. I, [2018-07-27T12:08:28.689869 #89817]  INFO -- : Can move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/test.out
  6. I, [2018-07-27T12:08:28.755624 #89817]  INFO -- : Can fix /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/qux/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
  7. I, [2018-07-27T12:08:28.760257 #89817]  INFO -- : Can move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png
  8. I, [2018-07-27T12:08:28.764470 #89817]  INFO -- : To cleanup these files run this command with DRY_RUN=false
  10. $ sudo gitlab-rake gitlab:cleanup:project_uploads DRY_RUN=false I, [2018-07-27T12:08:32.944414 #89936]  INFO -- : Looking for orphaned project uploads to clean up...
  11. D, [2018-07-27T12:08:33.293568 #89817] DEBUG -- : Processing batch of 500 project upload file paths, starting with /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out
  12. I, [2018-07-27T12:08:33.689869 #89817]  INFO -- : Did move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/test.out -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/test.out
  13. I, [2018-07-27T12:08:33.755624 #89817]  INFO -- : Did fix /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/qux/foo/bar/89a0f7b0b97008a4a18cedccfdcd93fb/foo.txt
  14. I, [2018-07-27T12:08:33.760257 #89817]  INFO -- : Did move to lost and found /opt/gitlab/embedded/service/gitlab-rails/public/uploads/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png -> /opt/gitlab/embedded/service/gitlab-rails/public/uploads/-/project-lost-found/foo/bar/1dd6f0f7eefd2acc4c2233f89a0f7b0b/image.png

Clean up project upload files from object storage

在 GitLab 11.2 中引入 .

如果对象存储库上载文件在 GitLab 数据库中不存在，请将其移动到丢失和找到的目录中.

  1. # omnibus-gitlab
  2. sudo gitlab-rake gitlab:cleanup:remote_upload_files
  4. # installation from source
  5. bundle exec rake gitlab:cleanup:remote_upload_files RAILS_ENV=production

Example output:

  1. $ sudo gitlab-rake gitlab:cleanup:remote_upload_files
  3. I, [2018-08-02T10:26:13.995978 #45011]  INFO -- : Looking for orphaned remote uploads to remove. Dry run...
  4. I, [2018-08-02T10:26:14.120400 #45011]  INFO -- : Can be moved to lost and found: @hashed/6b/DSC_6152.JPG
  5. I, [2018-08-02T10:26:14.120482 #45011]  INFO -- : Can be moved to lost and found: @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg
  6. I, [2018-08-02T10:26:14.120634 #45011]  INFO -- : To cleanup these files run this command with DRY_RUN=false
  1. $ sudo gitlab-rake gitlab:cleanup:remote_upload_files DRY_RUN=false I, [2018-08-02T10:26:47.598424 #45087]  INFO -- : Looking for orphaned remote uploads to remove...
  2. I, [2018-08-02T10:26:47.753131 #45087]  INFO -- : Moved to lost and found: @hashed/6b/DSC_6152.JPG -> lost_and_found/@hashed/6b/DSC_6152.JPG
  3. I, [2018-08-02T10:26:47.764356 #45087]  INFO -- : Moved to lost and found: @hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg -> lost_and_found/@hashed/79/02/7902699be42c8a8e46fbbb4501726517e86b22c56a189f7625a6da49081b2451/711491b29d3eb08837798c4909e2aa4d/DSC00314.jpg

Remove orphan artifact files

版本历史

如果发现磁盘上的作业工件文件多于应有的数量，则可以运行：

  1. sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files

该命令：

  • 扫描整个工件文件夹.
  • 检查哪些文件在数据库中仍然有记录.
  • 如果找不到数据库记录，那么将从磁盘中删除该文件.

默认情况下，此任务不会删除任何内容，但会显示它可以删除的内容. 如果您实际上要删除文件，请运行DRY_RUN=false的命令：

  1. sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files DRY_RUN=false

您还可以使用LIMIT要删除的文件数：

  1. sudo gitlab-rake gitlab:cleanup:orphan_job_artifact_files LIMIT=100

这只会从磁盘上删除多达 100 个文件. 您可以使用此功能删除一小集以进行测试.

如果提供DEBUG=1 ，则将看到被检测为孤立文件的每个文件的完整路径.

如果安装了ionice ，则任务将使用它来确保命令不会在磁盘上造成太大的负载. 您可以使用NICENESS配置NICENESS级别. 以下是有效水平，但请务必咨询man 1 ionice .

  • 0 or None
  • 1 or Realtime
  • 2 or Best-effort (default)
  • 3 or Idle

Remove expired ActiveSession lookup keys

在 GitLab 12.2 中引入 .

  1. # omnibus-gitlab
  2. sudo gitlab-rake gitlab:cleanup:sessions:active_sessions_lookup_keys
  4. # installation from source
  5. bundle exec rake gitlab:cleanup:sessions:active_sessions_lookup_keys RAILS_ENV=production

Container Registry garbage collection

容器注册表可以使用大量的磁盘空间. 为了清除未使用的层，注册表包括一个垃圾收集命令 .