监控REST API

Flink有一个监控API,可用于查询正在运行的作业的状态和统计信息,以及最近完成的作业。此监视API由Flink自己的仪表板使用,但也可用于自定义监视工具。

监控API是一个REST-ful API,它接受HTTP请求并使用JSON数据进行响应。

概览

监视API由作为Dispatcher一部分运行的Web服务器提供支持默认情况下,此服务器在post处侦听8081,可以在flink-conf.yamlvia中配置rest.port请注意,监视API Web服务器和Web仪表板Web服务器当前是相同的,因此在同一端口上一起运行。但是,它们会响应不同的HTTP URL。

对于多个Dispatchers(用于高可用性),每个Dispatcher将运行其自己的监视API实例,该实例提供有关已完成和正在运行的作业的信息,同时Dispatcher被选为集群Leader。

开发

REST API后台位于flink-runtime项目中。核心类是org.apache.flink.runtime.webmonitor.WebMonitorEndpoint设置服务器和请求路由。

我们使用NettyNetty Router库来处理REST请求和转换URL。之所以做出这样的选择是因为这种组合具有轻量级依赖性,并且Netty HTTP的性能非常好。

要添加新请求,Required

  • 添加一个新MessageHeaders类,作为新请求的接口,
  • 添加一个新AbstractRestHandler类,根据添加的类处理请求MessageHeaders
  • 添加处理程序org.apache.flink.runtime.webmonitor.WebMonitorEndpoint#initializeHandlers()一个很好的例子就是org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler使用了org.apache.flink.runtime.rest.messages.JobExceptionsHeaders

API

REST API是版本化的,通过在url前加上版本前缀,可以查询特定版本。前缀始终是形式v[version_number]例如,要访问/foo/bar一个版本1 将查询/v1/foo/bar

如果未指定版本,Flink将默认为支持请求最旧版本。

查询不受支持/不存在的版本将返回404错误。

注意如果群集在传统模式下运行,则REST API版本控制无效对于这种情况,请参阅下面的遗留API。

调度员

##### /cluster
动作: DELETE响应代码: 200 OK
关闭群集
  1. {}
  1. {}
##### /config
动作: GET响应代码: 200 OK
返回WebUI的配置。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:DashboardConfiguration", "properties" : { "refresh-interval" : { "type" : "integer" }, "timezone-name" : { "type" : "string" }, "timezone-offset" : { "type" : "integer" }, "flink-version" : { "type" : "string" }, "flink-revision" : { "type" : "string" } }}
##### /jars
动作: GET响应代码: 200 OK
返回先前通过'/ jars / upload'上传的所有jar的列表。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarListInfo", "properties" : { "address" : { "type" : "string" }, "files" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarListInfo:JarFileInfo", "properties" : { "id" : { "type" : "string" }, "name" : { "type" : "string" }, "uploaded" : { "type" : "integer" }, "entry" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarListInfo:JarEntryInfo", "properties" : { "name" : { "type" : "string" }, "description" : { "type" : "string" } } } } } } } }}
##### /jars/upload
动作: POST响应代码: 200 OK
将jar上传到集群。jar必须作为多部分数据发送。确保“Content-Type”标头设置为“application / x-java-archive”,因为某些http库默认情况下不添加标头。使用'curl'你可以通过'curl -X POST -H“上传jar。期望:” - F“jarfile =#path / to / flink-job.jar”http:// hostname:port / jars / upload'。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarUploadResponseBody", "properties" : { "filename" : { "type" : "string" }, "status" : { "type" : "string", "enum" : [ "success" ] } }}
##### /jars/:jarid
动作: DELETE响应代码: 200 OK
删除之前通过'/ jars / upload'上传的jar。
路径参数
- jarid - 标识jar的字符串值。上传jar时会返回一个路径,其中文件名是ID。此值等同于上载的jar(/ jars)列表中的id字段。
  1. {}
  1. {}
##### /jars/:jarid /plan
动作: GET响应代码: 200 OK
返回先前通过'/ jars / upload'上传的jar中包含的作业的数据流计划。
路径参数
- jarid - 标识jar的字符串值。上传jar时会返回一个路径,其中文件名是ID。此值等同于上载的jar(/ jars)列表中的id字段。
查询参数
- entry-class(可选):字符串值,指定入口点类的完全限定名称。覆盖jar文件清单中定义的类。- parallelism (可选):正整数值,指定作业所需的并行度。- program-args (可选):字符串值,指定程序或计划的参数。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobPlanInfo", "properties" : { "plan" : { "type" : "any" } }}
##### /jars/:jarid /run
动作: POST响应代码: 200 OK
通过运行先前通过'/ jars / upload'上传的jar来提交作业。
路径参数
- jarid - 标识jar的字符串值。上传jar时会返回一个路径,其中文件名是ID。此值等同于上载的jar(/ jars)列表中的id字段。
查询参数
- program-args (可选):字符串值,指定程序或计划的参数。- entry-class(可选):字符串值,指定入口点类的完全限定名称。覆盖jar文件清单中定义的类。- parallelism (可选):正整数值,指定作业所需的并行度。- allowNonRestoredState (可选):布尔值,指定如果保存点包含无法映射回作业的状态,是否应拒绝作业提交。- savepointPath (可选):字符串值,指定要从中还原作业的保存点的路径。
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarRunRequestBody", "properties" : { "entryClass" : { "type" : "string" }, "programArgs" : { "type" : "string" }, "parallelism" : { "type" : "integer" }, "allowNonRestoredState" : { "type" : "boolean" }, "savepointPath" : { "type" : "string" } }}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:webmonitor:handlers:JarRunResponseBody", "properties" : { "jobid" : { "type" : "any" } }}
##### /jobmanager/config
动作: GET响应代码: 200 OK
返回群集配置。
  1. {}
  1. { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:ClusterConfigurationInfoEntry", "properties" : { "key" : { "type" : "string" }, "value" : { "type" : "string" } } }}
##### /jobmanager/metrics
动作: GET响应代码: 200 OK
提供对JobManager指标的访问。
查询参数
- get (可选):以逗号分隔的字符串值列表,用于选择特定指标。
  1. {}
  1. { "type" : "any"}
##### /jobs
动作: GET响应代码: 200 OK
返回所有作业及其当前状态的概述。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:messages:webmonitor:JobIdsWithStatusOverview", "properties" : { "jobs" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:messages:webmonitor:JobIdsWithStatusOverview:JobIdWithStatus", "properties" : { "id" : { "type" : "any" }, "status" : { "type" : "string", "enum" : [ "CREATED", "RUNNING", "FAILING", "FAILED", "CANCELLING", "CANCELED", "FINISHED", "RESTARTING", "SUSPENDING", "SUSPENDED", "RECONCILING" ] } } } } }}
##### /jobs
动作: POST响应代码: 202 Accepted
提交工作。此调用主要供Flink客户端使用。此调用需要一个multipart / form-data请求,其中包含序列化JobGraph,jar和分布式缓存工件的文件上载以及JSON有效负载的名为“request”的属性。
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobSubmitRequestBody", "properties" : { "jobGraphFileName" : { "type" : "string" }, "jobJarFileNames" : { "type" : "array", "items" : { "type" : "string" } }, "jobArtifactFileNames" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobSubmitRequestBody:DistributedCacheFile", "properties" : { "entryName" : { "type" : "string" }, "fileName" : { "type" : "string" } } } } }}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobSubmitResponseBody", "properties" : { "jobUrl" : { "type" : "string" } }}
##### /jobs/metrics
动作: GET响应代码: 200 OK
提供对聚合作业指标的访问。
查询参数
- get (可选):以逗号分隔的字符串值列表,用于选择特定指标。- agg(可选):应该计算的以逗号分隔的聚合模式列表。可用的聚合是:“min,max,sum,avg”。- jobs (可选):以逗号分隔的32个字符的十六进制字符串列表,用于选择特定的作业。
  1. {}
  1. { "type" : "any"}
##### /jobs/overview
动作: GET响应代码: 200 OK
返回所有作业的概述。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:messages:webmonitor:MultipleJobsDetails", "properties" : { "jobs" : { "type" : "array", "items" : { "type" : "any" } } }}
##### /jobs/:jobid
动作: GET响应代码: 200 OK
返回作业的详细信息。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobDetailsInfo", "properties" : { "jid" : { "type" : "any" }, "name" : { "type" : "string" }, "isStoppable" : { "type" : "boolean" }, "state" : { "type" : "string", "enum" : [ "CREATED", "RUNNING", "FAILING", "FAILED", "CANCELLING", "CANCELED", "FINISHED", "RESTARTING", "SUSPENDING", "SUSPENDED", "RECONCILING" ] }, "start-time" : { "type" : "integer" }, "end-time" : { "type" : "integer" }, "duration" : { "type" : "integer" }, "now" : { "type" : "integer" }, "timestamps" : { "type" : "object", "additionalProperties" : { "type" : "integer" } }, "vertices" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobDetailsInfo:JobVertexDetailsInfo", "properties" : { "id" : { "type" : "any" }, "name" : { "type" : "string" }, "parallelism" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ] }, "start-time" : { "type" : "integer" }, "end-time" : { "type" : "integer" }, "duration" : { "type" : "integer" }, "tasks" : { "type" : "object", "additionalProperties" : { "type" : "integer" } }, "metrics" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo", "properties" : { "read-bytes" : { "type" : "integer" }, "read-bytes-complete" : { "type" : "boolean" }, "write-bytes" : { "type" : "integer" }, "write-bytes-complete" : { "type" : "boolean" }, "read-records" : { "type" : "integer" }, "read-records-complete" : { "type" : "boolean" }, "write-records" : { "type" : "integer" }, "write-records-complete" : { "type" : "boolean" } } } } } }, "status-counts" : { "type" : "object", "additionalProperties" : { "type" : "integer" } }, "plan" : { "type" : "string" } }}
##### /jobs/:jobid
动作: PATCH响应代码: 202 Accepted
终止工作。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
查询参数
- mode(可选):指定终止模式的字符串值。支持的值包括:“取消,停止”。
  1. {}
  1. {}
##### /jobs/:jobid /accumulators
动作: GET响应代码: 200 OK
返回作业的所有任务的累加器,汇总在各个子任务中。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
查询参数
- includeSerializedValue (可选):布尔值,指定序列化用户任务累加器是否应包含在响应中。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobAccumulatorsInfo", "properties" : { "job-accumulators" : { "type" : "array", "items" : { "type" : "any" } }, "user-task-accumulators" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobAccumulatorsInfo:UserTaskAccumulator", "properties" : { "name" : { "type" : "string" }, "type" : { "type" : "string" }, "value" : { "type" : "string" } } } }, "serialized-user-task-accumulators" : { "type" : "object", "additionalProperties" : { "type" : "any" } } }}
##### /jobs/:jobid /checkpoints
动作: GET响应代码: 200 OK
返回作业的检查点统计信息。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics", "properties" : { "counts" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics:Counts", "properties" : { "restored" : { "type" : "integer" }, "total" : { "type" : "integer" }, "in_progress" : { "type" : "integer" }, "completed" : { "type" : "integer" }, "failed" : { "type" : "integer" } } }, "summary" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics:Summary", "properties" : { "state_size" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics", "properties" : { "min" : { "type" : "integer" }, "max" : { "type" : "integer" }, "avg" : { "type" : "integer" } } }, "end_to_end_duration" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics" }, "alignment_buffered" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics" } } }, "latest" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics:LatestCheckpoints", "properties" : { "completed" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics:CompletedCheckpointStatistics", "properties" : { "id" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ] }, "is_savepoint" : { "type" : "boolean" }, "trigger_timestamp" : { "type" : "integer" }, "latest_ack_timestamp" : { "type" : "integer" }, "state_size" : { "type" : "integer" }, "end_to_end_duration" : { "type" : "integer" }, "alignment_buffered" : { "type" : "integer" }, "num_subtasks" : { "type" : "integer" }, "num_acknowledged_subtasks" : { "type" : "integer" }, "tasks" : { "type" : "object", "additionalProperties" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatistics", "properties" : { "id" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ] }, "latest_ack_timestamp" : { "type" : "integer" }, "state_size" : { "type" : "integer" }, "end_to_end_duration" : { "type" : "integer" }, "alignment_buffered" : { "type" : "integer" }, "num_subtasks" : { "type" : "integer" }, "num_acknowledged_subtasks" : { "type" : "integer" } } } }, "external_path" : { "type" : "string" }, "discarded" : { "type" : "boolean" } } }, "savepoint" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics:CompletedCheckpointStatistics" }, "failed" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics:FailedCheckpointStatistics", "properties" : { "id" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ] }, "is_savepoint" : { "type" : "boolean" }, "trigger_timestamp" : { "type" : "integer" }, "latest_ack_timestamp" : { "type" : "integer" }, "state_size" : { "type" : "integer" }, "end_to_end_duration" : { "type" : "integer" }, "alignment_buffered" : { "type" : "integer" }, "num_subtasks" : { "type" : "integer" }, "num_acknowledged_subtasks" : { "type" : "integer" }, "tasks" : { "type" : "object", "additionalProperties" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatistics" } }, "failure_timestamp" : { "type" : "integer" }, "failure_message" : { "type" : "string" } } }, "restored" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointingStatistics:RestoredCheckpointStatistics", "properties" : { "id" : { "type" : "integer" }, "restore_timestamp" : { "type" : "integer" }, "is_savepoint" : { "type" : "boolean" }, "external_path" : { "type" : "string" } } } } }, "history" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics", "properties" : { "id" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ] }, "is_savepoint" : { "type" : "boolean" }, "trigger_timestamp" : { "type" : "integer" }, "latest_ack_timestamp" : { "type" : "integer" }, "state_size" : { "type" : "integer" }, "end_to_end_duration" : { "type" : "integer" }, "alignment_buffered" : { "type" : "integer" }, "num_subtasks" : { "type" : "integer" }, "num_acknowledged_subtasks" : { "type" : "integer" }, "tasks" : { "type" : "object", "additionalProperties" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatistics" } } } } } }}
##### /jobs/:jobid /checkpoints/config
动作: GET响应代码: 200 OK
返回检查点配置。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointConfigInfo", "properties" : { "mode" : { "type" : "any" }, "interval" : { "type" : "integer" }, "timeout" : { "type" : "integer" }, "min_pause" : { "type" : "integer" }, "max_concurrent" : { "type" : "integer" }, "externalization" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointConfigInfo:ExternalizedCheckpointInfo", "properties" : { "enabled" : { "type" : "boolean" }, "delete_on_cancellation" : { "type" : "boolean" } } } }}
##### /jobs/:jobid /checkpoints/details/:checkpointid
动作: GET响应代码: 200 OK
返回检查点的详细信息。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- checkpointid - 标识检查点的长值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:CheckpointStatistics", "properties" : { "id" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ] }, "is_savepoint" : { "type" : "boolean" }, "trigger_timestamp" : { "type" : "integer" }, "latest_ack_timestamp" : { "type" : "integer" }, "state_size" : { "type" : "integer" }, "end_to_end_duration" : { "type" : "integer" }, "alignment_buffered" : { "type" : "integer" }, "num_subtasks" : { "type" : "integer" }, "num_acknowledged_subtasks" : { "type" : "integer" }, "tasks" : { "type" : "object", "additionalProperties" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatistics", "properties" : { "id" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ] }, "latest_ack_timestamp" : { "type" : "integer" }, "state_size" : { "type" : "integer" }, "end_to_end_duration" : { "type" : "integer" }, "alignment_buffered" : { "type" : "integer" }, "num_subtasks" : { "type" : "integer" }, "num_acknowledged_subtasks" : { "type" : "integer" } } } } }}
##### /jobs/:jobid /checkpoints/details/:checkpointid /subtasks/:vertexid
动作: GET响应代码: 200 OK
返回任务及其子任务的检查点统计信息。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- checkpointid - 标识检查点的长值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatisticsWithSubtaskDetails", "properties" : { "id" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "IN_PROGRESS", "COMPLETED", "FAILED" ] }, "latest_ack_timestamp" : { "type" : "integer" }, "state_size" : { "type" : "integer" }, "end_to_end_duration" : { "type" : "integer" }, "alignment_buffered" : { "type" : "integer" }, "num_subtasks" : { "type" : "integer" }, "num_acknowledged_subtasks" : { "type" : "integer" }, "summary" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatisticsWithSubtaskDetails:Summary", "properties" : { "state_size" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics", "properties" : { "min" : { "type" : "integer" }, "max" : { "type" : "integer" }, "avg" : { "type" : "integer" } } }, "end_to_end_duration" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics" }, "checkpoint_duration" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatisticsWithSubtaskDetails:CheckpointDuration", "properties" : { "sync" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics" }, "async" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics" } } }, "alignment" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:TaskCheckpointStatisticsWithSubtaskDetails:CheckpointAlignment", "properties" : { "buffered" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics" }, "duration" : { "type" : "object", "$ref" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:MinMaxAvgStatistics" } } } } }, "subtasks" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:checkpoints:SubtaskCheckpointStatistics", "properties" : { "index" : { "type" : "integer" }, "status" : { "type" : "string" } } } } }}
##### /jobs/:jobid /config
动作: GET响应代码: 200 OK
返回作业的配置。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "any"}
##### /jobs/:jobid /exceptions
动作: GET响应代码: 200 OK
返回作业已观察到的不可恢复的异常。截断标志定义是否发生了更多异常,但未列出,因为否则响应会变得太大。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobExceptionsInfo", "properties" : { "root-exception" : { "type" : "string" }, "timestamp" : { "type" : "integer" }, "all-exceptions" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobExceptionsInfo:ExecutionExceptionInfo", "properties" : { "exception" : { "type" : "string" }, "task" : { "type" : "string" }, "location" : { "type" : "string" }, "timestamp" : { "type" : "integer" } } } }, "truncated" : { "type" : "boolean" } }}
##### /jobs/:jobid /execution-result
动作: GET响应代码: 200 OK
返回作业执行的结果。允许访问作业的执行时间以及此作业创建的所有累加器。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:JobExecutionResultResponseBody", "properties" : { "status" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:queue:QueueStatus", "required" : true, "properties" : { "id" : { "type" : "string", "required" : true, "enum" : [ "IN_PROGRESS", "COMPLETED" ] } } }, "job-execution-result" : { "type" : "any" } }}
##### /jobs/:jobid /metrics
动作: GET响应代码: 200 OK
提供对作业指标的访问。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
查询参数
- get (可选):以逗号分隔的字符串值列表,用于选择特定指标。
  1. {}
  1. { "type" : "any"}
##### /jobs/:jobid /plan
动作: GET响应代码: 200 OK
返回作业的数据流计划。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobPlanInfo", "properties" : { "plan" : { "type" : "any" } }}
##### /jobs/:jobid /rescaling
动作: PATCH响应代码: 200 OK
触发重新调整作业。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
查询参数
- parallelism (强制项):正整数值,指定所需的并行度。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:TriggerResponse", "properties" : { "request-id" : { "type" : "any" } }}
##### /jobs/:jobid /rescaling/:triggerid
动作: GET响应代码: 200 OK
返回重新缩放 算子操作的状态。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- triggerid - 32个字符的十六进制字符串,用于标识异步 算子操作触发器ID。返回ID然后触发 算子操作。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:AsynchronousOperationResult", "properties" : { "status" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:queue:QueueStatus", "properties" : { "id" : { "type" : "string", "required" : true, "enum" : [ "IN_PROGRESS", "COMPLETED" ] } } }, "operation" : { "type" : "any" } }}
##### /jobs/:jobid /savepoints
动作: POST响应代码: 202 Accepted
触发保存点,然后可选择取消作业。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:savepoints:SavepointTriggerRequestBody", "properties" : { "target-directory" : { "type" : "string" }, "cancel-job" : { "type" : "boolean" } }}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:TriggerResponse", "properties" : { "request-id" : { "type" : "any" } }}
##### /jobs/:jobid /savepoints/:triggerid
动作: GET响应代码: 200 OK
返回保存点 算子操作的状态。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- triggerid - 32个字符的十六进制字符串,用于标识异步 算子操作触发器ID。返回ID然后触发 算子操作。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:AsynchronousOperationResult", "properties" : { "status" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:queue:QueueStatus", "properties" : { "id" : { "type" : "string", "required" : true, "enum" : [ "IN_PROGRESS", "COMPLETED" ] } } }, "operation" : { "type" : "any" } }}
##### /jobs/:jobid /vertices/:vertexid
动作: GET响应代码: 200 OK
返回任务的详细信息,并为每个子任务提供摘要。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexDetailsInfo", "properties" : { "id" : { "type" : "any" }, "name" : { "type" : "string" }, "parallelism" : { "type" : "integer" }, "now" : { "type" : "integer" }, "subtasks" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexDetailsInfo:VertexTaskDetail", "properties" : { "subtask" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ] }, "attempt" : { "type" : "integer" }, "host" : { "type" : "string" }, "start_time" : { "type" : "integer" }, "end-time" : { "type" : "integer" }, "duration" : { "type" : "integer" }, "metrics" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo", "properties" : { "read-bytes" : { "type" : "integer" }, "read-bytes-complete" : { "type" : "boolean" }, "write-bytes" : { "type" : "integer" }, "write-bytes-complete" : { "type" : "boolean" }, "read-records" : { "type" : "integer" }, "read-records-complete" : { "type" : "boolean" }, "write-records" : { "type" : "integer" }, "write-records-complete" : { "type" : "boolean" } } } } } } }}
##### /jobs/:jobid /vertices/:vertexid /accumulators
动作: GET响应代码: 200 OK
返回在所有子任务中聚合的任务的用户定义累加器。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexAccumulatorsInfo", "properties" : { "id" : { "type" : "string" }, "user-accumulators" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:UserAccumulator", "properties" : { "name" : { "type" : "string" }, "type" : { "type" : "string" }, "value" : { "type" : "string" } } } } }}
##### /jobs/:jobid /vertices/:vertexid /backpressure
动作: GET响应代码: 200 OK
返回作业的背压信息,并在必要时启动背压采样。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexBackPressureInfo", "properties" : { "status" : { "type" : "string", "enum" : [ "deprecated", "ok" ] }, "backpressure-level" : { "type" : "string", "enum" : [ "ok", "low", "high" ] }, "end-timestamp" : { "type" : "integer" }, "subtasks" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexBackPressureInfo:SubtaskBackPressureInfo", "properties" : { "subtask" : { "type" : "integer" }, "backpressure-level" : { "type" : "string", "enum" : [ "ok", "low", "high" ] }, "ratio" : { "type" : "number" } } } } }}
##### /jobs/:jobid /vertices/:vertexid /metrics
动作: GET响应代码: 200 OK
提供对任务指标的访问。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
查询参数
- get (可选):以逗号分隔的字符串值列表,用于选择特定指标。
  1. {}
  1. { "type" : "any"}
##### /jobs/:jobid /vertices/:vertexid /subtasks/accumulators
动作: GET响应代码: 200 OK
返回任务的所有子任务的所有用户定义的累加器。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtasksAllAccumulatorsInfo", "properties" : { "id" : { "type" : "any" }, "parallelism" : { "type" : "integer" }, "subtasks" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtasksAllAccumulatorsInfo:SubtaskAccumulatorsInfo", "properties" : { "subtask" : { "type" : "integer" }, "attempt" : { "type" : "integer" }, "host" : { "type" : "string" }, "user-accumulators" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:UserAccumulator", "properties" : { "name" : { "type" : "string" }, "type" : { "type" : "string" }, "value" : { "type" : "string" } } } } } } } }}
##### /jobs/:jobid /vertices/:vertexid /subtasks/metrics
动作: GET响应代码: 200 OK
提供对聚合子任务度量标准的访问。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
查询参数
- get (可选):以逗号分隔的字符串值列表,用于选择特定指标。- agg(可选):应该计算的以逗号分隔的聚合模式列表。可用的聚合是:“min,max,sum,avg”。- subtasks (可选):以逗号分隔的整数范围列表(例如“1,3,5-9”)以选择特定的子任务。
  1. {}
  1. { "type" : "any"}
##### /jobs/:jobid /vertices/:vertexid /subtasks/:subtaskindex
动作: GET响应代码: 200 OK
返回子任务的当前或最新执行尝试的详细信息。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。- subtaskindex - 标识子任务的正整数值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskExecutionAttemptDetailsInfo", "properties" : { "subtask" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ] }, "attempt" : { "type" : "integer" }, "host" : { "type" : "string" }, "start-time" : { "type" : "integer" }, "end-time" : { "type" : "integer" }, "duration" : { "type" : "integer" }, "metrics" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo", "properties" : { "read-bytes" : { "type" : "integer" }, "read-bytes-complete" : { "type" : "boolean" }, "write-bytes" : { "type" : "integer" }, "write-bytes-complete" : { "type" : "boolean" }, "read-records" : { "type" : "integer" }, "read-records-complete" : { "type" : "boolean" }, "write-records" : { "type" : "integer" }, "write-records-complete" : { "type" : "boolean" } } } }}
##### /jobs/:jobid /vertices/:vertexid /subtasks/:subtaskindex /attempts/:attempt
动作: GET响应代码: 200 OK
返回子任务执行尝试的详细信息。在发生故障/恢复时会发生多次执行尝试。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。- subtaskindex - 标识子任务的正整数值。- attempt - 标识执行尝试的正整数值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskExecutionAttemptDetailsInfo", "properties" : { "subtask" : { "type" : "integer" }, "status" : { "type" : "string", "enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ] }, "attempt" : { "type" : "integer" }, "host" : { "type" : "string" }, "start-time" : { "type" : "integer" }, "end-time" : { "type" : "integer" }, "duration" : { "type" : "integer" }, "metrics" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo", "properties" : { "read-bytes" : { "type" : "integer" }, "read-bytes-complete" : { "type" : "boolean" }, "write-bytes" : { "type" : "integer" }, "write-bytes-complete" : { "type" : "boolean" }, "read-records" : { "type" : "integer" }, "read-records-complete" : { "type" : "boolean" }, "write-records" : { "type" : "integer" }, "write-records-complete" : { "type" : "boolean" } } } }}
##### /jobs/:jobid /vertices/:vertexid /subtasks/:subtaskindex /attempts/:attempt/accumulators
动作: GET响应代码: 200 OK
返回子任务执行尝试的累加器。在发生故障/恢复时会发生多次执行尝试。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。- subtaskindex - 标识子任务的正整数值。- attempt - 标识执行尝试的正整数值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:SubtaskExecutionAttemptAccumulatorsInfo", "properties" : { "subtask" : { "type" : "integer" }, "attempt" : { "type" : "integer" }, "id" : { "type" : "string" }, "user-accumulators" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:UserAccumulator", "properties" : { "name" : { "type" : "string" }, "type" : { "type" : "string" }, "value" : { "type" : "string" } } } } }}
##### /jobs/:jobid /vertices/:vertexid /subtasks/:subtaskindex /metrics
动作: GET响应代码: 200 OK
提供对子任务度量标准的访问。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。- subtaskindex - 标识子任务的正整数值。
查询参数
- get (可选):以逗号分隔的字符串值列表,用于选择特定指标。
  1. {}
  1. { "type" : "any"}
##### /jobs/:jobid /vertices/:vertexid / subtasktimes
动作: GET响应代码: 200 OK
返回任务的所有子任务的时间相关信息。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo", "properties" : { "id" : { "type" : "string" }, "name" : { "type" : "string" }, "now" : { "type" : "integer" }, "subtasks" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:SubtasksTimesInfo:SubtaskTimeInfo", "properties" : { "subtask" : { "type" : "integer" }, "host" : { "type" : "string" }, "duration" : { "type" : "integer" }, "timestamps" : { "type" : "object", "additionalProperties" : { "type" : "integer" } } } } } }}
##### /jobs/:jobid /vertices/:vertexid / taskmanagers
动作: GET响应代码: 200 OK
返回TaskManager聚合的任务信息。
路径参数
- jobid - 标识作业的32个字符的十六进制字符串值。- vertexid - 标识作业顶点的32个字符的十六进制字符串值。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexTaskManagersInfo", "properties" : { "id" : { "type" : "any" }, "name" : { "type" : "string" }, "now" : { "type" : "integer" }, "taskmanagers" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:JobVertexTaskManagersInfo:TaskManagersInfo", "properties" : { "host" : { "type" : "string" }, "status" : { "type" : "string", "enum" : [ "CREATED", "SCHEDULED", "DEPLOYING", "RUNNING", "FINISHED", "CANCELING", "CANCELED", "FAILED", "RECONCILING" ] }, "start-time" : { "type" : "integer" }, "end-time" : { "type" : "integer" }, "duration" : { "type" : "integer" }, "metrics" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:metrics:IOMetricsInfo", "properties" : { "read-bytes" : { "type" : "integer" }, "read-bytes-complete" : { "type" : "boolean" }, "write-bytes" : { "type" : "integer" }, "write-bytes-complete" : { "type" : "boolean" }, "read-records" : { "type" : "integer" }, "read-records-complete" : { "type" : "boolean" }, "write-records" : { "type" : "integer" }, "write-records-complete" : { "type" : "boolean" } } }, "status-counts" : { "type" : "object", "additionalProperties" : { "type" : "integer" } } } } } }}
##### /overview
动作: GET响应代码: 200 OK
返回Flink集群的概述。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:legacy:messages:ClusterOverviewWithVersion", "properties" : { "taskmanagers" : { "type" : "integer" }, "slots-total" : { "type" : "integer" }, "slots-available" : { "type" : "integer" }, "jobs-running" : { "type" : "integer" }, "jobs-finished" : { "type" : "integer" }, "jobs-cancelled" : { "type" : "integer" }, "jobs-failed" : { "type" : "integer" }, "flink-version" : { "type" : "string" }, "flink-commit" : { "type" : "string" } }}
##### /savepoint-disposal
动作: POST响应代码: 200 OK
触发保存点的废弃处理。
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:job:savepoints:SavepointDisposalRequest", "properties" : { "savepoint-path" : { "type" : "string" } }}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:TriggerResponse", "properties" : { "request-id" : { "type" : "any" } }}
##### /savepoint-disposal/:triggerid
动作: GET响应代码: 200 OK
返回保存点处理 算子操作的状态。
路径参数
- triggerid - 32个字符的十六进制字符串,用于标识异步 算子操作触发器ID。返回ID然后触发 算子操作。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:handler:async:AsynchronousOperationResult", "properties" : { "status" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:queue:QueueStatus", "properties" : { "id" : { "type" : "string", "required" : true, "enum" : [ "IN_PROGRESS", "COMPLETED" ] } } }, "operation" : { "type" : "any" } }}
##### / taskmanagers
动作: GET响应代码: 200 OK
返回所有TaskManager的概述。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagersInfo", "properties" : { "taskmanagers" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagerInfo", "properties" : { "id" : { "type" : "any" }, "path" : { "type" : "string" }, "dataPort" : { "type" : "integer" }, "timeSinceLastHeartbeat" : { "type" : "integer" }, "slotsNumber" : { "type" : "integer" }, "freeSlots" : { "type" : "integer" }, "hardware" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:instance:HardwareDescription", "properties" : { "cpuCores" : { "type" : "integer" }, "physicalMemory" : { "type" : "integer" }, "freeMemory" : { "type" : "integer" }, "managedMemory" : { "type" : "integer" } } } } } } }}
##### / taskmanagers /metrics
动作: GET响应代码: 200 OK
提供对聚合TaskManager指标的访问。
查询参数
- get (可选):以逗号分隔的字符串值列表,用于选择特定指标。- agg(可选):应该计算的以逗号分隔的聚合模式列表。可用的聚合是:“min,max,sum,avg”。- taskmanagers (可选):以逗号分隔的32个字符的十六进制字符串列表,用于选择特定的TaskManager。
  1. {}
  1. { "type" : "any"}
##### / taskmanagers /:taskmanagerid
动作: GET响应代码: 200 OK
返回TaskManager的详细信息。
路径参数
- taskmanagerid - 标识TaskManager的32个字符的十六进制字符串。
  1. {}
  1. { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagerDetailsInfo", "properties" : { "id" : { "type" : "any" }, "path" : { "type" : "string" }, "dataPort" : { "type" : "integer" }, "timeSinceLastHeartbeat" : { "type" : "integer" }, "slotsNumber" : { "type" : "integer" }, "freeSlots" : { "type" : "integer" }, "hardware" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:instance:HardwareDescription", "properties" : { "cpuCores" : { "type" : "integer" }, "physicalMemory" : { "type" : "integer" }, "freeMemory" : { "type" : "integer" }, "managedMemory" : { "type" : "integer" } } }, "metrics" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagerMetricsInfo", "properties" : { "heapUsed" : { "type" : "integer" }, "heapCommitted" : { "type" : "integer" }, "heapMax" : { "type" : "integer" }, "nonHeapUsed" : { "type" : "integer" }, "nonHeapCommitted" : { "type" : "integer" }, "nonHeapMax" : { "type" : "integer" }, "directCount" : { "type" : "integer" }, "directUsed" : { "type" : "integer" }, "directMax" : { "type" : "integer" }, "mappedCount" : { "type" : "integer" }, "mappedUsed" : { "type" : "integer" }, "mappedMax" : { "type" : "integer" }, "memorySegmentsAvailable" : { "type" : "integer" }, "memorySegmentsTotal" : { "type" : "integer" }, "garbageCollectors" : { "type" : "array", "items" : { "type" : "object", "id" : "urn:jsonschema:org:apache:flink:runtime:rest:messages:taskmanager:TaskManagerMetricsInfo:GarbageCollectorInfo", "properties" : { "name" : { "type" : "string" }, "count" : { "type" : "integer" }, "time" : { "type" : "integer" } } } } } } }}
##### / taskmanagers /:taskmanagerid /metrics
动作: GET响应代码: 200 OK
提供对TaskManager指标的访问。
路径参数
- taskmanagerid - 标识TaskManager的32个字符的十六进制字符串。
查询参数
- get (可选):以逗号分隔的字符串值列表,用于选择特定指标。
  1. {}
  1. { "type" : "any"}

仅当群集在传统模式下运行时,此部分才相关

以下是可用请求列表,其中包含示例JSON响应。所有请求都是样本表单http://hostname:8081/jobs,下面我们仅列出URL 路径部分。

尖括号中的值是变量,例如http://hostname:8081/jobs/<jobid>/exceptions必须请求例如http://hostname:8081/jobs/7684be6004e4e955c2a558a9bc463f65/exceptions

  • /config
  • /overview
  • /jobs/overview
  • /jobs/<jobid>
  • /jobs/<jobid>/vertices
  • /jobs/<jobid>/config
  • /jobs/<jobid>/exceptions
  • /jobs/<jobid>/accumulators
  • /jobs/<jobid>/vertices/<vertexid>
  • /jobs/<jobid>/vertices/<vertexid>/subtasktimes
  • /jobs/<jobid>/vertices/<vertexid>/taskmanagers
  • /jobs/<jobid>/vertices/<vertexid>/accumulators
  • /jobs/<jobid>/vertices/<vertexid>/subtasks/accumulators
  • /jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>
  • /jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>
  • /jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>/accumulators
  • /jobs/<jobid>/plan
  • /jars/upload
  • /jars
  • /jars/:jarid
  • /jars/:jarid/plan
  • /jars/:jarid/run

一般

/config

有关监视API和服务器设置的一些信息。

样本结果:

  1. {
  2. "refresh-interval": 3000,
  3. "timezone-offset": 3600000,
  4. "timezone-name": "Central European Time",
  5. "flink-version": "1.7-SNAPSHOT",
  6. "flink-revision": "8124545 @ 16.09.2015 @ 15:38:42 CEST"
  7. }

/overview

Flink集群状态的简单摘要。

样本结果:

  1. {
  2. "taskmanagers": 17,
  3. "slots-total": 68,
  4. "slots-available": 68,
  5. "jobs-running": 0,
  6. "jobs-finished": 3,
  7. "jobs-cancelled": 1,
  8. "jobs-failed": 0
  9. }

工作概览

/jobs/overview

使用小摘要概述所有作业。

样本结果:

  1. {
  2. "jobs":[
  3. {
  4. "jid": "7684be6004e4e955c2a558a9bc463f65",
  5. "name": "Flink Java Job at Wed Sep 16 18:08:21 CEST 2015",
  6. "state": "FINISHED",
  7. "start-time": 1442419702857,
  8. "end-time": 1442419975312,
  9. "duration":272455,
  10. "last-modification": 1442419975312,
  11. "tasks": {
  12. "total": 6,
  13. "pending": 0,
  14. "running": 0,
  15. "finished": 6,
  16. "canceling": 0,
  17. "canceled": 0,
  18. "failed": 0
  19. }
  20. },
  21. {
  22. "jid": "49306f94d0920216b636e8dd503a6409",
  23. "name": "Flink Java Job at Wed Sep 16 18:16:39 CEST 2015",
  24. ...
  25. }]
  26. }

正在运行或已完成的工作的详细信息

/jobs/<jobid>

一个作业的摘要,列出数据流计划,状态,状态转换的时间戳,每个顶点( 算子)的聚合信息。

样本结果:

  1. {
  2. "jid": "ab78dcdbb1db025539e30217ec54ee16",
  3. "name": "WordCount Example",
  4. "state":"FINISHED",
  5. "start-time":1442421277536,
  6. "end-time":1442421299791,
  7. "duration":22255,
  8. "now":1442421991768,
  9. "timestamps": {
  10. "CREATED": 1442421277536, "RUNNING": 1442421277609, "FAILING": 0, "FAILED": 0, "CANCELLING": 0, "CANCELED": 0, "FINISHED": 1442421299791, "RESTARTING": 0
  11. },
  12. "vertices": [ {
  13. "id": "19b5b24062c48a06e4eac65422ac3317",
  14. "name": "CHAIN DataSource (at getTextDataSet(WordCount.java:142) ...",
  15. "parallelism": 2,
  16. "status": "FINISHED",
  17. "start-time": 1442421277609,
  18. "end-time": 1442421299469,
  19. "duration": 21860,
  20. "tasks": {
  21. "CREATED": 0, "SCHEDULED": 0, "DEPLOYING": 0, "RUNNING": 0, "FINISHED": 2, "CANCELING": 0, "CANCELED": 0, "FAILED": 0
  22. },
  23. "metrics": {
  24. "read-bytes": 0, "write-bytes": 37098, "read-records": 0, "write-records": 3312
  25. }
  26. }, {
  27. "id": "f00c89b349b5c998cfd9fe2a06e50fd0",
  28. "name":"Reduce (SUM(1), at main(WordCount.java:67)",
  29. "parallelism": 2,
  30. ....
  31. }, {
  32. "id": "0a36cbc29102d7bc993d0a9bf23afa12",
  33. "name": "DataSink (CsvOutputFormat (path: /tmp/abzs, delimiter: ))",
  34. ...
  35. } ],
  36. "status-counts": {
  37. "CREATED": 0, "SCHEDULED": 0, "DEPLOYING": 0, "RUNNING": 0, "FINISHED": 3, "CANCELING": 0, "CANCELED": 0, "FAILED": 0
  38. },
  39. "plan": {
  40. // see plan details below
  41. }
  42. }

/jobs/<jobid>/vertices

目前一样 /jobs/<jobid>

/jobs/<jobid>/config

作业使用的用户定义的执行配置。

样本结果:

  1. {
  2. "jid": "ab78dcdbb1db025539e30217ec54ee16",
  3. "name": "WordCount Example",
  4. "execution-config": {
  5. "execution-mode": "PIPELINED",
  6. "restart-strategy": "Restart deactivated",
  7. "job-parallelism": -1,
  8. "object-reuse-mode": false
  9. }
  10. }

/jobs/<jobid>/exceptions

工作中观察到的不可恢复的异常。truncated标志定义是否发生了更多异常,但未列出,因为否则响应会变得太大。

样本结果:

  1. {
  2. "root-exception": "java.io.IOException: File already exists:/tmp/abzs/2\n\tat org.apache.flink.core.fs.local.LocalFileSystem. ...",
  3. "all-exceptions": [ {
  4. "exception": "java.io.IOException: File already exists:/tmp/abzs/1\n\tat org.apache.flink...",
  5. "task": "DataSink (CsvOutputFormat (path: /tmp/abzs, delimiter: )) (1/2)",
  6. "location": "localhost:49220"
  7. }, {
  8. "exception": "java.io.IOException: File already exists:/tmp/abzs/2\n\tat org.apache.flink...",
  9. "task": "DataSink (CsvOutputFormat (path: /tmp/abzs, delimiter: )) (2/2)",
  10. "location": "localhost:49220"
  11. } ],
  12. "truncated":false
  13. }

/jobs/<jobid>/accumulators

聚合的用户累加器加上作业累加器。

样本结果:

  1. {
  2. "job-accumulators":[],
  3. "user-task-accumulators": [ {
  4. "name": "avglen",
  5. "type": "DoubleCounter",
  6. "value": "DoubleCounter 61.5162972"
  7. },
  8. {
  9. "name": "genwords",
  10. "type": "LongCounter",
  11. "value": "LongCounter 37500000"
  12. } ]
  13. }

/jobs/<jobid>/vertices/<vertexid>

有关一个特定顶点的信息,以及每个子任务的摘要。

样本结果:

  1. {
  2. "id": "dceafe2df1f57a1206fcb907cb38ad97",
  3. "name": "CHAIN DataSource -> Map -> FlatMap -> Combine(SUM(1))",
  4. "parallelism": 2,
  5. "now": 1442424002154,
  6. "subtasks": [ {
  7. "subtask":0,
  8. "status": "FINISHED",
  9. "attempt": 0,
  10. "host": "localhost",
  11. "start-time": 1442421093762,
  12. "end-time": 1442421386680,
  13. "duration": 292918,
  14. "metrics": {
  15. "read-bytes": 0, "write-bytes": 12684375, "read-records": 0, "write-records": 1153125
  16. }
  17. }, {
  18. "subtask": 1,
  19. "status": "FINISHED",
  20. "attempt": 0,
  21. "host": "localhost",
  22. "start-time": 1442421093774,
  23. "end-time": 1442421386267,
  24. "duration": 292493,
  25. "metrics": {
  26. "read-bytes": 0, "write-bytes": 12684375, "read-records": 0, "write-records": 1153125
  27. }
  28. } ]
  29. }

/jobs/<jobid>/vertices/<vertexid>/subtasktimes

此请求返回给定顶点的所有子任务的状态转换的时间戳。例如,这些可用于在子任务之间创建时间线比较。

样本结果:

  1. {
  2. "id": "dceafe2df1f57a1206fcb907cb38ad97",
  3. "name": "CHAIN DataSource -> Map -> Combine(SUM(1))",
  4. "now":1442423745088,
  5. "subtasks": [ {
  6. "subtask": 0,
  7. "host": "localhost",
  8. "duration": 292924,
  9. "timestamps": {
  10. "CREATED": 1442421093741, "SCHEDULED": 1442421093756, "DEPLOYING": 1442421093762, "RUNNING": 1442421094026, "FINISHED": 1442421386680, "CANCELING": 0, "CANCELED": 0, "FAILED": 0
  11. }
  12. }, {
  13. "subtask": 1,
  14. "host": "localhost",
  15. "duration": 292494,
  16. "timestamps": {
  17. "CREATED": 1442421093741, "SCHEDULED": 1442421093773, "DEPLOYING": 1442421093774, "RUNNING": 1442421094013, "FINISHED": 1442421386267, "CANCELING": 0, "CANCELED": 0, "FAILED": 0
  18. }
  19. } ]
  20. }

/jobs/<jobid>/vertices/<vertexid>/taskmanagers

一个特定顶点的TaskManager统计信息。这是由返回的子任务统计信息的聚合/jobs/<jobid>/vertices/<vertexid>

样本结果:

  1. {
  2. "id": "fe20bcc29b87cdc76589ca42114c2499",
  3. "name": "Reduce (SUM(1), at main(WordCount.java:72)",
  4. "now": 1454348282653,
  5. "taskmanagers": [ {
  6. "host": "ip-10-0-43-227:35413",
  7. "status": "FINISHED",
  8. "start-time": 1454347870991,
  9. "end-time": 1454347872111,
  10. "duration": 1120,
  11. "metrics": {
  12. "read-bytes": 32503056, "write-bytes": 9637041, "read-records": 2906087, "write-records": 849467
  13. },
  14. "status-counts": {
  15. "CREATED": 0, "SCHEDULED": 0, "DEPLOYING": 0, "RUNNING": 0, "FINISHED": 18, "CANCELING": 0, "CANCELED": 0, "FAILED": 0
  16. }
  17. },{
  18. "host": "ip-10-0-43-227:41486",
  19. "status": "FINISHED",
  20. "start-time": 1454347871001,
  21. "end-time": 1454347872395,
  22. "duration": 1394,
  23. "metrics": {
  24. "read-bytes": 32389499, "write-bytes": 9608829, "read-records": 2895999, "write-records": 846948
  25. },
  26. "status-counts": {
  27. "CREATED": 0, "SCHEDULED": 0, "DEPLOYING": 0, "RUNNING": 0, "FINISHED": 18, "CANCELING": 0, "CANCELED": 0, "FAILED": 0
  28. }
  29. } ]
  30. }

/jobs/<jobid>/vertices/<vertexid>/accumulators

用于特定顶点的聚合的用户定义累加器。

样本结果:

  1. {
  2. "id": "dceafe2df1f57a1206fcb907cb38ad97",
  3. "user-accumulators": [ {
  4. "name": "avglen", "type": "DoubleCounter", "value": "DoubleCounter 123.03259440000001"
  5. }, {
  6. "name": "genwords", "type": "LongCounter", "value": "LongCounter 75000000"
  7. } ]
  8. }

/jobs/<jobid>/vertices/<vertexid>/subtasks/accumulators

获取给定顶点的所有子任务的所有用户定义的累加器。这些是由请求以聚合形式返回的各个累加器/jobs/<jobid>/vertices/<vertexid>/accumulators

样本结果:

  1. {
  2. "id": "dceafe2df1f57a1206fcb907cb38ad97",
  3. "parallelism": 2,
  4. "subtasks": [ {
  5. "subtask": 0,
  6. "attempt": 0,
  7. "host": "localhost",
  8. "user-accumulators": [ {
  9. "name": "genwords", "type": "LongCounter", "value": "LongCounter 62500000"
  10. }, {
  11. "name": "genletters", "type": "LongCounter", "value": "LongCounter 1281589525"
  12. } ]
  13. }, {
  14. "subtask": 1,
  15. "attempt": 0,
  16. "host": "localhost",
  17. "user-accumulators": [ {
  18. "name": "genwords", "type": "LongCounter", "value": "LongCounter 12500000"
  19. }, {
  20. "name": "genletters", "type": "LongCounter", "value": "LongCounter 256317905"
  21. } ]
  22. } ]
  23. }

/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>

特定子任务的当前或最新执行尝试的摘要。请参阅下面的示例。

/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>

特定子任务的特定执行尝试的摘要。在发生故障/恢复时会发生多次执行尝试。

样本结果:

  1. {
  2. "subtask": 0,
  3. "status": "FINISHED",
  4. "attempt": 0,
  5. "host": "localhost",
  6. "start-time": 1442421093762,
  7. "end-time": 1442421386680,
  8. "duration": 292918,
  9. "metrics": {
  10. "read-bytes": 0, "write-bytes": 12684375, "read-records": 0, "write-records": 1153125
  11. }
  12. }

/jobs/<jobid>/vertices/<vertexid>/subtasks/<subtasknum>/attempts/<attempt>/accumulators

在一次特定执行尝试期间为一个特定子任务收集累加器(在故障/恢复的情况下发生多次尝试)。

样本结果:

  1. {
  2. "subtask": 0,
  3. "attempt": 0,
  4. "id": "b22f94d91bf41ddb",
  5. "user-accumulators": [ {
  6. "name": "genwords", "type":"LongCounter", "value":"LongCounter 62500000"
  7. }, {
  8. "name": "genletters", "type": "LongCounter", "value": "LongCounter 1281589525"
  9. }, {
  10. "name": "avglen", "type": "DoubleCounter", "value": "DoubleCounter 102.527162"
  11. } ]
  12. }

/jobs/<jobid>/plan

作业的数据流计划。该计划也包含在工作总结(/jobs/<jobid>)中。

样本结果:

  1. {
  2. "jid":"ab78dcdbb1db025539e30217ec54ee16",
  3. "name":"WordCount Example",
  4. "nodes": [ {
  5. "id": "f00c89b349b5c998cfd9fe2a06e50fd0",
  6. "parallelism": 2,
  7. "operator": "GroupReduce",
  8. "operator_strategy": "Sorted Group Reduce",
  9. "description": "Reduce (SUM(1), at main(WordCount.java:67)",
  10. "inputs": [ {
  11. "num": 0,
  12. "id":"19b5b24062c48a06e4eac65422ac3317",
  13. "ship_strategy": "Hash Partition on [0]",
  14. "local_strategy":"Sort (combining) on [0:ASC]",
  15. "exchange":"pipelined"
  16. } ],
  17. "optimizer_properties": {
  18. "global_properties": [
  19. { "name":"Partitioning", "value":"HASH_PARTITIONED" },
  20. { "name":"Partitioned on", "value":"[0]" },
  21. { "name":"Partitioning Order", "value":"(none)" },
  22. { "name":"Uniqueness", "value":"not unique" }
  23. ],
  24. "local_properties": [
  25. { "name":"Order", "value":"[0:ASC]" },
  26. { "name":"Grouped on", "value":"[0]" },
  27. { "name":"Uniqueness", "value":"not unique" }
  28. ],
  29. "estimates": [
  30. { "name":"Est. Output Size", "value":"(unknown)" },
  31. { "name":"Est. Cardinality", "value":"(unknown)" }
  32. ],
  33. "costs": [
  34. { "name":"Network", "value":"(unknown)" },
  35. { "name":"Disk I/O", "value":"(unknown)" },
  36. { "name":"CPU", "value":"(unknown)" },
  37. { "name":"Cumulative Network", "value":"(unknown)" },
  38. { "name":"Cumulative Disk I/O", "value":"(unknown)" },
  39. { "name":"Cumulative CPU","value":"(unknown)" }
  40. ],
  41. "compiler_hints": [
  42. { "name":"Output Size (bytes)", "value":"(none)" },
  43. { "name":"Output Cardinality", "value":"(none)" },
  44. { "name":"Avg. Output Record Size (bytes)", "value":"(none)" },
  45. { "name":"Filter Factor", "value":"(none)" }
  46. ]
  47. }
  48. },
  49. {
  50. "id": "19b5b24062c48a06e4eac65422ac3317",
  51. "parallelism": 2,
  52. "operator": "Data Source -> FlatMap -> GroupCombine",
  53. "operator_strategy":" (none) -> FlatMap -> Sorted Combine",
  54. "description":"DataSource (at getTextDataSet(WordCount.java:142) (org.apache.flink.api.java.io.TextInputFormat)) -> FlatMap (FlatMap at main(WordCount.java:67)) -> Combine(SUM(1), at main(WordCount.java:67)",
  55. "optimizer_properties": {
  56. ...
  57. }
  58. },
  59. {
  60. "id": "0a36cbc29102d7bc993d0a9bf23afa12",
  61. "parallelism": 2,
  62. "operator": "Data Sink",
  63. "operator_strategy": "(none)",
  64. "description": "DataSink (CsvOutputFormat (path: /tmp/abzs, delimiter: ))",
  65. "inputs":[ {
  66. "num": 0,
  67. "id": "f00c89b349b5c998cfd9fe2a06e50fd0",
  68. "ship_strategy": "Forward",
  69. "exchange": "pipelined"
  70. } ],
  71. "optimizer_properties": {
  72. ...
  73. }
  74. } ]
  75. }

取消工作

取消工作

DELETE要求/jobs/:jobid/cancel

触发取消工作,成功的结果是{}

使用Savepoint取消作业

保存点成功后触发保存点并取消作业。

GET请求/jobs/:jobid/cancel-with-savepoint/触发保存点到默认保存点目录并取消作业。

GET请求/jobs/:jobid/cancel-with-savepoint/target-directory/:targetDirectory触发保存点到给定目标目录并取消作业。

由于保存点可能需要一些时间才能完成,因此异步执行此 算子操作。此请求的结果是正在进行的取消的位置。

样本触发结果:

  1. {
  2. "status": "accepted",
  3. "request-id": 1,
  4. "location": "/jobs/:jobid/cancel-with-savepoint/in-progress/1"
  5. }
监测进展

取消的进度必须由用户监控

  1. /jobs/:jobid/cancel-with-savepoint/in-progress/:requestId

触发结果返回请求ID。

####### 进行中

  1. {
  2. "status": "in-progress",
  3. "request-id": 1
  4. }

#######成功

  1. {
  2. "status": "success",
  3. "request-id": 1,
  4. "savepoint-path": "<savepointPath>"
  5. }

savepointPath点到保存点的外部路径,其可用于恢复保存点。

#######失败

  1. {
  2. "status": "failed",
  3. "request-id": 1,
  4. "cause": "<error message>"
  5. }

提交程序

可以通过REST API和Web前端上传,运行和列出Flink程序。

上传新的JAR文件

发送POST请求,/jars/upload将您的jar文件作为多部分数据发送到该jarfile文件下。还要确保多部分数据包含Content-Type文件本身,某些http库默认情况下不添加标头。

多部分有效负载应该开始

  1. ------BoundaryXXXX
  2. Content-Disposition: form-data; name="jarfile"; filename="YourFileName.jar"
  3. Content-Type: application/x-java-archive
运行程序(POST)

发送POST请求/jars/:jarid/runjarid参数是配置的Web前端上载目录(配置Keysweb.upload.dir)中程序JAR的文件名

您可以指定以下查询参数(全部可选):

  • 程序参数program-args=arg1 arg2 arg3
  • 要执行的主要类entry-class=EntryClassName.class
  • 默认并行度parallelism=4
  • 要从中恢复的保存点路径savepointPath=hdfs://path/to/savepoint
  • 允许非恢复状态allowNonRestoredState=true如果呼叫成功,您将收到包含已提交作业的ID的响应。

示例:使用保存点运行程序

请求:

  1. POST: /jars/MyProgram.jar/run?savepointPath=/my-savepoints/savepoint-1bae02a80464&allowNonRestoredState=true

响应:

  1. {"jobid": "869a9868d49c679e7355700e0857af85"}