Metadata API Reference

Reference documentation for the Kubeflow Metadata API

api/service.proto

Kubeflow Metadata Service REST API

Version: v1alpha1

Default request content-types: application/json

Default response content-types: application/json

Schemes: http, https

Summary

Paths

GET /api/v1alpha1/artifact_types

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiListArtifactTypesResponse

POST /api/v1alpha1/artifact_types

Tags: MetadataService

Uses default content-types: application/json

ml_metadataArtifactType

Uses default content-types: application/json

• 200 OK

A successful response.

apiCreateArtifactTypeResponse

DELETE /api/v1alpha1/artifact_types/{name}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiGetArtifactTypeResponse

GET /api/v1alpha1/artifact_types/{name}/artifacts

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiListArtifactsResponse

DELETE /api/v1alpha1/artifact_types/{name}/artifacts/{id}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

GET /api/v1alpha1/artifact_types/{name}/artifacts/{id}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiGetArtifactResponse

NOTE: The order of the following RPC methods affects the order of matching a particular HTTP path. So put a more specific path pattern before a generic one. For example, GET /api/v1alpha1/artifact_types/{parent}/artifacts should appear before GET /api/v1alpha1/artifact_types/{name} to be possibly matched.

POST /api/v1alpha1/artifact_types/{parent}/artifacts

Tags: MetadataService

Uses default content-types: application/json

The Artifact to create. Note that Artifact.type_id is ignored.

ml_metadataArtifact

Uses default content-types: application/json

• 200 OK

A successful response.

apiCreateArtifactResponse

GET /api/v1alpha1/artifacts

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiListArtifactsResponse

POST /api/v1alpha1/events

Tags: MetadataService

Uses default content-types: application/json

ml_metadataEvent

Uses default content-types: application/json

• 200 OK

A successful response.

List events based on an artifact or execution id.

GET /api/v1alpha1/events/artifacts/{name}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiListEventsResponse

List events based on an artifact or execution id.

GET /api/v1alpha1/events/executions/{name}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiListEventsResponse

GET /api/v1alpha1/execution_types

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiListExecutionTypesResponse

POST /api/v1alpha1/execution_types

Tags: MetadataService

Uses default content-types: application/json

ml_metadataExecutionType

Uses default content-types: application/json

• 200 OK

A successful response.

apiCreateExecutionTypeResponse

DELETE /api/v1alpha1/execution_types/{name}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiGetExecutionTypeResponse

GET /api/v1alpha1/execution_types/{name}/executions

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiListExecutionsResponse

DELETE /api/v1alpha1/execution_types/{name}/executions/{id}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

GET /api/v1alpha1/execution_types/{name}/executions/{id}

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiGetExecutionResponse

POST /api/v1alpha1/execution_types/{parent}/executions

Tags: MetadataService

Uses default content-types: application/json

The Execution to create. Note that Execution.type_id is ignored.

ml_metadataExecution

Uses default content-types: application/json

• 200 OK

A successful response.

apiCreateExecutionResponse

GET /api/v1alpha1/executions

Tags: MetadataService

Uses default content-types: application/json

• 200 OK

A successful response.

apiListExecutionsResponse

Schema definitions

apiCreateArtifactResponse: object

• artifact: ml_metadataArtifact
• Newly created artifact with id.

apiCreateArtifactTypeResponse: object

• artifact_type: ml_metadataArtifactType
• Newly created artifact type with id.

apiCreateExecutionResponse: object

• execution: ml_metadataExecution
• Newly created execution with id.

apiCreateExecutionTypeResponse: object

• execution_type: ml_metadataExecutionType
• Newly created execution type with id.

apiGetArtifactResponse: object

• artifact: ml_metadataArtifact

apiGetArtifactTypeResponse: object

• artifact_type: ml_metadataArtifactType

apiGetExecutionResponse: object

• execution: ml_metadataExecution

apiGetExecutionTypeResponse: object

• execution_type: ml_metadataExecutionType

apiListArtifactsResponse: object

• artifacts: object[]

• ml_metadataArtifact

apiListArtifactTypesResponse: object

• artifact_types: object[]

• ml_metadataArtifactType

apiListEventsResponse: object

• events: object[]

• ml_metadataEvent

• artifacts: object

• ml_metadataArtifact

• executions: object

• ml_metadataExecution

apiListExecutionsResponse: object

• executions: object[]

• ml_metadataExecution

apiListExecutionTypesResponse: object

• execution_types: object[]

• ml_metadataExecutionType

EventPath: object

A simple path (e.g. {step{key:"foo"}}) can name an artifact in the context of an execution.

• steps: object[]
• A simple path (e.g. {step{key:"foo"}}) can name an artifact in the context of an execution.

PathStep

ml_metadataAnyArtifactStructType: object

Every ArtifactStruct is a member of this type.

ml_metadataArtifact: object

• id: string (int64)
• The id of the artifact.

• type_id: string (int64)
• The id of an ArtifactType. Type must be specified when an artifact is created, and it cannot be changed.

• uri: string
• The uniform resource identifier of the physical artifact. May be empty if there is no physical artifact.

• properties: object
• Properties of the artifact. Properties must be specified in the ArtifactType.

ml_metadataValue

• custom_properties: object
• User provided custom properties which are not defined by its type.

ml_metadataValue

ml_metadataArtifactStructType: object

The type of an ArtifactStruct. An artifact struct type represents an infinite set of artifact structs. It can specify the input or output type of an ExecutionType. See the more specific types referenced in the message for more details.

• simple: ml_metadataArtifactType

• union_type: ml_metadataUnionArtifactStructType

• intersection: ml_metadataIntersectionArtifactStructType

• list: ml_metadataListArtifactStructType

• none: ml_metadataNoneArtifactStructType

• any: ml_metadataAnyArtifactStructType

• tuple: ml_metadataTupleArtifactStructType

• dict: ml_metadataDictArtifactStructType

ml_metadataArtifactType: object

• id: string (int64)
• The id of the type. 1-1 relationship between type names and IDs.

• name: string
• The name of the type.

• properties: object
• The schema of the type. Properties are always optional in the artifact. Properties of an artifact type can be expanded but not contracted (i.e., you can add columns but not remove them).

ml_metadataPropertyType

ml_metadataDictArtifactStructType: object

• properties: object
• Underlying properties for the type.

ml_metadataArtifactStructType

• none_type_not_required: boolean (boolean)
• If true, then if properties["foo"] can be None, then that key is not required.

• extra_properties_type: ml_metadataArtifactStructType
• Extra keys are allowed that are not specified in properties. These keys must have the type specified below. If this is not specified, then extra properties are not allowed.

ml_metadataEvent: object

An event represents a relationship between an artifact ID and an execution. There are four kinds of events, relating to both input and output, as well as declared versus undeclared. For example, the DECLARED_INPUT events are part of the signature of an execution. For example, consider: my_execution({"data":[3,7],"schema":8}) Where 3, 7, and 8 are artifact_ids. Assuming execution_id is 12, this becomes: {artifact_id:3, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"data"},{"index":0}]}} {artifact_id:7, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"data"},{"index":1}]}} {artifact_id:8, execution_id: 12, type:DECLARED_INPUT, path:{step:[{"key":"schema"}]}} The INPUT is an artifact actually read by the execution. The OUTPUT is an artifact actually written by the execution. The DECLARED_OUTPUT are the artifacts that are the "official" output. For example, the trainer may output multiple caches of the parameters (as OUTPUT objects), but then finally write the SavedModel as a DECLARED_OUTPUT. TODO(martinz): add a type for Event, similar to ArtifactType.

• artifact_id: string (int64)
• The artifact id is required for an event, and should refer to an existing artifact.

• execution_id: string (int64)
• The execution_id is required for an event, and should refer to an existing execution.

• path: EventPath
• The path in an artifact struct, or the name of an artifact.

• type: ml_metadataEventType
• The type of an event.

• milliseconds_since_epoch: string (int64)

ml_metadataEventType: string , x ∈ { UNKNOWN (default) , DECLARED_OUTPUT , DECLARED_INPUT , INPUT , OUTPUT }

Events distinguish between an artifact that is written by the execution (possibly as a cache), versus artifacts that are part of the declared output of the Execution. For more information on what DECLARED_ means, see the comment on the message.

ml_metadataExecution: object

• id: string (int64)
• The id of the execution.

• type_id: string (int64)
• The id of an ExecutionType. The ExecutionType must be specified and cannot be changed.

• last_known_state: ml_metadataExecutionState
• The last known state of an execution.

• properties: object
• Properties of the Execution. Properties must be specified in the ExecutionType.

ml_metadataValue

• custom_properties: object
• User provided custom properties which are not defined by its type.

ml_metadataValue

ml_metadataExecutionState: string , x ∈ { UNKNOWN (default) , NEW , RUNNING , COMPLETE , FAILED }

TODO(martinz): consider adding INVALIDATED as a state.

ml_metadataExecutionType: object

• id: string (int64)
• The id of the type. 1-1 relationship between type names and IDs.

• name: string
• The name of the type.

• properties: object
• The schema of the type. Properties are always optional in the execution.

ml_metadataPropertyType

• input_type: ml_metadataArtifactStructType
• The ArtifactStructType of the input. For example: { "dict":{ "properties":{ "schema":{ "union_type":{ "none":{}, "simple":{…schema type…} }, }, "data":{ "simple":{…data_type…} } } } } That would be an optional schema field with a required data field.

• output_type: ml_metadataArtifactStructType

ml_metadataIntersectionArtifactStructType: object

A member of this type must satisfy all constraints. This primarily useful not as an end-user type, but something calculated as an intermediate type in the system.

For example, suppose you have a method: def infer_my_input_type(a): # try to infer the input type of this method. use_in_method_x(a) # with input type x_input use_in_method_y(a) # with input type y_input

Given this information, you know that infer_my_input_type has type {"intersection":{"constraints":[x_input, y_input]}}.

IntersectionArtifactStructType intersection_type = {"constraints":[ {"dict":{"properties":{"schema":{"any":{}}}, "extra_properties":{"any":{}}}}, {"dict":{"properties":{"data":{"any":{}}}, "extra_properties":{"any":{}}}}]} Since the first constraint requires the dictionary to have a schema property, and the second constraint requires it to have a data property, this is equivalent to: ArtifactStructType other_type = {"dict":{"properties":{"schema":{"any":{}},"data":{"any":{}}}}, "extra_properties":{"any":{}}}

• constraints: object[]

• ml_metadataArtifactStructType

ml_metadataListArtifactStructType: object

Represents an ArtifactStruct list type with homogeneous elements.

• element: ml_metadataArtifactStructType
• Every entry in the list must be of this type. Note: if this type is Any, then the list can have arbitrary elements.

ml_metadataNoneArtifactStructType: object

ml_metadataPropertyType: string , x ∈ { UNKNOWN (default) , INT , DOUBLE , STRING }

TODO(martinz): consider moving this inside some message, to avoid having literals directly in apo package.

ml_metadataTupleArtifactStructType: object

An ordered list of heterogenous artifact structs. The length of the list is fixed. Each position in the list can have a different type.

• elements: object[]

• ml_metadataArtifactStructType

ml_metadataUnionArtifactStructType: object

Represents a union of types.

• candidates: object[]
• An artifact struct matches this type if it matches any of the candidates. If candidates is empty, this is a bottom type (matches no artifacts).

ml_metadataArtifactStructType

ml_metadataValue: object

A value in properties.

• int_value: string (int64)

• double_value: number (double)

• string_value: string

PathStep: object

• index: string (int64)

• key: string