Katib

Reference documentation for Katib

Table of Contents

Top

api.proto

Katib GRPC API v1beta1

AlgorithmSetting

HP or NAS algorithm settings.

FieldTypeLabelDescription
namestring
valuestring

AlgorithmSpec

HP or NAS algorithm specification.

FieldTypeLabelDescription
algorithm_namestring
algorithm_settingsAlgorithmSettingrepeated

DeleteObservationLogReply

DeleteObservationLogRequest

FieldTypeLabelDescription
trial_namestring

EarlyStoppingRule

EarlyStoppingRule represents single early stopping rule.

FieldTypeLabelDescription
namestringName of the rule. Usually, metric name.
valuestringValue of the metric.
comparisonComparisonTypeCorrelation between name and value, one of equal, less or greater
start_stepint32Defines quantity of intermediate results that should be received before applying the rule. If start step is empty, rule is applied from the first recorded metric.

EarlyStoppingSetting

Early stopping algorithm settings.

FieldTypeLabelDescription
namestring
valuestring

EarlyStoppingSpec

Early stopping algorithm specification.

FieldTypeLabelDescription
algorithm_namestring
algorithm_settingsEarlyStoppingSettingrepeated

Experiment

Structure for a single Experiment.

FieldTypeLabelDescription
namestringName for the Experiment.
specExperimentSpecExperiment specification.

ExperimentSpec

Specification of an Experiment. Experiment represents a single optimization run over a feasible space. Each Experiment contains a configuration describing the feasible space, as well as a set of Trials. It is assumed that objective function f(x) does not change in the course of an Experiment.

FieldTypeLabelDescription
parameter_specsExperimentSpec.ParameterSpecs
objectiveObjectiveSpecObjective specification for the Experiment.
algorithmAlgorithmSpecHP or NAS algorithm specification for the Experiment.
early_stoppingEarlyStoppingSpecEarly stopping specification for the Experiment.
parallel_trial_countint32How many Trials can be processed in parallel.
max_trial_countint32Max completed Trials to mark Experiment as succeeded.
nas_configNasConfigNAS configuration for the Experiment.

ExperimentSpec.ParameterSpecs

List of ParameterSpec.

FieldTypeLabelDescription
parametersParameterSpecrepeated

FeasibleSpace

Feasible space for optimization. Int and Double type use Max/Min. Discrete and Categorical type use List.

FieldTypeLabelDescription
maxstringMax Value
minstringMinimum Value
liststringrepeatedList of Values.
stepstringStep for double or int parameter

GetEarlyStoppingRulesReply

FieldTypeLabelDescription
early_stopping_rulesEarlyStoppingRulerepeated

GetEarlyStoppingRulesRequest

FieldTypeLabelDescription
experimentExperiment
trialsTrialrepeated
db_manager_addressstring

GetObservationLogReply

FieldTypeLabelDescription
observation_logObservationLog

GetObservationLogRequest

FieldTypeLabelDescription
trial_namestring
metric_namestring
start_timestringThe start of the time range. RFC3339 format
end_timestringThe end of the time range. RFC3339 format

GetSuggestionsReply

FieldTypeLabelDescription
parameter_assignmentsGetSuggestionsReply.ParameterAssignmentsrepeated
algorithmAlgorithmSpec
early_stopping_rulesEarlyStoppingRulerepeated

GetSuggestionsReply.ParameterAssignments

FieldTypeLabelDescription
assignmentsParameterAssignmentrepeated

GetSuggestionsRequest

FieldTypeLabelDescription
experimentExperiment
trialsTrialrepeatedall completed trials owned by the experiment.
request_numberint32The number of Suggestion you request at one time. When you set 3 to request_number, you can get three Suggestions at one time.

GraphConfig

GraphConfig contains a config of DAG

FieldTypeLabelDescription
num_layersint32Number of layers
input_sizesint32repeatedDimensions of input size
output_sizesint32repeatedDimensions of output size

Metric

FieldTypeLabelDescription
namestring
valuestring

MetricLog

FieldTypeLabelDescription
time_stampstringRFC3339 format
metricMetric

NasConfig

NasConfig contains a config of NAS job

FieldTypeLabelDescription
graph_configGraphConfigConfig of DAG
operationsNasConfig.OperationsList of Operation

NasConfig.Operations

FieldTypeLabelDescription
operationOperationrepeated

ObjectiveSpec

Objective specification.

FieldTypeLabelDescription
typeObjectiveTypeType of optimization.
goaldoubleGoal of optimization, can be empty.
objective_metric_namestringPrimary metric name for the optimization.
additional_metric_namesstringrepeatedList of additional metrics to record from Trial. This can be empty if we only care about the objective metric.

Observation

FieldTypeLabelDescription
metricsMetricrepeated

ObservationLog

FieldTypeLabelDescription
metric_logsMetricLogrepeated

Operation

Config for operations in DAG

FieldTypeLabelDescription
operation_typestringType of operation in DAG
parameter_specsOperation.ParameterSpecs

Operation.ParameterSpecs

List of ParameterSpec

FieldTypeLabelDescription
parametersParameterSpecrepeated

ParameterAssignment

FieldTypeLabelDescription
namestring
valuestring

ParameterSpec

Config for a hyperparameter. Katib will create each Hyper parameter from this config.

FieldTypeLabelDescription
namestringName of the parameter.
parameter_typeParameterTypeType of the parameter.
feasible_spaceFeasibleSpaceFeasibleSpace for the parameter.

ReportObservationLogReply

ReportObservationLogRequest

FieldTypeLabelDescription
trial_namestring
observation_logObservationLog

SetTrialStatusReply

SetTrialStatusRequest

FieldTypeLabelDescription
trial_namestring

Trial

Structure for a single Trial.

FieldTypeLabelDescription
namestringName for the Trial.
specTrialSpecTrial specification.
statusTrialStatusTrial status.

TrialSpec

Specification of a Trial. It represents Trial’s parameter assignments and objective.

FieldTypeLabelDescription
objectiveObjectiveSpecObjective specification for the Trial.
parameter_assignmentsTrialSpec.ParameterAssignmentsList of assignments generated for the Trial.

TrialSpec.ParameterAssignments

List of ParameterAssignment

FieldTypeLabelDescription
assignmentsParameterAssignmentrepeated

TrialStatus

Current Trial status. It contains Trial’s latest condition, start time, completion time, observation.

FieldTypeLabelDescription
start_timestringTrial start time in RFC3339 format
completion_timestringTrial completion time in RFC3339 format
conditionTrialStatus.TrialConditionTypeTrial current condition. It is equal to the latest Trial CR condition.
observationObservationThe best Trial observation in logs.

ValidateAlgorithmSettingsReply

Return INVALID_ARGUMENT Error if Algorithm Settings are not Valid

ValidateAlgorithmSettingsRequest

FieldTypeLabelDescription
experimentExperiment

ComparisonType

NameNumberDescription
UNKNOWN_COMPARISON0Unknown comparison, not used
EQUAL1Equal comparison, e.g. accuracy = 0.7
LESS2Less comparison, e.g. accuracy < 0.7
GREATER3Greater comparison, e.g. accuracy > 0.7

ObjectiveType

Direction of optimization. Minimize or Maximize.

NameNumberDescription
UNKNOWN0Undefined type and not used.
MINIMIZE1Minimize
MAXIMIZE2Maximize

ParameterType

Types of value for HyperParameter.

NameNumberDescription
UNKNOWN_TYPE0Undefined type and not used.
DOUBLE1Double float type. Use “Max/Min”.
INT2Int type. Use “Max/Min”.
DISCRETE3Discrete number type. Use “List” as float.
CATEGORICAL4Categorical type. Use “List” as string.

TrialStatus.TrialConditionType

Trial can be in one of 6 conditions. TODO (andreyvelich): Remove unused conditions.

NameNumberDescription
CREATED0
RUNNING1
SUCCEEDED2
KILLED3
FAILED4
EARLYSTOPPED5
UNKNOWN6

DBManager

DBManager service defines APIs to manage Katib database.

Method NameRequest TypeResponse TypeDescription
ReportObservationLogReportObservationLogRequestReportObservationLogReplyReport a log of Observations for a Trial. The log consists of timestamp and value of metric. Katib store every log of metrics. You can see accuracy curve or other metric logs on UI.
GetObservationLogGetObservationLogRequestGetObservationLogReplyGet all log of Observations for a Trial.
DeleteObservationLogDeleteObservationLogRequestDeleteObservationLogReplyDelete all log of Observations for a Trial.

EarlyStopping

EarlyStopping service defines APIs to manage Katib Early Stopping algorithms

Method NameRequest TypeResponse TypeDescription
GetEarlyStoppingRulesGetEarlyStoppingRulesRequestGetEarlyStoppingRulesReply
SetTrialStatusSetTrialStatusRequestSetTrialStatusReply

Suggestion

Suggestion service defines APIs to manage Katib Suggestion from HP or NAS algorithms

Method NameRequest TypeResponse TypeDescription
GetSuggestionsGetSuggestionsRequestGetSuggestionsReply
ValidateAlgorithmSettingsValidateAlgorithmSettingsRequestValidateAlgorithmSettingsReply

Scalar Value Types

.proto TypeNotesC++JavaPythonGoC#PHPRuby
doubledoubledoublefloatfloat64doublefloatFloat
floatfloatfloatfloatfloat32floatfloatFloat
int32Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.int32intintint32intintegerBignum or Fixnum (as required)
int64Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.int64longint/longint64longinteger/stringBignum
uint32Uses variable-length encoding.uint32intint/longuint32uintintegerBignum or Fixnum (as required)
uint64Uses variable-length encoding.uint64longint/longuint64ulonginteger/stringBignum or Fixnum (as required)
sint32Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.int32intintint32intintegerBignum or Fixnum (as required)
sint64Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.int64longint/longint64longinteger/stringBignum
fixed32Always four bytes. More efficient than uint32 if values are often greater than 2^28.uint32intintuint32uintintegerBignum or Fixnum (as required)
fixed64Always eight bytes. More efficient than uint64 if values are often greater than 2^56.uint64longint/longuint64ulonginteger/stringBignum
sfixed32Always four bytes.int32intintint32intintegerBignum or Fixnum (as required)
sfixed64Always eight bytes.int64longint/longint64longinteger/stringBignum
boolboolbooleanbooleanboolboolbooleanTrueClass/FalseClass
stringA string must always contain UTF-8 encoded or 7-bit ASCII text.stringStringstr/unicodestringstringstringString (UTF-8)
bytesMay contain any arbitrary sequence of bytes.stringByteStringstr[]byteByteStringstringString (ASCII-8BIT)

Last modified 11.11.2020: Katib v1beta1 documentation update (#2312) (ba726043)