tasks.proto

—————– First class Plugins

flyteidl.core.Resources

[flyteidl.core.Resources proto]

A customizable interface to convey resources requested for a container. This can be interpretted differently for different container engines.

{
  "requests": [],
  "limits": []
}
requests

(flyteidl.core.Resources.ResourceEntry) The desired set of resources requested. ResourceNames must be unique within the list.

limits

(flyteidl.core.Resources.ResourceEntry) Defines a set of bounds (e.g. min/max) within which the task can reliably run. ResourceNames must be unique within the list.

flyteidl.core.Resources.ResourceEntry

[flyteidl.core.Resources.ResourceEntry proto]

Encapsulates a resource name and value.

{
  "name": "...",
  "value": "..."
}
name

(flyteidl.core.Resources.ResourceName) Resource name.

value

(string) Value must be a valid k8s quantity. See https://github.com/kubernetes/apimachinery/blob/master/pkg/api/resource/quantity.go#L30-L80

Enum flyteidl.core.Resources.ResourceName

[flyteidl.core.Resources.ResourceName proto]

Known resource names.

UNKNOWN

(DEFAULT)

CPU

GPU

MEMORY

STORAGE

flyteidl.core.RuntimeMetadata

[flyteidl.core.RuntimeMetadata proto]

Runtime information. This is losely defined to allow for extensibility.

{
  "type": "...",
  "version": "...",
  "flavor": "..."
}
type

(flyteidl.core.RuntimeMetadata.RuntimeType) Type of runtime.

version

(string) Version of the runtime. All versions should be backward compatible. However, certain cases call for version checks to ensure tighter validation or setting expectations.

flavor

(string) optional It can be used to provide extra information about the runtime (e.g. python, golang… etc.).

Enum flyteidl.core.RuntimeMetadata.RuntimeType

[flyteidl.core.RuntimeMetadata.RuntimeType proto]

OTHER

(DEFAULT)

FLYTE_SDK

flyteidl.core.TaskMetadata

[flyteidl.core.TaskMetadata proto]

Task Metadata

{
  "discoverable": "...",
  "runtime": "{...}",
  "timeout": "{...}",
  "retries": "{...}",
  "discovery_version": "...",
  "deprecated_error_message": "..."
}
discoverable

(bool) Indicates whether the system should attempt to lookup this task’s output to avoid duplication of work.

runtime

(flyteidl.core.RuntimeMetadata) Runtime information about the task.

timeout

(google.protobuf.Duration) The overall timeout of a task including user-triggered retries.

retries

(flyteidl.core.RetryStrategy) Number of retries per task.

discovery_version

(string) Indicates a logical version to apply to this task for the purpose of discovery.

deprecated_error_message

(string) If set, this indicates that this task is deprecated. This will enable owners of tasks to notify consumers of the ending of support for a given task.

flyteidl.core.TaskTemplate

[flyteidl.core.TaskTemplate proto]

A Task structure that uniquely identifies a task in the system Tasks are registered as a first step in the system.

{
  "id": "{...}",
  "type": "...",
  "metadata": "{...}",
  "interface": "{...}",
  "custom": "{...}",
  "container": "{...}"
}
id

(flyteidl.core.Identifier) Auto generated taskId by the system. Task Id uniquely identifies this task globally.

type

(string) A predefined yet extensible Task type identifier. This can be used to customize any of the components. If no extensions are provided in the system, Flyte will resolve the this task to its TaskCategory and default the implementation registered for the TaskCategory.

metadata

(flyteidl.core.TaskMetadata) Extra metadata about the task.

interface

(flyteidl.core.TypedInterface) A strongly typed interface for the task. This enables others to use this task within a workflow and gauarantees compile-time validation of the workflow to avoid costly runtime failures.

custom

(google.protobuf.Struct) Custom data about the task. This is extensible to allow various plugins in the system.

container

(flyteidl.core.Container) Known target types that the system will guarantee plugins for. Custom SDK plugins are allowed to set these if needed. If no corresponding execution-layer plugins are found, the system will default to handling these using built-in handlers.

flyteidl.core.ContainerPort

[flyteidl.core.ContainerPort proto]

Defines port properties for a container.

{
  "container_port": "..."
}
container_port

(uint32) Number of port to expose on the pod’s IP address. This must be a valid port number, 0 < x < 65536.

flyteidl.core.Container

[flyteidl.core.Container proto]

{
  "image": "...",
  "command": [],
  "args": [],
  "resources": "{...}",
  "env": [],
  "config": [],
  "ports": []
}
image

(string) Container image url. Eg: docker/redis:latest

command

(string) Command to be executed, if not provided, the default entrypoint in the container image will be used.

args

(string) These will default to Flyte given paths. If provided, the system will not append known paths. If the task still needs flyte’s inputs and outputs path, add $(FLYTE_INPUT_FILE), $(FLYTE_OUTPUT_FILE) wherever makes sense and the system will populate these before executing the container.

resources

(flyteidl.core.Resources) Container resources requirement as specified by the container engine.

env

(flyteidl.core.KeyValuePair) Environment variables will be set as the container is starting up.

config

(flyteidl.core.KeyValuePair) Allows extra configs to be available for the container. TODO: elaborate on how configs will become available.

ports

(flyteidl.core.ContainerPort) Ports to open in the container. This feature is not supported by all execution engines. (e.g. supported on K8s but not supported on AWS Batch)