flytekit.configuration package

Submodules

flytekit.configuration.auth module

flytekit.configuration.auth.ASSUMABLE_IAM_ROLE = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is the role the SDK will use by default to execute workflows. For example, in AWS this should be an IAM role string.

flytekit.configuration.auth.KUBERNETES_SERVICE_ACCOUNT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is the kubernetes service account that will be passed to workflow executions.

flytekit.configuration.aws module

flytekit.configuration.common module

class flytekit.configuration.common.FlyteBoolConfigurationEntry(section, key, default=None, validator=None, fallback=None)[source]

Bases: flytekit.configuration.common._FlyteConfigurationEntry

class flytekit.configuration.common.FlyteConfigurationFile(location=None)[source]

Bases: object

get_bool(section, key, default=None)[source]
Parameters
  • section (Text) –

  • key (Text) –

  • default (bool) –

Return type

bool

get_int(section, key, default=None)[source]
Parameters
  • section (Text) –

  • key (Text) –

  • default (int) –

Return type

int

get_string(section, key, default=None)[source]
Parameters
  • section (Text) –

  • key (Text) –

  • default (Text) –

Return type

Text

reset_config(location)[source]
Parameters

location (Text) –

class flytekit.configuration.common.FlyteIntegerConfigurationEntry(section, key, default=None, validator=None, fallback=None)[source]

Bases: flytekit.configuration.common._FlyteConfigurationEntry

class flytekit.configuration.common.FlyteRequiredBoolConfigurationEntry(section, key, validator=None)[source]

Bases: flytekit.configuration.common._FlyteRequiredConfigurationEntry

class flytekit.configuration.common.FlyteRequiredIntegerConfigurationEntry(section, key, validator=None)[source]

Bases: flytekit.configuration.common._FlyteRequiredConfigurationEntry

class flytekit.configuration.common.FlyteRequiredStringConfigurationEntry(section, key, validator=None)[source]

Bases: flytekit.configuration.common._FlyteRequiredConfigurationEntry

class flytekit.configuration.common.FlyteRequiredStringListConfigurationEntry(section, key, validator=None)[source]

Bases: flytekit.configuration.common._FlyteRequiredConfigurationEntry

class flytekit.configuration.common.FlyteStringConfigurationEntry(section, key, default=None, validator=None, fallback=None)[source]

Bases: flytekit.configuration.common._FlyteConfigurationEntry

class flytekit.configuration.common.FlyteStringListConfigurationEntry(section, key, default=None, validator=None, fallback=None)[source]

Bases: flytekit.configuration.common._FlyteConfigurationEntry

flytekit.configuration.common.format_section_key(section, key)[source]
Parameters
  • section (Text) –

  • key (Text) –

Return type

Text

flytekit.configuration.gcp module

flytekit.configuration.internal module

flytekit.configuration.internal.look_up_version_from_image_tag(tag)[source]
Looks up the image tag from environment variable (should be set from the Dockerfile).

FLYTE_INTERNAL_IMAGE should be the environment variable.

This function is used when registering tasks/workflows with Admin. When using the canonical Python-based development cycle, the version that is used to register workflows and tasks with Admin should be the version of the image itself, which should ideally be something unique like the sha of the latest commit.

Parameters

tag (Text) – e.g. somedocker.com/myimage:someversion123

Return type

Text

flytekit.configuration.platform module

flytekit.configuration.resources module

flytekit.configuration.resources.DEFAULT_CPU_LIMIT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

If not specified explicitly when constructing a task, this limit will be applied as the default. Follows Kubernetes CPU request/limit format.

flytekit.configuration.resources.DEFAULT_CPU_REQUEST = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

If not specified explicitly when constructing a task, this request will be applied as the default. Follows Kubernetes CPU request/limit format.

flytekit.configuration.resources.DEFAULT_GPU_LIMIT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

If not specified explicitly when constructing a task, this limit will be applied as the default. Follows Kubernetes GPU request/limit format.

flytekit.configuration.resources.DEFAULT_GPU_REQUEST = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

If not specified explicitly when constructing a task, this request will be applied as the default. Follows Kubernetes GPU request/limit format.

flytekit.configuration.resources.DEFAULT_MEMORY_LIMIT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

If not specified explicitly when constructing a task, this limit will be applied as the default. Follows Kubernetes memory request/limit format.

flytekit.configuration.resources.DEFAULT_MEMORY_REQUEST = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

If not specified explicitly when constructing a task, this request will be applied as the default. Follows Kubernetes memory request/limit format.

flytekit.configuration.resources.DEFAULT_STORAGE_LIMIT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

If not specified explicitly when constructing a task, this limit will be applied as the default. Follows Kubernetes storage request/limit format.

flytekit.configuration.resources.DEFAULT_STORAGE_REQUEST = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

If not specified explicitly when constructing a task, this request will be applied as the default. Follows Kubernetes storage request/limit format.

flytekit.configuration.sdk module

flytekit.configuration.sdk.EXECUTION_ENGINE = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is a comma-delimited list of package strings, in order, for resolving execution behavior.

TODO: Explain how this would be used to extend the SDK

flytekit.configuration.sdk.LAUNCH_PLAN_NAME_FORMAT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is a Python format string which the SDK will use to generate names for launch plans. Any template portion of the string can only include ‘module’ or ‘name’. So ‘{name}’ is valid, but ‘{key}’ is not. If not specified, we fall back to the configuration for flytekit.configuration.sdk.NAME_FORMAT

flytekit.configuration.sdk.LOCAL_SANDBOX = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is the path where SDK will place files during local executions and testing. The SDK will not automatically clean up data in these directories.

flytekit.configuration.sdk.LOGGING_LEVEL = <flytekit.configuration.common.FlyteIntegerConfigurationEntry object>

This is the default logging level for the Python logging library and will be set before user code runs. Note that this configuration is special in that it is a runtime setting, not a compile time setting. This is the only runtime option in this file.

flytekit.configuration.sdk.NAME_FORMAT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is a Python format string which the SDK will use to generate names for discovered entities. The default is ‘{module}.{name}’ which will result in strings like ‘package.module.name’. Any template portion of the string can only include ‘module’ or ‘name’. So ‘{name}’ is valid, but ‘{key}’ is not.

flytekit.configuration.sdk.PARQUET_ENGINE = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is the parquet engine to use when reading data from parquet files.

flytekit.configuration.sdk.ROLE = <flytekit.configuration.common.FlyteRequiredStringConfigurationEntry object>

This is the role the SDK will use by default to execute workflows. For example, in AWS this should be an IAM role string.

flytekit.configuration.sdk.SDK_PYTHON_VENV = <flytekit.configuration.common.FlyteStringListConfigurationEntry object>

This is a list of commands/args which will be prefixed to the entrypoint command by SDK.

flytekit.configuration.sdk.TASK_NAME_FORMAT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is a Python format string which the SDK will use to generate names for tasks. Any template portion of the string can only include ‘module’ or ‘name’. So ‘{name}’ is valid, but ‘{key}’ is not. If not specified, we fall back to the configuration for flytekit.configuration.sdk.NAME_FORMAT

flytekit.configuration.sdk.TYPE_ENGINES = <flytekit.configuration.common.FlyteStringListConfigurationEntry object>

This is a comma-delimited list of package strings, in order, for resolving type behavior.

TODO: Explain how this would be used to extend the SDK

flytekit.configuration.sdk.WORKFLOW_NAME_FORMAT = <flytekit.configuration.common.FlyteStringConfigurationEntry object>

This is a Python format string which the SDK will use to generate names for workflows. Any template portion of the string can only include ‘module’ or ‘name’. So ‘{name}’ is valid, but ‘{key}’ is not. If not specified, we fall back to the configuration for flytekit.configuration.sdk.NAME_FORMAT

flytekit.configuration.sdk.WORKFLOW_PACKAGES = <flytekit.configuration.common.FlyteStringListConfigurationEntry object>

This is a comma-delimited list of packages that SDK tools will use to discover entities for the purpose of registration and execution of entities.

flytekit.configuration.statsd module

Module contents

class flytekit.configuration.TemporaryConfiguration(new_config_path, internal_overrides=None)[source]

Bases: object

flytekit.configuration.set_flyte_config_file(config_file_path)[source]
Parameters

config_file_path (Text) –