flytekit.tools package

Submodules

flytekit.tools.lazy_loader module

class flytekit.tools.lazy_loader.LazyLoadPlugin(plugin_name, plugin_requirements, related_modules)[source]

Bases: object

LAZY_LOADING_PLUGINS = {'hive_sensor': ['hmsclient>=0.0.1,<1.0.0'], 'schema': ['numpy>=1.14.0,<2.0.0', 'pandas>=0.22.0,<1.0.0', 'pyarrow>=0.11.0,<1.0.0'], 'sidecar': ['k8s-proto>=0.0.2,<1.0.0'], 'spark': ['pyspark>=2.4.0,<3.0.0']}
classmethod get_extras_require()[source]
Return type

dict[Text,list[Text]]

flytekit.tools.lazy_loader.lazy_load_module(module)[source]
Parameters

module (Text) –

Return type

_types.ModuleType

flytekit.tools.module_loader module

flytekit.tools.module_loader.iterate_modules(pkgs)[source]
flytekit.tools.module_loader.iterate_registerable_entities_in_order(pkgs, ignore_entities=None, include_entities=None, detect_unreferenced_entities=True)[source]

This function will iterate all discovered entities in the given package list. It will then attempt to topologically sort such that any entity with a dependency on another comes later in the list. Note that workflows can reference other workflows and launch plans. :param list[Text] pkgs: :param set[type] ignore_entities: If specified, ignore these entities while doing a topological sort. All other

entities will be taken. Only one of ignore_entities or include_entities can be set.

Parameters
  • include_entities (set[type]) – If specified, include these entities while doing a topological sort. All other entities will be ignored. Only one of ignore_entities or include_entities can be set.

  • detect_unreferenced_entities (bool) – If true, we will raise exceptions on entities not included in the package configuration.

Return type

module, Text, flytekit.common.mixins.registerable.RegisterableEntity

flytekit.tools.module_loader.load_workflow_modules(pkgs)[source]

Load all modules and packages at and under the given package. Used for finding workflows/tasks to register.

Parameters

pkgs (list[Text]) – List of dot separated string containing paths folders (packages) containing the modules (python files)

:raises ImportError

flytekit.tools.subprocess module

flytekit.tools.subprocess.check_call(cmd_args, **kwargs)[source]

Module contents