Skip to content

Releases: flyteorg/flyte

Flyte v1.16.0-b4 milestone release

30 Jul 22:50
@flyte-bot flyte-bot
v1.16.0-b4
bb266c5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Flyte v1.16.0-b4 milestone release Pre-release
Pre-release

Flyte 1.16.0-b4

This is a re-release of b3, only to pick up an updated console image.

Full Changelog: v1.16.0-b2...v1.16.0-b3

Loading
0 Join discussion

Flyte v1.16.0-b3 milestone release

29 Jul 11:03
@flyte-bot flyte-bot
v1.16.0-b3
e407c2e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Flyte v1.16.0-b3 milestone release Pre-release
Pre-release

Flyte 1.16.0-b3

Release highlights

Added

Changed

Fixed

Dependencies

  • Bumped github.com/go-viper/mapstructure/v2 from 2.1.0 to 2.3.0 (#6512) by @dependabot[bot]

Full Changelog

v1.16.0-b2...v1.16.0-b3

Contributors

Sovietaced, dependabot, and 9 other contributors
Loading
0 Join discussion

Flyte v1.16.0-b2 milestone release

05 Jun 22:16
@flyte-bot flyte-bot
v1.16.0-b2
c710fb6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Flyte v1.16.0-b2 milestone release Pre-release
Pre-release

Flyte 1.16.0-b2 release notes

What's Changed

Contributors

pingsutw and flyte-bot
Loading
0 Join discussion

Flyte v1.15.3 milestone release

22 Apr 02:49
@flyte-bot flyte-bot
v1.15.3
08a8d43
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Flyte v1.15.3 milestone release Latest
Latest

Flyte 1.15.3 release notes

Added

  • Made ArrayNode taskPhaseVersion increment retries configurable. #6374 by @hamersaw
  • Added support for setting s3.endpoint in flyte-core. #6401 by @dwo
  • Added OIDC configuration option to allow mismatched discovery and issuer URLs. #5712 by @ddl-ebrown

Changed

  • Changed QPS configuration parameter type to float64. #6303 by @pingsutw
  • Refactored code to utilize built-in max/min functions for simplification. #6379 by @evenevent

Fixed

Contributors

We thank the following contributors for their work in this release:
@fg91, @hamersaw, @pingsutw, @evenevent, @Future-Outlier, @Sovietaced, @dwo, @flyte-bot, @ddl-ebrown

Full Changelog: v1.15.1...v1.15.3

Contributors

dwo, Sovietaced, and 7 other contributors
Assets 3
Loading

flytectl/v0.9.5

07 Apr 18:30
@flyte-bot flyte-bot
flytectl/v0.9.5
f2a1ad7
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
flytectl/v0.9.5

Changelog

  • 2c84109 Add instructions on how to yank a flytectl release (#6105)
  • 754ab74 Bump github.com/golang-jwt/jwt/v5 from 5.2.1 to 5.2.2 in /flyteidl (#6361)
  • 222aff2 Bump golang.org/x/net from 0.27.0 to 0.33.0 in /flytestdlib (#6214)
  • 4bf0f82 Bump jinja2 from 3.1.4 to 3.1.5 in /flytectl/docs (#6126)
  • b779bed Bump jinja2 from 3.1.5 to 3.1.6 in /flytectl/docs (#6310)
  • 5aedd11 Dep: Upgrade kubeflow training operator (#6294)
  • b8fb68d Fix: Upgrade stow to fix listing error files (#6159)
  • d5499e8 Remove mockery fork (#6280)
  • 62ca2e3 Remove mockery fork logic (#6288)
  • 34d124d Update flytectl_update_workflow-execution-config.rst (#5865)
  • fd9a378 Upgrade golang.org/x/crypto lib to address vulnerability (#6133)
  • c0eb6de fix: change to cmd ./bin/flytectl demo start (#6290)
Loading

Flyte v1.15.1 milestone release

28 Mar 22:39
@flyte-bot flyte-bot
v1.15.1
4164ac4
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Flyte v1.15.1 milestone release

Flyte 1.15.1 release notes

New Features & Enhancements

  • Per-Launch Plan Notifications: Added support for custom notification templates tied to specific launch plans (#6064).

  • AWS Secret Manager Integration: Environment variables can now reference secrets stored in AWS Secrets Manager (#6316).

  • Dynamic Pod Configuration: Improved merging of Ray head/worker node pod templates (#6232) and resource limits for Kubernetes plugins (Dask, Kubeflow) (#6264).

  • Array Node Improvements: Added support for bound inputs (#6276) and preemption handling (#6259).

  • Agent Enhancements: Added structured logging for agents (#6344) and reserved fields in execution payloads (#6321).

  • Connector Plugin: Introduced a new Connector plugin framework for extensibility (#6332).

Bug Fixes

  • Error Handling: Fixed critical errors in delete operations (#6269), array node metrics (#6291), and fastcache handling for failed nodes (#6318).

  • Dynamic Log Links: Enabled dynamic log links for Kubeflow plugins to improve debugging (#6284).

  • Preemption Handling: Addressed edge cases in pod preemption logic for worker nodes (#6259).

Security Patches

Documentation Updates

  • Streaming Decks: Added documentation for streaming deck visualization in Flyte fundamentals (#6251, #6272).

  • Slurm GPU Clusters: New guides for setting up Slurm agent (#6231) and GPU clusters (#6273).

  • Dynamic Log Links: Documented usage for Kubernetes-based task plugins (#6285).

  • Launch Plan Chaining: Explained how to chain launch plans effectively (#6317).

Other Improvements & Maintenance

  • CI/CD: Aligned Go versions in CI with go.mod (#5736), removed mockery fork (#6280, #6288), and improved codecov reporting (#6299).

Contributor Experience

  • Updated contribution docs for flytectl (#6290) and streamlined development workflows.

New Contributors

A warm welcome to first-time contributors:
@rustco, @mwaylonis, @sjtucoder, @bra-fsn, @CtfChan, @vlada-dudr, @jamestwebber, @0yukali0 🎉

Full changelog

Read more

Contributors

EngHabu, honnix, and 25 other contributors
Loading

Flyte v1.15.0 milestone release

15 Feb 03:06
@flyte-bot flyte-bot
v1.15.0
ee919f8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Flyte v1.15.0 milestone release

Flyte 1.15.0 release notes

Added

Streaming Decks (#2779)

Decks is a Flyte feature that provides customizable visualization tools for tasks, offering both default and user-defined insights
For use cases like Hyperparameter Optimization, seeing the metrics, data, logs, and links on the Flyte Deck while the task is running is extremely useful. However, the Flyte Deck HTML is uploaded to the remote bucket only after the task succeeds.
With this change, flytepropeller now sends the Deck URI to flyteadmin once the task starts running. This feature is also helpful for debugging, identifying your task’s dependencies and even looking at the source code that is being executed.

Screenshot 2025-02-14 at 2.10.27 PM

Define shared memory in the task definition

Several ML libraries make use of shared memory, so this is now easier to do in Flyte. This new argument can be either a bool or a string value, the former means "memory backed volumes are sized to node allocatable memory", whereas the latter means that a volume of that size will be defined in the podspec used to run the task. For example, the simple case:

import os
from flytekit import task

@task(shared_memory=True)
def t1() -> bool:
    # /dev/shm is available in the task
    ...

or a more complex case:

import os
from flytekit import task

@task(shared_memory="10Gi")
def t1() -> bool:
    # /dev/shm of size 10Gi is available in the task
    ...

Progress bars to help visualize package creation and upload

Flytekit includes now progress tracking, including package creation/compression progress visualization.

Here's how the progress bar looks like:

image

And after files are compressed and uploaded:

image

Debug @dynamic tasks

Advancing the debug story we've been chipping away, we now support debugging @dynamic tasks.

Changed

Better Secrets handling (#3048)

Flyte enables tasks to requests secrets managed by Kubernetes or by external systems.
If you want to map and mount a Kubernetes secret with an environment variable, flytekit required you to export an environment variable as _FSEC_<GROUP>_<GROUP_VERSION>_<KEY> with GROUP corresponding to the Secret name, and KEY being the actual key to retrieve from the secret. This was inconvenient and too verbose.
Starting with this release, you can easily configure secrets from environment variables::

@task(secret_requests=[Secret(..., env_var="MY_ENVVAR", mount_requirement=Secret.MountType.ENV_VAR)
def hello():
    ...

Or from files:

@task(secret_requests=[Secret(..., env_var="MY_TOKEN_PATH", mount_requirement=Secret.MountType.FILE)
def hello():
    ...

Override Pod template using .with_overrides (#2981)

Flyte lets you specify default resources for task executions to enable deterministic scheduling behavior.
The .with_overrides method allows you to override the global resource defaults dynamically:

@workflow
def my_pipeline(x: typing.List[int]) -> int:
    return square_1(x=count_unique_numbers_1(x=x)).with_overrides(limits=Resources(cpu="6", mem="500Mi"))

This method has been available for Resources only. Starting with this release, you can now override the Pod template too.

Similar to Resources, you can instruct flytepropeller to use a custom Pod template for the task executions, setting the template at different levels: from namespace-wide to the task configuration.
Starting with this release, you can override dynamically the Pod template when the task is invoked in the workflow:

@task
def say_hello() -> str:
    return "Hello, World!"

@workflow
def hello_world_wf() -> str:
    res = say_hello().with_overrides(limits=Resources(cpu="2", mem="600Mi"),pod_template=PodTemplate(
        primary_container_name="primary-nelson",
        labels={"lKeyA": "lValA", "lKeyB": "lValB"},
        annotations={"aKeyA": "aValA", "aKeyB": "aValB"},
        pod_spec=V1PodSpec(
            containers=[
                V1Container(
                    name="primary-nelson",
                    image="arbaobao/flyte-test-images:pythonpath5",
                    env=[V1EnvVar(name="eKeyC", value="eValC"), V1EnvVar(name="eKeyD", value="eValD")],
                ),
                V1Container(
                    name="primary-nelson2",
                    image="arbaobao/flyte-test-images:pythonpath5",
                    env=[V1EnvVar(name="eKeyC", value="eValC"), V1EnvVar(name="eKeyD", value="eValD")],
                ),
            ],
        )
    ))
    return res

This change provides more flexible pod configuration management while maintaining compatibility with existing resource override patterns.

Introducing Caching Policies (#3129)

Flyte provides robust caching mechanisms to optimize workflow performance. Two key features users can leverage today include:

a. Task Output Caching with Deterministic Inputs

Users can enable task-level caching by setting cache=True in the @task decorator. This ensures that tasks with identical inputs reuse previously computed outputs, saving time and resources.

Example:

@task(cache=True, cache_version="1.0")  
def square(n: int) -> int:  
    return n * n

b. Custom Hashing for Offloaded Objects (e.g., DataFrames)

Flyte allows users to define custom hash functions for non-deterministic objects (like pandas DataFrames) using Annotated types and HashMethod. This ensures caching works even for complex data types.

Example:

def hash_pandas_dataframe(df: pd.DataFrame) -> str:  
    return str(pd.util.hash_pandas_object(df))  

@task  
def data_loader(cache=True, cache_version="1.0",
) -> Annotated[pd.DataFrame, HashMethod(hash_pandas_dataframe)]:  
    return pd.DataFrame(...)

This release brings a refactoring of the Flyte’s caching system into a unified Cache object, deprecating legacy parameters and adding advanced features:

1. Unified Cache Configuration Object

All caching parameters (existing and new) are now grouped under a single Cache class:

class Cache:  
    version: Optional[str] = None # Replaces `cache_version`  
    serialize: bool = False      # Replaces `cache_serialize`  
    ignored_inputs: Union[Tuple[str, ...], str] = ()  # New: Exclude inputs from cache key  
    salt: str = ""           New: Add unique salt to cache key  
    policies: Optional[Union[List[CachePolicy], CachePolicy]] = None  # New: Dynamic versioning

Deprecation Notice: Legacy parameters (e.g., cache_version, cache_serialize) are deprecated but remain supported.

2. Advanced Cache Key Customization

  • Cache Policies: Define dynamic versioning logic via CachePolicy protocols.

  • Policies generate version strings through the get_version method. Those are then concatenated and hashed (SHA-256) to form the final cache key.

In case a Cache object is defined, we require the definition of policies

  • Salt: Add a unique string to the cache key to avoid collisions (e.g., differentiate between teams sharing the same task).

  • Ignored Inputs: Exclude specific inputs (e.g., request_id) from cache key calculation using ignored_inputs.

c. Migration Path

Existing code using cache=True or cache_version will continue to work, but users are encouraged to migrate to the Cache object:

@task(cache=Cache(version="1.0", serialize=True, ignored_inputs=("debug_mode",)))  
def my_task(...) -> ...:

Faster GitIgnore checks

Algorithmic speed-ups are always welcome and this is exactly what happened in flyteorg/flytekit#3007. Enjoy the massive speedup!

Full changelog

  • docs: Update datatype mapping for polars Dataframe and LazyFrame by @HansBambel in #6109
  • Add custom info field to ExternalResourceInfo by @hamersaw in #6115
  • improve error message in the agent plugin by @pingsutw in #6114
  • upstream ArrayNode DataMode + BindingData LiteralOffloadedMetadata idl changes by @pvditt in #6074
  • Pass CustomInfo metadata through ArrayNode ExternalResourceInfo (#591) by @hamersaw in #6116
  • [Docs] Add new page for jupyter notebook interactive mode by @Mecoli1219 in #6036
  • Use child dir for branch taken by @andrewwdye in #6120
  • auto-update contributors by @flyte-bot in #6047
  • Improve resource manager test coverage by @luckyarthur in #5973
  • Add nodeName template var and update doc by @ketian-indeed in #6088
  • Allow flytepropeller manager to have different resource request from flyte propeller by @cpaulik in #5974
  • Unpin flytectl from single-binary tests by @eapolinario in #6113
  • Add new sandbox-bundled-offloaded-functional-tests for offloading literals unit test by @Mecoli1219 in #6111
  • chore: fix some function names in comment by @loselarry in #6122
  • Flyteadmin digest comparison should rely on database semantics by @popojk in #6058
  • Upgrade golang.org/x/crypto lib to address vulnerability by @katrogan in #6133
  • chore: using dom...
Read more

Contributors

EngHabu, cpaulik, and 35 other contributors
Loading

Flyte v1.14.1 milestone release

17 Dec 20:18
@flyte-bot flyte-bot
v1.14.1
8a5f54c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Flyte v1.14.1 milestone release

Flyte 1.14.1 Release Notes

  • Update flytestdlib and affected tools (copilot) for missing config.

What's Changed

New Contributors

Full Changelog: v1.14.0...v1.14.1

Contributors

eapolinario, katrogan, and 5 other contributors
Loading

flytectl/v0.9.4

11 Dec 18:15
@flyte-bot flyte-bot
flytectl/v0.9.4
2a7d363
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
flytectl/v0.9.4

Changelog

  • b3330ba Adopt protogetter (#5981)
  • 47edd99 Fix Union type with dataclass ambiguous error and support superset comparison (#5858)
  • 636cc23 Fix remaining misuses of capturing the default file descriptors in flytectl unit tests (#5950)
  • 121665d Fix: customize demo cluster port (#5969)
  • ffa1672 Regenerate flytectl docs (#5914)
  • 86c63f7 Remove explict go toolchain versions (#5723)
  • b9900fe Update aws-go-sdk to v1.47.11 to support EKS Pod Identity (#5796)
  • e4d29f8 Upstream contributions from Union.ai (#5769)
  • 9abfbda [Flyte][3][flytepropeller][Attribute Access][flytectl] Binary IDL With MessagePack (#5763)
  • 704f8eb fix: align the default config output (#5947)
Loading

Flyte v1.14.0 milestone release

06 Dec 23:25
@flyte-bot flyte-bot
v1.14.0
c91fb69
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
Flyte v1.14.0 milestone release

Flyte 1.14.0 Release Notes

Added

Support for FlyteDirectory as input to ContainerTask (#5715)

A significant update to the flytekit storage interface enables downloading multi-part blobs. This allows Flyte to copy a FlyteDirectory as an input to ContainerTasks, enabling higher flexibility in workflow development with raw containers.

Fixed

Better handling of CORS in TLS connections (#5855)

When using Flyte with TLS certificates, CORS options were not enabled causing issues like the Flyte console UI not showing any project when multiple were created. This scenario came with relative frequency among users evaluating Flyte in non-production environments. Now, CORS will be enabled on TLS connections too.

Changed

Enhanced flexibility for Ray plugin configurations (#5933)

This release makes the configuration of RayJobs more flexible, letting you pass Pod specs independently for each Ray node type: Worker, Head, and Submitter. This enables you to declare configuration for each group to better align with your infrastructure requirements.

Example:

ray_config = RayJobConfig(
    worker_node_config=
    [WorkerNodeConfig(group_name="test_group", replicas=3, min_replicas=0, max_replicas=10, 
    k8s_pod=V1PodSpec (containers=[{name="ray-worker", 
    resources=V1ResourceRequirements(requests={....}, limits={....})])
  ],
    head_node_config=HeadNodeConfig(k8s_pod=V1PodSpec(containers=[{name="ray-head", 
    resources=V1ResourceRequirements(requests={....}, limits={....}])),
    runtime_env={"pip": ["numpy"]},
    enable_autoscaling=True,
    shutdown_after_job_finishes=True,
)

Breaking

  • As Python 3.8 hit the End of Life period in October 2024, starting with this release, flytekit requires Python >=3.9.

  • The flyte-core Helm chart doesn't set the appProtocol in the Service manifests by default anymore. This might affect deployments that use the Istio service mesh.

Full changelog

Read more

Contributors

ctso, bgedik, and 47 other contributors
Loading