Skip to content

Limit the number of fleet consolidation attempts #3624

@r4victor

Description

@r4victor

If a user creates a fleet with nodes.min > 0, dstack server will be trying to maintain nodes.min forever. Even if the project has no offers that can be provisioned (e.g. no backends or backends using dstack Sky creds with 0 balance), dstack will still try to create missing instances forever. Currently, the provisioning interval is 10m meaning every 10min dstack creates nodes.min InstanceModels and events.

We should either terminate the fleet after some number of failed consolidation attempts or introduce a new status/flag that would keep the fleet active while disabling any further consolidation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions