Workflow on Dapr Docs

Workflow overview

Mon, 01 Jan 0001 00:00:00 +0000

Dapr workflow makes it easy for developers to write business logic and integrations in a reliable way. Since Dapr workflows are stateful, they support long-running and fault-tolerant applications, ideal for orchestrating microservices. Dapr workflow works seamlessly with other Dapr building blocks, such as service invocation, pub/sub, state management, and bindings.

The durable, resilient Dapr Workflow capability:

Offers a built-in workflow runtime for driving Dapr Workflow execution.
Provides SDKs for authoring workflows in code, using any language.
Provides HTTP and gRPC APIs for managing workflows (start, query, pause/resume, raise event, terminate, purge).

Some example scenarios that Dapr Workflow can perform are:

Features and concepts

Mon, 01 Jan 0001 00:00:00 +0000

Now that you’ve learned about the workflow building block at a high level, let’s deep dive into the features and concepts included with the Dapr Workflow engine and SDKs. Dapr Workflow exposes several core features and concepts which are common across all supported languages.

Note

For more information on how workflow state is managed, see the workflow architecture guide.

Workflows

Dapr Workflows are functions you write that define a series of tasks to be executed in a particular order. The Dapr Workflow engine takes care of scheduling and execution of the tasks, including managing failures and retries. If the app hosting your workflows is scaled out across multiple machines, the workflow engine load balances the execution of workflows and their tasks across multiple machines.

Workflow versioning

Mon, 01 Jan 0001 00:00:00 +0000

Versioning

There are many scenarios where it is necessary to introduce changes to workflow code while workflows are actively running. In cases where these changes are non-deterministic, a versioning strategy is required so that existing workflows can continue executing with the original code, while new workflows use the updated version.

Workflows can be versioned using two complementary approaches and it’s intended that both are used in conjunction with one another. These are:

Workflow patterns

Mon, 01 Jan 0001 00:00:00 +0000

Dapr Workflows simplify complex, stateful coordination requirements in microservice architectures. The following sections describe several application patterns that can benefit from Dapr Workflows.

Task chaining

In the task chaining pattern, multiple steps in a workflow are run in succession, and the output of one step may be passed as the input to the next step. Task chaining workflows typically involve creating a sequence of operations that need to be performed on some data, such as filtering, transforming, and reducing.

Workflow architecture

Mon, 01 Jan 0001 00:00:00 +0000

Dapr Workflows allow developers to define workflows using ordinary code in a variety of programming languages. The workflow engine runs inside of the Dapr sidecar and orchestrates workflow code deployed as part of your application. Dapr Workflows are built on top of Dapr Actors providing durability and scalability for workflow execution.

This article describes:

The architecture of the Dapr Workflow engine
How the workflow engine interacts with application code
How the workflow engine fits into the overall Dapr architecture
How different workflow backends can work with workflow engine

For more information on how to author Dapr Workflows in your application, see How to: Author a workflow.

How to: Author a workflow

Mon, 01 Jan 0001 00:00:00 +0000

This article provides a high-level overview of how to author workflows that are executed by the Dapr Workflow engine.

Note

If you haven’t already, try out the workflow quickstart for a quick walk-through on how to use workflows.

Author workflows as code

Dapr Workflow logic is implemented using general purpose programming languages, allowing you to:

Use your preferred programming language (no need to learn a new DSL or YAML schema).
Have access to the language’s standard libraries.
Build your own libraries and abstractions.
Use debuggers and examine local variables.
Write unit tests for your workflows, just like any other part of your application logic.

The Dapr sidecar doesn’t load any workflow definitions. Rather, the sidecar simply drives the execution of the workflows, leaving all the workflow activities to be part of the application.

How to: Manage workflows

Mon, 01 Jan 0001 00:00:00 +0000

Now that you’ve authored the workflow and its activities in your application, you can start, terminate, rerun, and get information about the workflow using the CLI or API calls.

Managing Workflows with the Dapr CLI

The Dapr CLI provides commands for managing workflow instances in both self-hosted and Kubernetes environments.

Multi Application Workflows

Mon, 01 Jan 0001 00:00:00 +0000

It is often the case that a single workflow spans multiple applications, microservices, or programming languages. This is where an activity or a child workflow will be executed on a different application than the one hosting the parent workflow.

Some scenarios where this is useful include:

A Machine Learning (ML) training activity must be executed on GPU-enabled machines, while the rest of the workflow runs on CPU-only orchestration machines.
Activities need access to sensitive data or credentials that are only available to particular identities or locales.
Different parts of the workflow need to be executed in different trust zones or networks.
Different parts of the workflow need to be executed in different geographic regions due to data residency requirements.
An involved business process spans multiple teams or departments, each owning their own application.
Implementation of a workflow spans different programming languages based on team expertise or existing codebases.
Different team boundaries or microservice ownership.

The diagram below shows an example scenario of a complex workflow that orchestrates across multiple applications that are written in different languages. Each applications’ main steps and activities are:

History Retention Policy

Mon, 01 Jan 0001 00:00:00 +0000

Dapr workflow state is stored in the [actor state store]{{ %ref workflow-architecture.md#state-store-usage %}}. By default, Dapr Workflows retains the complete history of workflow state changes indefinitely. This means historical workflows can be queried and inspected at any time. Running many workflows or workflows which generate large amounts of state change history can lead to increased storage usage, which can eventually fill up the state store disk space.

To help manage storage usage, Dapr Workflows supports configuring a history retention policy. The retention policy defines how long to retain workflow state change history before it is deleted. Workflows will only be eligible for deletion once it has reached a terminal state (Completed, Failed, or Terminated). Each workflow terminal state can have a custom retention duration, as well as a default retention duration for any terminal states not explicitly configured. The duration is defined as a Go duration string (for example, 72h for 72 hours, or 30m for 30 minutes).

Workflow Execution Concurrency

Mon, 01 Jan 0001 00:00:00 +0000

You can configure the maximum concurrent workflows and activities that can be executed at any one time with the following configuration. These limits are imposed on a per sidecar basis, meaning that if you have 10 replicas of your workflow app, the effective limit is 10 times the configured value.

Setting these limits can help prevent resource exhaustion on your Dapr sidecar and application, or to drain down a backlog of workflows if there had been a spike in activity causing resource contention. These limits do not distinguish between different workflow or activity definitions, so they apply to all workflows and activities running in the sidecar.