Skip to main content

orbitra.dags.airflow.client

OrbitraDagsClient: list, trigger and follow DAG runs through the Airflow REST API (/api/v2). The client asks its TokenCredential for a token on every request, so a long wait outlives the token it started with. get_dags_client builds one from the Orbitra settings and the Orbitra login of an environment. Every API failure is a :class:DagsError with a one-line message: a missing DAG, run or task names the id, a refused token says how to log in again, and anything else carries the HTTP status and the server’s detail.

Functions

get_dags_client

Build an OrbitraDagsClient for the Airflow of an Orbitra environment, authenticated as the Orbitra login. Args:
  • environment: The Orbitra environment; resolved from the settings sources when None.
Returns:
  • A client; no token is requested until its first call.
Raises:
  • DagsError: The environment’s settings do not say where its Airflow API is.
Examples:

Classes

DagsError

A call to the Airflow API failed; the message says why in one line.

OrbitraDagsClient

A client of the Airflow REST API of an Orbitra environment. Methods:

close

Close the HTTP connections the client keeps open.

get_dag_run

Read a DAG run. Args:
  • dag_id: The DAG id.
  • dag_run_id: The run id.
Returns:
  • The run as it is now.

get_task_logs

Read the full log of one try of a task instance. Args:
  • dag_id: The DAG id.
  • dag_run_id: The run id.
  • task_id: The task id.
  • try_number: The try to read; None reads the task instance’s current try.
  • map_index: The map index of a mapped task instance; -1 for a task that is not mapped.
Returns:
  • The log lines, oldest first.

list_dag_runs

List the runs of a DAG, most recent first. Args:
  • dag_id: The DAG id.
  • limit: How many runs to return, from 1 to 100.
  • offset: How many runs to skip, for pagination.
  • states: Only return runs in one of these states (queued, running, success, failed, in any case).
  • start_time: Only return runs that started at or after this time. A naive datetime is read as UTC.
  • end_time: Only return runs that started at or before this time. A naive datetime is read as UTC.
Returns:
  • One page of runs.
Raises:
  • ValueError: limit, offset or a state is invalid; nothing is requested.

list_dags

List the DAGs the server knows, sorted by DAG id; stale DAGs (whose file is gone) are left out. Args:
  • name_like: Only list DAGs whose id contains this text, case-insensitively.
Returns:
  • Every matching DAG, read across all pages.

list_task_instances

List the task instances of a DAG run, every map index of a mapped task included. Args:
  • dag_id: The DAG id.
  • dag_run_id: The run id.
Returns:
  • Every task instance of the run, read across all pages.

trigger_dag_run

Trigger a run of a DAG now, the way the Airflow UI does. Args:
  • dag_id: The DAG to run.
  • conf: The run configuration (dag_run.conf).
Returns:
  • The id of the new run.
Examples:

wait_for_dag_run

Poll a DAG run until it ends in success or failed. Args:
  • dag_id: The DAG id.
  • dag_run_id: The run id.
  • poll_interval: Seconds between two reads of the run.
  • timeout: Seconds to wait before giving up; None waits until the run ends.
  • raise_on_failure: Raise when the run ends failed; False returns the failed run instead.
Returns:
  • The finished run.
Raises:
  • DagsError: The run failed (with raise_on_failure) or the timeout passed.