Skip to main content

orbitra.dags.airflow.operators.uv

Run a Python callable inside a virtual environment built from a uv.lock.

Functions

default_venv_cache_path

Return the virtual environment cache directory configured in the Orbitra DAGs settings. Returns:
  • The orbitra_dags_venv_cache setting (environment ORBITRA_DAGS_VENV_CACHE, .env or
  • [tool.orbitra]; default ~/.cache/orbitra-dags/venvs) with ~ expanded.

Classes

UvVirtualenvOperator

Run a function in a virtual environment built with uv sync from a uv.lock. The task venv is defined by a directory holding pyproject.toml and uv.lock (the “uv project”). At run time the operator creates a venv (uv venv), installs the locked dependency set into it (uv sync --frozen ...) and executes the callable in a subprocess of that venv, like PythonVirtualenvOperator does. Because the lock pins every package, two runs always install the same versions. Project discovery. uv_project_dir may be absolute, relative to the DAG bundle root, or relative to the DAG file’s directory. When omitted the operator walks up from the DAG file’s directory until it finds a directory containing both files, stopping at the bundle root (or the filesystem root under dag.test()), the same way uv itself discovers a project. Discovery runs on the worker because bundle paths only exist there. DAGs deployed as a zip work too: the archive is extracted once per content hash into a bundles folder next to the venv cache, and the project is looked up inside it before the directories above the archive. Supported dependencies. This version installs packages published to a package index (public or private registry) and pinned by uv.lock; the project itself is a set of dependencies and is not installed (--no-install-project). Dependencies from local paths, directories, workspace members, editable installs, git or direct URLs are rejected before any venv is built, with an error naming them. The check covers exactly what the task selects (groups, extras and transitive dependencies). Venv cache. By default venvs are cached in a venv-<hash> folder under the directory given by the Orbitra DAGs setting orbitra_dags_venv_cache (environment ORBITRA_DAGS_VENV_CACHE, .env or [tool.orbitra] in pyproject.toml; default ~/.cache/orbitra-dags/venvs). The hash covers the SHA-256 of uv.lock and pyproject.toml, the resolved project directory, the sync options, groups, extras, Python version, system_site_packages and the pinned package set, so any change to the lock produces a new venv. Set the orbitra_dags_venv_cache_key setting (ORBITRA_DAGS_VENV_CACHE_KEY) to any new value to force a rebuild of all cached venvs. Pass venv_cache_path=None to build a throw-away venv on every run. Source relocation. Instead of pasting the function into the temporary script, the operator compiles it with the absolute path of the DAG file and its original line numbers. Breakpoints set in the DAG file fire inside the venv subprocess (with subprocess debugging enabled in the IDE) and tracebacks point at the DAG file. Azure SDK logs. The azure logger of the subprocess is set to the orbitra_dags_azure_log_level setting (ORBITRA_DAGS_AZURE_LOG_LEVEL or [tool.orbitra] of the uv project’s pyproject.toml; default WARNING) before the function runs, so the SDKs’ HTTP request/response dumps stay out of the task log. dag.test(). Before Airflow 3.5, Variable.get, Connection.get and XCom pulls from inside the venv hang under dag.test() (apache/airflow#65587). There they fail at once with a hint instead: pass the value as a templated argument ("{{ var.json.get('my_var', {}) }}"), which works everywhere. On the runtime they work. The function must be defined with def (no lambdas, no generators), all imports must happen inside it and it may not reference variables from the enclosing scope. Arguments and the return value travel through the configured serializer (pickle by default). The results of a mapped task (step.expand(...)) arrive as a list. Args:
  • python_callable: Function to run inside the venv.
  • uv_project_dir: Directory with pyproject.toml + uv.lock (templated). None enables automatic discovery as described above.
  • uv_sync_options: Flags for uv sync; see DEFAULT_UV_SYNC_OPTIONS.
  • uv_groups: [dependency-groups] to include on top of the defaults (--group X).
  • uv_extras: [project.optional-dependencies] extras to include (--extra X).
  • venv_cache_path: Parent directory for cached venvs (templated). Defaults to the orbitra_dags_venv_cache setting, read on the worker at run time; None disables caching.
  • python_version: Python version for the venv. It must satisfy the project’s requires-python, otherwise uv recreates the venv and system_site_packages is lost.
  • serializer: "pickle" (default), "cloudpickle" or "dill". With system_site_packages=False the chosen library must be part of the lock.
  • system_site_packages: Whether the venv sees the worker’s site-packages (Airflow, Orbitra SDKs). Defaults to True. Works both when the worker runs on the system interpreter (the venv is created with --system-site-packages) and when the worker itself is a virtual environment, as in local development or CI: its site-packages are then appended to the task venv’s sys.path through a .pth file, after the locked packages, so the lock always wins on conflicts.
  • expect_airflow: Whether Airflow is expected inside the venv. Defaults to True.
  • op_args: Positional arguments for the callable.
  • op_kwargs: Keyword arguments for the callable.
  • string_args: Strings exposed to the callable through the global virtualenv_string_args.
  • templates_dict: Templated values made available in the callable’s context.
  • templates_exts: File extensions resolved while templating.
  • skip_on_exit_code: Exit code(s) that mark the task as skipped instead of failed.
  • env_vars: Extra environment variables for the subprocess.
  • inherit_env: Whether the subprocess inherits the worker’s environment. Defaults to True.
  • **kwargs: Any other BaseOperator argument.
Raises:
  • UvProjectContractError: At run time, if the selected dependencies fall outside the supported contract described above.
  • ValueError: If requirements, pip_install_options, index_urls or index_urls_from_connection_ids are passed. Dependencies and indexes come from the lock; credentials for private indexes are read by uv from UV_INDEX_<NAME>_USERNAME / UV_INDEX_<NAME>_PASSWORD on the worker.
Methods:

execute

Resolve the uv project, load the pinned package set and run the callable in the venv. The pinned package set (uv export) is validated against the supported contract (registry packages only, see the class docstring) and stored as the operator’s effective dependency list before the parent decides which Airflow context keys are serializable, so a lock without apache-airflow or pendulum behaves like an isolated environment.

execute_callable

Validate the serializer against the lock, then build (or reuse) the venv and run the callable.

get_python_source

Return the callable’s source relocated to the DAG file, followed by the azure logger level statement. See orbitra.dags.airflow._source.relocate_source for the relocation. The second line sets the azure logger of the venv subprocess to the orbitra_dags_azure_log_level setting (WARNING by default) before the function is called, which keeps the Azure SDKs’ HTTP request/response dumps out of the task log. Under dag.test() before Airflow 3.5 a third line makes supervisor requests from the venv (Variable.get, Connection.get, XCom pulls) fail at once instead of hanging; see orbitra.dags.airflow._dag_test.

uv_project_path

Absolute path of the resolved uv project directory. Resolved lazily on first access (normally by execute, which also knows the bundle root).