orbitra.dags.airflow.operators.uv
Run a Python callable inside a virtual environment built from a uv.lock.
Functions
default_venv_cache_path
- The
orbitra_dags_venv_cachesetting (environmentORBITRA_DAGS_VENV_CACHE,.envor [tool.orbitra]; default~/.cache/orbitra-dags/venvs) with~expanded.
Classes
UvVirtualenvOperator
Run a function in a virtual environment built with uv sync from a uv.lock.
The task venv is defined by a directory holding pyproject.toml and uv.lock (the “uv
project”). At run time the operator creates a venv (uv venv), installs the locked dependency
set into it (uv sync --frozen ...) and executes the callable in a subprocess of that venv, like
PythonVirtualenvOperator does. Because the
lock pins every package, two runs always install the same versions.
Project discovery. uv_project_dir may be absolute, relative to the DAG bundle root, or
relative to the DAG file’s directory. When omitted the operator walks up from the DAG file’s
directory until it finds a directory containing both files, stopping at the bundle root (or the
filesystem root under dag.test()), the same way uv itself discovers a project. Discovery runs on
the worker because bundle paths only exist there. DAGs deployed as a zip work too: the archive is
extracted once per content hash into a bundles folder next to the venv cache, and the project is
looked up inside it before the directories above the archive.
Supported dependencies. This version installs packages published to a package index (public or
private registry) and pinned by uv.lock; the project itself is a set of dependencies and is not
installed (--no-install-project). Dependencies from local paths, directories, workspace members,
editable installs, git or direct URLs are rejected before any venv is built, with an error naming them.
The check covers exactly what the task selects (groups, extras and transitive dependencies).
Venv cache. By default venvs are cached in a venv-<hash> folder under the directory given by
the Orbitra DAGs setting orbitra_dags_venv_cache (environment ORBITRA_DAGS_VENV_CACHE, .env
or [tool.orbitra] in pyproject.toml; default ~/.cache/orbitra-dags/venvs). The hash covers the SHA-256 of
uv.lock and pyproject.toml, the resolved project directory, the sync options, groups,
extras, Python version, system_site_packages and the pinned package set, so any change to the lock
produces a new venv. Set the orbitra_dags_venv_cache_key setting (ORBITRA_DAGS_VENV_CACHE_KEY) to
any new value to force a rebuild of all cached venvs. Pass venv_cache_path=None to build a
throw-away venv on every run.
Source relocation. Instead of pasting the function into the temporary script, the operator
compiles it with the absolute path of the DAG file and its original line numbers. Breakpoints set in
the DAG file fire inside the venv subprocess (with subprocess debugging enabled in the IDE) and
tracebacks point at the DAG file.
Azure SDK logs. The azure logger of the subprocess is set to the orbitra_dags_azure_log_level
setting (ORBITRA_DAGS_AZURE_LOG_LEVEL or [tool.orbitra] of the uv project’s pyproject.toml;
default WARNING) before the function runs, so the SDKs’ HTTP request/response dumps stay out of the task log.
dag.test(). Before Airflow 3.5, Variable.get, Connection.get and XCom pulls from inside the venv hang
under dag.test() (apache/airflow#65587). There they fail at once with a hint instead: pass the value as a
templated argument ("{{ var.json.get('my_var', {}) }}"), which works everywhere. On the runtime they work.
The function must be defined with def (no lambdas, no generators), all imports must happen
inside it and it may not reference variables from the enclosing scope. Arguments and the return
value travel through the configured serializer (pickle by default). The results of a mapped task
(step.expand(...)) arrive as a list.
Args:
python_callable: Function to run inside the venv.uv_project_dir: Directory withpyproject.toml+uv.lock(templated).Noneenables automatic discovery as described above.uv_sync_options: Flags foruv sync; seeDEFAULT_UV_SYNC_OPTIONS.uv_groups:[dependency-groups]to include on top of the defaults (--group X).uv_extras:[project.optional-dependencies]extras to include (--extra X).venv_cache_path: Parent directory for cached venvs (templated). Defaults to theorbitra_dags_venv_cachesetting, read on the worker at run time;Nonedisables caching.python_version: Python version for the venv. It must satisfy the project’srequires-python, otherwise uv recreates the venv andsystem_site_packagesis lost.serializer:"pickle"(default),"cloudpickle"or"dill". Withsystem_site_packages=Falsethe chosen library must be part of the lock.system_site_packages: Whether the venv sees the worker’s site-packages (Airflow, Orbitra SDKs). Defaults toTrue. Works both when the worker runs on the system interpreter (the venv is created with--system-site-packages) and when the worker itself is a virtual environment, as in local development or CI: its site-packages are then appended to the task venv’ssys.paththrough a.pthfile, after the locked packages, so the lock always wins on conflicts.expect_airflow: Whether Airflow is expected inside the venv. Defaults toTrue.op_args: Positional arguments for the callable.op_kwargs: Keyword arguments for the callable.string_args: Strings exposed to the callable through the globalvirtualenv_string_args.templates_dict: Templated values made available in the callable’s context.templates_exts: File extensions resolved while templating.skip_on_exit_code: Exit code(s) that mark the task as skipped instead of failed.env_vars: Extra environment variables for the subprocess.inherit_env: Whether the subprocess inherits the worker’s environment. Defaults toTrue.**kwargs: Any otherBaseOperatorargument.
UvProjectContractError: At run time, if the selected dependencies fall outside the supported contract described above.ValueError: Ifrequirements,pip_install_options,index_urlsorindex_urls_from_connection_idsare passed. Dependencies and indexes come from the lock; credentials for private indexes are read by uv fromUV_INDEX_<NAME>_USERNAME/UV_INDEX_<NAME>_PASSWORDon the worker.
execute
uv export) is validated against the supported contract (registry
packages only, see the class docstring) and stored as the operator’s effective dependency list
before the parent decides which Airflow context keys are serializable, so a lock without
apache-airflow or pendulum behaves like an isolated environment.
execute_callable
get_python_source
azure logger level statement.
See orbitra.dags.airflow._source.relocate_source for the relocation. The second line sets the azure
logger of the venv subprocess to the orbitra_dags_azure_log_level setting (WARNING by default) before
the function is called, which keeps the Azure SDKs’ HTTP request/response dumps out of the task log.
Under dag.test() before Airflow 3.5 a third line makes supervisor requests from the venv (Variable.get,
Connection.get, XCom pulls) fail at once instead of hanging; see orbitra.dags.airflow._dag_test.
uv_project_path
execute, which also knows the bundle root).