Metadata-Version: 2.1
Name: aidatalake-ray-python-sdk
Version: 26.9.0.1
Summary: AiDataLake Ray Python SDK and aidatalake CLI
Home-page: https://github.com/huaweicloud/huaweicloud-sdk-python-v3
Author: Huawei
License: Apache-2.0
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE

# AiDataLake Ray Python SDK

Python SDK and `aidatalake` command line tool for managing **Ray jobs** on
Huawei Cloud AIDataLake.

The SDK is a thin, purpose-built layer on top of the generated
[`huaweicloudsdkaidatalake`](https://pypi.org/project/huaweicloudsdkaidatalake/)
package: it handles configuration, credentials and output rendering, and it
exposes exactly the four operations most users need — **submit**, **status**,
**list** and **terminate** Ray jobs. `show` and `cancel` are kept as aliases
of `status` and `terminate` for scripts written against earlier versions.

> **Released through Huawei's internal artifact repository.** This project is
> distributed as the `aidatalake-ray-python-sdk` **source distribution
> (sdist)** via the internal artifact repository (`hwcloud_artifact` → OBS); it
> is not published to public PyPI. Installing it gives you both the
> `aidatalake` command and the `aidatalake_ray_pysdk` Python package.

---

## Table of contents

1. [Installation](#installation)
2. [Quick start](#quick-start)
3. [Configuration](#configuration)
4. [Logging in](#logging-in)
5. [Temporary (STS) credentials](#temporary-sts-credentials)
6. [Automatic token refresh with a credential provider](#automatic-token-refresh-with-a-credential-provider)
7. [Command reference](#command-reference)
8. [Output formats](#output-formats)
9. [Exit codes and errors](#exit-codes-and-errors)
10. [Using the Python API](#using-the-python-api)
11. [Development and testing](#development-and-testing)

---

## Installation

The package lives on Huawei's **internal** artifact repository, not on public
PyPI — point `pip` at the internal mirror (normally preconfigured in the build
and developer environments):

```bash
pip install aidatalake-ray-python-sdk
```

This installs:

- the `aidatalake` command line tool, and
- the `aidatalake_ray_pysdk` Python package.

The only hard dependency is `huaweicloudsdkaidatalake>=3.1.216`, which is
installed automatically. PyYAML is **optional** — see
[Output formats](#output-formats).

The version is read from the installed distribution's metadata, so
`aidatalake --version` always reports the version you actually installed.

This project is licensed under the
[Apache License 2.0](LICENSE).

---

## Quick start

```bash
# One-time: store your credentials locally (see "Configuration").
export HUAWEICLOUD_SDK_AK=your_ak
export HUAWEICLOUD_SDK_SK=your_sk
export HUAWEICLOUD_SDK_PROJECT_ID=your_project_id
export AIDATALAKE_ENDPOINT=https://aidatalake.<region>.myhuaweicloud.com
export AIDATALAKE_WORKSPACE_ID=your_workspace_id

aidatalake login

# Then use the CLI without flags.
aidatalake job submit --name my-job --entrypoint "python train.py"
aidatalake job status <job_id>
aidatalake job list --status RUNNING --limit 10
aidatalake job terminate <job_id>
```

Each command also accepts the settings as flags (e.g. `--workspace-id`), or
reads them from the credential file written by
[`aidatalake login`](#logging-in).

---

## Configuration

Configuration is resolved, in order of precedence, from:

1. the `--flag` passed on this invocation,
2. the matching environment variable (`AIDATALAKE_*` / `HUAWEICLOUD_SDK_*`),
3. the credential file written by `aidatalake login`.

| Environment variable                    | Flag                          | Required | Description |
|-----------------------------------------|-------------------------------|----------|-------------|
| `AIDATALAKE_ENDPOINT`                   | `--endpoint`                  | yes      | AIDataLake API endpoint URL. |
| `HUAWEICLOUD_SDK_AK`                    | `--ak`                        | yes*     | Access Key ID. |
| `HUAWEICLOUD_SDK_SK`                    | `--sk`                        | yes*     | Secret Access Key. |
| `AIDATALAKE_WORKSPACE_ID`               | `--workspace-id`              | yes      | Workspace ID. |
| `HUAWEICLOUD_SDK_PROJECT_ID`            | `--project-id`                | yes      | Project ID for request signing. Distinct from the workspace ID. |
| `AIDATALAKE_REGION`                     | `--region`                    | no       | Region id. The endpoint alone is enough. |
| `HUAWEICLOUD_SDK_SECURITY_TOKEN`        | `--security-token`            | no       | Temporary STS security token. |
| `AIDATALAKE_CREDENTIAL_PROVIDER`        | `--credential-provider`       | no       | Dotted path to a credential provider class that supplies and renews credentials. |
| `AIDATALAKE_CREDENTIAL_PROVIDER_CONFIG` | `--credential-provider-config`| no       | Provider configuration as inline JSON or a JSON file path. |
| `AIDATALAKE_CREDENTIALS_FILE`           | —                            | no       | Override the credential file location (default `~/.config/aidatalake/credentials.json`). |

\* AK/SK are required unless a `--credential-provider` is used (which supplies
them instead), or unless they were stored with `aidatalake login`.

> **`project_id` vs `workspace_id`**: these are two independent, required
> identifiers and neither is a substitute for the other. `project_id` scopes
> the IAM-signed request; `workspace_id` addresses the Ray workspace in the
> URL. If a command reports one of them missing, set it explicitly — the CLI
> never falls back between the two.

> **Prefer environment variables (or `login`) over flags for secrets.**
> `--ak`/`--sk` passed on the command line are visible to other processes
> (e.g. via `ps`).

---

## Logging in

`aidatalake login` stores the configuration once so later commands need no
flags:

```bash
aidatalake login \
    --endpoint https://aidatalake.cn-north-7.myhuaweicloud.com \
    --ak AK --sk SK \
    --workspace-id ws-123 --project-id proj-456

aidatalake job list          # no flags needed
aidatalake logout            # removes the stored credentials
```

Every value falls back to the matching environment variable, so a login that
only needs to store the AK/SK is just:

```bash
export AIDATALAKE_ENDPOINT=... AIDATALAKE_WORKSPACE_ID=... HUAWEICLOUD_SDK_PROJECT_ID=...
aidatalake login --ak AK --sk SK
```

Behavior worth knowing:

- Credentials are written to `~/.config/aidatalake/credentials.json` with
  **owner-only permissions** (directory `0o700`, file `0o600`) because the file
  holds the AK/SK in clear text. The write is atomic, and the temporary file is
  created `0o600` before any secret is written to it.
- Secrets are **never echoed back**: the confirmation lists which fields were
  stored, not their values.
- Re-running `login` with different values is rejected unless `--force` is
  given, so an accidental re-run cannot silently replace working credentials.
  Re-running it with *identical* values is a no-op.
- A login that stores only part of the configuration says so — it prints the
  settings still required per command instead of pretending to be complete.
- `logout` is safe to run when not logged in.

An explicit `--flag` or environment variable always wins over the stored file,
so a stored login never shadows a per-invocation override.

---

## Temporary (STS) credentials

`HUAWEICLOUD_SDK_SECURITY_TOKEN` / `--security-token` supplies a temporary
security token obtained from STS. When set, the SDK sends it as the
`X-Security-Token` header and folds it into the signed request headers.

- **Short-lived on purpose**: the token expires after its configured lifetime
  (typically minutes to hours). Pass a static token only for the length of a
  session — for anything longer, use a credential provider (below), which
  renews it automatically.
- Use it when you have an STS session instead of a permanent AK/SK pair. The
  AK/SK still come from `HUAWEICLOUD_SDK_AK`/`HUAWEICLOUD_SDK_SK`.

### Permanent vs temporary credentials

The two are distinguished by the security token:

- **Permanent AK/SK** (no token) — never expires. This is what plain
  `--ak`/`--sk` (or a `CredentialSnapshot` with `security_token=None`)
  represents.
- **Temporary (STS) credentials** (token set) — the token lapses after its
  configured lifetime. The AK/SK pair stays the same, but the token must be
  renewed before it expires, which is what the credential provider (below)
  does automatically.

---

## Automatic token refresh with a credential provider

A credential provider supplies the AK/SK/security token and renews them when
they expire, so long-running invocations survive a token rotation:

```bash
aidatalake --credential-provider mypackage.MyCredentialProvider job list
```

The named class must be a `DynamicCredentialProvider` subclass. It is
instantiated with the `--credential-provider-config` dict (inline JSON or a
file path); the request-signing `projectId` always comes from
`--project-id` / `HUAWEICLOUD_SDK_PROJECT_ID` (required, independently of the
workspace ID), never from `--credential-provider-config`.

```python
# mypackage.py
from aidatalake_ray_pysdk import CredentialSnapshot, DynamicCredentialProvider

class MyCredentialProvider(DynamicCredentialProvider):
    def __init__(self, config=None):
        config = config or {}
        super().__init__(self.supply, project_id=config.get('projectId'))

    def supply(self):
        # Call your STS / metadata service here.
        ak, sk, token, expires_at_ms = fetch_temporary_credentials()
        return CredentialSnapshot(ak, sk, token, expired_at_ms=expires_at_ms)
```

Renewal happens two ways:

- **Before signing**: a snapshot within 30 seconds of its expiry (or already
  past it) is replaced from the supplier, so an expired token is never sent.
- **On rejection**: if the service answers `401`/`403` (the token lapsed early,
  or another process rotated it), the request is retried **once** with freshly
  supplied credentials. A second rejection is reported as an auth error (exit
  code 4). Without a provider there is no retry.

`--credential-provider` cannot be combined with
`--ak`/`--sk`/`--security-token` — the provider owns the credentials.

### How refresh works

The provider caches one credential snapshot (an immutable
`CredentialSnapshot` carrying AK/SK/token plus an expiry time) and consults the
supplier again once it expires. Two things trigger a refresh:

1. **Before signing (proactive).** A snapshot is treated as expired *30 seconds
   before* its stated lifetime ends (the refresh-advance window), so an
   expired token is never sent — it is replaced in time to be signed into the
   next request. A snapshot with an unknown lifetime
   (`expired_at_ms=0`) counts as already expired, so the supplier is consulted
   on every call.
2. **On rejection (reactive).** If the service answers `401`/`403` — the token
   lapsed early, or another process rotated it — the client is rebuilt from a
   forcibly refreshed snapshot and the request is retried **exactly once**. A
   second rejection is an auth error (exit code 4). Without a provider there
   is no retry: a static `401`/`403` is reported directly.

The retry is exactly once, never a loop: a genuine permission problem would
otherwise retry forever.

```
Command starts
   │
   ├─ Provider present? ───────────────────── No ──┐
   │     Yes                                      │
   │                                              ▼
   │  client = build_client(cfg)      BasicCredentials(ak, sk, project_id)
   │     └─ to_basic_credentials():        + with_security_token(token)
   │          snapshot expired? (30 s ahead)    (STS header + signed in)
   │              ├─ Yes → supplier.get() → fresh snapshot → new credentials
   │              └─ No  → cached snapshot
   │
   ├─ call(client) ──────────────────────────► request
   │                                             │
   │                             Service returns 401/403?
   │                                ├─ Yes + provider → rebuild with forced
   │                                │     fresh snapshot → retry once
   │                                │        second rejection → AUTH_ERROR (exit 4)
   │                                └─ No → error translated and reported
   │
   └─ Result printed
```

---

## Command reference

```
aidatalake login [--force]
aidatalake logout
aidatalake job submit --name NAME --entrypoint CMD [options]
aidatalake job status|show JOB_ID [--job-id JOB_ID]
aidatalake job list [filters]
aidatalake job terminate|cancel JOB_ID [--job-id JOB_ID]
```

Every command accepts the global options from
[Configuration](#configuration) (`--endpoint`, `--ak`, `--sk`,
`--workspace-id`, `--project-id`, `--region`, `--security-token`,
`--credential-provider`, `--credential-provider-config`) and the output
options from [Output formats](#output-formats), placed anywhere on the command
line (e.g. `aidatalake --json job list`).

### `aidatalake job submit`

```bash
aidatalake job submit --name NAME --entrypoint "python train.py" \
    --endpoint-name ray \
    --pip "numpy==2.0" --working-dir "obs://bucket/code" \
    --env MODE=prod --metadata team=ml
```

| Option                | Description |
|-----------------------|-------------|
| `--name`, `-n`        | **Required.** Job name (used for listing and filtering). |
| `--entrypoint`        | **Required.** Command the Ray cluster executes. |
| `--description`       | Optional job description. |
| `--endpoint-name`     | Ray endpoint (cluster) to run on. |
| `--client-token`      | Idempotency token (sent as `X-Client-Token`); a retry with the same token is not submitted twice. |
| `--runtime-env`       | `RuntimeEnv.config` as inline JSON. |
| `--pip`               | Extra pip package to install (repeatable). |
| `--working-dir`       | Working directory for the entrypoint. |
| `--env`               | Environment variable `KEY=value` for the job (repeatable). |
| `--metadata`          | Free-form job metadata `key=value` (repeatable). |
| `--resource-config`   | `RayResourceConfig` as inline JSON (head/worker resources). |
| `--strategy-json`     | `RayJobStrategy` (retry/timeout) as inline JSON. |
| `--config-json`       | Free-form `RayJobConfig` body config as inline JSON or a JSON file. |
| `--config-yaml`       | The YAML sibling of `--config-json` (file or inline; needs PyYAML). Mutually exclusive with `--config-json`. |
| `-w`, `--wait`        | Block until the job reaches a terminal state, then exit `0` on `SUCCEEDED` or `10` otherwise. |
| `--timeout`           | Maximum seconds to wait with `--wait` before giving up (exit code `3`). Without it, `--wait` polls until the job leaves the active set. |

`--config-json` / `--config-yaml` are the escape hatch for the deep-nested
fields of `RayJobConfig` (`job_config`, `image`), in addition to the dedicated
flags for `runtime_env`, `resource_config`, `rayjob_strategy` and `metadata`.
**Any key that is not a declared field is rejected** — a misspelling fails the
submit loudly instead of being silently dropped.

With `--wait`, the CLI polls the job every few seconds until it leaves the
active set (`PENDING`, `QUEUED`, `RUNNING`, `CANCELING`) and then exits `0` on
`SUCCEEDED`, `10` on any other terminal status, or `3` when `--timeout`
elapsed first. Progress goes to stderr when a machine-readable format was
requested, so stdout stays a single parseable document. The job id is printed
whether or not `--wait` was given, so the caller can always act on the job:

```bash
aidatalake job submit --name NAME --entrypoint "python train.py" --wait --timeout 1800
```

### `aidatalake job status` / `show`

Shows the details of one job. The job id may be given positionally or with
`--job-id`. `show` is an alias of `status`.

```bash
aidatalake job status JOB_ID
aidatalake job status --job-id JOB_ID
```

### `aidatalake job list`

Lists one page of jobs, newest first.

```bash
aidatalake job list
aidatalake job list --status RUNNING --status FAILED --limit 10
aidatalake job list --name "my-job"
aidatalake job list --create-time-after 1735689600000
```

Filters (`--limit`, `--marker`, `--name`, `--id`, `--endpoint-name`,
`--federation-name`, `--create-time-before`, `--create-time-after`,
repeatable `--status`) combine with AND; multiple `--status` values are OR'd.

Valid `--status` values: `QUEUED PENDING RUNNING CANCELING CANCELED FAILED
QUEUED_TIMEOUT RUNNING_TIMEOUT SUCCEEDED`.

Pagination: in text output the footer prints `Next page: aidatalake job list
--marker <marker>` (and the previous page) when more pages exist.

### `aidatalake job terminate` / `cancel`

Cancels a running job. `cancel` is an alias of `terminate`.

```bash
aidatalake job terminate JOB_ID
```

Cancellation is asynchronous — the job typically reports `CANCELING` first and
settles on `CANCELED`, and it stays visible in `list` output. The API has no
delete operation.

---

## Output formats

Every command accepts `-o/--output` with one of:

| Format      | Use |
|-------------|-----|
| `text` (default) | Human-readable `key: value` lines. |
| `json`      | Machine-parseable JSON document. |
| `yaml`      | Human-readable YAML document. |
| `table`     | Aligned column table (no prose footer) — rows only, so it stays parseable. |

`--json` is a shorthand for `-o json`. Both flags may appear anywhere on the
command line (`aidatalake --json job list`).

```bash
aidatalake job status JOB_ID -o text
aidatalake job status JOB_ID -o json
aidatalake job list --limit 3 -o table
```

> YAML output (and `--config-yaml`) requires PyYAML, which is **not** a hard
> dependency: install it with `pip install pyyaml` when you need it. Without
> it, requesting YAML is a clean user error, never a traceback.

---

## Exit codes and errors

| Code | Meaning |
|------|---------|
| `0`  | Success. |
| `1`  | User error (bad input, missing resource). |
| `2`  | Infrastructure error (connection, 5xx). |
| `3`  | Timeout — `job submit --wait --timeout SECONDS` elapsed before the job reached a terminal state. |
| `4`  | Authentication/authorization error. |
| `10` | Job reached a terminal state other than `SUCCEEDED` (e.g. `job submit --wait`). |

Errors are reported on **stderr**, and a failed command leaves **stdout
empty** — a caller always parses exactly one document from stdout. With
`-o json`/`-o yaml`, the error is a machine-readable envelope:

```json
{"error": {"code": "resource_not_found",
           "message": "...",
           "resolution": "Verify the resource ID or name...",
           "exit_code": 1}}
```

`code` lets automation key on the failure type instead of scraping the
message. The values are `resource_not_found`, `name_conflict`,
`auth_unauthorized`, `auth_forbidden`, `auth_token_expired`,
`operation_timeout`, `workload_failed`, `insufficient_capacity`,
`connection_error`, `rate_limited`, `missing_config` and `invalid_config`.
With the default `text` format the error is a plain `error: ...` line.

For example, `404 → exit 1`, `401/403 → exit 4`, `429 → exit 2`, HTTP 5xx and
transport failures → `exit 2`.

---

## Using the Python API

The same building blocks that power the CLI are importable:

```python
from aidatalake_ray_pysdk import ClientConfig, build_client, submit_job

cfg = ClientConfig(
    endpoint='https://...',
    ak='AK',
    sk='SK',
    workspace_id='ws-123',
    project_id='proj-456',       # required; independent of the workspace ID
    security_token='STS-token',  # optional temporary credentials
)
client = build_client(cfg)

response = submit_job(
    client, cfg.workspace_id,
    name='my-job',
    entrypoint='python train.py',
)
print(response.job_id)
```

With a credential provider, the AK/SK are omitted and the token is renewed
from the supplier as it expires:

```python
from aidatalake_ray_pysdk import (
    ClientConfig, CredentialSnapshot, DynamicCredentialProvider, build_client,
)

provider = DynamicCredentialProvider(
    lambda: CredentialSnapshot('AK', 'SK', security_token='STS-token',
                               expired_at_ms=expiry_epoch_ms),
    project_id='proj-456',
)
client = build_client(
    ClientConfig(endpoint='https://...', workspace_id='ws-123',
                 project_id='proj-456', credential_provider=provider))
```

`build_client(cfg, force_refresh=True)` bypasses the provider's cached snapshot
— this is what the CLI's 401/403 retry uses.

The credential store behind `aidatalake login` is available to callers too:

```python
from aidatalake_ray_pysdk import (
    load_credentials, missing_settings, resolve_config, save_credentials,
)

save_credentials({'endpoint': 'https://...', 'ak': 'AK', 'sk': 'SK',
                  'workspace_id': 'ws-123', 'project_id': 'proj-456'})
already = load_credentials()      # {} when not logged in
missing_settings(already)         # settings still needed, e.g. ['--project-id (...)']

# Thread the stored values into a config resolution:
cfg = resolve_config(args, stored=already)
```

---

## Development and testing

The test suite lives under `aidatalake_ray_pysdk_test/` and exercises the CLI
against an in-process HTTP mock server, so it runs **with no network and no
live AIDataLake endpoint**:

```bash
python -m pytest aidatalake_ray_pysdk_test/tests -q
```

Layout:

```
aidatalake_ray_pysdk/        # the shipped package
  cli.py                     # argparse tree, handlers, output and error reporting
  config.py                  # setting precedence and client construction
  credentials.py             # static and self-renewing credential providers
  auth.py                    # the local credential store (login/logout)
  jobs.py                    # Ray job request/response assembly
  render.py                  # JSON / YAML / table / text output helpers
  errors.py                  # exit codes, error identifiers, SDK translation
aidatalake_ray_pysdk_test/   # the test suite (not shipped in the sdist)
.build_config/build.sh       # release build: tests, then the sdist published as aidatalake-ray-python-sdk-${SDK_PACKAGE_VERSION}.tar.gz for OBS
.cloudbuild/ray-python-sdk.yml # releases the tarball via hwcloud_artifact
```
