Skip to content

Use TraceML With Your Stack

Run your existing Hugging Face Trainer, PyTorch Lightning, or RF-DETR script with traceml run train.py. TraceML instruments standard training automatically. For custom PyTorch loops, MONAI, Ray, Accelerate, and DeepSpeed, follow the setup in the matching guide below.

Choose Your Integration

Choose your framework to open its setup guide. Zero-code launch uses traceml run train.py for the standard training paths.

Framework Setup Example
Hugging Face Trainer Zero-code launch Example
PyTorch Lightning Zero-code launch Example
RF-DETR Zero-code launch Example
Custom PyTorch loop Explicit step boundary Example
Hugging Face Accelerate Explicit step boundary Example
MONAI Handler setup Example
Ray Train Trainer adapter Example
DeepSpeed Explicit step boundary Example
W&B / MLflow Summary export Example

Integration support matrix

Tested coverage, versions, and limitations

CI tested identifies configurations checked by the linked job and tests. Documented recipe and experimental identify coverage without equivalent end-to-end validation. See each framework guide for the requirements that apply to your run.

Stack Recommended API Tested dependency range CPU / GPU Single- / multi-process Multi-node Emitted signals Example Validation level Limitations
Hugging Face Trainer traceml run train.py (standard Trainer); init() + TraceMLTrainerCallback for manual setup CI resolves torch>=2.5.0, transformers>=4.46.1, and accelerate>=0.26.0; not a pinned compatibility matrix. CPU: CI tested — real Trainer run
GPU: Not claimed — no GPU job
Single: CI tested — CPU only
Multi: CI tested — two CPU/Gloo ranks
Experimental — no reproducible job CPU launcher tests assert step, forward, backward, optimizer, and DataLoader fetch events. The automatic two-rank test verifies both ranks and aligned optimizer steps. GPU H2D and memory signals are not asserted in CI. huggingface_trainer_minimal.py CI tested
CI job: test-integration; automatic single-process and two-rank real Trainer launcher tests
Hugging Face limitations · #141 · #320 · #323
PyTorch Lightning traceml run train.py (standard Trainer.fit); init() + TraceMLCallback for manual setup CI resolves lightning>=2.6.0 (not pinned; this is not a broad version matrix). The cited tests run under both the lightning and the legacy pytorch_lightning namespace. CPU: CI tested — Ubuntu integration job
GPU: Documented recipe — no GPU job
Single: CI tested — real Trainer.fit() CPU run
Multi: CI tested — two CPU/Gloo ranks
Experimental — no reproducible job CPU tests assert step, forward, backward, optimizer, and DataLoader fetch events. The automatic two-rank test verifies both ranks and aligned optimizer steps. H2D is GPU-only and not asserted in CI. lightning_minimal.py CI tested
CI job: test-integration; real Trainer.fit() automatic single-process and two-rank launcher tests
Lightning guide · #141 · #320 · #323
RF-DETR traceml run train.py (standard model.train); rfdetr.init() for manual setup Dedicated CPU CI pins rfdetr[train]==1.10.1. A reproducible one- and four-T4 case study uses RF-DETR development commit 0ed5be8 with torch 2.9.1; this is not a broad compatibility range. CPU: CI tested — dedicated RF-DETR CPU job
GPU: Documented recipe — one- and four-T4 case study; no CUDA CI
Single: CI tested — parity and real Nano smoke
Multi: CI tested — two CPU/Gloo ranks in CI; four-T4 NCCL case study
Experimental — no physical multi-node validation CPU tests assert training step, inner-model forward, backward, optimizer and DataLoader fetch, excluding evaluation. The T4 case study records H2D and CUDA phase timing on one and four GPUs; CUDA memory remains unvalidated. rfdetr_minimal.py CI tested
CI job: test-rfdetr; RF-DETR parity, Nano and Gloo smoke tests
RF-DETR limitations · #141 · #323
Custom PyTorch loop traceml.init() + traceml.trace_step(model) CI resolves torch>=2.5.0 (not pinned; this is not a broad version matrix). CPU: CI tested — Ubuntu integration job
GPU: Not claimed — no GPU job
Single: CI tested — launcher smoke run
Multi: Not claimed — no matrix evidence
Not claimed — no matrix evidence The cited smoke test asserts a final summary; it does not declare a per-signal contract. quickstart.py CI tested
CI job: test-integration; real PyTorch launcher smoke test
Distributed Training · #141 · #320 · #323
Hugging Face Accelerate traceml.init() + traceml.trace_step(accelerator.unwrap_model(model)) CI resolves torch>=2.5.0 and accelerate>=0.26.0; not a pinned compatibility matrix. CPU: CI tested — real Accelerator.prepare/backward run
GPU: Not claimed — no GPU job
Single: CI tested — CPU only
Multi: Documented recipe — no real distributed job
Experimental — no reproducible job step, forward, backward, optimizer, and dataloader fetch on the cited real Accelerate run. accelerate_minimal.py CI tested
CI job: test-integration; real Accelerate prepare/backward test
Accelerate limitations · #141 · #320 · #323
MONAI traceml_ai.integrations.monai.init() + TraceMLHandler in train_handlers A dedicated CPU CI job pins monai==1.6. with pytorch-ignite==0.5., and reruns the same tests at the declared floor pytorch-ignite==0.4.11. The one-T4 spleen notebook ran MONAI 1.6.0, Ignite 0.5.5 and torch 2.11.0+cu130. This is not a broad compatibility matrix. CPU: CI tested — dedicated MONAI CPU job
GPU: Documented recipe — one-T4 spleen notebook; no CUDA CI
Single: CI tested — real SupervisedTrainer runs, plus the documented example end to end
Multi: Not claimed — no distributed job
Not claimed — no distributed job tests/integrations/test_telemetry_conformance.py asserts step time, forward, backward, optimizer and the engine's own fetch bracket on a CPU run. The same job runs examples/integrations/monai_minimal.py under traceml run, with and without gradient accumulation, and fails if the summary loses Input Wait, forward, backward or optimizer time, or if its step count disagrees with the optimizer updates the trainer made. H2D and step memory are GPU-only and not asserted in CI. The one-T4 spleen notebook's table shows H2D for every run, and each run's summary carries step memory; neither value is validated. monai_minimal.py CI tested
CI job: test-monai; real MONAI SupervisedTrainer tests, rerun at the Ignite floor
MONAI limitations · #141 · #320 · #323 · #463
Ray Train TraceMLTorchTrainer + TraceMLRayConfig Documented install requirement: ray[train]>=2.0.0. No real-Ray CI-tested range. CPU: Documented recipe — no real-Ray job
GPU: Documented recipe — no GPU job
Single: Documented recipe — local Ray recipe
Multi: Documented recipe — worker recipe only
Experimental — no reproducible job Guide documents step, forward, backward, optimizer, and wrapped Ray Data fetch timing; no real-Ray CI signal assertion exists. torchtrainer_minimal.py documented recipe Ray guide · #141 · #320 · #323
DeepSpeed traceml.init() + traceml.trace_step(model_engine.module) Documented install requirement: deepspeed>=0.10.0. No real-DeepSpeed CI-tested range. CPU: Unsupported — the guide requires CUDA
GPU: Documented recipe — no CUDA CI job
Single: Documented recipe — no real engine run
Multi: Documented recipe — no real distributed job
Experimental — no reproducible job Guide documents DataLoader fetch, H2D, forward, backward, optimizer, step, and memory; no real-DeepSpeed CI signal assertion exists. deepspeed_minimal.py documented recipe DeepSpeed limitations · #141 · #320 · #323
W&B / MLflow traceml.summary() No W&B or MLflow dependency is installed or run in CI. CPU: Documented recipe — tracker client not run
GPU: Not applicable — tracker export is device-independent
Single: Documented recipe — no tracker-client CI run
Multi: Not claimed — no matrix evidence
Not claimed — no matrix evidence No tracker telemetry is emitted by TraceML; the documented integration exports TraceML's compact summary dictionary. summary_logging_minimal.py documented recipe W&B / MLflow guide · #323

Evidence limits

  • This matrix makes no GPU, multi-process, or multi-node claim without a corresponding reproducible job.
  • Tested configurations and recorded examples do not establish compatibility with every version, device, or training strategy.