Project Overview
Ray is an open-source distributed computing framework and AI compute engine built on Python. It provides a core parallel runtime — a distributed object store, a task scheduler, and an actor system — plus a unified suite of ML libraries: Ray Tune for hyperparameter search, Ray Train for distributed model training, Ray Serve for online model serving, and Ray Data for scalable data preprocessing.
The project is actively maintained by Anyscale and a wide open-source community. If you are running ML training, hyperparameter search, model serving, or large-scale data preprocessing in Python, Ray is worth a serious evaluation.
Key Challenges Addressed
Python's GIL prevents true thread-level parallelism. Scaling Python workloads across many machines historically meant adopting a JVM-based framework like Spark or hand-rolling multiprocessing and inter-process communication. Ray's core mechanism is a task-and-actor model: decorate functions with @ray.remote to run them in parallel across cores and machines, and decorate classes with @ray.remote to make them stateful distributed actors. The framework handles scheduling, fault tolerance, and data movement between nodes.
Specific problems Ray targets:
- CPU-bound Python workloads that can't escape the GIL on a single machine
- Coordinating multi-GPU or multi-node training jobs without manually configuring process groups
- Running many hyperparameter trials in parallel with early stopping and resource-aware scheduling
- Serving ML models at low latency with dynamic batching and autoscaling
- Streaming large datasets into training jobs without blocking GPU computation on I/O
Getting Started
Install Ray with pip:
pip install "ray[default]"
For ML library support, install the relevant extras:
pip install "ray[train,tune,serve,data]"
A minimal parallel task example:
import ray
ray.init()
@ray.remote
def square(x):
return x * x
futures = [square.remote(i) for i in range(10)]
results = ray.get(futures)
Sharp edges you will hit immediately:
- Object store memory: Ray allocates a fraction of available RAM to its Plasma object store by default. If you leave this unset on a memory-constrained machine, Ray silently spills objects to disk and your jobs run dramatically slower with no obvious warning. Set
object_store_memoryexplicitly inray.init()before running any real workload. - Cluster address binding: When joining worker nodes,
ray start --addressmust use the head node's IP as seen from the workers — not127.0.0.1. Binding to localhost causes workers to connect but tasks never schedule, and jobs hang indefinitely. - Python version consistency: All nodes must run the same Python minor version. Mixing 3.10 on the head node with 3.11 on workers causes serialization failures with error messages that don't clearly identify the version mismatch as the root cause.
Features and Use Cases
Ray Core gives you @ray.remote tasks and actors. Use tasks for stateless parallel CPU work. Use actors for long-lived stateful services — parameter servers, shared feature caches, or streaming ingest loops — that accumulate state across many calls.
Ray Tune runs distributed hyperparameter search. You define a training function, a search space, and an optimization algorithm — Bayesian search via Optuna, population-based training, or ASHA for early stopping — and Tune schedules trials across your cluster. It integrates with MLflow and Weights & Biases for experiment tracking with no extra code.
Ray Train scales model training across GPUs and nodes. It wraps PyTorch DDP, DeepSpeed, and Hugging Face Accelerate so you can go from one GPU to a multi-node cluster by changing a ScalingConfig rather than rewriting your training loop. A TransformersTrainer handles distributed fine-tuning of Hugging Face models directly.
Ray Serve deploys models as HTTP endpoints with dynamic batching (@serve.batch), model composition (chaining multiple models in a pipeline), and request-driven autoscaling. For a transformer model that benefits from batched inference, Serve's batching decorator is the direct path to throughput improvement without a separate serving framework.
Ray Data preprocesses large datasets in a streaming fashion, overlapping data reading with GPU computation so accelerators stay fed. It reads Parquet, CSV, JSON, Delta Lake, and cloud storage without loading the full dataset into memory.
Ecosystem and Dependencies
Ray integrates broadly with the Python ML stack:
- PyTorch: Ray Train wraps
torch.distributednatively. You keep your existing training loop and add Ray scaffolding around it, without reconfiguring process groups manually. - XGBoost and LightGBM: both have native Ray integrations for distributing tree training across workers with no estimator code changes beyond wrapping the estimator.
- Hugging Face Transformers: Ray Train's
TransformersTrainerhandles distributed fine-tuning end to end. - MLflow and Weights & Biases: Ray Tune callbacks log metrics to both platforms automatically.
- KubeRay: the ray-project/kuberay operator manages Ray clusters on Kubernetes, including head and worker node lifecycle and cluster autoscaling.
Architectural Overview
Ray's architecture has three layers.
Distributed object store (Plasma): every object returned by a remote task lives in shared memory local to the node that produced it. When another task needs it, Ray transfers it over the network. For large NumPy arrays or tensors, this bypasses Python pickling — the data stays as raw bytes in shared memory and is read directly by the consuming worker.
Scheduler (GCS + distributed schedulers): the Global Control Store manages cluster metadata — node membership, task lineage, and actor locations. The distributed scheduler assigns tasks to workers based on resource requirements declared on the decorator. @ray.remote(num_gpus=1) tasks only land on nodes with a free GPU slot. You can declare custom resources to model specialized hardware such as TPUs or FPGAs.
Actor model: actors are long-lived Python processes with their own private state. Calls queue in a mailbox and execute serially inside the actor unless you use async methods. This is how Ray implements parameter servers, shared feature stores, and other stateful pipeline components that would otherwise require external coordination infrastructure.
The key design choice: Ray is imperative, not declarative. You write ordinary Python and the framework infers the execution graph at runtime by tracking object dependencies between tasks. This is more flexible than static graph frameworks but harder to optimize ahead of execution.
Pros and Cons
Pros
- Gradual adoption: you can parallelize a single function in five lines without committing to a framework rewrite.
- First-class GPU scheduling: resource declarations at the task and actor level make multi-GPU allocation explicit and automatic across the cluster.
- Unified stack: Core, Tune, Train, Serve, and Data share the same cluster and object store. Data does not need to be serialized between pipeline stages.
- Active development with frequent releases and a large contributor base.
- Python-native: no JVM, no YAML DAG definitions, no domain-specific language. Distributed code looks like ordinary Python with decorators.
Cons
- Distributed debugging is hard: remote task stack traces are surfaced asynchronously and out of context. Production debugging requires familiarity with the Ray dashboard and distributed logging tooling.
- Head node is a single point of failure by default. High-availability head node configuration exists but adds meaningful operational complexity.
- Object store tuning is mandatory for production. Default memory allocations are wrong for most real workloads; you will spend time calibrating
object_store_memory,plasma_directory, and disk-spill thresholds. - API churn: Ray Train and Ray Data APIs have changed significantly across major versions. Upgrading Ray often requires code changes, not just dependency bumps.
- Actor cold-start latency: spawning a new actor involves process creation and environment setup, which can take multiple seconds. Do not use actors in paths where request latency matters.
Comparison and Alternatives
GitHub stars
Weekly star gains
Full metrics details
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
Ray
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
173 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub GraphQL current stargazer cohort, reconstructed from starredAt events and totalCount retrieved in one stable pagination walk. GitHub does not expose historical unstar events, so this is not an exact ledger of past star counts.
133 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Issues opened
Weekly total
Full metrics details
GitHub API observation. Historical values render only when the source provides a real observation for that day.
Ray
GitHub API observation. Historical values render only when the source provides a real observation for that day.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub API observation. Historical values render only when the source provides a real observation for that day.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub API observation. Historical values render only when the source provides a real observation for that day.
131 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub API observation. Historical values render only when the source provides a real observation for that day.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub API observation. Historical values render only when the source provides a real observation for that day.
101 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Issues closed
Weekly total
Full metrics details
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
Ray
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
131 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
101 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Pull requests opened
Weekly total
Full metrics details
GitHub API observation. Historical values render only when the source provides a real observation for that day.
Ray
GitHub API observation. Historical values render only when the source provides a real observation for that day.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub API observation. Historical values render only when the source provides a real observation for that day.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub API observation. Historical values render only when the source provides a real observation for that day.
131 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub API observation. Historical values render only when the source provides a real observation for that day.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub API observation. Historical values render only when the source provides a real observation for that day.
101 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Pull requests closed
Weekly total
Full metrics details
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
Ray
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
131 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
101 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Pull requests merged
Weekly total
Full metrics details
GitHub API observation. Historical values render only when the source provides a real observation for that day.
Ray
GitHub API observation. Historical values render only when the source provides a real observation for that day.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub API observation. Historical values render only when the source provides a real observation for that day.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub API observation. Historical values render only when the source provides a real observation for that day.
131 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub API observation. Historical values render only when the source provides a real observation for that day.
143 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub API observation. Historical values render only when the source provides a real observation for that day.
177 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub API observation. Historical values render only when the source provides a real observation for that day.
101 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Open/closed pull request ratio
Weekly average
Full metrics details
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
Ray
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
173 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub Search pull-request totals queried for exact UTC days; rolling values are sums of proven daily counts.
133 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Open/closed issues ratio
Weekly average
Full metrics details
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
Ray
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
173 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub Search issue totals queried for exact UTC days; rolling values are sums of proven daily counts.
133 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Releases
Weekly total
Full metrics details
GitHub release published_at events bucketed by UTC day; drafts are excluded.
Ray
GitHub release published_at events bucketed by UTC day; drafts are excluded.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Dask
GitHub release published_at events bucketed by UTC day; drafts are excluded.
178 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Apache Spark
GitHub release published_at events bucketed by UTC day; drafts are excluded.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Horovod
GitHub release published_at events bucketed by UTC day; drafts are excluded.
173 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Celery
GitHub release published_at events bucketed by UTC day; drafts are excluded.
180 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Prefect
GitHub release published_at events bucketed by UTC day; drafts are excluded.
133 observed daily rows. Missing days are not fabricated.
| Date | Value |
|---|
Hotness score
Weekly average
Full metrics details
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
Ray
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
Dask
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
178 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
Apache Spark
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
Horovod
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
173 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
Celery
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
Prefect
Derived only from GitHub GraphQL starredAt events after daily star totals are reconciled.
133 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.40 × Hot today + 0.40 × Hot week + 0.20 × Breakout.
- Same-day multiplier = stars 1d ÷ max(1, stars 7d ÷ 7)
- Weekly multiplier = stars 7d ÷ max(1, stars 30d ÷ 30 × 7)
- Fortnight multiplier = stars 14d ÷ max(1, stars 90d ÷ 90 × 14)
- Hot today = clamp(70 × log-scale(stars 1d, 1000) + 30 × breakout-scale(same-day, 4))
- Hot week = clamp(75 × log-scale(stars 7d, 5000) + 25 × breakout-scale(weekly, 3))
- Breakout = clamp(35 × breakout-scale(same-day, 4) + 40 × breakout-scale(weekly, 3) + 25 × breakout-scale(fortnight, 2.5))
| Date | Stars | Stars 1d | Stars 7d | Stars 14d | Stars 30d | Stars 90d | Same-day multiplier | Weekly multiplier | Fortnight multiplier | Hot today | Hot week | Breakout | Stored score |
|---|
Reliability score
Weekly average
Full metrics details
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
Ray
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
Dask
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
178 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
Apache Spark
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
Horovod
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
173 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
Celery
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
180 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
Prefect
Derived from the displayed continuity, closure, shipping, liveness, and support-burden components.
133 observed daily rows. Missing days are not fabricated.
Per-day formula: 0.30 × Continuity + 0.30 × Closure + 0.20 × Shipping + 0.10 × Liveness + 0.10 × Support burden.
- Continuity = 35% active ratio 90d + 20% active ratio 30d + 15% PR efficiency 90d + 10% PR efficiency 30d + 10% liveness + 10% observed-history coverage
- Closure = 55% PR merge efficiency 90d + 45% issue close efficiency 90d
- Shipping = 100 × (20% × release ratio 30d + 30% × release ratio 90d + 50% × release ratio 180d); ratios are releases ÷ 2, 6, and 12, capped at 1
- Support burden = 100 − 2 × open issues per 1,000 stars
- Liveness = 100 × exp(−pushed days ago ÷ 120)
Stars, issues, pull requests, and releases are daily observations. Pushed-days and license are repository snapshot-derived inputs and are not independent historical GitHub events.
| Date | Issues opened | Issues closed | PRs opened | PRs merged | Releases | Stars | Open issues | Pushed days ago | Continuity | Closure | Shipping | Liveness | Support burden | License | Observed days | Missing days | Needs healing | Stored score |
|---|
Several projects overlap with Ray depending on your use case:
| Project | Repo | Hotness (~6mo) | PRs merged (~6mo) | Star growth (~6mo) |
|---|---|---|---|---|
| Ray | https://github.com/ray-project/ray | 35.86 | 686 | 3340 |
| Dask | https://github.com/dask/dask | metrics pending | 0 | 9 |
| Apache Spark | https://github.com/apache/spark | 4.56 | 0 | 5783 |
| Horovod | https://github.com/horovod/horovod | metrics pending | 0 | metrics pending |
| Celery | https://github.com/celery/celery | 87.24 | 235 | 1846 |
| Prefect | https://github.com/PrefectHQ/prefect | 79.09 | 1568 | 1398 |
Dask is the closest direct comparison. Both parallelize Python across cores and machines and integrate with NumPy, Pandas, and scikit-learn. Dask is more mature for pure dataframe and array workloads; Ray has better GPU scheduling, a more complete ML training and serving story, and a more ergonomic actor model for stateful services.
Apache Spark is the established choice for large-scale batch data processing. It has a mature SQL engine, broad structured data support, and decades of production use. For GPU-accelerated ML workloads, Spark requires more integration work and lacks native GPU scheduling at the task level.
Horovod is narrower: distributed deep learning training via ring-allreduce, and nothing else. If all you need is data-parallel distributed training and you are already committed to TensorFlow or PyTorch, Horovod is simpler. If you need anything beyond training — serving, tuning, data pipelines — Ray Train gives a more complete platform.
Celery is a mature task queue for application-level async work. It has no concept of GPU resources, actor state, or shared object stores. Use it for sending emails or processing uploads. Don't use it for ML training pipelines.
Prefect is a workflow orchestration tool, not a compute engine. It schedules and monitors pipelines but delegates actual computation to external systems like Kubernetes jobs or Dask. Use it when your primary need is scheduling visibility and retry logic, not parallel compute.
The best overall alternative to Ray is Dask. It has a more stable API surface, deeper documentation for data science workflows, and broader adoption in the scientific Python community — making it the lower-risk starting point if you do not specifically need Ray's GPU scheduling or actor model.
Conclusion
If you are a machine learning engineer running distributed training, hyperparameter search, or model serving at scale, Ray gives you a single Python framework that covers the full pipeline — from data preprocessing through deployment — with documentation that covers each layer in depth. The shared object store eliminates the serialization overhead that accumulates when you stitch together separate training, tuning, and serving systems.
Do not run Ray in production without explicitly setting object_store_memory in ray.init(): left at the default, Ray silently falls back to disk-spill under memory pressure and your training throughput collapses with no clear warning in the logs. Start by running ray status on your cluster to confirm available CPU, GPU, and memory resources before submitting your first large job.