Skip to content
YOTOP

Platform

The software that makes GPUs easy to use.

Raw GPUs are only the beginning. The YOTOP Platform is the software layer that manages clusters, orchestrates GPUs, schedules jobs, and serves models — so your teams consume compute as a service instead of operating it.

Cluster managementGPU orchestrationJob scheduling & workloadsModel deployment & inferenceToken FactoryModel API Gateway

Group 01

Manage

Control every cluster and GPU from one place.

Unified Cluster Management and GPU Resource Orchestration give operators and platform teams a single view of capacity — and the controls to allocate it.

1.1Unified Cluster Management

One inventory. One health model.

See every cluster, node, and GPU in one console — health, configuration, and lifecycle state included. Cluster observability surfaces GPU, fabric, and storage signals before they turn into failed jobs.

  • Inventory across clusters and sites
  • Node lifecycle: provisioning, ready, draining, maintenance
  • Cluster observability with alerting
  • Role-based access for operators and tenants
operations / fleet overview
Illustrative
  • Site 1

    2 clusters

    Healthy
  • Site 2

    1 cluster

    Maintenance
  • Site 3

    2 clusters

    1 node draining
GPU utilization · fleetsample

● site-3/cluster-b/node-07 · GPU error → drained

● site-1 · firmware rollout 3/8 nodes

● queue: training · 2 jobs pending

1.2GPU Resource Orchestration

Pool capacity. Allocate by policy.

GPU orchestration treats capacity across clusters as one pool. Allocate GPUs to teams and tenants by policy, move work to healthy capacity when hardware fails, and scale inference out when demand rises.

  • Cross-cluster GPU pooling
  • Allocation by team, tenant, or project
  • Automatic failover to healthy nodes
  • Partitioning and sharing where hardware supports it
Illustrative
Unified control plane
cluster-a24 GPU
cluster-b24 GPU
cluster-c18 GPU

Place work where it runs best

  • 01 › Training job placed on 16 GPUs in one fabric domain
  • 02 › Fine-tuning queue admitted under team quota
  • 03 › Inference replicas pinned across two clusters
  • Training
  • Fine-tuning
  • Inference
  • Available

Group 02

Run

Keep the right jobs on the right GPUs.

Job Scheduling and AI Workload Management turn shared capacity into high utilization — without teams fighting over GPUs.

2.1Job Scheduling

Workload scheduling that understands AI.

Distributed training needs every GPU at once, close together on the fabric. Inference needs guaranteed headroom. Batch jobs should fill the gaps. YOTOP scheduling encodes those rules on Kubernetes or Slurm.

  • Gang scheduling for multi-node jobs
  • Topology-aware placement
  • Priorities, preemption, and fair-share
  • Backfill of idle capacity
  • InferenceGuaranteed
  • TrainingHigh · gang
  • Fine-tuningFair-share
  • Batch / evalPreemptible

Policy

Scheduler

priority
quota
topology
preemption

Shared GPU pool

Serving capacity is protected; training runs gang-scheduled in one fabric domain; batch backfills what is left.

2.2AI Workload Management

Every job visible, recoverable, and accountable.

Track jobs from submission to completion. When a node fails, checkpointed jobs resume on healthy capacity. Usage rolls up by team and project, so GPU-hours connect to the work they produced.

  • Job queues and lifecycle tracking
  • Checkpoint-aware restart after failures
  • Versioned container environments
  • Usage and utilization by team and project
workloads / jobs
Illustrative
  • pretrain-7b-run-12Runningcheckpoint 41 · 2 min ago
  • ft-support-v4Runningepoch 3 of 4
  • pretrain-ablation-3Queuedwaiting for 64 GPUs in one fabric domain
  • eval-nightlyBackfilledrunning on idle capacity
  • ft-legal-v2Resumedrestarted from checkpoint after node drain

Group 03

Serve

Put models in front of users — and keep them fast.

Model Deployment, Inference Services, Token Factory, and the Model API Gateway take a trained model to a production endpoint.

3.1Model Deployment

From checkpoint to endpoint.

Package a model from the registry, roll it out behind a canary, and promote it when it's healthy. Rollbacks are a configuration change, and every deployment is versioned and reproducible.

  • Registry-based, containerized deployment
  • Canary rollouts with automatic rollback
  • Versioned serving configurations
  • Autoscaling on load and queue depth
  1. Registry
  2. Package
  3. Canary
  4. Production
  5. Autoscale
deployments / support-assistant
Illustrative
Model
assistant-v3
Source
model registry
Replicas
4 · min 2 · max 8
Scale on
queue depth
Rollout
canary 10% → 100%

Traffic split

v2 · 90%v3 · 10%

Replicas

3.2Inference Services

Inference optimization, built in.

Latency, throughput, and cost per token pull in different directions. Inference Services tune runtime, batching, and parallelism for each model — interactive endpoints for latency, batch endpoints for token throughput.

  • Optimized serving runtimes per model
  • Continuous batching and KV-cache management
  • Tensor, pipeline, or data parallelism by model size
  • Quantization where evaluation shows quality holds

Optimize serving for

illustrative
Batching
Continuous batching
Replicas
Autoscaled on queue depth
Parallelism
Matched to model size
Scaling signal
Queue depth + latency target

Typical fit: Most production APIs

3.3Token Factory

Buy outcomes, not servers.

Token Factory delivers generation as a metered service. Hosted models run on optimized serving stacks over pooled GPU capacity, and you plan and pay for token throughput instead of provisioning GPUs.

  • High token throughput

    Serving stacks tuned for sustained generation

  • Metered by design

    Usage attributed to model, key, and tenant

  • Shared or dedicated

    Pooled capacity, or reserved throughput for steady demand

token-factory / usage
Illustrative
Tokens generated · 14 dayssample data
  • assistant-v352%
  • summarize-v229%
  • embed-v119%

3.4Model API Gateway

One API for every model you run.

Applications call one endpoint. The Unified Model API Gateway handles authentication, routing between models and versions, rate limits, and usage reporting — so swapping or upgrading a model doesn't mean rewriting an integration.

  • Single endpoint across models and deployments
  • API keys, scopes, and per-key rate limits
  • Version routing for safe upgrades
  • Usage and errors by application
model-api / quickstart
Illustrative
# One endpoint, one key, every model you serve
curl https://<gateway-host>/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "assistant-v3", "messages": [...]}'

Authenticate

keys · scopes

Route

model → deployment

Meter

usage per key

Interface visuals on this page are illustrative and use sample data. They show how platform capabilities are organized, not a specific product release or measured performance.

Next step

See the platform on your workloads.

Tell us what you run and where. We'll walk your team through management, scheduling, and serving on YOTOP.