Platform
The software that makes GPUs easy to use.
Raw GPUs are only the beginning. The YOTOP Platform is the software layer that manages clusters, orchestrates GPUs, schedules jobs, and serves models — so your teams consume compute as a service instead of operating it.
Group 01
Manage
Control every cluster and GPU from one place.
Unified Cluster Management and GPU Resource Orchestration give operators and platform teams a single view of capacity — and the controls to allocate it.
1.1Unified Cluster Management
One inventory. One health model.
See every cluster, node, and GPU in one console — health, configuration, and lifecycle state included. Cluster observability surfaces GPU, fabric, and storage signals before they turn into failed jobs.
- Inventory across clusters and sites
- Node lifecycle: provisioning, ready, draining, maintenance
- Cluster observability with alerting
- Role-based access for operators and tenants
- Healthy
Site 1
2 clusters
- Maintenance
Site 2
1 cluster
- 1 node draining
Site 3
2 clusters
● site-3/cluster-b/node-07 · GPU error → drained
● site-1 · firmware rollout 3/8 nodes
● queue: training · 2 jobs pending
1.2GPU Resource Orchestration
Pool capacity. Allocate by policy.
GPU orchestration treats capacity across clusters as one pool. Allocate GPUs to teams and tenants by policy, move work to healthy capacity when hardware fails, and scale inference out when demand rises.
- Cross-cluster GPU pooling
- Allocation by team, tenant, or project
- Automatic failover to healthy nodes
- Partitioning and sharing where hardware supports it
Place work where it runs best
- 01 › Training job placed on 16 GPUs in one fabric domain
- 02 › Fine-tuning queue admitted under team quota
- 03 › Inference replicas pinned across two clusters
- Training
- Fine-tuning
- Inference
- Available
Group 02
Run
Keep the right jobs on the right GPUs.
Job Scheduling and AI Workload Management turn shared capacity into high utilization — without teams fighting over GPUs.
2.1Job Scheduling
Workload scheduling that understands AI.
Distributed training needs every GPU at once, close together on the fabric. Inference needs guaranteed headroom. Batch jobs should fill the gaps. YOTOP scheduling encodes those rules on Kubernetes or Slurm.
- Gang scheduling for multi-node jobs
- Topology-aware placement
- Priorities, preemption, and fair-share
- Backfill of idle capacity
- InferenceGuaranteed
- TrainingHigh · gang
- Fine-tuningFair-share
- Batch / evalPreemptible
Policy
Scheduler
priority
quota
topology
preemption
Shared GPU pool
Serving capacity is protected; training runs gang-scheduled in one fabric domain; batch backfills what is left.
2.2AI Workload Management
Every job visible, recoverable, and accountable.
Track jobs from submission to completion. When a node fails, checkpointed jobs resume on healthy capacity. Usage rolls up by team and project, so GPU-hours connect to the work they produced.
- Job queues and lifecycle tracking
- Checkpoint-aware restart after failures
- Versioned container environments
- Usage and utilization by team and project
- pretrain-7b-run-12training256 GPURunningcheckpoint 41 · 2 min ago
- ft-support-v4fine-tuning16 GPURunningepoch 3 of 4
- pretrain-ablation-3training64 GPUQueuedwaiting for 64 GPUs in one fabric domain
- eval-nightlybatch8 GPUBackfilledrunning on idle capacity
- ft-legal-v2fine-tuning8 GPUResumedrestarted from checkpoint after node drain
Group 03
Serve
Put models in front of users — and keep them fast.
Model Deployment, Inference Services, Token Factory, and the Model API Gateway take a trained model to a production endpoint.
3.1Model Deployment
From checkpoint to endpoint.
Package a model from the registry, roll it out behind a canary, and promote it when it's healthy. Rollbacks are a configuration change, and every deployment is versioned and reproducible.
- Registry-based, containerized deployment
- Canary rollouts with automatic rollback
- Versioned serving configurations
- Autoscaling on load and queue depth
- Model
- assistant-v3
- Source
- model registry
- Replicas
- 4 · min 2 · max 8
- Scale on
- queue depth
- Rollout
- canary 10% → 100%
Traffic split
Replicas
3.2Inference Services
Inference optimization, built in.
Latency, throughput, and cost per token pull in different directions. Inference Services tune runtime, batching, and parallelism for each model — interactive endpoints for latency, batch endpoints for token throughput.
- Optimized serving runtimes per model
- Continuous batching and KV-cache management
- Tensor, pipeline, or data parallelism by model size
- Quantization where evaluation shows quality holds
Optimize serving for
illustrative- Batching
- Continuous batching
- Replicas
- Autoscaled on queue depth
- Parallelism
- Matched to model size
- Scaling signal
- Queue depth + latency target
Typical fit: Most production APIs
3.3Token Factory
Buy outcomes, not servers.
Token Factory delivers generation as a metered service. Hosted models run on optimized serving stacks over pooled GPU capacity, and you plan and pay for token throughput instead of provisioning GPUs.
High token throughput
Serving stacks tuned for sustained generation
Metered by design
Usage attributed to model, key, and tenant
Shared or dedicated
Pooled capacity, or reserved throughput for steady demand
- assistant-v352%
- summarize-v229%
- embed-v119%
3.4Model API Gateway
One API for every model you run.
Applications call one endpoint. The Unified Model API Gateway handles authentication, routing between models and versions, rate limits, and usage reporting — so swapping or upgrading a model doesn't mean rewriting an integration.
- Single endpoint across models and deployments
- API keys, scopes, and per-key rate limits
- Version routing for safe upgrades
- Usage and errors by application
# One endpoint, one key, every model you serve
curl https://<gateway-host>/v1/chat/completions \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "assistant-v3", "messages": [...]}'Authenticate
keys · scopes
Route
model → deployment
Meter
usage per key
Interface visuals on this page are illustrative and use sample data. They show how platform capabilities are organized, not a specific product release or measured performance.
Next step
See the platform on your workloads.
Tell us what you run and where. We'll walk your team through management, scheduling, and serving on YOTOP.
