COSMOS runs on two layers of software, and knowing which one you are looking at saves a great deal of confusion.
The testbed services own the hardware. They power nodes on and off, load disk images, hand out addresses, hold the reservation calendar, and know which radio is plugged into which port. They are long-running services on the infrastructure, reached through an aggregate manager, and they are the same services whether you drive them by hand or from a script.
The orchestration framework is how an experiment is expressed and run. You write one declarative description of what you need — two nodes with USRPs, an attenuator at 40 dB, this software staged, these results collected — and the framework works out whether that is possible on a given sandbox, asks the services for it, runs the experiment, collects the artifacts and tears down.
The framework does not replace the services. It calls them. omf load is still what images a node; the framework wraps it so that imaging is one line in a description instead of a step you remember to do.
| Experiment orchestration | The current framework: descriptions, cosmos-run, the role library. Start here if you are running experiments. |
| Testbed services | Inventory, PXE, Frisbee, chassis management, scheduling and the rest — what the framework calls, and what you talk to directly if you are operating the testbed. |
Everything below is a real session on a COSMOS console. Nothing is installed by you: the framework is already there.
seskar@console-grid:~$ cosmos-run sdr/ofdm-ber-grid.description.yml --resolve-only
PLAY [Resolve the experiment onto live nodes] **********************************
"msg": "ofdm-ber-grid on grid: tx=node18-2.grid.cosmos-lab.org; rx=node18-19.grid.cosmos-lab.org "
PLAY RECAP *********************************************************************
localhost : ok=10 changed=0 unreachable=0 failed=0 skipped=1
That answers "could this run here, and on which nodes" without touching anything — it does not even need a reservation. Drop --resolve-only and the same command runs the experiment.
The description it just read is a couple of dozen lines of YAML naming what the experiment needs rather than how to obtain it. The orchestration pages cover the language in full.
Three properties drove the design, and each one is visible in how the system behaves when something goes wrong.
The inventory is live, so the experiment is resolved fresh every run. Nodes go offline out-of-band — someone re-images one, a radio stops answering. A host list captured last week is a list of hopeful guesses. The framework queries the inventory service on every run and tells you what it found:
'ofdm-ber-grid' cannot run on sb4 right now.
4 powered-on node(s) were considered.
- tx: pinned nodes belong to another subdomain
needed 1, found 0
node18-2.grid.cosmos-lab.org
-> in grid, not sb4
This experiment is pinned to specific hardware and does not run
anywhere else. Run it from console.grid.cosmos-lab.org.
That is a real failure message. It says what was wanted, what was found, why they did not match, and what to do — because "no hosts matched" is the least useful thing a system can say.
Descriptions are portable where the hardware allows and pinned where it does not. An experiment that needs any two nodes with a USRP says so and runs on whichever sandbox can satisfy it. One that needs a specific radio behind a specific attenuator says that instead, and refuses elsewhere rather than running somewhere that looks similar.
Every run records what it was. Each produces a manifest naming the description's hash, the collection version, the nodes actually chosen and the wall-clock time, next to the artifacts. Reproducing a result six months later does not depend on anyone's memory of which node they used.
Worth stating plainly, because these are deliberate limits rather than gaps:
omf. Imaging, power and resource bookkeeping belong to the testbed services; the framework wraps the existing commands.--resolve-only, which reads only the inventory and touches no hardware.Operators: the framework ships as two Debian packages, cosmos-orchestration (the collection and cosmos-run ) and cosmos-experiments (the descriptions the tutorials name), already installed on every console. Users do not install anything. See installation and packaging.
Last verified: 2026-08-03 on console.grid.cosmos-lab.org.