Real messages, real causes. Every entry below has actually happened; several cost hours before the cause was found, which is why the wording is preserved verbatim — searching for the text you are looking at should land here.
'x' cannot run on <sandbox> right now'needs-impossible' cannot run on grid right now.
63 powered-on node(s) were considered.
- worker: no candidate provides gpu (model=)
needed 3, found 0
each requirement, counted independently:
gpu: satisfied by 0 of 63 candidate(s) <-- nothing provides this
Working as designed. The arrow marks the requirement at fault — with four requirements and one impossible, this points straight at it. Either ask for less, or move sandbox.
pinned nodes belong to another subdomain- tx: pinned nodes belong to another subdomain
needed 1, found 0
node18-2.grid.cosmos-lab.org
-> in grid, not sb4
This experiment is pinned to specific hardware and does not run
anywhere else. Run it from console.grid.cosmos-lab.org.
You are on the wrong console. The message names the right one. Waiting will not help — pinned means pinned.
does not match '^[a-z][a-z0-9-]*$'The top-level name takes hyphens; resource and parameter names take underscores. needs_impossible is invalid as a name, valid as a resource.
No description at <name>, and no such file under: …The name is relative to ~/cosmos-experiments then /usr/share/cosmos-experiments. Both are listed in the error. Check the subdirectory — it is sdr/ofdm-ber-sb5.description.yml, not ofdm-ber-sb5.description.yml.
WARNING - using ~/cosmos-experiments/… which DIFFERS from the packaged …cosmos-run: WARNING - using /home/seskar/cosmos-experiments/sdr/ofdm-ber-sb5.description.yml
which DIFFERS from the packaged /usr/share/cosmos-experiments/sdr/ofdm-ber-sb5.description.yml
Delete your copy to use the packaged one.
Take this seriously. Your home copy wins, and it goes on winning after the package is updated. A stale copy kept a transmit gain of 70 after the package moved to 89, and the resulting mismatch was diagnosed for hours as failing hardware. If you did not mean to keep a fork, delete your copy.
Two forms exist and they behave differently:
cosmos-run x.yml -e amplitude=0.06 # sets the variable
cosmos-run x.yml -e '{"cosmos_parameters": {"amplitude": 0.06}}' # goes through the validator
Both work. The second is checked against choices, min and max; the first is not. Prefer the second when the value matters.
UHD is not available on <node>UHD is not available on node3-20.grid.cosmos-lab.org. This experiment needs
an SDR image (for example ubuntu2404-uhd4.9-gr3.10.ndz or a baseline-sdr
image). The node is running something else - re-image it, or let node_ready
do so with an sdr.usrp requirement.
Someone re-imaged the node, or it never had the right image. Check whether they are still using it before taking it back — judge on recent omf load activity, not on who, since sessions stay attached for weeks.
Connection refused right after imagingThe node is almost certainly still in the PXE stage image. A successful omf load does not prove the node booted what you loaded.
ssh root@node1-2 cat /.omf-image-name # what it actually booted
omf tell -a offh -t node1-2 && sleep 25 && omf tell -a on -t node1-2
No UHD Devices Found on a node that definitely has a radioOn sb6, each X310 is on a private point-to-point 10 GbE run whose port comes up with no address, so UHD cannot see it:
ip addr add 192.168.40.1/24 dev enp1s0 && ip link set enp1s0 up mtu 9000
uhd_find_devices --args="addr=192.168.40.2"
On the grid, the X310s are rfdevice resources in MIMO racks, not attached to any node — omf load does not apply to them. Probe the rack addresses (10.10.23.x, 10.10.24.x) and pin the one you want.
Both of these kill it at construction, so the symptom surfaces several tasks later as "the transmitter is not running", pointing at nothing.
internal is not a valid time sourceRuntimeError: AssertionError: assertion failed:
internal is not a valid time source.
possible values are: [none, external, _external_, mimo].
A USRP2/N210 rejects internal, though GNU Radio Companion offers it.
Board 0 may not be getting a PPS signalset_time_unknown_pps() blocks waiting for a pulse that a radio with no external reference will never produce. GRC emits it whenever the block's sync option is "Unknown PPS". Two radios measuring a BER need no common time base — use "don't sync".
This is its own discipline; the short version, in the order to check.
Transmit overdrive. Compression preserves power while destroying the constellation, so the received level and the computed effective SNR both look perfect and only the BER is wrong. Halve the drive.
Measured on sb5, QPSK: amplitude 0.06 gives 3.8e-6, amplitude 0.15 gives 7.5e-2 — and both report ~24 dB effective SNR.
Ask whether you reached a floor or the end of the axis. 64-QAM needs about 25.6 dB for 1e-5; a sweep stopping at 20 dB shows a "floor" that is the end of the sweep.
Compare your asked SNR against the dB above floor the run prints. Beyond that the link has no more SNR to give, every point measures the same thing, and the scatter is the noise of a saturated axis. More samples never fix this.
--check fails on something that has not happened yetSteps that cannot be simulated are skipped, and steps that depend on their output are skipped with them. If you see a genuine failure under --check, that is a bug worth reporting — a dry run should report, not fail.
cosmos-run x.yml -v # more; repeat for more still
cosmos-run x.yml --version # which copy is actually running
cosmos-run x.yml -- --step # anything after -- goes to ansible-playbook
--version is the first thing to check when a fix does not seem to have taken effect: it names the copy in use and any it is shadowing.
Last verified: 2026-08-04 on console.sb5.cosmos-lab.org.