After your experiment has run and produced data on the node(s), this tutorial
shows you how to query it, pull it off the testbed, and visualize it.
This tutorial picks up where the basic experiment tutorials leave off:
your experiment has run, it produced data on the node(s), and now you
want to query it, pull it off the testbed, or visualise it.
There are three common data destinations on COSMOS: node-local disk (fast
scratch during the run), shared NFS (visible to multiple nodes and durable
across re-images), and your own laptop or off-testbed storage for long-term
archival. The tutorial walks through each in turn, then covers on-console
Python analysis and long-term archiving options.
After completing this tutorial you will be able to:
/mnt/datadump, /home).scp and rsync.| Difficulty | Beginner |
| Estimated time | 30 min |
| Domain / sandbox | Any COSMOS domain with a running experiment (examples use sb1.cosmos-lab.org) |
| Topic group | Getting started |
| Last verified | Not re-tested (migrated 2026-06-20) |
Background knowledge
Account & access
Devices / nodes
| Resource | Role | Qty | Notes |
|---|---|---|---|
| Any COSMOS node | Data collection and analysis host | 1+ | Must have a completed experiment whose output is on local disk or NFS |
Disk images
| Image | Load onto | Provides |
|---|---|---|
| TODO: verify | TODO: verify | Baseline images have /data mounted from a partition with sufficient free space; console hosts have Python and matplotlib preinstalled |
Software components
| Component | Version | Source |
|---|---|---|
| Python 3 | as installed on image/console | preinstalled on console hosts and most baseline images |
| pandas | as installed | preinstalled or pip install pandas |
| matplotlib | as installed | preinstalled on console; pip install matplotlib otherwise |
| NumPy | as installed | preinstalled or pip install numpy |
| zstd | as installed | preinstalled on most images; apt install zstd otherwise |
| rsync / scp | standard | available on console and most images |
Spectrum / RF / special
Not applicable.
Not applicable. This tutorial is about data retrieval and analysis, not about
a specific RF or network topology. The general connectivity is:
bucket1.cosmos-lab.org:/mnt/pool/fivehundred mounted at /mnt/datadump on BED nodes and several other sandboxes; svm-cl1-01.orbit-lab.org:/home_dirs/orbit mounted at /home on Nebula and some other domains.ssh <username>@console.sb1.cosmos-lab.org
ssh root@node1-1
ls /data/
mount | grep nfs
df -h /mnt/datadump
By convention, place experiment output under /data on the node (the
baseline disk images have /data mounted from a partition with plenty
of free space):
# typical experiment skeleton — replace with your real binary
mkdir -p /data/myexp-$(date +%Y%m%d-%H%M)
cd /data/myexp-*
./run_my_experiment.sh > stdout.log 2> stderr.log
Tag the run with a timestamp so multiple back-to-back runs don't
overwrite each other.
Helpful conventions:
manifest.json orREADME.txt in the run directory. Future-you needs to know whatzstd -T0 raw.bin typically halvesMost BED nodes and several other sandboxes mount the bulk-storage NFS
share at /mnt/datadump (backed by bucket1.cosmos-lab.org,
1.7 TiB) so multiple nodes can write into the same directory tree:
# on every recording node
mkdir -p /mnt/datadump/myexp-$(date +%Y%m%d-%H%M)/$(hostname)
./recorder --out /mnt/datadump/myexp-…/$(hostname)/capture.bin
For Nebula and a few other domains, /home is mounted from the NetApp
cluster (svm-cl1-01.orbit-lab.org:/home_dirs/orbit) and is the right place for
per-user shared state.
Check what's mounted before you write:
$ mount | grep nfs
bucket1.cosmos-lab.org:/mnt/pool/fivehundred on /mnt/datadump type nfs4 …
svm-cl1-01.orbit-lab.org:/home_dirs/orbit on /home type nfs4 …
$ df -h /mnt/datadump
Filesystem Size Used Avail Use% Mounted on
bucket1.cosmos-lab.org:/mnt/pool/fivehundred 1.7T 534G 1.2T 31% /mnt/datadump
If the path you want isn't mounted, talk to the staff — adding a domain
to an existing NFS share is a quick fix on the storage side (a few
minutes), but it's a co-ordinated change.
The simplest reliable workflow: leave the data on /data or NFS during
the run, then scp (or rsync) it out at the end.
From your laptop, with the testbed VPN/SSH working:
# scp a single file
scp seskar@console.sb1.cosmos-lab.org:/data/myexp-2026-05-12/capture.bin .
# rsync a whole directory tree (resumable, doesn't re-copy what you already have)
rsync -avP \
console.sb1.cosmos-lab.org:/data/myexp-2026-05-12/ \
./myexp-2026-05-12/
If the data is on NFS via console, the same paths work because the
console mounts the same share.
For really large pulls, switch to rsync -avPz --compress-level=1 to
turn on lightweight stream compression — saves WAN time at the cost of
a bit of CPU.
Once the data is local, Python + pandas + matplotlib is the standard
analysis stack:
import pandas as pd, matplotlib.pyplot as plt
# Example: a CSV of throughput vs. time
df = pd.read_csv('throughput.csv', names=['time_s', 'mbps'])
df.plot(x='time_s', y='mbps', marker='o')
plt.xlabel('Time (s)')
plt.ylabel('Throughput (Mb/s)')
plt.grid()
plt.title('iperf3 throughput')
plt.savefig('throughput.png', dpi=150)
For raw IQ captures (e.g. from a USRP), use NumPy with the correct
dtype:
import numpy as np
# UHD complex int16: interleaved I, Q as int16
iq = np.fromfile('capture.bin', dtype=np.int16)
iq = iq[0::2].astype(np.float32) + 1j * iq[1::2].astype(np.float32)
print(iq.shape, iq.dtype) # → (N,) complex64
print('Power (dBFS):', 10*np.log10(np.mean(np.abs(iq)**2) + 1e-20))
For 5G NR DCI captures from ngscope, see the project's own README —
it ships a Python loader.
Sometimes you want to glance at a result without copying it out. The
console hosts have Python and matplotlib installed; you can render to a
PNG and view it over an SSH+X tunnel, or just save it and grab the PNG
with scp:
# on console
ssh -X console.sb1.cosmos-lab.org
$ cd /data/myexp-…
$ python3 -c "
import pandas as pd, matplotlib.pyplot as plt
df = pd.read_csv('throughput.csv', names=['t','mbps'])
df.plot(x='t', y='mbps', grid=True)
plt.savefig('throughput.png', dpi=150)
"
# Save the PNG to your laptop:
$ exit
$ scp console.sb1.cosmos-lab.org:/data/myexp-…/throughput.png .
For results that need to be shareable (papers, reproducibility):
/mnt/datadump/published/<paper-id>/ is a convenient placeREADME.md describing how toAfter completing this tutorial you should be able to confirm:
ls /data/myexp-*/ on the node shows the expected output files (logs, captures, CSVs).df -h /mnt/datadump confirms NFS is mounted and has available space when using shared storage.scp / rsync command completes without errors and the files appear locally.throughput.png (or equivalent plot file).omf tell -a offh -t system:topo:allres
Only run omf save if you customized the disk image. Experiment output already copied
off the testbed does not need to be saved in the image.
omf save -n <node> # only if you changed the image and want to keep it
| Symptom | Likely cause | Fix |
|---|---|---|
/mnt/datadump not mounted on a node |
Domain not configured for NFS bulk storage | Check with mount \| grep nfs; contact testbed staff to add the domain to the share |
scp fails with "Connection refused" or "Permission denied" |
VPN not active or SSH key not registered | Ensure your COSMOS VPN is up; verify your SSH key is loaded in the portal |
rsync re-copies files already transferred |
Permissions or timestamps differ between source and destination | Add --checksum to rsync to compare by content, not mtime |
Python ModuleNotFoundError for pandas/numpy |
Library not installed on analysis machine | Run pip install pandas numpy matplotlib in your virtual environment |
| NFS write hangs or node appears stuck | NFS server unreachable (network blip) | Check connectivity: ping bucket1.cosmos-lab.org; if the NFS mount is wedged, contact testbed staff |
svm-cl1-01.orbit-lab.org in /home path |
Legacy hostname still present in mount table | Verify current NFS server hostname with testbed staff |
Author(s): COSMOS team · Last verified: Not re-tested (migrated 2026-06-20) · Tested image/release: TODO: verify · Tags: data, analysis, python, nfs, scp, getting-started