Deploy the Open Network Automation Platform (ONAP) on a bare-metal Kubernetes cluster provisioned from COSMOS testbed nodes, then bring up selected ONAP subsystems (mariadb-galera, portal) via Helm.
ONAP is an open-source platform for design, creation, and lifecycle orchestration of network services. It was formed from the merger of the Linux Foundation's OPEN-O and AT&T's ECOMP projects. The ONAP architecture has two major frameworks — a design-time environment and a run-time environment — each composed of many separate subsystems.
This tutorial walks through a simplified ONAP deployment on COSMOS: no OpenStack, Kubernetes only. Pre-built disk images (latest-onap-control.ndz and latest-onap-worker.ndz) are loaded onto testbed nodes with the helper script onaptutorial-min.rb. A cluster-creation script then bootstraps Kubernetes using RKE (Rancher Kubernetes Engine), and selected ONAP Helm charts are deployed into the cluster.
The example uses two nodes (one controller, one worker) drawn from the grid domain, but the setup script accepts multiple controllers and workers for larger deployments.
After completing this tutorial you will be able to:
onaptutorial-min.rb.helm ls and kubectl get nodes.| Difficulty | Advanced |
| Estimated time | 90+ min (includes ~12 min for make all; make onap) |
| Domain / sandbox | grid.cosmos-lab.org (example nodes: node13-8, node14-7) |
| Topic group | Cloud & Edge |
| Last verified | Not re-tested (migrated 2026-06-20) |
Background knowledge
Account & access
Devices / nodes
| Resource | Role | Qty | Notes |
|---|---|---|---|
| node14-7 | Kubernetes controller | 1 | Runs control plane, etcd; latest-onap-control.ndz loaded |
| node13-8 | Kubernetes worker | 1 | Runs ONAP workloads; latest-onap-worker.ndz loaded |
The scripts accept multiple controllers and workers — scale up by adjusting the
-cand-warguments.
Disk images
| Image | Load onto | Provides |
|---|---|---|
| latest-onap-control.ndz | Controller node(s) | Pre-configured Kubernetes control-plane environment for ONAP |
| latest-onap-worker.ndz | Worker node(s) | Pre-configured Kubernetes worker environment for ONAP |
Software components
| Component | Version | Source |
|---|---|---|
| RKE (Rancher Kubernetes Engine) | v0.2.1 | Downloaded by create-kube-cluster.sh from GitHub |
| Kubernetes | v1.13.5-rancher1-2 | Deployed by RKE; pinned in cluster.yml |
| Helm | TODO: verify | Pre-installed in controller image or installed separately |
| ONAP OOM Helm charts | El Alto (5.0.0) | Pre-cloned in ~/oom/ on controller image |
| MariaDB Galera | 5.0.0 (Helm chart) | Deployed as demo-mariadb-galera release |
| ONAP Portal | 5.0.0 (Helm chart) | Deployed as part of demo release |
Spectrum / RF / special
Not applicable.
Environment notes
-- Openstack installed
-- VM cloud image ubuntu 18.04-server-image
Two (or more) testbed nodes on the grid domain form a bare-metal Kubernetes cluster. The COSMOS console acts as the management station from which onaptutorial-min.rb images the nodes and create-kube-cluster.sh bootstraps the cluster over the control-plane network. All kubectl and helm commands are then run directly on the controller node via SSH.
Reserve at least 2 nodes on the grid domain (see Prerequisites) and log into the console:
ssh <username>@console.grid.cosmos-lab.org
Determine the topology group assigned to your reservation (e.g. group-black). The group name appears in the format system:topo:group-<name>.
Image the nodes using the helper script — this loads latest-onap-control.ndz onto the first node and latest-onap-worker.ndz onto the second, then powers them on:
onaptutorial-min.rb -g group-nogroup
Expected output:
msherman@console:~$ onaptutorial-min.rb -g group-black
Imaging:
Controller:latest-onap-control.ndz -> [node14-7.grid.orbit-lab.org]
Worker:latest-onap-worker.ndz -> [node13-8.grid.orbit-lab.org]
2019-10-21 14:22:02 -0400 Telling set group-black: offh
2019-10-21 14:22:23 -0400 Loading image: latest-onap-control.ndz for node(s) node14-7.grid.orbit-lab.org
2019-10-21 14:22:25 -0400 Loading image: latest-onap-worker.ndz for node(s) node13-8.grid.orbit-lab.org
2019-10-21 14:26:21 -0400 Telling set group-black: on
Check for error messages; if any appear, repeat the command.
Verify both nodes are powered on:
omf stat -t system:topo:group-nogroup
Expected output:
msherman@console:~$ omf stat -t system:topo:group-black
INFO NodeHandler: OMF Experiment Controller 5.4 (git 861d645)
INFO NodeHandler: Slice ID: default_slice (default)
INFO NodeHandler: Experiment ID: default_slice-2019-10-21t14.26.36.553-04.00
INFO NodeHandler: Message authentication is disabled
INFO property.resetDelay: resetDelay = 300 (Fixnum)
INFO property.resetTries: resetTries = 1 (Fixnum)
INFO property.nodes: nodes = "system:topo:group-black" (String)
INFO property.summary: summary = false (FalseClass)
INFO Topology: Loaded topology 'system:topo:group-black'.
Talking to the CMC service, please wait
-----------------------------------------------
Node: node13-8.grid.orbit-lab.org State: POWERON
Node: node14-7.grid.orbit-lab.org State: POWERON
-----------------------------------------------
INFO EXPERIMENT_DONE: Event triggered. Starting the associated tasks.
INFO NodeHandler:
INFO NodeHandler: Shutting down experiment, please wait...
INFO NodeHandler:
INFO run: Experiment default_slice-2019-10-21t14.26.36.553-04.00 finished after 0:5
If a node is not in state POWERON, run:
omf tell -a on -t system:topo:group-nogroup
Confirm the nodes respond to ping from the console:
ping node13-8
Note the controller and worker node names — you will need them in the next step.
On the console, create a working directory and download the cluster-creation script:
mkdir onap-test && cd onap-test
wget https://wiki.cosmos-lab.org/raw-attachment/wiki/tutorials/orchestration-example/create-kube-cluster.sh
chmod 755 create-kube-cluster.sh
Run the script, substituting the actual controller and worker node names assigned to your reservation:
./create-kube-cluster.sh -c "node1" -w "node2"
The script (create-kube-cluster.sh) performs the following steps automatically:
#!/bin/bash
rm -f cluster.yml
rm -f kube_config_cluster.yml
rm -f rke
rm -f config
omf tell -t all -a offh
sleep 60
wget https://github.com/rancher/rke/releases/download/v0.2.1/rke_linux-amd64
mv rke_linux-amd64 rke
chmod 754 rke
usage () {
echo "Usage:"
echo " ./$(basename $0) -c \"cNode1 cNode2 ... cnodeN\" -w \"wNode1 wnode2 ... wnodeN\""
echo "Note: controllers hostnames and workers hostnames are to be enclosed in \"\""
exit 0
}
if [[ ( $# == "--help") || $# == "-h" ]]
then
usage
exit 0
fi
if [ "$#" -lt 2 ]; then
echo "Missing Kubernetes control and worker nodes"
usage
fi
echo "# An example of an HA Kubernetes cluster for ONAP" >> cluster.yml
echo "nodes:" >> cluster.yml
while getopts c:w: option
do
case "${option}"
in
c) CONTROLLERS=${OPTARG};;
w) WORKERS=${OPTARG};;
esac
done
IFS=' ' read -ra C <<< "$CONTROLLERS"
IFS=' ' read -ra W <<< "$WORKERS"
echo "Testing node availability. This might take some time"
for i in "${C[@]}"; do
while ! ping -c 1 -n -w 1 $i &> /dev/null
do
printf "%c" "."
done
echo "127.0.0.1 localhost" > hosts
echo "`ping $i -c 1 | grep "PING" | grep '('|awk '{gsub(/[()]/,""); print $3}'` ${i}" >> hosts
scp hosts root@$i:/etc/hosts
done
for i in "${W[@]}"; do
while ! ping -c 1 -n -w 1 $i &> /dev/null
do
printf "%c" "."
done
echo "127.0.0.1 localhost" > hosts
echo "`ping $i -c 1 | grep "PING" | grep '('|awk '{gsub(/[()]/,""); print $3}'` ${i}" >> hosts
scp hosts root@$i:/etc/hosts
done
echo "Availability check successful"
for i in "${C[@]}"; do
echo "- address: `ping $i -c 1 | grep "PING" | grep '('|awk '{gsub(/[()]/,""); print $3}'`" >> cluster.yml
echo ' port: "22"' >> cluster.yml
echo " role:" >> cluster.yml
echo " - controlplane" >> cluster.yml
echo " - etcd" >> cluster.yml
echo " hostname_override: `ping $i -c 1 | grep 'PING' | awk '{print $2}' | awk -F . '{print $1}'`" >> cluster.yml
echo " user: root" >> cluster.yml
echo " ssh_key_path: '~/.ssh/id_rsa'" >> cluster.yml
done
echo "# worker nodes start " >> cluster.yml
for i in "${W[@]}"; do
echo "- address: `ping $i -c 1 | grep "PING" | grep '('|awk '{gsub(/[()]/,""); print $3}'`" >> cluster.yml
echo ' port: "22"' >> cluster.yml
echo " role:" >> cluster.yml
echo " - worker" >> cluster.yml
echo " hostname_override: `ping $i -c 1 | grep 'PING' | awk '{print $2}' | awk -F . '{print $1}'`" >> cluster.yml
echo " user: root" >> cluster.yml
echo " ssh_key_path: '~/.ssh/id_rsa'" >> cluster.yml
done
echo 'services:
kube-api:
service_cluster_ip_range: 10.43.0.0/16
pod_security_policy: false
always_pull_images: false
kube-controller:
cluster_cidr: 10.42.0.0/16
service_cluster_ip_range: 10.43.0.0/16
kubelet:
cluster_domain: cluster.local
cluster_dns_server: 10.43.0.10
fail_swap_on: false
network:
plugin: canal
authentication:
strategy: x509
ssh_key_path: "~/.ssh/id_rsa"
ssh_agent_auth: false
authorization:
mode: rbac
ignore_docker_version: false
kubernetes_version: "v1.13.5-rancher1-2"
private_registries:
- url: nexus3.onap.org:10001
user: docker
password: docker
is_default: true
cluster_name: "onap"
restore:
restore: false
snapshot_name: ""' >> cluster.yml
./rke up
for i in "${C[@]}"; do
scp kube_config_cluster.yml root@$i:~/.kube/config
done
exit 0
When rke up completes, the kubeconfig is copied to the controller node.
SSH into the controller node:
ssh root@node1
Set the default Kubernetes namespace to onap:
kubectl config set-context --current --namespace=onap
Verify the cluster nodes are registered and ready:
kubectl get nodes -o=wide
On the controller node, initialize Tiller and wait for it to roll out:
kubectl -n kube-system create serviceaccount tiller
kubectl create clusterrolebinding tiller --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
helm init --service-account tiller
kubectl -n kube-system rollout status deploy/tiller-deploy
Edit the override file to enable only the subsystems you need. In the example below, mariadb-galera and portal are enabled; all others are disabled:
cd overrides/
//edit the onap-all.yaml file and set the ONAP components to false
nano onap-all.yaml
Example onap-all.yaml contents (mariadb-galera and portal enabled):
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
###################################################################
# This override file enables helm charts for all ONAP applications.
###################################################################
cassandra:
enabled: false
mariadb-galera:
enabled: true
aaf:
enabled: false
aai:
enabled: false
appc:
enabled: false
cds:
enabled: false
clamp:
enabled: false
cli:
enabled: false
consul:
enabled: false
contrib:
enabled: false
dcaegen2:
enabled: false
dmaap:
enabled: false
esr:
enabled: false
log:
enabled: false
sniro-emulator:
enabled: false
oof:
enabled: false
msb:
enabled: false
multicloud:
enabled: false
nbi:
enabled: false
policy:
enabled: false
pomba:
enabled: false
portal:
enabled: true
robot:
enabled: false
sdc:
enabled: false
sdnc:
enabled: false
so:
enabled: false
uui:
enabled: false
vfc:
enabled: false
vid:
enabled: false
vnfsdk:
enabled: false
modeling:
enabled: false
// start server
helm serve &
//hit enter if the process is running to let the process run in background
// add repository
helm repo add local http://127.0.0.1:8879
This step packages all ONAP Helm charts and makes them available in the local repository. This takes 12 or more minutes — be patient.
cd ~/oom/kubernetes
make all; make onap
Expected output (abridged):
root@node:make all; make onap
[common]
make[1]: Entering directory '/root/oom/kubernetes'
make[2]: Entering directory '/root/oom/kubernetes/common'
[common]
make[3]: Entering directory '/root/oom/kubernetes/common'
==> Linting common
[INFO] Chart.yaml: icon is recommended
...
Update Complete. Happy Helming!
Saving 34 charts
Downloading aaf from repo http://127.0.0.1:8879
Downloading aai from repo http://127.0.0.1:8879
Downloading appc from repo http://127.0.0.1:8879
Downloading cassandra from repo http://127.0.0.1:8879
Downloading cds from repo http://127.0.0.1:8879
Downloading clamp from repo http://127.0.0.1:8879
Downloading cli from repo http://127.0.0.1:8879
Downloading common from repo http://127.0.0.1:8879
Downloading consul from repo http://127.0.0.1:8879
Downloading contrib from repo http://127.0.0.1:8879
Downloading dcaegen2 from repo http://127.0.0.1:8879
Downloading dmaap from repo http://127.0.0.1:8879
Downloading esr from repo http://127.0.0.1:8879
Downloading log from repo http://127.0.0.1:8879
Downloading sniro-emulator from repo http://127.0.0.1:8879
Downloading mariadb-galera from repo http://127.0.0.1:8879
Downloading msb from repo http://127.0.0.1:8879
Downloading multicloud from repo http://127.0.0.1:8879
Downloading nbi from repo http://127.0.0.1:8879
Downloading nfs-provisioner from repo http://127.0.0.1:8879
Downloading pnda from repo http://127.0.0.1:8879
Downloading policy from repo http://127.0.0.1:8879
Downloading pomba from repo http://127.0.0.1:8879
Downloading portal from repo http://127.0.0.1:8879
Downloading oof from repo http://127.0.0.1:8879
Downloading robot from repo http://127.0.0.1:8879
Downloading sdc from repo http://127.0.0.1:8879
Downloading sdnc from repo http://127.0.0.1:8879
Downloading so from repo http://127.0.0.1:8879
Downloading uui from repo http://127.0.0.1:8879
Downloading vfc from repo http://127.0.0.1:8879
Downloading vid from repo http://127.0.0.1:8879
Downloading vnfsdk from repo http://127.0.0.1:8879
Downloading modeling from repo http://127.0.0.1:8879
Deleting outdated charts
==> Linting onap
Lint OK
1 chart(s) linted, no failures
Successfully packaged chart and saved it to: /root/oom/kubernetes/dist/packages/onap-5.0.0.tgz
make[1]: Leaving directory '/root/oom/kubernetes'
root@node:~/oom/kubernetes#
helm deploy demo local/onap --namespace onap -f ~/overrides/onap-all.yaml
Expected output:
root@node:~/oom/kubernetes# helm deploy demo local/onap --namespace onap -f ~/overrides/onap-all.yaml
fetching local/onap
release "demo" deployed
release "demo-cassandra" deployed
release "demo-mariadb-galera" deployed
root@node:~/oom/kubernetes#
helm ls
Expected output:
NAME REVISION UPDATED STATUS CHART APP VERSION NAMESPACE
demo 1 Sun Oct 20 02:14:55 2019 DEPLOYED onap-5.0.0 El Alto onap
demo-cassandra 1 Sun Oct 20 02:14:55 2019 DEPLOYED cassandra-5.0.0 onap
demo-mariadb-galera 1 Sun Oct 20 02:14:56 2019 DEPLOYED mariadb-galera-5.0.0 onap
kubectl get nodes -o=wide shows all controller and worker nodes in Ready state.helm ls lists the demo, demo-cassandra, and demo-mariadb-galera releases all with STATUS: DEPLOYED.kubectl get pods -n onap shows the MariaDB Galera and Portal pods running (allow several minutes for images to pull and containers to start).When your experiment is complete, release the nodes:
omf tell -a offh -t system:topo:allres
No omf save is needed unless you modified the disk images and want to preserve the changes.
| Symptom | Likely cause | Fix |
|---|---|---|
onaptutorial-min.rb reports errors during imaging |
Node not reachable or frisbee timeout | Re-run the command; check omf stat to confirm nodes are in POWERON before retrying |
Node not in POWERON after imaging |
Power sequencing issue | Run omf tell -a on -t system:topo:group-<name> and recheck omf stat |
ping node13-8 fails from console |
Node not yet booted | Wait 60–90 seconds after POWERON and retry |
rke up fails with SSH errors |
SSH key not present on nodes or wrong hostname | Confirm ~/.ssh/id_rsa is present on the console and the node hostnames match what ping resolves |
make all; make onap fails |
Helm server not running or missing OOM repo | Confirm helm serve & is running and helm repo list shows local; re-run helm repo add local http://127.0.0.1:8879 |
helm deploy shows no releases |
Namespace onap not created |
Run kubectl create namespace onap then retry |
Pods stuck in ImagePullBackOff |
Cannot reach nexus3.onap.org:10001 |
Verify outbound internet connectivity from the worker node; check the private registry credentials in cluster.yml |
Author(s): COSMOS team · Last verified: Not re-tested (migrated 2026-06-20) · Tested image/release: latest-onap-control.ndz / latest-onap-worker.ndz; ONAP El Alto (5.0.0); Kubernetes v1.13.5-rancher1-2 · Tags: onap, kubernetes, helm, cloud, orchestration, nfv