| | 171 | |
| | 172 | == Per Experiment Dynamic Files == |
| | 173 | |
| | 174 | These files end up in {{{/space/local/containers}} (softlnked from {{{/var/containers}}} on the physical nodes and in {{{/var/containers}}} on the container nodes. Presumably this directoty gets populated by the experiment startup script invocation {{{sudo /share/containers/setup/hv/bootstrap /proj/Deter/exp/2x4-cont/containers/site.conf}}}. |
| | 175 | |
| | 176 | {{{ |
| | 177 | total 56 |
| | 178 | drwxr-xr-x 2 root root 4096 Feb 15 08:06 bin |
| | 179 | drwxr-xr-x 10 glawler root 4096 Feb 15 08:10 config |
| | 180 | -rw-r--r-- 1 root root 0 Feb 15 08:10 configured |
| | 181 | -rw-r--r-- 1 root root 9 Feb 15 08:06 eid |
| | 182 | drwxr-xr-x 2 root root 4096 Feb 15 08:06 etc |
| | 183 | drwxr-xr-x 2 root root 4096 Feb 15 08:09 images |
| | 184 | drwxr-xr-x 7 root root 4096 Feb 15 08:06 launch |
| | 185 | drwxr-xr-x 2 root root 4096 Feb 15 08:10 lib |
| | 186 | drwxr-xr-x 2 root root 4096 Feb 15 08:10 log |
| | 187 | drwxr-xr-x 2 root root 4096 Feb 15 08:07 mnt |
| | 188 | drwxr-xr-x 6 root root 4096 Feb 15 08:06 packages |
| | 189 | -rw-r--r-- 1 root root 6 Feb 15 08:06 pid |
| | 190 | drwxr-xr-x 7 root root 4096 Feb 15 08:06 setup |
| | 191 | drwxr-xr-x 4 root root 4096 Feb 15 08:10 vde |
| | 192 | drwxr-xr-x 2 root root 4096 Feb 15 08:10 vmchannel |
| | 193 | }}} |
| | 194 | |
| | 195 | ** bin/info.py** - spits out basic information to stdout: |
| | 196 | |
| | 197 | {{{ |
| | 198 | [(buffynet) glawler@a:/var/containers/bin]$ ./info.py --all |
| | 199 | NODE=a, PROJECT=Deter, EXPERIMENT=2x4-cont |
| | 200 | TYPE=qemu |
| | 201 | NAME=inf000, INET=10.0.0.4, MASK=255.255.255.0, MAC=00:00:00:00:00:04 |
| | 202 | NAME=control0, INET=172.16.183.35, MASK=255.240.0.0, MAC=00:66:00:00:b7:23 |
| | 203 | CONTROL=control0, INET=172.16.183.35, MASK=255.240.0.0, MAC=00:66:00:00:b7:23 |
| | 204 | }}} |
| | 205 | |
| | 206 | **/var/containers/launch/hv/hv** - container startup script. This is invoked after the emulab-configured event, which is triggered after the control network interface is configured. This is where it all starts. ***What does HV stand for?** "Hypervisor" maybe? This only runs on the pnode I'm assuming. |
| | 207 | |
| | 208 | **/etc/rc.local** - on the containers mounts the appropriate directories and calls a container-type specific bootstrap script and sets up routes (by calling /var/containers/launch/routes.py /var/containers/config) and calls the start command from the orginal NS file (if it exists). |
| | 209 | |
| | 210 | == /share/containers/ Files == |
| | 211 | |
| | 212 | Documentation for select files from the containers repo is below. |
| | 213 | |
| | 214 | **setup/hv/bootstrap** - This is the file that gets executed as the DETER start_cmd on the pnodes once the experiment is swapped in. It reads the experiment-specific conf file then sets up the local containers working directory (in {{{/share/containers}}). It then installs various packages, both for DETER and platform specific things ("deter_data" and a VDE build). It then runs {{{var/containers/setup/hv/hv}}} to do actual work. This file is described below. |
| | 215 | |
| | 216 | **/var/containers/setup/hv/hv** - This script is invoked by the start_cmd script on the pnode after the node has booted and before the containers are up. It untars the ${EXP}/containers dir into {{{/var/containers/config}}} and writes the /etc/{init,init.d} containers control/start/stop script. It then reads the {{{children}}} file. It then calls the per-child setup script ({{/setup/hv/*}}} (although I don't see these). Finally it invokes the /etc/init/containers platform script. (On Ubuntu it uses "start containers".) This maps to /var/containers/launch/hv/hv, described below. |
| | 217 | |
| | 218 | **/var/containers/launch/hv/hv** - The script invoked by {{./setup/hv/hv}} on the pnode. It sets up tap pipes and bridges for connecting VDE switches between pnodes. It creates the vde switches. It then calls {{{/var/containers/launch/%s/hv}}} %s=child name for all existing files. There is one of these files for each type of container supported, embedded_pnode, qemu, openvz, and process. The qemu version is explained below. |
| | 219 | |
| | 220 | **/var/containers/launch/qemu/hv** - this file is invoked at the end of the launch sequence on a pnode to spawn the qemu containers that will run on that pnode. (The VDE networking is setup before this script is invoked.) Log is written to {{{/var/containers/log/hv_qemu.log}}}. Builds something called "vncboot" (don't know why this is needed). The script then reads the per-experiment YAML config files, mounts the qemu image on the phost (via qemu-nbd), then writes the config information to the qemu disk. This assumes an Ubuntu image, which is bad. There doesn't seem to be any OS/platform checking here at all. (This file writes rc.local to the qemu image.) Then it starts up some IRC-based control channel with a server running on boss. ??? There is no ircd running on boss now, so is this some sort of legacy command/control channel for containers? |
| | 221 | |
