| Version 3 (modified by , 9 years ago) (diff) |
|---|
This wiki page documents the files created by the DETER containerization system. I've created a simple containerized experiment, then looked at all the files generated, documenting the contents of the files below.
The sample experiment is Deter,muppetnet. The NS file and containerization command are:
set ns [new Simulator]
source tb_compat.tcl
set opt(netSpeed) 100Mb
set allNodes "beaker beauregard brewster drbunsenhoneydew floyd lips mousey rizzo pops kermit piggy fozzie drteeth animal janice"
set studNodes "beaker beauregard brewster drbunsenhoneydew kermit piggy fozzie"
set mayhemNodes "floyd lips mousey rizzo pops drteeth animal janice"
foreach node $allNodes {
set $node [$ns node]
}
# link between LANS
set rainbow [$ns duplex-link "fozzie" "janice" 100Mb 0.0ms DropTail]
# two LANs
set studebaker [$ns make-lan $studNodes $opt(netSpeed) 0.0ms]
set electricmayhem [$ns make-lan $mayhemNodes $opt(netSpeed) 0.0ms]
$ns rtproto Static
$ns run
[glawler@users:~/src/nsfiles]$ /share/containers/containerize.py DETER muppetnet ./muppetnet.ns
This creates the directory /proj/Deter/exp/muppetnet which contains a containers directory. This directory has the following files:
-rw-r--r-- 1 glawler Deter 9668 Feb 12 07:25 annotated.xml -rw-r--r-- 1 glawler Deter 142 Feb 12 07:25 assignment -rw-r--r-- 1 glawler Deter 68 Feb 12 07:25 backend_config.yaml drwxr-xr-x 2 glawler Deter 1024 Feb 12 07:25 children -rw-r--r-- 1 glawler Deter 7246 Feb 12 07:26 config.tgz -rw-r--r-- 1 glawler Deter 5660 Feb 12 07:25 converted.xml -rw-r--r-- 1 glawler Deter 2432 Feb 12 07:25 embedding.yaml -rw-r--r-- 1 glawler Deter 662 Feb 12 07:25 experiment.tcl -rw-r--r-- 1 glawler Deter 0 Feb 12 07:25 ghosts -rw-r--r-- 1 glawler Deter 793 Feb 12 07:26 hosts -rw-r--r-- 1 glawler Deter 6 Feb 12 07:26 hostsw.pickle -rw-r--r-- 1 glawler Deter 1315 Feb 12 07:25 maverick_url -rw-r--r-- 1 glawler Deter 1690 Feb 12 07:25 openvz_guest_url -rw-r--r-- 1 glawler Deter 11063 Feb 12 07:25 partitioned.xml -rw-r--r-- 1 glawler Deter 816 Feb 12 07:25 phys_topo.ns -rw-r--r-- 1 glawler Deter 1120 Feb 12 07:25 phys_topo.xml -rw-r--r-- 1 glawler Deter 16 Feb 12 07:25 pid_eid -rw-r--r-- 1 glawler Deter 54 Feb 12 07:25 pnode_types.yaml -rw-r--r-- 1 glawler Deter 10 Feb 12 07:25 qemu_users.yaml drwxr-xr-x 2 glawler Deter 512 Feb 12 07:26 route -rw-r--r-- 1 glawler Deter 205 Feb 12 07:26 shaping.yaml -rw-r--r-- 1 glawler Deter 732 Feb 12 07:25 site.conf drwxr-xr-x 2 glawler Deter 512 Feb 12 07:26 software drwxr-xr-x 2 glawler Deter 512 Feb 12 07:26 switch -rw-r--r-- 1 glawler Deter 708 Feb 12 07:26 switch_extra.yaml -rw-r--r-- 1 glawler Deter 6 Feb 12 07:26 swlocs.pickle -rw-r--r-- 1 glawler Deter 36759 Feb 12 07:26 topo.xml -rw-r--r-- 1 glawler Deter 3136 Feb 12 07:26 traffic_shaping.pickle -rw-r--r-- 1 glawler Deter 66961 Feb 12 07:26 visualization.png -rw-r--r-- 1 glawler Deter 3 Feb 12 07:26 wirefilters.yaml
Notes on these files:
annotated.xml - This is an XML file that describes the nodes and substrates (networks) of the experiment. The node description includes name, interfaces, container type, and node type (pc, etc).
assignment - Shows container assignment to physical node. e.g. node "foo" runs on physical node 1.
backend_config.yaml - "{server: 'boss.isi.deterlab.net:6667', unique_id: fdb56b6d989ff6a4}"
children - Not sure. Is a directory that has files which describe a DAG - relationships between pnodes, openvz, and container nodes. Mostly empty files in this experiment though as the network topology is minimal.
config.tgz - This directory, ./containers tarred and gzipped.
converted.xml - the same as annotated.xml minus the container information added.
embedding.yaml - container and pnode relationships in YAML.
experiment.tcl - The NS file which generated the experiment.
ghosts - empty file
hosts - /etc/hosts which includes the container nodes.
hostsw.pickle - ??? (dp0\n. Looks like a pickle file (binary python marhshalling format), but does not "un-pickle" into any data. Hits EOF.
maverick_url - URLs to container images. Default location is http://scratch/benito/.... Used for qemu images, I think.
openvz_guest_url - URLs to openvz images. Default location /share/containers/images/...
partitioned.xml - Another XML file that is used in the partitioning process. See annotated.xml
phys_topo.ns, phys_topo.xml - physical NS file/topology/nodes of experiment.
pid_eid - group and experiment ID of experiment
pnode_types.yaml - The machine types of the pnodes, "pc2133, bpc2133, MicroCloud?", in YAML.
qemu_users.yaml - list of users for qemu nodes? Just contains [glawler] for this experiment, so it's not an exhaustive list...
route - routing info for containers. directory with one file per node, showing route info. e.g. 10.0.0.0/8 10.0.1.7.
shaping.yaml - traffic shaping for container networks.
site.conf - configuration settings for containers: example:
[containers] switch_shaping = true qemu_host_hw = pc2133,bpc2133,MicroCloud openvz_guest_url = %(exec_root)s/images/ubuntu-10.04-x86.tar.gz exec_root = /share/containers qemu_host_os = Ubuntu1204-64-STD openvz_host_os = CentOS6-64-openvz maverick_url = http://scratch/benito/pangolinbz.img.bz2 xmlrpc_server = boss.isi.deterlab.net:3069 url_base = http://www.isi.deterlab.net/ grandstand_port = 4919 backend_server = boss.isi.deterlab.net:6667 openvz_template_dir = %(exec_root)s/images/ attribute_prefix = containers switched_containers = qemu,process
software - empty directory
switch - directory of per switch node (pnodes) files. Each file describes vlan creation, numbering, and switch port/vlan assignment for containers on that node. Does this get fed to the {{{vde_switch}}es on the pnodes?
switch_extra.yaml - YAML file which looks like it contains information about gluing together the VDE switches - extra interfaces (taps), double pipes (attaching switches together I think), and tap pipes (for attaching tap ifaces to the switch)
swlocs.pickle - another python pickle file. But hits EOF before it is decoded. No idea what this is for.
topo.xml - Containers topology in YAML format. Has substrates (networks). For nodes it shows interfaces and addressing (ip/MAC) as well as which vde_switch the node is attached to. Nodes also contain some VM info like disk size, template/image path/name, vhost.
traffic_shaping.pickle - Looks like, oddly, traffic shaping information in the python pickle format:
>>> import pickle
>>> import pprint
>>> fd = open('traffic_shaping.pickle', 'rb')
>>> d = pickle.load(fd)
>>> pprint.pprint(d)
>>> pprint.pprint(d)
{u'animal': {u'10.0.0.1': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.2': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.3': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.4': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.5': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.6': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.8': {'bandwidth': None, 'delay': None, 'loss': None}},
...
u'rizzo': {u'10.0.0.1': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.2': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.3': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.5': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.6': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.7': {'bandwidth': None, 'delay': None, 'loss': None},
u'10.0.0.8': {'bandwidth': None, 'delay': None, 'loss': None}}}
visualization.png - image of the topology of the container networks.
wirefilters.yaml - empty YAML file: {}
