Changes between Version 7 and Version 8 of ReferenceGuide


Ignore:
Timestamp:
Oct 18, 2012 4:03:28 PM (7 years ago)
Author:
Ted Faber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • ReferenceGuide

    v7 v8  
    55This document describes the details of the commands and data structures that make up the containers system.  The [UsersGuide User Guide /Tutotial] provides useful context about the workflows and goals of the system that inform these technical details.
    66
    7 == containerize.py ==
     7== Commands ==
     8
     9This describes the command line interface to the containers system.
     10
     11=== containerize.py ===
    812
    913The {{{containerize.py}}} command creates a DETER experiment made up of containers.  The {{{containerize.py}}} program is available from {{{/share/containers/containerize.py}}} on {{{users.isi.deterlab.net}}}.  A sample invocation is:
     
    7175More detailed examples are available in [UsersGuide the tutorial]
    7276
    73 == container_image.py ==
     77=== container_image.py ===
    7478
    7579The {{{container_image.py}}} command draws a picture of the topology of an experiment.  This is helpful in keeping track of how virtual nodes are connected.  {{{containerize.py}}} calls this internally and stores the output in the [ReferenceGuide#Per-ExperimentDirectory per-experiment directory] (unless {{{--no-image}}} is used.
     
    106110}}}
    107111
    108 
    109 == Per-experiment Directory ==
    110 
    111 == Site Configuration File ==
     112== Configuration Files ==
     113
     114=== Per-experiment Directory ===
     115
     116When an experiment is containerized, the data necessary to create it is stored in {{{/proj/}}''project''{{{/exp/}}}''experiment''{{{containers}}}.  The
     117{{{/proj/}}''project''{{{/exp/}}}''experiment'' is created by DETER when the experiment is created, and used by experimenters for a variety of things.
     118
     119There are a few files in the per-experiment directory that most experimenters can use:
     120
     121  {{{experiment.tcl}}}::
     122   If the topology was passed to [UsersGuide#containerize.py containerize.py] as an ns file, this is a copy of that input file.  Useful for seeing what the experimenter asked for, or as a basis for new experiments.
     123  {{{experiment.xml}}}::
     124   The analogue of {{{experiment.tcl}}} is the topology was given as [http://fedd.deterlab.net/wiki/TopDl topdl].  The topdl input file.
     125  {{{visualization.png}}}::
     126   A drawing of the virtual topology in png format.  Generated by [ReferenceGuide#container_image.py container_image.py]
     127  {{{hosts}}}::
     128   The host to IP mapping that will be installed on each node as {{{/etc/hosts}}}.
     129  {{{site.conf}}}:
     130   A clone of the [ReferenceGuide#SiteConfigurationFile site configuration file] that holds the global variables that the container creation will use.  Values overridden on the command line invocation of [ReferenceGuide#containerize.py containerize.py] will be present in this file.
     131
     132The rest of this directory is primarily of interest to developers.  It includes:
     133
     134 {{{annotated.xml}}}::
     135   First version of the input topology after default container types have been added.  Input to the partitioning step.
     136 {{{assignment}}}::
     137   A yaml representation of the partition to virtual node mapping.
     138 {{{backend_config.yaml}}}::
     139   The server and channel to use for [ReferenceGuide#Grandstand grandstand] communication.  Encoded in yaml.
     140 {{{children}}}::
     141   Directory containing the assignment, including all the levels of nested hypervisors.
     142 {{{config.tgz}}}::
     143   The contents of the per-experiment directory (except {{{config.tgz}}}) for distribution into the experiment.
     144 {{{embedding.yaml}}}::
     145   A yaml-encoded representation of the children sub-directory
     146 {{{ghosts}}}::
     147   Containers that are initially not started in the experiment.
     148 {{{maverick_url}}}:
     149   Yaml encoding of the qemu images to be used on each node.
     150 {{{openvz_guest_url}}}::
     151   Yaml encoding of the openvz templates to be used on each node.
     152 {{{partitioned.xml}}}::
     153   Output of the partitioning process.  A copy of {{{annotated.xml}}} that has been decorated with the partitions.
     154 {{{phys_topo.ns}}}::
     155   The ns2 file used to create the DETER experiment.
     156 {{{phys_topo.xml}}}::
     157   The topdl file used to generate {{{phystopo.ns}}}.
     158 {{{pid_eid}}}::
     159   The DETER project and experiment name under which this topology will be created.  Broken out into {{{/var/containers/pid}}} and {{{/var/containers/eid}}} on virtual nodes inside the topology.
     160 {{{route}}}::
     161   A directory containing the routing tables for each node
     162 {{{shaping.yaml}}}::
     163   Yaml-encoded data about the per-network and per-node loss, delay, and capacity parameters.
     164 {{{switch}}}::
     165   A directory containing the [http://wiki.virtualsquare.org/wiki/index.php/VDE VDE switch] topology for the experiment.
     166 {{{switch_extra.yaml}}}::
     167   Yaml-encoded extra switch configuration information.  Mostly VDE switch configuration esoterica.
     168 {{{topo.xml}}}::
     169   The final topology representation from which the physical topology is extracted.  Includes the virtual topology as well.  This file can be used as input to [ReferenceGuide#container_image.py container_image.py].
     170 {{{traffic_shaping.pickle}}}:
     171   [http://docs.python.org/library/pickle.html Pickled] information for configuring endnode traffic shaping.
     172 {{{wirefilters.yaml}}}::
     173   Specific parameters for configuring the delay elements in VDE switched topologies that implement traffic shaping.  [ReferenceGuide#Interconnections:VDEswitchesandlocalnetworking See below].
     174
     175=== Site Configuration File ===
     176
     177The site configuration file controls how all experiments are containerized across DETER.  The contents are primarily of interest to developers, but researchers may occasionally find the need to specify their own.  The {{{--config}}} parameter to [ReferenceGuide#containerize.py containerize.py] does that.
    112178
    113179The site configuration file is an attribute value pair file parsed by a [file:///usr/local/share/doc/python2.7/library/configparser.html python ConfigParser] that sets overall container parameters.  Many of these have legacy internal names.