Changes between Version 26 and Version 27 of NeoContainers


Ignore:
Timestamp:
Oct 13, 2015 12:27:15 PM (9 years ago)
Author:
Geoff Lawler
Comment:

formatting.

Legend:

Unmodified
Added
Removed
Modified

  • NeoContainers

    v26 v27  
    1414users: > git clone https://github.com/deter-project/config_server.git
    1515}}}
     161. Create an experiment in which to run your containers. There are two modes: using the existing containers system or not using it. Neo-continainers uses the existing containers system to figure out more complex network topologies. If you just want containers "hanging off" your physical nodes and can compute IP addresses for your containers by hand, you do not need to use the existing containers system at all. Use 2a for an existing containers experiment. Use 2b for a standard NS-file based DETER experiment.
     17    a. Create an experiment using the existing containers system. Creat a containerized experiment with an NS file and the {{{/share/containers/containerize.py}}} script.
    1618
    17 2. Create an experiment in which to run your containers. There are two modes: using the existing containers system or not using it. Neo-continainers uses the existing containers system to figure out more complex network topologies. If you just want containers "hanging off" your physical nodes and can compute IP addresses for your containers by hand, you do not need to use the existing containers system at all. Use 2a for an existing containers experiment. Use 2b for a standard NS-file based DETER experiment.
    18 
    19 2a. Create an experiment using the existing containers system. Creat a containerized experiment with an NS file and the {{{/share/containers/containerize.py}}} script.
    20 
    21 In your NS file for each container in the experiment, specify  {{{image_os}}}, {{{image_type}}}, {{{image_name}}}, and {{{image_url}}} via the {{{tb-add-node-attribute}}} syntax. Details on each attribute is given below.
    22 
    23 * {{{image_os}}} - This is really just to distinguish Windows from non-Windows nodes. If the {{{image_os}}} starts with "{{{windows}}}", the image will be treated as a Windows node. Otherwise it'll be assumed to be some sort of Unix-y container.
    24 * {{{image_type}}} - This setting describes the containerization tech of the node. Currently this is *always* set to "{{{vagrant}}}" as Vagrant is the only package used to spin up the containers.
    25 * {{{image_name}}} - The name of the image. Any containers that share a name will also share an image.
    26 * {{{image_url}}} - A URL must be specified which the neo-containers system uses to download the container image. This URL must be resolvable from the experiment nodes. The image will only be downloaded once as long as the {{{image_name}}}s are the same for each container. Existing and supported images are Ubuntu 14.04 64 @ {{{http://scratch/containers/deter_ub1404_64_vb.box}}} and Windows 7 @ {{{http://scratch/containers/deter_win7.box}}}.
    27 
    28 Here is an example that creates Windows and Ubuntu 14.04 containers:
     19        In your NS file for each container in the experiment, specify  {{{image_os}}}, {{{image_type}}}, {{{image_name}}}, and {{{image_url}}} via the {{{tb-add-node-attribute}}} syntax. Details on each attribute is given below.
     20            * {{{image_os}}} - This is really just to distinguish Windows from non-Windows nodes. If the {{{image_os}}} starts with "{{{windows}}}", the image will be treated as a Windows node. Otherwise it'll be assumed to be some sort of Unix-y container.
     21            * {{{image_type}}} - This setting describes the containerization tech of the node. Currently this is *always* set to "{{{vagrant}}}" as Vagrant is the only package used to spin up the containers.
     22            * {{{image_name}}} - The name of the image. Any containers that share a name will also share an image.
     23            * {{{image_url}}} - A URL must be specified which the neo-containers system uses to download the container image. This URL must be resolvable from the experiment nodes. The image will only be downloaded once as long as the {{{image_name}}}s are the same for each container. Existing and supported images are Ubuntu 14.04 64 @ {{{http://scratch/containers/deter_ub1404_64_vb.box}}} and Windows 7 @ {{{http://scratch/containers/deter_win7.box}}}.
     24        Here is an example that creates Windows and Ubuntu 14.04 containers:
    2925{{{
    3026set r2d2 [$ns node]
    … …  
    4137}}}
    4238
    43 2b. Create an experiment without using the existing containers system. Just create an NS file with a fully connected network. Use the PNODE-BASE image for all machines on which you want to run containers.
    44 
    45 Create a JSON file which describes your containers. It's a list of containers. For each container you must specify the {{{host}}} (machine it runs on), {{{interfaces}}} in addition to the parameters from 2a above.
    46 
     39    b. Create an experiment without using the existing containers system. Just create an NS file with a fully connected network. Use the PNODE-BASE image for all machines on which you want to run containers. Create a JSON file which describes your containers. It's a list of containers. For each container you must specify the {{{host}}} (machine it runs on), {{{interfaces}}} in addition to the parameters from 2a above.
    4740{{{
    4841[
    … …  
    9386]
    9487}}}
    95 
    96 This example creates four containers on two host nodes.
    97 
    98 If using this mode, skip steps 3 and 4.
     88    This example creates four containers on two host nodes.[[BR]] If using this mode, skip steps 3 and 4.
    9989
    100903. Use the NS file to create a containerized experiment using the existing containers scripts (on users): {{{/share/containers/containerize.py [group] [experiment] [ns file]}}}. Note that the experiment must currently be created in the {{{Deter}}} group as that's where the custom pnode disk images are. This will change.
    101 
    102914. Modify the NS file generated by {{{containerize.py}}} to have a new image for the pnode machines. Navigate to the new experiment page and click {{{Modify Experiment}}}. Change the OS type of the pnodes to {{{PNODE_BASE}}} and the hardware type to {{{MicroCloud}}}. I.e. for each pnode in the NS file, make the lines have the form:
    10392{{{
    … …  
    10594    tb-set-hardware ${pnode(0000)} MicroCloud
    10695}}}
    107 Remove all existing {{{tb-set-node-startcmd}}} lines as these start the old containers system. This is no longer used.
     96    Remove all existing {{{tb-set-node-startcmd}}} lines as these start the old containers system. This is no longer used.
    10897
    109 The final NS file will look something like this.
    110 
     98    The final NS file will look something like this.
    11199{{{
    112100set ns [new Simulator]
    … …  
    1251135. If your experiment does not rely on the existing containers system, you need to tell DETER about your containers before swapping in the experiment so that it can allocate control network addresses for your containers. (The existing containers system does this for you, but since you've not run {{{containerize.py}}}, you must do this yourself.)
    126114
    127 On users (or anywhere that can talk to {{{chef.isi.deterlab.net}}}:
     115    On users (or anywhere that can talk to {{{chef.isi.deterlab.net}}}:
    128116{{{
    129117> cd [your config_server repository]/bin
    … …  
    131119}}}
    132120
    133 Note that you only have to do this once per experiment. You do not have to do this before each swap in! Just once to reserve control net addresses from DETER.
     121    Note that you only have to do this once per experiment. You do not have to do this before each swap in! Just once to reserve control net addresses from DETER.
    134122
    135123
    … …  
    1381267. Populate the configuration database that runs on {{{chef.isi.deterlab.net}}} by running the database population scripts {{{load_containers_db.sh}}} and {{{load_config_db.sh}}} (This will automated in the future.) This should be run from a physical node in the experiment. I use {{{pnode-0000}}} in the example below.
    139127
    140 On a single pnode:
     128    On a single pnode:
    141129{{{
    142130> ssh pnode-0000.${EXPID}.${PROJID}
    … …  
    145133}}}
    146134
    147 If you are using the existing containers system also load the containers information.
    148 
     135    If you are using the existing containers system also load the containers information.
    149136{{{
    150137> ./load_containers_db.sh -p ${PROJID} -e ${EXPID}
    151138}}}
    152139
    153 At this point, the Chef server and configuration database knows everything it needs to about your experiment and the nodes within it.
     140    At this point, the Chef server and configuration database knows everything it needs to about your experiment and the nodes within it.
    154141
    1551428. Let Chef configure the nodes. Bootstrap and configure the pnodes. To configure/bootstrap the node use the {{{bootstrap_node.sh}}} script. The script needs to know which role the node plays in the experiment. There are currently three roles: {{{pnode}}}, {{{container}}}, and {{{win-container}}}.
    156143
    157 On all the pnodes:
     144    On all the pnodes:
    158145{{{
    159146> ssh pnode-0000.${EXPID}.${PROJID}
    … …  
    161148> ./bootstrap_node.sh -r pnode
    162149}}}
     150    The {{{pnode}}} role will spawn the containers and configure them.
    163151
    164 The {{{pnode}}} role will spawn the containers and configure them.
    165 
    166 Once nodes are bootstrapped, simply running {{{sudo chef-client}}} will re-configure the nodes (both pnodes and the containers) if something should go wrong.
     152    Once nodes are bootstrapped, simply running {{{sudo chef-client}}} will re-configure the nodes (both pnodes and the containers) if something should go wrong.
    167153
    1681549. Remove experiment data from the configuration database once the experiment is complete.
    169155
    170 On a machine that can talk to {{{chef.isi.deterlab.net}}}:
     156    On a machine that can talk to {{{chef.isi.deterlab.net}}}:
    171157{{{
    172158> cd [your config_server repository]/bin
    … …  
    174160}}}
    175161
    176 An alternate way to do this is just to make a call on the {{{config_server}}} directly:
    177 
     162    An alternate way to do this is just to make a call on the {{{config_server}}} directly:
    178163{{{
    179164curl http://chef:5320/exp/${PROJID}/${EXPID}/delete