Changes between Initial Version and Version 1 of PortingNotes


Ignore:
Timestamp:
Aug 29, 2013 10:12:04 AM (11 years ago)
Author:
Ted Faber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PortingNotes

    v1 v1  
     1[[TOC]]
     2
     3= Porting Notes =
     4
     5I recently (27-28 Aug 2013) ported the containers system to the EARS testbed.  This is a summary of what I did to get containers running.
     6
     7== Codebase ==
     8
     9This was largely straightforward.  I cloned from the git repository into /share/containers.  Files not in there are those that need compiled, the openvz images in /share/containers/images and the qemu image stored on scratch.
     10
     11Codebase install is completed by running make in /share/containers.  This points out that fedd needs to be installed.  After this install I made changes to the system (the topdl_converter [ReferenceGuide#SiteConfigurationFile site variable]) one can get away with just installing the deter package from fedd.
     12
     13Installing fedd also requires installing the {{{devel/py-asn1}}} and {{{math/metis4}}} packages - that's even if only the deter data handling package is installed.  The full fedd install also pulls in {{{net/py-zsi}}} port which includes {{{textproc/py-xml}}} as a dependency.
     14
     15Fedd also requires [http://abac.deterlab/net libabac], of course.  That requires {{{security/xmlsec1}}} to generate signed XML.
     16
     17There are ports in the testbed repo to install these, but I did them by hand for no good reason.
     18
     19== Images and Templates ==
     20
     21Openvz templates are comparatively small (less than half a GB) so nodes copy them directly from NFS.  They go into /share/containers/images .
     22
     23The qemu image is nearly a full GB and retrieved from a web server over TCP.  On DETERlab, that server is scratch.  On EARS I used boss.ears.deterlab.net.  I created a directory /usr/testbed/www/containers/ and put the image (pangolinbz.img.bz2) into it.
     24
     25We also needed the testbed image for running CentOS6 for containers.  That image was incorrectly configured to run in the new EARS event system model and needed to be upgraded to the new event system.  To do that, swap the original image in and:
     26
     27 * download pubsub sources from https://www.isi.deterlab.net/downloads/pubsub-0.9.9.zip
     28 * compile and install it (on the experiment node)
     29   * configure
     30   * make
     31   * sudo make install-client
     32 * pull the deter repo onto the node (including an appropriate defs file for it).
     33 * make an object dir to work in (mkdir obj)
     34 * from obj (on the experiment node)
     35   * ${DETER_REPO_PATH}/configure --with-TBDEFS=${DETER_REPO_PATH}/${DEFS_FILE}
     36   * make client
     37   * sudo make client-install
     38 * Image it over the original image.
     39
     40At this point experiments will swap in, but the containerized nodes point at DETER's standard software repos.  If you've made a DETER install with mirrors on a machine called scratch, you're done.
     41
     42== Modifying Container Images ==
     43
     44Modifying the openvz images is the easier:
     45
     46{{{
     47$ cd /share/containers/images
     48$ mkdir edit
     49$ tar -C edit -xzvf ${TEMPLATE}.tar.gz
     50# make changes to the tree in edit (this is a comment, not a root shell)
     51$ tar -C edit -czvf ${TEMPLATE} .
     52}}}
     53
     54(That's a dot at the end of the second tar command).
     55
     56To modify the qemu image.  Swap in a containerized qemu experiment.  This is to get a machine with the qemu tools on it.  You can acquire such a machine any way you want, but swapping in does it directly.
     57
     58Log into that experiment node, and pick a local directory on a filesystem with a few gigs of space.  I used /tmp.
     59
     60Now on that machine:
     61
     62{{{
     63$ wget http://boss.ears.deterlab.net/containers/pangolinbz.img.bz2
     64$ bunzip2 pangolinbz.img.bz2
     65$ qemu-nbd -P 1 -p 1024 ./pangolinbz.img &
     66$ nbd-client localhost 1024 /dev/nbd0
     67$ mount -o noatime /dev/nbd0 /mnt
     68# make changes to filesystem in /mnt (this is a comment, not a root shell)
     69$ umount /mnt
     70$ nbd-client -d /dev/nbd0
     71# that should kill the qemu-nbd process.  If not you can kill it by hand)
     72$ bzip2 ./pangolinbz.img
     73# ship ./pangolinbz.img where it needs to go
     74}}}
     75
     76
     77=== Specific EARS changes to containers ===
     78
     79The specific changes for EARS were to change /etc/environment to include
     80
     81{{{
     82http_proxy="http://192.168.253.1:8888"
     83}}}
     84
     85And to put standard sources.list for the various ubuntu distributions in /etc/apt
     86
     87For the 10.04 based distros, add a file called /etc/apt/apt.conf.d/proxy that contains:
     88
     89{{{
     90Acquire::http::Proxy "http://192.168.253.1:8888";
     91}}}
     92
     93Note the trailing semi-colon.
     94
     95On the qemu image, keys for the TOR repo has to be added to /etc/apt/trustdb.gpg and /etc/apt/trusted.gpg
     96
     97Those detail will vary from install to install.
     98
     99
     100