wiki:ConfigDB

Version 6 (modified by Geoff Lawler, 9 years ago) (diff)

--

Config Server API

The config server is a RESTful API. The server (by default) listens on port 5000. All messages are in JSON and have a top level status entry. If status is not zero, the only other entry in the message will be a message field, which contains an error message. Otherwise the message will contain the information requested.

  • URL: /this_is_a_bad_URL. Bad URLs return a non-zero status and error message.
    • Example: unknown URL:
      > curl http://localhost:5000/borkborkbork
      {
        "message": "404: Not Found",
        "status": 1
      }
      
    • Example: Bad request. A valid URL, but with bad "arguments".
      > curl http://localhost:5000/user/jjh
      {
        "message": "user jjh not found",
        "status": 1
      }
      
    • And another Invalid request.
      > curl http://localhost:5000/computer/bork
      {
        "message": "No computer 'bork' found.",
        "status": 1
      }
      
  • URL: /pnodes. Return names of all physical nodes in the experiment. Each entry contains a type, control_name, and exp_name fields.
    • Example
      > curl http://localhost:5000/pnodes
      {
        "pnodes": [
          {
            "control_name": "cpc9",
            "exp_name": "pnode-0000",
            "type": "pnode"
          },
          {
            "control_name": "cpc93",
            "exp_name": "pnode-0001",
            "type": "pnode"
          },
          {
            "control_name": "cpc90",
            "exp_name": "config",
            "type": "pnode"
          }
        ],
        "status": 0
      }
      
  • URL: /computer/<node_name> - return information about computer given. What the information is is dependent on the type of computer. Currently supported: pnode and qemu. Examples of both are given below:
    • QEMU:
      > curl http://localhost:5000/computer/glory
      {
        "computer": {
          "arch": "x86_64",
          "image_url": "http://scratch/benito/pangolinbz.img.bz2",
          "interfaces": [
            {
              "address": "10.0.1.3",
              "bcast": null,
              "capacity": {
                "kind": "max",
                "rate": 100000.0,
                "units": "Kb/s"
              },
              "mac": "00:00:00:00:00:07",
              "mask": "255.255.255.0",
              "name": "inf000",
              "tap": false,
              "tun": false
            },
            {
              "address": "172.16.100.130",
              "bcast": null,
              "capacity": null,
              "mac": "00:66:00:00:64:82",
              "mask": "255.240.0.0",
              "name": "control0",
              "tap": false,
              "tun": false
            }
          ],
          "name": "glory",
          "os": "Ubuntu1204-64-STD",
          "type": "qemu"
        },
        "status": 0
      }
      
    • Pnode:
      > curl http://localhost:5000/computer/pnode-0000
      {
        "computer": {
          "control_name": "cpc9",
          "exp_name": "pnode-0000",
          "type": "pnode"
        },
        "status": 0
      }
      
  • URL: /embed/<pnode_name>. Returns a list of containers ("embeds") that run on that pnode.
    • Example:
      > curl http://localhost:5000/embed/pnode-0000
      {
        "embeds": [
          "giles",
          "buffy",
          "xander",
          "angel"
        ],
        "status": 0
      }
      
  • URL: /user/<login>. Return the /etc/passwd type information for a user:email, gid, password, etc.
    • Example
      > curl http://localhost:5000/user/glawler
      {
        "email": "\"glawler@tislabs.com\"",
        "gid": 1,
        "home": "/users/glawler",
        "login": "glawler",
        "name": "\"Geoff Lawler\"",
        "password": "$1$25502626$SJgRY/qEUFJWWIB1rkFqW1",
        "root": 1,
        "shell": "bash",
        "status": 0,
        "uid": 10467,
        "user": "glawler"
      } 
      
  • URL: /vde/<node name>. Return information about VDE switches (and ports) that should exist on the given pnode.
    • Example:
      curl http://config:5000/vde/pnode-0000
      {
        "ports": [
          {
            "port": 1,
            "vlan": 1
          },
          {
            "port": 2,
            "vlan": 1
          }
        ],
        "status": 0,
        "switch": {
          "management": null,
          "socket": null,
          "tag": "hv"
        }
      }
      

Config DB Tables (Schema)

Current schema:

CREATE TABLE container_conf (                  
        id INTEGER NOT NULL,                   
        switch_shaping BOOLEAN,                
        qemu_host_hw VARCHAR(32),              
        openvz_guest_url VARCHAR(64),          
        exec_root VARCHAR(64),                 
        qemu_host_os VARCHAR(32),              
        openvz_host_os VARCHAR(32),            
        qemu_url VARCHAR(64),                  
        xmlrpc_server_host VARCHAR(32),        
        xmlrpc_server_port INTEGER,            
        url_base VARCHAR(64),                  
        grandstand_port INTEGER,               
        backend_server_host VARCHAR(64),       
        backend_server_port INTEGER,           
        openvz_template_dir VARCHAR(64),       
        attribute_prefix VARCHAR(32),          
        switched_containers VARCHAR(32),       
        topdl_converter VARCHAR(128),          
        default_dest VARCHAR(32),              
        default_router VARCHAR(32),            
        node_log VARCHAR(128),                 
        PRIMARY KEY (id),                      
        CHECK (switch_shaping IN (0, 1))       
);                                             
CREATE TABLE computer (                        
        id INTEGER NOT NULL,                   
        name VARCHAR(64),                      
        PRIMARY KEY (id)                       
);                                             
CREATE TABLE hosts (                           
        id INTEGER NOT NULL,                   
        address VARCHAR(50),                   
        name VARCHAR(32),                      
        PRIMARY KEY (id)                       
);                                             
CREATE TABLE groups (                          
        id INTEGER NOT NULL,                   
        name VARCHAR(32),                      
        gid INTEGER,                           
        PRIMARY KEY (id)                       
);                                             
CREATE TABLE mounts (                          
        id INTEGER NOT NULL,                   
        remote VARCHAR(128),                   
        local VARCHAR(128),                    
        type VARCHAR(32),                      
        options VARCHAR(64),                   
        PRIMARY KEY (id)                                      
);                                                            
CREATE TABLE capacity (                                       
        id INTEGER NOT NULL,                                  
        rate FLOAT,                                           
        units VARCHAR(8),                                     
        kind VARCHAR(8),                                      
        PRIMARY KEY (id)                                      
);                                                            
CREATE TABLE users (                                          
        id INTEGER NOT NULL,                                  
        login VARCHAR(64),                                    
        password VARCHAR(64),                                 
        uid INTEGER,                                          
        gid INTEGER,                                          
        root INTEGER,                                         
        name VARCHAR(64),                                     
        home VARCHAR(64),                                     
        email VARCHAR(64),                                    
        shell VARCHAR(64),                                    
        PRIMARY KEY (id),                                     
        FOREIGN KEY(gid) REFERENCES groups (id)               
);                                                            
CREATE TABLE switch_vde (                                     
        id INTEGER NOT NULL,                                  
        computer_id INTEGER,                                  
        socket VARCHAR(64),                                   
        management VARCHAR(64),                               
        tag VARCHAR(16),                                      
        PRIMARY KEY (id),                                     
        FOREIGN KEY(computer_id) REFERENCES computer (id)     
);                                                            
CREATE TABLE computer_embedded_pnode (                        
        id INTEGER NOT NULL,                                  
        computer_id INTEGER,                                  
        realname VARCHAR(64),                                 
        PRIMARY KEY (id),                                     
        FOREIGN KEY(computer_id) REFERENCES computer (id)     
);                                                            
CREATE TABLE computer_openvz (                                
        id INTEGER NOT NULL,                                  
        template VARCHAR(128),                                
        computer_id INTEGER,                                  
        PRIMARY KEY (id),                                     
        FOREIGN KEY(computer_id) REFERENCES computer (id)     
);                                                            
CREATE TABLE guest_host_embed (                               
        id INTEGER NOT NULL,                                  
        guest INTEGER,                                                      
        host INTEGER,                                                       
        PRIMARY KEY (id),                                                   
        FOREIGN KEY(guest) REFERENCES computer (id),                        
        FOREIGN KEY(host) REFERENCES computer (id)                          
);                                                                          
CREATE TABLE computer_linux_container (                                     
        id INTEGER NOT NULL,                                                
        computer_id INTEGER,                                                
        PRIMARY KEY (id),                                                   
        FOREIGN KEY(computer_id) REFERENCES computer (id)                   
);                                                                          
CREATE TABLE computer_qemu (                                                
        id INTEGER NOT NULL,                                                
        arch VARCHAR(12),                                                   
        image_url VARCHAR(128),                                             
        os VARCHAR(64),                                                     
        computer_id INTEGER,                                                
        PRIMARY KEY (id),                                                   
        FOREIGN KEY(computer_id) REFERENCES computer (id)                   
);                                                                          
CREATE TABLE computer_pnode (                                               
        id INTEGER NOT NULL,                                                
        computer_id INTEGER,                                                
        realname VARCHAR(64),                                               
        PRIMARY KEY (id),                                                   
        FOREIGN KEY(computer_id) REFERENCES computer (id)                   
);                                                                          
CREATE TABLE substrate (                                                    
        id INTEGER NOT NULL,                                                
        name VARCHAR(32),                                                   
        capacity_id INTEGER,                                                
        vlan INTEGER,                                                       
        multicast VARCHAR(15),                                              
        control BOOLEAN,                                                    
        PRIMARY KEY (id),                                                   
        UNIQUE (name),                                                      
        FOREIGN KEY(capacity_id) REFERENCES capacity (id),                  
        CHECK (control IN (0, 1))                                           
);                                                                          
CREATE TABLE switch_port (                                                  
        id INTEGER NOT NULL,                                                
        port INTEGER,                                                       
        vlan INTEGER,                                                       
        switch_id INTEGER,                                                  
        PRIMARY KEY (id),                                                   
        FOREIGN KEY(switch_id) REFERENCES switch_vde (id)                   
);                                                                          
CREATE TABLE interface (                                        
        id INTEGER NOT NULL,                                    
        address VARCHAR(50),                                    
        mask VARCHAR(15),                                       
        bcast VARCHAR(15),                                      
        name VARCHAR(16),                                       
        mac VARCHAR(17),                                        
        capacity_id INTEGER,                                    
        tap BOOLEAN,                                            
        tun BOOLEAN,                                            
        computer_id INTEGER,                                    
        switch_port_id INTEGER,                                 
        substrate_id INTEGER,                                   
        PRIMARY KEY (id),                                       
        UNIQUE (address),                                       
        UNIQUE (mac),                                           
        FOREIGN KEY(capacity_id) REFERENCES capacity (id),      
        CHECK (tap IN (0, 1)),                                  
        CHECK (tun IN (0, 1)),                                  
        FOREIGN KEY(computer_id) REFERENCES computer (id),      
        FOREIGN KEY(switch_port_id) REFERENCES switch_port (id),
        FOREIGN KEY(substrate_id) REFERENCES substrate (id)     
);                                                              
CREATE TABLE route (                                            
        id INTEGER NOT NULL,                                    
        default_route VARCHAR(18),                              
        default_dest VARCHAR(18),                               
        extra_route VARCHAR(18),                                
        interface_id INTEGER,                                   
        PRIMARY KEY (id),                                       
        FOREIGN KEY(interface_id) REFERENCES interface (id)     
);                                                              

Config request path

Given the API/URls the request sequence should go as follows:

Pnodes

  1. http://config:5000/computer/<name>. The pnode checks the "type" field and sees that it's a pnode. So it requests information about containers embedded in it next.
  2. http://config:5000/embed/pnode-0000. The pnode gets a list of nodes embedded in it.
  3. For each embedded node: http://config:5000/computer/<embedded node name> to get the container specific information (like path to qemu image) and embedded node interfaces so it can setup the virtual networking prior to launching the containers.

Nodes (containers)

  1. http://config:5000/computer/<name>. To get at least the hostname and maybe the networking information if the system used to spawn the containers does not set that.
  2. http://config:5000/moutns. Know what remote file systems to mount locally. Again, this may be taken care of by the vm spawn system (it can be when using Vagrant for instance.)
  3. http://config:5000/users. Get the account information to create the users.
  4. TBD/other cool stuff.