3.0 Starting a new system
3.1 Before you start
Before you start a new Legion system, you should consider what type of set-up will best suit your needs. Primary considerations are:
- What kind of system do you need? How many machines do you anticipate using? Do you only use local hosts or do you also use remote machines? Possible configurations might include:
- a. A single host system: one Legion machine with one or more host objects (Figure 1). This is the simplest system.
Figure 1: Single host system
- b. A multihost system: multiple Legion hosts linked together and sharing local resources (Figure 2). This can include homogeneous or heterogeneous platforms, as well as non-Legion machines. The machines do not need to be in physical proximity.
Figure 2: Multihost system
- c. A multidomain system: multiple Legion domains connected together and sharing each others' resources (Figure 3).
Figure 3: Multidomain system
- Will you be using Legion security? This is an important consideration, since a secure Legion system cannot be cleanly shut down and restarted. If you are using security, you should decide who will be the "admin," the system administrator. The system should be started up by the admin, since he or she will have special privileges on the core objects created in the new system. (These privileges do not extend to other users' objects, however.)
- What kind of host objects will you be using? Options include:
- a. Basic host object: it resides on its host and manages and guards its host's resources. This is the template for the other host objects.
See Host and vault objects in the Basic User manual for discussion of basic host objects. Section 11.0 in this manual discusses host-vault pairings and adding new hosts.
- b. PCD host object: it resides on its host, manages and guards its host's resources, and uses a process control daemon to regulate ownership of all Legion processes executed on that host. If you use a PCD host as your bootstrap host, the start-up process will be slightly different.
The daemon requires root privileges to start and to run. The PCD host object is useful if outside users will be running processes on your host, but can only be used if Legion security is enabled. Each user's processes will be tracked and accounted for. For more information, please see "Process control daemon host objects".
- c. Batch queue host object: it resides on its host, manages and guards its host's resources, and submits Legion jobs to the local queueing system.
This is the best choice for hosts that use a queue management system, although the PCD host object is more secure and has better accounting. For more information, please see "Batch queue host objects".
- d. Virtual host object: it resides on a different host, represents and guards its host's resources, but does not run normal Legion objects. A virtual host cannot be used as a bootstrap host: it is added to an already running system.
A virtual host object is used for running Legion jobs on unsupported platforms. The host object resides on a supported platform and runs native jobs with standard Legion tools on the target host machine. It can be used for scheduling, resource selection, and transparent execution on the target machine. For more information, please see "Virtual hosts".
- The LegionClass object (the root of the Legion binding mechanism and the parent class of many metaclasses) will be placed here.
- The $LEGION_OPR/LegionClass.config file will be created on this host and will contain the LegionClass object address, which must be globally known to all Legion objects. The file must be available when other objects are started in the system.
$ legion_setup_state Creating OPR directory /home/xxx/OPR/. Saving LegionClass configuration file: /home/xxx/OPR/LegionClass.config LegionClass host name = your.startup.host.name LegionClass port number = 7899 LegionClass timestamp = 898198093 $
$ legion_startup Starting meta-class object: LegionClass Continue (y=yes, Y=yes to all, n=no, N=no to all, v=verbose, V=verbose all)? Y Starting meta-class object: BootstrapMetaClass Starting class object: BindingAgentClass Starting class object: CommandLineClass Starting class object: UnixHostClass Starting class object: UnixVaultClass Starting class object: UnixImplementationClass Starting class object: UnixImplementationCacheClass Starting class object: ContextClass Legion first-time system startup complete Run "legion_initialize" $
Creating host object BootstrapHostObject on "your.current.host.name" Continue (y=yes, Y=yes to all, n=no, N=no to all, v=verbose, V=verbose all)? Y Configuring wellknown binaries for host "1.01.07.0100..."
Creating vault object BootstrapVaultObject on "your.bootstrap.host.name" Setting BootstrapHost and BootstrapVault restrictions Added 1 host(s) to vault's compatibility set Added 1 vault(s) to host's compatibility set
Creating an ImplementationCache Creating an implementation (ContextObject) for ContextClass Creating the root context object
Adding "BootstrapHost" to the hosts context Adding the alias "your.bootstrap.host.name" for BootstrapHost to the hosts context Adding "BootstrapVault" to the vaults context
Creating the context "/users" to contain user-objects Creating context "users" in parent "/". New context LOID = "1.3622260c.05.08000000.000001fc..."
Creating the initial system-admin user object, "/users/admin" Please select a Legion password for "/users/admin": New Legion password: xxxx Retype password: xxxx 1.3622260c.6b000000.01000000.000001fc0c... Please enter the Legion password for "/users/admin" to continue: Password: xxxx
Changing ownership of all objects to "/users/admin" 1.3622260c.01..000001fc0cbe1846763f895a... 1.3622260c.02..000001fc0b3b16eb8b2dde29... [...etc.] Changed ownership of 43 objects.
Creating initial ACLs files for all core objects in /home/xxxx/OPR/init_acls Creating ACL for /class/AuthenticationObjectClass class Creating ACL for /class/BasicFileClass class [...etc.]
Creating context "tmp" in parent "/". New context LOID = "1.362dabd6.05.09000000.000001fc..." [...etc.] Done creating acl files
$ legion_starthost -L <path_to_Legion> \ -O <path_to_OPR> -U <user> -A <arch> <new_host_name> \ /vaults/vault-<new_host_name>
$ legion_starthost -L <path_to_Legion> \ -O <path_to_OPR> -U <user> -A <arch> -B PCDUnixHost \ <new_host_name> /vaults/vault-<new_host_name>
$ legion_starthost -L <path_to_Legion> \ -O <path_to_OPR> -U <user> -A <arch> \ -B BatchQueueHost <new_host_name> \ /vaults/vault-<new_host_name> $ legion_update_attributes /hosts/<new_host_name> \ -a "host_queue_type(`<Queue_type>')"
$ source <path_to_globally_visable_setup_script>/setup.[sh|csh] $ legion_login /users/admin [run legion_login only if you have enabled Legion security] $ legion_init_arch
$ source <path_to_globally_visible_setup_script>/setup.[sh|csh]
$ source <path_to_globally_visible_setup_script>/setup.[sh|csh] $ legion_login /users/<user_name> $ legion_cd /home/<user_name>
3.2 Set up the environment
A properly set-up environment is crucial for working in the Legion system. The start-up process uses certain Legion-specific environment variables, which must be correctly set before starting applications and running command-line utility programs. You must set these variables each time you starting working in Legion. Without a properly set environment, programs cannot communicate with other objects in the system, and the program may terminate with an error, never return a value, or fail in a more spectacular fashion. If this occurs, try setting your environment properly and starting over.
You must have /bin/ksh installed in your system. There are a number of Legion scripts that will look for ksh, and if it is not installed in your system you will get error messages.
To prepare the environment in a newly started or restarted system, set the $LEGION and $LEGION_OPR variables and source the script that is appropriate for your shell.1
The environment must be properly set in each shell in which you plan to run Legion commands. Check to be sure that environment variables are properly set (we suggest $LEGION/../OPR for the OPR root directory path).
If necessary, run the following.
3.3 Starting a single host system
If you have downloaded the source code, you must compile the system before you can start it (see section 2.5).
There are three steps in this procedure: configure the system with legion_setup_state, start it with legion_startup, and, finally, initialize it with legion_initialize.
3.3.1 Choose a bootstrap host
You must first choose a bootstrap host. Start and shut down your system from here.2 The bootstrap host will have the following features:
If you choose a PCD host as your bootstrap, the start-up procedure is slightly different than for a basic host object.
3.3.2 Set up and configure the system
You must first set up the initial state for core Legion system objects. Legion system objects are persistent, and can save and restore their own state. Some of these objects must have their state initialized before they run for the first time. After the initial start-up, these objects will manage their own state and configuration, if the system is properly maintained.
Enter the legion_setup_state command to configure the system:
This program will return your start-up host name, a port number for the LegionClass object, and a time. If you do not want to use the default settings, use the -i flag to run the command in an interactive mode. Your output will look something like this:
The script creates the $LEGION_OPR directory and several sub-directories, populating them with initial states for several core system objects. The timestamp sets the starting time for the system: Legion objects use a timestamp to guarantee each object's unique identity. The current time is measured in seconds since January 1, 1970.
3.3.3 Start the system
The legion_startup script provides prompts asking whether or not to start each component. It's best to answer "yes to all" (Y). The verbose option allows you to see more detailed information, as the script works, about debugging. (This can be large amounts of information, so use this option only if you are searching for a problem.)
To start the main core system objects, enter the legion_startup command.
Legion will start several classes on your host. Your output will look something like this
This indicates that the major class objects were started: LegionClass is the top-level metaclass and is the parent of every other object in the system. BootstrapMetaClass is the class object for the bootstrap class objects, which are those class objects that have instances that need to get created in the initialization phase; those class objects are UnixVaultClass, UnixHostClass, UnixImplementationClass, and UnixImplementationCacheClass.
The new class objects will start instances in the new system: BindingAgentClass parents binding objects, CommandLine- Class parents command-line objects, etc.
3.3.4 Initialize the system
Like legion_startup, the legion_initialize script will provide prompts asking whether or not to perform each task, and it is generally best to use the "yes to all" option (Y). To initialize Legion, enter
The output shows the system creating and tagging the key ingredients of a new system. It is too long to reproduce in full here, but we'll look at some selected actions.
The first line shows the system creating a bootstrap host object on your current host (a host object manages a host, so the bootstrap host object manages the bootstrap host).
A bootstrap vault object is automatically created on your current host (a vault object manages a vault, which stores Legion object's permanent states). This guarantees that the bootstrap host object has a compatible vault object. All host objects must be paired with at least one compatible vault object (i.e., a vault that it can "see" -- see "About host-vault pairs").
Implementation objects represent and manage the implementation cache (used to allow Legion processes to take place in different architectures) and context space.
More implementation objects are created as the process creates new object classes.
Two context names are added to the /hosts context (BootstrapHost and your.bootstrap.host.name): both names refer to the Bootstrap host object (see Figure 4) but only one is added to the /vaults context.
Figure 4: Context paths for the bootstrap host and vault objects  |
This object will manage a portion of the persistent storage mechanism for the Bootstrap host (please see section 7.0 in the Basic User Manual for a discussion of Legion hosts and vaults). The /impls context contains names of the default implementation objects (see Implementation model). These can all be viewed with context-related commands or the GUI once the system has been completely started.
3.3.5 Set the environment
Source the script that sets up environment variables for using the context space.
You must source this script in each shell in which you plan to run Legion commands, including the shell in which you ran legion_initialize. Otherwise you will not be able to run any Legion utilities or commands.
3.3.6 Choose your security setting
If you wish enable Legion security, run the legion_init_security command. You'll have to decide now whether or not you want security, since the command will not run properly unless you run it immediately after initializing the system. If you don't wish to use it, just skip over this section. However, none of your processes will be protected.
You will be given a system administrator user id called admin. The administrator has root privileges in the system and can create new users, modify security settings, etc. He/she also has ownership of all currently existing objects in the new system but not any other objects (i.e., any objects that other users create).
Several events take place when you run this command, and we will discuss each in turn.
The /users context holds Legion user ids, starting with admin:
This admin is the system administrator's user id: anyone logging in as admin has root privileges. You will have to enter the password three times during the legion_init_security process and when logging in.
Legion assigns ownership of all existing objects to admin. Objects created by other users will belong to those users.
The access control lists (ACLs) protect objects against unauthorized use. Only an object's creator can use the object, unless the creator specifies otherwise. The initial ACL files allow only the admin to use the core objects.
Several new context objects are created and placed in the root context and given ACLs.
ACL files are then created for all previously existing objects.
At this point, security has been enabled and is running. You must now log in as admin. Note that you need to include the /users path.
You are now ready to start working. For more information, see "About Legion security" and "Using security features".
3.4 Starting a multihost system
This is a two-part process. First, you have to have a running single host system, as laid out in section 3.3. Second, you add new host objects on the desired machines. Since you will be starting processes on the target hosts from the bootstrap host be sure that you can run rsh/ssh on the bootstrap host as well as on the target hosts from the bootstrap host without having to enter a password. You can set up a .rhosts file for rsh or an authorized_keys files for ssh to accomplish this (see the rsh and ssh man pages for more information).
3.4.1 Set up
You'll need to set the proper environment variable on the bootstrap host and the remote host(s) so that you can run Legion commands on a remote host using rsh or ssh.
Set these variables on the bootstrap host before you start the new system (i.e., before you run legion_startup). Please note that you only need to follow these steps on the bootstrap host: you will need to install the Legion binaries on any other machines that you add to your system, but you do not need to start more Legion systems.
To add additional hosts and users, copy the $LEGION_OPR/setup.[sh|csh] scripts to a globally accessible location. Hosts can share an NFS-mounted Legion tree, but for best results you should place the OPRs on a local disk.
3.4.2 Create a new vault
If the new host will not be compatible with your existing vaults, create a new vault object with the legion_startvault command.
See section 11.4 for more information on new vaults.
3.4.3 Create a new host
Use the legion_starthost command to create a new host object on the desired host. (See section 11.3 for more information on new hosts.)
The same command, with the -B flag, will start a new PCD host object. (See section 12.1 for more information on adding PCD hosts.)
The same command, with the -B flag, will start a new batch queue host object. You'll need to update the host object's attributes to include the queue type. (See section 13.1 for more information on adding batch queue hosts).
Adding a virtual host is more complicated: see section 14.0.
If the new host has a new architecture, you now need to add implementations of the core objects for the new architecture. Log in to the new machine and run the following:
Repeat these steps for each additional host. We suggest that you customize these steps and write a script to simplify the process, especially if you need to bring up a big net.
3.4.4 Adding new users to a secure net
To add users to a secure net, run:
$ legion_create_user <new_user_name>
If you are working on a PCD host, follow up with these steps: |
You will need to wait an hour for new user account(s) to be available.
3.4.5 Working in the new net
Users can work in an insecure net by entering:
User can work in a secure net by entering:
Please note that neither of these procedures will open a separate shell.
3.5 Making a set-up script for users
We suggest that Legion system administrators prepare a set-up script for users to source when starting work in Legion. You can run the legion_make_setup_script command to automatically generate a set-up script for your system. This script sets the environment variables for Legion users. The usage is:
legion_make_setup_script
[-o <script basename>]
[-OPR <OPR dir name>]
[-L <$LEGION dir name>]
[-debug] [-help]
1.Bourne Shell is not directly supported by our implementation of Legion, due to the use of alias to implement some Legion commands (notably legion_set_context). Bash, however, is supported.
2. Once fully operational, Legion does not automatically shut down and restart: the system is intended to stay up.
[Home] [General] [Documentation] [Software] [Testbeds] [Et Cetera] [Map/Search]
|
legion@Virginia.edu
http://legion.virginia.edu/