3.0 Starting a new system
3.1 Before you start
Before you start a new Legion system, consider what type of set-up will best suit your needs. Primary considerations include:
- What kind of system do you need? How many machines do you anticipate using? Do you only use local hosts or do you also use remote machines? Possible configurations might include:
- a. A single host system: one Legion machine with one or more host objects (Figure 1). This is the simplest system.
- b. A multihost system: multiple Legion hosts linked together and sharing local resources (Figure 2). This can include homogeneous or heterogeneous platforms, as well as non-Legion machines. The machines do not need to be in physical proximity.
- c. A multidomain system: multiple Legion domains connected together and sharing each others' resources (Figure 3).
Figure 3: Multidomain system
- Will you be using Legion security? This is an important consideration, since a secure Legion system cannot be cleanly shut down and restarted. If you are using security, you should decide who will be the "admin," the system administrator. The system should be started up by the admin, since he or she will have special privileges on the core objects created in the new system. (These privileges do not extend to other users' objects, however.)
- What kind of host objects will you be using? Options include:
- a. Basic host object: it resides on its host and manages and guards its host's resources. This is the template for the other host objects.
See Host and vault objects in the Basic User manual for discussion of basic host objects. In this manual, see section 11.0 for a discussion of host-vault pairings and adding new hosts.
- b. PCD host object: it resides on its host, manages and guards its host's resources, and uses a process control daemon to regulate ownership of all Legion processes executed on that host. If you use a PCD host as your bootstrap host, the start-up process will be slightly different.
The daemon requires root privileges to start and to run. The PCD host object is useful if outside users will be running processes on your host, but can only be used if Legion security is enabled. Each user's processes will be tracked and accounted for. For more information, please see "Process control daemon host objects".
- c. Batch queue host object: it resides on its host, manages and guards its host's resources, and submits Legion jobs to the local queueing system.
This is the best choice for hosts that use a queue management system, although the PCD host object is more secure and has better accounting. For more information, please see "Batch queue host objects".
- d. Virtual host object: it resides on a different host, represents and guards its host's resources, but does not run normal Legion objects. A virtual host cannot be used as a bootstrap host: it is added to an already running system.
A virtual host object is used for running Legion jobs on unsupported platforms. The host object resides on a supported platform and runs native jobs with standard Legion tools on the target host machine. It can be used for scheduling, resource selection, and transparent execution on the target machine. For more information, please see "Virtual hosts".
Figure 2: Multihost system
|
3.2 Set up the environment
A properly set-up environment is crucial for working in the Legion system. The start-up process uses certain Legion-specific environment variables, which must be correctly set before starting applications and running command-line utility programs. You must set these variables each time you starting working in Legion. Without a properly set environment, programs cannot communicate with other objects in the system, and the program may terminate with an error, never return a value, or fail in a more spectacular fashion. If this occurs, try setting your environment properly and starting over.
You must have /bin/ksh installed in your system. There are a number of Legion scripts that will look for ksh, and if it is not installed in your system you will get error messages.
To prepare the environment in a newly started or restarted system, set the $LEGION and $LEGION_OPR variables and source the script that is appropriate for your shell.1
The environment must be properly set in each shell in which you plan to run Legion commands. Check to be sure that environment variables are properly set (we suggest $LEGION/../OPR for the OPR root directory path).
If necessary, run the following.
3.3 Starting a single host system
There are three steps in this procedure: configure the system with legion_setup_state, start it with legion_startup, and, finally, initialize it with legion_initialize.
3.3.1 Choose a bootstrap host
You must first choose a bootstrap host. Start and shut down your system from here.2 The bootstrap host will have the following features:
- The LegionClass object (the root of the Legion binding mechanism and the parent class of many metaclasses) will be placed here.
- The $LEGION_OPR/LegionClass.config file will be created on this host and will contain the LegionClass object address, which must be globally known to all Legion objects. The file must be available when other objects are started in the system.
If you choose a PCD host as your bootstrap, the start-up procedure is slightly different than for a basic host object.
3.3.2 Set up and configure the system
You must first set up the initial state for core Legion system objects. Legion system objects are persistent, and can save and restore their own state. Some of these objects must have their state initialized before they run for the first time. After the initial start-up, these objects will manage their own state and configuration, if the system is properly maintained.
Enter the legion_setup_state command to configure the system. This program will return your start-up host name, a port number for the LegionClass object, and a time. If you do not want to use the default settings, use the -i flag to run the command in an interactive mode. Your output will look something like this:
$ legion_setup_state Creating OPR directory /home/xxx/OPR/. Saving LegionClass configuration file: /home/xxx/OPR/LegionClass.config LegionClass host name = your.startup.host.name LegionClass port number = 7899 LegionClass timestamp = 898198093 $
The script creates the $LEGION_OPR directory and several sub-directories, populating them with initial states for several core system objects. The timestamp sets the starting time for the system: Legion objects use a timestamp to guarantee each object's unique identity. The current time is measured in seconds since January 1, 1970.
3.3.3 Start the system
The legion_startup script provides prompts asking whether or not to start each component. It's best to answer "yes to all" (Y). The verbose option allows you to see more detailed information, as the script works, about debugging. (This can be large amounts of information, so use this option only if you are searching for a problem.)
To start the main core system objects, enter the legion_startup command. Legion will start several classes on your host. Your output will look something like this:
$ legion_startup Starting meta-class object: LegionClass Continue (y=yes, Y=yes to all, n=no, N=no to all, v=verbose, V=verbose all)? Y Starting meta-class object: BootstrapMetaClass Starting class object: BindingAgentClass Starting class object: CommandLineClass Starting class object: UnixHostClass Starting class object: UnixVaultClass Starting class object: UnixImplementationClass Starting class object: UnixImplementationCacheClass Starting class object: ContextClass Legion first-time system startup complete Run "legion_initialize" $
The output shows the major class objects starting. LegionClass is the top-level metaclass and is the parent of every other object in the system. BootstrapMetaClass is the class object for the bootstrap class objects, which are those class objects that have instances that need to get created in the initialization phase; those class objects are UnixVaultClass, UnixHostClass, UnixImplementationClass, and UnixImplementationCacheClass.
The new class objects will start instances in the new system: BindingAgentClass parents binding objects, CommandLine- Class parents command-line objects, etc.
3.3.4 Initialize the system
Like legion_startup, the legion_initialize script will provide prompts asking whether or not to perform each task, and it is generally best to use the "yes to all" option (Y). To initialize Legion, enter
The output shows the system creating and tagging the key ingredients of a new system. It is too long to reproduce in full here, but we'll look at some selected actions.
Creating host object BootstrapHostObject on "your.current.host.name" Continue (y=yes, Y=yes to all, n=no, N=no to all, v=verbose, V=verbose all)? Y Configuring wellknown binaries for host "1.01.07.0100..."
The first line shows the system creating a bootstrap host object on your current host (a host object manages a host, so the bootstrap host object manages the bootstrap host).
Creating vault object BootstrapVaultObject on "your.bootstrap.host.name" Setting BootstrapHost and BootstrapVault restrictions Added 1 host(s) to vault's compatibility set Added 1 vault(s) to host's compatibility set
A bootstrap vault object is automatically created on your current host (a vault object manages a vault, which stores Legion object's permanent states). This guarantees that the bootstrap host object has a compatible vault object. All host objects must be paired with at least one compatible vault object (i.e., a vault that it can "see").3
Creating an ImplementationCache Creating an implementation (ContextObject) for ContextClass Creating the root context object
Implementation objects represent and manage the implementation cache (used to allow Legion processes to take place in different architectures) and context space.
Adding "BootstrapHost" to the hosts context Adding the alias "your.bootstrap.host.name" for BootstrapHost to the hosts context Adding "BootstrapVault" to the vaults context
More implementation objects are created as the process creates new object classes.
Two context names are added to the /hosts context, BootstrapHost and your.bootstrap.host.name. Both names refer to the Bootstrap host object but only one is added to the /vaults context (Figure 4).
Figure 4: Context paths for the bootstrap host and vault objects  |
This object will manage a portion of the persistent storage mechanism for the Bootstrap host. The /impls context contains names of the default implementation objects (see Implementation model). These can all be viewed with context-related commands or the GUI once the system has been completely started.
3.3.5 Set the environment
Source the script that sets up environment variables for using the context space.
You must source this script in each shell in which you plan to run Legion commands, including the shell in which you ran legion_initialize. Otherwise you will not be able to run any Legion utilities or commands.
3.3.6 Choose your security setting
If you wish enable Legion security, run the legion_init_security command. You'll have to decide now whether or not you want security, since the command will not run properly unless you run it immediately after initializing the system. If you don't wish to use it, just skip over this section. However, none of your processes will be protected.
Several events take place when you run this command.
$ legion_init_security Creating the context "/users" to contain user-objects Creating the initial system-admin user object, "/users/admin" Please select a Legion password for "/users/admin": New Legion password: xxxx Retype password: xxxx 1.399b330d.6f000000.01000000.000001fc0cd... Please enter the Legion password for "/users/admin" to continue: Enter Password: xxxx You have successfully logged in.
First, Legion creates a /users context. This context contains all Legion user ids. Since you need a user id to work in a secure system, you are automatically assigned a system administrator user id called admin. Anyone logged in as admin has root privileges in the system and can create new users, modify security settings, etc. The admin user also has ownership of all existing objects in the new system but not any future objects that other users create.
You must create a password for admin. You'll be asked to enter it three times during the legion_init_security process.
Once you're logged in, Legion gives you ownership of all existing objects in the system.
Changing ownership of all objects to "/users/admin" 1.3622260c.01..000001fc0cbe1846763f895a... 1.3622260c.02..000001fc0b3b16eb8b2dde29... [...etc.] Changed ownership of 63 objects.
After this point, any new objects created will belong to whoever created them.
Legion then configures security for the new system's resources. It creates access control lists (ACLs) for all existing core classes and their instances.
Configuring security for the default collection Creating initial ACLs files for all core objects in /home/spw4s/OPR/init_acls Creating ACL for /class/AuthenticationObjectClass class Creating ACL for /class/BasicFileClass class Creating ACL for /class/BasicSchedulerClass class Creating ACL for /class/BatchQueueMetaClass class [...etc.] Creating ACLs for instances of /class/BasicSchedulerClass Creating ACL for 1.399b330d.68000000.01000000.000001fc0b3da560 eff6580b840f3e7a76b4c82beb9b421ce47ff465557c914bc4bb3ba85140b3 444091bdf45dca6e50deac309b02d420b631b886619ea276de1372260b Creating ACLs for instances of /class/BatchQueueMetaClass Creating ACL for 1.399b330d.73000000..000001fc0cd9a1202afc0753 365c8441c69ffcbcd9ccd235c5c72603707b855aaf543dc6632731d9322861 8948049c13dba35de9727993c1e4abe7467c232cb16c05c831 ... already done Creating ACLs for instances of /class/BootstrapMetaClass Creating ACL for 1.399b330d.04..000001fc0e608816a569dbc1d05034 34eedcd9d7cbb97112871ff09e32482308466f094531d7b332007536be821a 598bb4aa4cbdbd9731592bdc06167c028403f5ab8945 ... already done [...etc.]
The access control lists (ACLs) protect objects against unauthorized use. Only an object's creator can use the object, unless the creator specifies otherwise. The initial ACL files allow only the admin to use the core objects.
Setting ACL for /class/AuthenticationObjectClass class Setting ACL for /class/BasicFileClass class Setting ACL for /class/BasicSchedulerClass class [...etc.] All acls set. You have successfully logged out. $
When all necessary ACLs have been set, you are logged out.
At this point, security has been enabled and is running. You must now log in as admin. Note that you need to include the /users path.
Your system is now ready to go. For more information about security see "About Legion security" and "Using security features".
3.4 Starting a multihost system
This is a two-part process. First, you have to have a running single host system, as laid out in section 3.3. Second, you add new host objects on the desired machines. Since you will be starting processes on the target hosts from the bootstrap host be sure that you can run rsh/ssh on the bootstrap host as well as on the target hosts from the bootstrap host without having to enter a password. You can set up a .rhosts file for rsh or an authorized_keys files for ssh to accomplish this (see the rsh and ssh man pages for more information).
3.4.1 Set up
You'll need to set the proper environment variable on the bootstrap host and the remote host(s) so that you can run Legion commands on a remote host using rsh or ssh.
Set these variables on the bootstrap host before you start the new system (i.e., before you run legion_startup). Please note that you only need to follow these steps on the bootstrap host: you will need to install the Legion binaries on any other machines that you add to your system, but you do not need to start more Legion systems.
To add additional hosts and users, copy the $LEGION_OPR/setup.[sh|csh] scripts to a globally accessible location. Hosts can share an NFS-mounted Legion tree, but for best results you should place the OPRs on a local disk.
3.4.2 Create a new vault
If the new host will not be compatible with your existing vaults, create a new vault object with the legion_startvault command.
There are several flags that you can use to set $LEGION, $LEGION_OPR, architectures, etc. Please see page 40 in the Reference Manual for more information about this command.
See section 11.4 in this manual for more information on new vaults.
3.4.3 Create a new host
Use the legion_starthost command to create a new host object on the desired host. For example, to start a new host object on myNewHost.DNS.name, you would enter:
The same command, with the -B flag, will start a new PCD host object.
$ legion_starthost -B PCDUnixHost \ MyNewPCDHost.DNS.name /vaults/BootstrapVault \ hosts/myNewPCDHost
See also section 12.1 for more information on adding PCD hosts.
The same flag can be used to start a new batch queue host object. You'll need to update the host object's attributes to include the queue type.
$ legion_starthost -B BatchQueueHost \ MyNewBQHost.DNS.name /vaults/BootstrapVault \ hosts/myNewBQHost $ legion_update_attributes /hosts/myNewBQHost \ -a "host_queue_type(`<Queue_type>')"
See section 13.1 for more information on adding batch queue hosts.
There are several other flags and options with this command described on page 39 in the Reference Manual. See section 11.3 for more information on new hosts.
If the new host has a new architecture, you now need to add implementations of the core objects for the new architecture. Log in to the new machine and run the following:
$ source <path_to_globally_visable_setup_script>/setup.[sh|csh] $ legion_login /users/admin [run legion_login only if you have enabled Legion security] $ legion_init_arch
Repeat these steps for each additional host. We suggest that you customize these steps and write a script to simplify the process, especially if you need to bring up a big net.
3.4.4 Adding new users to a secure net
If you have not initialized security, there are no user accounts.
Only admin can add users to a secure net. Run the legion_create_user command with the new user's name:
$ legion_create_user <new user name>
We suggest that you put all users in the /users context. I.e.,
New user accounts are available immediately after creation.
If you are working on a PCD host, follow up with these steps: |
3.4.5 Working in the new net
Users can work in an insecure net by entering:
User can work in a secure net by entering:
$ source <path_to_globally_visible_setup_script>/setup.[sh|csh] $ legion_login /users/<user_name> $ legion_cd /home/<user_name>
Please note that neither of these procedures will open a separate shell.
3.5 Making a set-up script for users
We suggest that Legion system administrators prepare a set-up script for users to source when starting work in Legion. You can run the legion_make_setup_script command to automatically generate a set-up script for your system. This script sets the environment variables for Legion users. The usage is:
legion_make_setup_script
[-o <script basename>]
[-OPR <OPR dir name>]
[-L <$LEGION dir name>]
[-debug] [-help]
1. Bourne Shell is not directly supported by our implementation of Legion, due to the use of alias to implement some Legion commands (notably legion_set_context). Bash, however, is supported. back
2. Once fully operational, Legion does not automatically shut down and restart: the system is intended to stay up. back
[Home] [General] [Documentation] [Software] [Testbeds] [Et Cetera] [Map/Search]
|
legion@Virginia.edu
http://legion.virginia.edu/