PyLoadL

Data Access API

Synopsis

from pyloadl import *
  
version = ll_version()

#
# Data Access API functions
#

query = ll_query(JOBS|MACHINES|CLUSTER|WLMSTAT|MATRIX|CLASSES|RESERVATIONS|
                 MCLUSTERS|BLUE_GENE|FAIRSHARE)

return = ll_set_request(query, QUERY_ALL|QUERY_JOBID|
                               QUERY_STEPID|QUERY_GROUP| 
	 		       QUERY_CLASS|QUERY_HOST| 
		 	       QUERY_STARTDATE|QUERY_ENDDATE| 
			       QUERY_PROCID|QUERY_RESERVATION_ID|
                               QUERY_BG_JOB|QUERY_BG_PARTITION|
                               QUERY_BG_BASE_PARTITION|QUERY_TOP_DOG,
                               filter_list, ALL_DATA|Q_LINE|STATUS_LINE)
 
object ,num_objs, err_code = ll_get_objs(query, LL_STARTD|LL_SCHEDD|LL_CM|LL_MASTER|
                             LL_STARTER|LL_HISTORY_FILE, hostname)

return = ll_reset_request(object)

object = ll_next_obj(object)

return = ll_free_objs(object)

return = ll_deallocate(query)

result = ll_get_data(object, LLAPI_Specification)

Description

A small example script using the Data Access API :

from pyloadl import *

#
# Query Job information
#

query = ll_query(JOBS)

retval = ll_set_request(query, QUERY_ALL, "", ALL_DATA)
job, num, err = ll_get_objs(query, LL_CM, "")

if num > 0:

   while PyCObjValid(job):

       # Get User Information from credential object

       cred  = ll_get_data(job, LL_JobCredential)
       owner = ll_get_data(cred, LL_CredentialUserName)
		
       # Loop through all steps for this job

       step = ll_get_data(job, LL_JobGetFirstStep)
       while PyCObjValid(step):

         id = ll_get_data(step, LL_StepID)

         step = ll_get_data(job, LL_JobGetNextStep)

       job = ll_next_obj(query)

# Free and deallocate objects used by LoadLeveler

ll_free_objs(job)
ll_deallocate(query)

API Functions

The Data Access API has the following functions below and are used in this order :

• ll_query to initialise the query object.
• ll_set_request to filter the objects you want to query.
• ll_get_objs to retrieve a list of objects from a LoadLeveler daemon/history file.
• ll_get_data to retrieve specific data from an object.
• ll_next_obj to retrieve the next object in the list.

• ll_free_objs to free the list of objects you received.
• ll_deallocate to end the query.

ll_query

ll_query is the first call you make to access the API, it establishes the type of information you receive.

 query = ll_query( JOBS | MACHINES | CLUSTER | WLMSTAT | MATRIX | CLASSES | 
                   RESERVATIONS | MCLUSTERS | BLUE_GENE | FAIRSHARE )

Note:

1. CLASSES query requires LoadLeveler 3.2+
2. RESERVATIONS query requires LoadLeveler 3.3+
3. BLUE_GENE query requires LoadLeveler 3.3.1+ and a Blue Gene system
4. MCLUSTERS query requires LoadLeveler 3.3.1+ and a Multi-Cluster environment
5. FAIRSHARE query requires LoadLeveler 3.3.2+

ll_set_request

ll_set_request is used to determine the range of data returned by ll_get_objs.

 return = ll_set_request(query, QueryFlags, ObjectFilter, DataFilter) 

Parameters :

1. query
   

The object returned from ll_query

2. QueryFlags
   

The permissable Query Flags depends on the type of query being made., the flags are :

Query Flags
Flag	Description
QUERY_ALL	Query all jobs
QUERY_JOBID	Query by job ID
QUERY_STEPID	Query by step ID
QUERY_USER	Query by user ID
QUERY_GROUP	Query by LoadLeveler group
QUERY_CLASS	Query by LoadLeveler class
QUERY_HOST	Query by machine name
QUERY_STARTDATE	Query by job start dates
QUERY_ENDDATE	Query by job end dates
QUERY_PROCID	Query by process id
QUERY_RESERVATION_ID	Query by reservation id
QUERY_BG_JOB	Query by Blue Gene type jobs only
QUERY_BG_PARTITION	Query defined Blue Gene partitions
QUERY_BG_BASE_PARTITION	Query Blue Gene base partitions
QUERY_TOP_DOG	Query by job steps that are top dogs

They can be used with the following query types:

Query Type	Permitted Flags
JOBS	QUERY_ALL QUERY_JOBID QUERY_STEPID QUERY_USER QUERY_GROUP QUERY_CLASS QUERY_HOST QUERY_STARTDATE QUERY_ENDDATE QUERY_PROCID QUERY_RESERVATION_ID
MACHINES	QUERY_ALL QUERY_HOST
CLUSTER	QUERY_ALL
CLASSES	QUERY_ALL QUERY_CLASS
WLMSTAT	QUERY_STEPID
MATRIX	QUERY_ALL QUERY_HOST
RESERVATIONS	QUERY_ALL QUERY_USER QUERY_GROUP QUERY_HOST QUERY_RESERVATION_ID
MCLUSTERS	QUERY_ALL
BLUE_GENE	QUERY_ALL QUERY_BG_BASE_PARTITION QUERY_BG_PARTITION
FAIRSHARE	QUERY_ALL QUERY_USER QUERY_GROUP



3. ObjectFilter

Specifies the search criteria:

Query Flag	Filter
QUERY_ALL	None
QUERY_JOBID	List of job IDs e.g ["host.jobid1", "host.jobid2"]
QUERY_STEPID	List of step IDs e.g ["host.jobid.stepid"]
QUERY_USER	List of user IDs e.g ["mike", "mark", "mary"]
QUERY_CLASS	List of LoadLeveler class names
QUERY_GROUP	List of LoadLeveler group names
QUERY_HOST	List of LoadLeveler host names
QUERY_STARTDATE or QUERY_ENDDATE	Two start dates or two end dates having the format MM/DD/YYYY e.g ["01/14/2003", "02/23/2003"]
QUERY_PROCID	List containing a single process id, this is automatically terminated with a NULL by PyLoadL eg ["procid"]
QUERY_RESERVATION_ID	List containing a single reservation id
QUERY_LOCAL	N/A
QUERY_BG_JOB	Query by bluegene type jobs only. List of job IDs e.g ["host.jobid1", "host.jobid2"]
QUERY_BG_BASE_PARTITION	List of BG Base Partitions, ["all"] for all base partitions
QUERY_BG_PARTITION	List of BG Partitions, ["all"] for all partitions
QUERY_TOP_DOG	List of Step IDs




NOTE :


If when using QUERY_PROCID some of the return fields don't return a value then check that APAR IY48329 is installed; it also appeared not work at all with GANG scheduling.

4. DataFilter
   

Filters the amount of data you get back from the query. Permitted values are:

• ALL_DATA
  
Return everything
• Q_LINE
  
Valid only for a JOBS query, not using the history file, returns the same data as llq -f
• STATUS_LINE
  
Valid only for a MACHINES query, returns the same data as llstatus -f

ll_reset_request

 return = ll_reset_request(query)

ll_get_objs

Sends a query request to LoadLeveler

 object, num_objs, err_code = ll_get_objs( query, query_daemon, host )

Returns a LL_element object, the number of query objects and an error code if one occurred. Possible error values are :

Parameters

ll_get_data

 data = ll_get_data(element, LLAPI specification)

This call differs from the IBM documentation, the data you request is returned as the return value and not as a third parameter eg

 load_avg = get_data(machine, LL_MachineLoadAverage) # Returns an integer

 mach_list = get_data(machine, LL_MachineStepList)   # Returns a list of strings

To know what you are getting you really need to know about the LoadLeveler Job Object Model. All of this is in the IBM LoadLeveler for AIX 5L: Using and Administering reference guide and html.

enum types

Returns from some query types may be, in C terms, enumerated types. In PyLoadL these all return as integers. Return values are shown below:

• LL_AdapterReqUsage
  
  

   SHARED, NOT_SHARED, SLICE_NOT_SHARED
  



• LL_StepHoldType
  
  

   NO_HOLD, HOLDTYPE_USER, HOLDTYPE_SYSTEM, HOLDTYPE_USERSYS
  



• LL_StepState
  
  

   STATE_IDLE, STATE_PENDING, STATE_STARTING, STATE_RUNNING,
   STATE_COMPLETE_PENDING, STATE_REJECT_PENDING, STATE_REMOVE_PENDING,
   STATE_VACATE_PENDING, STATE_COMPLETED, STATE_REJECTED, STATE_REMOVED,
   STATE_VACATED, STATE_CANCELED, STATE_NOTRUN, STATE_TERMINATED,
   STATE_UNEXPANDED, STATE_SUBMISSION_ERR, STATE_HOLD, STATE_DEFERRED,
   STATE_NOTQUEUED, STATE_PREEMPTED, STATE_PREEMPT_PENDING, 
   STATE_RESUME_PENDING
  

  
  

ll_next_obj

Returns the next object from the query object.

 job = ll_next_obj(query)

ll_free_objs

Frees up the the memory used by the ll_get_objs routine for LoadLeveler Data, this is a very important call especially for long running processes.

 return = ll_free_objs(query)

ll_deallocate

Frees the query object itself, this is the last LoadLeveler action.

 return = ll_deallocate(query)