LIB_RUN

There are a series of steps associated with running a program, which we will cover in detail, but here is an overview of the steps involved:

Find the executable file.
See if user has privileges to run the file. Under some circumstances, this may result in the user being told that the executable cannot be found.
Check for a run-handler. If one is defined, execute the handler passing the executable name to it. This requires jumping back to the first step.
Verify that the user has enough memory quota to load the executable. If running it detached, verify that there is enough quota to create a new process.
Verify there is enough physical memory to load the program.
Load and/or map the program into user memory.
Set current privileges to the union of the current process privileges and the program privileges.
Start program execution.

We will spend the next few articles addressing these steps.

The first two steps have to be done in tandem. We have to check the execution path, which may be multiple directories, to find an executable matching the specified program name. We will go with the first match we've found, whose protection allows the user to run it. Thus, we may execute the second or third (or...) one that we find if the first (or second...) is protected such that the user cannot run it. Of course, if a specific directory is specified, we only need to concern ourselves with that directory.

Another consideration that makes this more complex is that if a file extension is not provided, we have to determine which file to execute if the same name exists with multiple extensions. Something like prog.txt is probably a text file and therefore not executable. But prog.exe, prog.bin, and prog.php might all be. Obviously, we will exclude any file whose protection prohibits access. But when there are multiple choices, which one do we choose? That comes down to the executable priority list, which we will discuss in a bit. It defines which executable extensions have what priority. Higher priority extensions are chosen before lower priority extensions. The policy is handled by the LIB routines and not by the executive. In fact, the executive must be given an explicit device, path, and filename, including extension - all else is disambiguated by Starlet.

The situation is a bit complicated by the fact that, when searching a path, a higher priority extension in a later directory must take precedence over a lower priority extension in a directory earlier in the path. So we may need to make multiple passes through the path to find the best match taking both protections and extension priorities into consideration.

One last consideration is that the user may specify a file extension that doesn't match any of the extensions in the priority list - like prog.txt. It is possible that such a file might still be an exectuable. Remember when we talked about data streams back in articles 8 and 9? The default data stream is what most people think of in terms of file data. But UOS allows additional (or ancillary) data streams to be read from and written to on file. It took 135 articles to get to the topic, but we now start to make use of ancillary data streams. We use a data stream with the name "$RUN" (All UOS-reserved data stream names start with "$") - in this case, we will refer to it as the $RUN attribute of the file. If that attribute exists on a file, it indicates the name of the program (the "run handler") that will execute that file. So, if said data stream exists, we read the data for that stream, which is the name of the program to run. We then go back to the beginning step and find that program, and we include the name of the file in question at the beginning of the parameters passed to the run handler. Note that this program, once found, could also have a $RUN attribute. Also note that even if the file extension is known, the $RUN attribute overrides the defined extension priorities.

Briefly, what is the purpose of a run handler? UOS can only run certain types of executable files. UOS cannot directly run an executable created for Linux or Windows, for example. Nor can it run uncompiled source programs written for some interpreted languages such as BASIC or PHP. But we want to provide a way for users to seamlessly run these programs on UOS. This is where run handlers come in. They are programs that are directly executable by UOS, and take the given file and execute it on behalf of UOS. So, a Windows emulator can be associated with Windows executables to run Windows programs on UOS. Likewise, a BASIC interpreter can be associated with BASIC source files so that the interpreter can execute the BASIC program. These are just examples - there is no limit to the number of run handlers that UOS could run. In the future, we will address some run handlers that will be provided with UOS by default. In fact, we've already discussed one run handler at length: UCL. By default, files with the ".com" extension are executed by UCL. The name by which a run handler goes depends upon the way it operates. UCL is considered a shell (similar to bash on Linux). A run handler that executed Windows programs under UOS would be considered an emulator. On some operating systems, such as RSTS/E, such emulators are bundled together with shells and called "Run Time Systems". A run handler that executed source code, such as BASIC or PHP, would be considered an interpreter (sometimes also incorporating a shell). But, by whatever name or technology, anything that executes a non-UOS-binary on behalf of UOS is considered a run handler.

LIB_RUN is the interface to executive service that runs programs. It performs the first of the three steps described above. This routine has no equivalent in the VMS Starlet (there is a LIB$RUNPROGRAM there, but it works differently and we'll cover it in the future). Here is the user documentation:

LIB_RUN

Executes a program in the context of the current process.

Format

LIB_RUN programname, commandline, flags

Arguments

programname
The address of a SRB that points to the name of the program file to execute. The following rules determine which program is executed if the path and/or extension are omitted:

If a path isn't specified, the program must exist in the user's execution path.

The first instance of the program found in the execution path is the one that is executed, so if programs with the specified name exist in multiple directories in the execution path, you must specify the path in order to execute one that is later in the path.

If a file extension is not provided and multiple matching programs are found in a given directory, the one executed depends upon the current order of execution for file extensions. If a file with a higher extension priority exists in the path after a file with a lower extension priority, the one with the higher extension priority is run.

commandline
The address of a SRB that points to the command line to be passed to the program when execution begins.

flags
The address of a 64-bit value containing the flags that apply to program execution. The flags are:

Flag Meaning

RUN_ACCNT Run with accounting, otherwise program quota usage is not applied to user's account. If not set, the user must have the ACNT privilege.

RUN_AUTH Run with normal authorizations, otherwise run as a logged-out user. If not set, the user must have the IMPERSONATE privilege.

RUN_DEBUG Run the program in the debugger.

RUN_DUMP Create a dump file if the program ends abnormally.

Quotas:

None for the call, any/all for the running program.

Possible error codes

SS_NORMAL = Success
UOSErr_File_Not_Found = No matching program could be found
UOSErr_Infinite_Symbol_Recursion = extension priorities (sys$extensions symbol) contains an infinite recursion.

function RUN( _Program, Command_Line : string ;
    Accounting : boolean = True ; Authorize : boolean = False ;
    Debug : boolean = False ; Dump : boolean = False ) : int64 ;

var SRB1 : TSRB ;
    SRB2 : TSRB ;

begin
    Set_String( _Program, SRB1 ) ;
    Set_String( Command_Line, SRB2 ) ;
    Result := LIB_RUN( int64( @SRB1 ), int64( @SRB2 ), ord( Accounting ),
        ord( Authorize ), ord( Debug ), ord( Dump ) ) ;
end ;

This routine is added to the PasStarlet unit to provide a Pascal-friendly interface to the LIB_RUN routine.

function LIB_RUN( Prog, CL : int64 ; _Flags : int64 ) : int64 ;

var _Program, Command_Line : string ;
    SL : TStringList ;
    F : TCOM_UOS_File ;
    Flags : int64 ;
    I, P : integer ;
    Len : int64 ;
    Node, Access, Node2, Device, Path, Name, _Type, Version, S, Search_Path, Original : string ;
    This_Extension, Handler : string ;
    SRB1, SRB2 : TSRB ;
    Status : int64 ;
    Stream : int64 ;
    SysRequest : TString2I1_Request ;

begin
    // Setup...
    _Program := Get_String( PSRB( Prog )^ ) ;
    Command_Line := Get_String( PSRB( CL )^ ) ;
    Parse_Filename( _Program, Node, Access, Node2, Device, Path, Name, _Type, Version ) ;
    if( _Flags <> 0 ) then
    begin
        Flags := Pint64( _Flags )^ ;
    end ;

This code obtains the program name and command line, and then parses the filename, and obtains the run flags.

    if( _Type = '' ) then // Extension wasn't specified
    begin
        // Obtain extension priorities...
        SL := TStringList.Create ;
        Parse( LIB_Get_Symbol( 'sys$extensions' ) ) ;
        SL.Add( '.com=sys$system:ucl.exe' ) ;
        SL.Add( '.exe=' ) ;
    end ;

If no extension was provided, we load a stringlist with the extension priorities. This comes from the sys$extensions symbol, if defined. If the symbol isn't defined (or in case it doesn't contain the .exe or .com extensions), we append the standard extensions to the end of the list. This may cause duplicates in the list, but since the ones in sys$extensions are processed first, they will always take precedence over the standard extensions. From this, you can see that the .com extension takes precedence over the .exe extension, which might seem odd since .exe files will execute much faster than .com files. But this is so that .com files can be used to modify the behavior of a program on a given system. The format of sys$extensions is a semi-colon-delimited string of extension specifications (if specification includes semi-colons itself, the specification must be enclosed in quotes). Each specification has the format:
<extension> = <handler>
where extension is the file extension, including the dot (.) and handler is the fully-qualified filename of the program that is the handler for the specified extension. If no handler is specified (this should only be the case for UOS .exe files), the system executes the file directly, without using any handlers. If sys$extensions isn't defined, the default behavior will be as if it was defined with the following contents:
.com=sys$system:ucl.exe;.exe=

    // Find file...
    Nest_Level := 0 ;
    while( true ) do
    begin
        inc( Nest_Level ) ;
        if( Nest_Level > 16 ) then
        begin
            Result := UOSErr_Infinite_Symbol_Recursion ;
            if( SL <> nil ) then
            begin
                SL.Free ;
            end ;
            exit ;
        end ;

Here we have an infinite loop. This is to handle the situation where handlers can be nested ad infinitum. Each time we resolve to a given run handler, we need to see if that program has a run handler. If it does, we have to loop back and repeat the process for the that program. However, it is possible that the extension priorities have a recursion in them. Consider the following value for sys$extensions:
.php=sys$system:php.php
In this case, if the user tried to run a file with the .php extension, UOS will use the sys$system:php.php program. Unfortunately, since that is also the .php extension, the code would read through the extensions and try to use the sys$system:php.php program, which then causes us to loop back and try it all over again - a true infinite loop. Therefore, if we reach 16 levels of recursions, we'll assume that this is the case and exit with an error. But it could also simply indicate that there are a lot of run handlers and we hit 16 levels of run handling. Generally speaking, for performance sake, we'd limit this to 1 or 2 levels for any given extension. Anything beyond that is probably going to bog down due to overhead anyway. 16 levels ought to be more than sufficient. In the future, we'll discuss ways of changing these kinds of built-in limitations. Granted, we could check the simple case of recursion, but it is useful to simply limit the number of levels for the purpose of maintaining system performance. This code solves both issues.

        // Find appropriate file in path...
        Search_Path := Get_Path ;
        _Program := '' ;
        if( _Type <> '' ) then // Extension was specified
        begin
            if( ( Device <> '' ) or ( Path <> '' ) ) then
            begin
                if( Directory_Exists( Device + S + Path + Name + _Type + Version ) ) then
                begin
                    _Program := Device + S + Path + Name + _Type + Version ;
                end ;
            end else
            while( Search_Path <> '' ) do
            begin
                S := Trim_Quotes( Parse_Parameter( ';', Search_Path, 2 ) ) ;
                if( pos( '::', S ) = 0 ) then
                begin
                    S := Node + S ;
                end ;
                if( Directory_Exists( S + Path + Name + _Type + Version ) ) then
                begin
                    _Program := S + Path + Name + _Type + Version ;
                    break ;
                end ;
            end ; // while( Search_Path <> '' )
        end else

First, we have to find the specified file. We get the search path in case an explicit path isn't specified. And we clear _Program to indicate that the program wasn't found (if it is, we'll set the variable appropriately).

There are two different paths through the code to find the file: 1) if an extension is specified, and 2) if an extension is not specified. In this snippit of code we are handling the first case. If a device or path is explicitly specified, we don't use the search path and simply check to see if the file exists. Otherwise we clear the _Program variable and drop through to the code that follows the find-the-program code.

However, if a device and path are not specified, we have to search for the program. So we iterate through the path, seeing if the program exists in that directory. If found, we construct _Program from that path and the program file name, and break out of the loop. Note that if a node was specified in the program name but not in the current search path entry, we add the node to the search path entry.

        begin
            Original := Search_Path ;
            for I := 0 to SL.Count - 1 do
            begin
                This_Extension := SL[ I ] ;
                P := pos( '=', This_Extension ) ;
                Handler := copy( This_Extension, P + 1, length( This_Extension ) ) ;
                setlength( This_Extension, P - 1 ) ;
                while( Search_Path <> '' ) do
                begin
                    S := Trim_Quotes( Parse_Parameter( ';', Search_Path, 2 ) ) ;
                    if( pos( '::', S ) = 0 ) then
                    begin
                        S := Node + S ;
                    end ;
                    if( Directory_Exists( S + Path + Name + This_Extension + Version ) ) then
                    begin
                        _Program := S + Path + Name + This_Extension + Version ;
                        break ;
                    end ;
                end ; // while( Search_Path <> '' )
                if( _Program <> '' ) then // Found a match
                begin
                    break ;
                end ;
                Search_Path := Original ;
            end ; // for I := 0 to SL.Count - 1
        end ;

Now we process the case where the extension isn't specified. Here we save the search path, then loop through the extensions list. We parse the handler and extension from the current extension specification. Next we iterate through the search path looking for a file with the current extension in each entry in the path, just like we did in the previous code snippet. If found, we construct the program name and break out of this loop. After the loop, if _Program is set, we found a match and we exit the extension loop. Otherwise, we restore the search path, loop back for the next extension and try again. Note that if a match is found, Handler will be set to the run handler for this extension.

        if( _Program = '' ) then // File not found
        begin
            Result := UOSErr_File_Not_Found ;
            if( SL <> nil ) then
            begin
                SL.Free ;
            end ;
            exit ;
        end ;

When we drop down to this code after searching for the program, if _Program is null, the program wasn't found, so we exit with an error. Otherwise, we continue on to the next stage of the process - checking for a run handler.

        // Check for run handler (overrides the handler above)...
        F := Open_Binary_File( _Program, FAB_V_GET or FAB_V_SHRALL ) ;
        if( F = nil ) then // Open failure
        begin
            Result := LIB_Get_Exception( 0 ) ;
            if( SL <> nil ) then
            begin
                SL.Free ;
            end ;
            exit ;
        end ;
        Stream := F.Stream_Index( '$RUN' ) ;

Next we check for a run handler for the file, as defined by the $RUN property for the file (if any). If one is defined, it will override the default run handler for the program extension. All UOS files possess at least the default data stream (stream 0), but additional streams can be provided to associate additional meta data with the file. All streams are read and written similarly, but there is a stream interface to files that helps us to manage the data streams. Thus, to access a data stream, we first open the file. If there is an error, we exit with an error. Otherwise, we ask for the index for the data stream with the name "$RUN". This will be the $RUN property for that file.

        if( Stream > 0 ) then // There is a $RUN property
        begin
            Len := F.Stream_Length( Stream ) ;
            if( Len < 65536 ) then // Ignore if stream is > 8K in length
            begin
                setlength( Handler, Len ) ;
                Result := F.Read_Stream( Stream, 0, Len, int64( PChar( Handler ) ) ) ;
                if( Result <> 0 ) then // Error reading $RUN property
                begin
                    if( SL <> nil ) then
                    begin
                        SL.Free ;
                        F.Close( 0 ) ;
                        F.Free ;
                    end ;
                    exit ;
                end ;
                I := pos( #0, Handler ) ;
                if( I > 0 ) then
                begin
                    setlength( Handler, I - 1 ) ;
                end ;
                I := pos( #26, Handler ) ;
                if( I > 0 ) then
                begin
                    setlength( Handler, I - 1 ) ;
                end ;
            end ;
        end ;

If the stream index returned to us is 0, no stream with that name exists for the file. Otherwise, we know that there is a run handler defined, so we want to get it. We get the length of the $RUN data stream. If the length exceeds 64K bytes, it is almost certainly an error in the run handler specification. In this case, we ignore it and act as if the file has no $RUN property. The main data stream (index 0) has a little more flexibility than the other data streams, for instance: having a logical size. Ancillary data streams simply have a physical length and it is up to the code using them to handle a disparity between logical and physical size. Thus, if their clustersize is more than 64K, $RUN will always be ignored.

Next we read the data into a string and exit if there was an error. Otherwise, we terminate the string at the first null or control-Z character (whichever happens first). This illustrates one means of providing a logical size for a data stream, which always has to be done due to the aforementioned lack of a logical size being maintained on ancillary data streams. In the end, Handler will be the value of the $RUN property for the file.

        F.Close( 0 ) ;
        F.Free ;
        if( Handler = '' ) then // No handler = default handler
        begin
            break ;
        end ;

        // Found a run handler...
        if( ( Flags and RUN_DEBUG ) <> 0 ) then
        begin
            Flags := Flags and ( not RUN_DEBUG ) ;
            Command_Line := _Program + '/DEBUG ' + Command_Line
        end else
        begin
            Command_Line := '"'  + _Program + '" ' + Command_Line ;
        end ;
        _Program := Handler ;
    end ; // while( true )

Now that we have a run handler from the file (or not), we are done with the file and close it. If there is no handler, the program is assumed to be a UOS native image and we exit the loop. Otherwise, we now have to find the run handler program. So we prepend the command line with the name of the program and change _Program to be the run handler, and loop back to start the search process all over again. However, if the RUN_DEBUG flag is set, we add "/DEBUG" after the program name so the run handler knows to run the program in a debugger (assuming it has one).

    if( SL <> nil ) then
    begin
        SL.Free ;
    end ;
    if( ( Flags and RUN_DEBUG ) <> 0 ) then
    begin
        Command_Line := '"'  + _Program + '" ' + Command_Line ;
        _Program := '_sys$system:debug.exe'
    end ;

Once we are out of the loop, we clean up the stringlist. Then we check for the RUN_DEBUG flag. If set, we are going to treat the system debugger as if it were a run handler. In that case, we preprend the command line with the program name, and set _Program to the name of the system debugger program. Note that there is no attempt to check for a run handler for debug.exe, or to follow pathing. If we get here, we will hard-code the debugger and have the executive run that program.

    // At this point, we have an existing program to execute...
    Status := 0 ;
    fillchar( SysRequest, sizeof( SysRequest ), 0 ) ;
    SysRequest.Request.Subsystem :=  UOS_Subsystem_FIP ;
    SysRequest.Request.Request := UOS_FIP_Run ;
    SysRequest.Request.Length := sizeof( SysRequest ) - sizeof( TSystem_Request ) ;
    SysRequest.Request.Status := integer( @Status ) ;
    Set_String( _Program, SRB1 ) ;
    Set_String( Command_Line, SRB2 ) ;
    SysRequest.String1 := SRB1 ;
    SysRequest.String2 := SRB2 ;
    SysRequest.Integer1 := int64( @Flags ) ; // Pass modified flags
    Call_To_Ring0( integer( @SysRequest ) ) ;
    Result := Status ;
end ; // LIB_RUN

Next, we wrap up the current parameters and call the executive to execute the program.

    procedure Parse( S : string ) ;

    var X : string ;

    begin
        S := trim( S ) ;
        while( S <> '' ) do
        begin
            X := Parse_Parameter( ';', S, 2 ) ;
            if( ( pos( '=', X ) > 2 ) and ( X[ 1 ] = '.' ) ) then
            begin
                SL.Add( X ) ;
            end ;
        end ;
    end ;

This local routine parses a string, delimited by semi-colons, into a stringlist.

function Get_Path : string ;

var X, W : string ;

begin
    Result := LIB_Get_Symbol( 'path' ) ;
    if( Result = '' ) then
    begin
        Result := LIB_Get_Symbol( 'sys$disk:' ) ;
    end else
    begin
        while( Result <> '' ) do
        begin
            W := Parse_Parameter( ';', Result, 2 ) ;
            if( X <> '' ) then
            begin
                X := X + ';' ;
            end ;
            X := X + '"' + W + '"' ;
        end ;
        Result := X ;
    end ;
    Result := Result + ';sys$system:' ;
end ;

This function, used in the above code, returns the current search path, making sure that sys$system is included at the end. This allows the user to override sys$system and look elsewhere first, if desired. But sys$system is the default location for CUSPs, so we make sure to always look there if a program is not found anywhere else.

In the next article, we will take a look at the data stream interface used by the above code. After that, we'll look at the run service.