1 Introduction
2 Ground Rules

Building a File System
3 File Systems
4 File Content Data Structure
5 Allocation Cluster Manager
6 Exceptions and Emancipation
7 Base Classes, Testing, and More
8 File Meta Data
9 Native File Class
10 Our File System
11 Allocation Table
12 File System Support Code
13 Initializing the File System
14 Contiguous Files
15 Rebuilding the File System
16 Native File System Support Methods
17 Lookups, Wildcards, and Unicode, Oh My
18 Finishing the File System Class

The Init Program
19 Hardware Abstraction and UOS Architecture
20 Init Command Mode
21 Using Our File System
22 Hardware and Device Lists
23 Fun with Stores: Partitions
24 Fun with Stores: RAID
25 Fun with Stores: RAM Disks
26 Init wrap-up

The Executive
27 Overview of The Executive
28 Starting the Kernel
29 The Kernel
30 Making a Store Bootable
31 The MMC
32 The HMC
33 Loading the components
34 Using the File Processor
35 Symbols and the SSC
36 The File Processor and Device Management
37 The File Processor and File System Management
38 Finishing Executive Startup

Users and Security
39 Introduction to Users and Security
40 More Fun With Stores: File Heaps
41 File Heaps, part 2
42 SysUAF
43 TUser
44 SysUAF API

Terminal I/O
45 Shells and UCL
46 UOS API, the Application Side
47 UOS API, the Executive Side
48 I/O Devices
49 Streams
50 Terminal Output Filters
51 The TTerminal Class
52 Handles
53 Putting it All Together
54 Getting Terminal Input
55 QIO
56 Cooking Terminal Input
57 Putting it all together, part 2
58 Quotas and I/O

UCL
59 UCL Basics
60 Symbol Substitution
61 Command execution
62 Command execution, part 2
63 Command Abbreviation
64 ASTs
65 Expressions, Part 1
66 Expressions, Part 2: Support code
67 Expressions, part 3: Parsing
68 SYS_GETJPIW and SYS_TRNLNM
69 Expressions, part 4: Evaluation

UCL Lexical Functions
70 PROCESS_SCAN
71 PROCESS_SCAN, Part 2
72 TProcess updates
73 Unicode revisted
74 Lexical functions: F$CONTEXT
75 Lexical functions: F$PID
76 Lexical Functions: F$CUNITS
77 Lexical Functions: F$CVSI and F$CVUI
78 UOS Date and Time Formatting
79 Lexical Functions: F$CVTIME
80 LIB_CVTIME
81 Date/Time Contexts
82 SYS_GETTIM, LIB_Get_Timestamp, SYS_ASCTIM, and LIB_SYS_ASCTIM
83 Lexical Functions: F$DELTA_TIME
84 Lexical functions: F$DEVICE
85 SYS_DEVICE_SCAN
86 Lexical functions: F$DIRECTORY
87 Lexical functions: F$EDIT and F$ELEMENT
88 Lexical functions: F$ENVIRONMENT
89 SYS_GETUAI
90 Lexical functions: F$EXTRACT and F$IDENTIFIER
91 LIB_FAO and LIB_FAOL
92 LIB_FAO and LIB_FAOL, part 2
93 Lexical functions: F$FAO
94 File Processing Structures
95 Lexical functions: F$FILE_ATTRIBUTES
96 SYS_DISPLAY
97 Lexical functions: F$GETDVI
98 Parse_GetDVI
99 GetDVI
100 GetDVI, part 2
101 GetDVI, part 3
102 Lexical functions: F$GETJPI
103 GETJPI
104 Lexical functions: F$GETSYI
105 GETSYI
106 Lexical functions: F$INTEGER, F$LENGTH, F$LOCATE, and F$MATCH_WILD
107 Lexical function: F$PARSE
108 FILESCAN
109 SYS_PARSE
110 Lexical Functions: F$MODE, F$PRIVILEGE, and F$PROCESS
111 File Lookup Service
112 Lexical Functions: F$SEARCH
113 SYS_SEARCH
114 F$SETPRV and SYS_SETPRV
115 Lexical Functions: F$STRING, F$TIME, and F$TYPE
116 More on symbols
117 Lexical Functions: F$TRNLNM
118 SYS_TRNLNM, Part 2
119 Lexical functions: F$UNIQUE, F$USER, and F$VERIFY
120 Lexical functions: F$MESSAGE
121 TUOS_File_Wrapper
122 OPEN, CLOSE, and READ system services

UCL Commands
123 WRITE
124 Symbol assignment
125 The @ command
126 @ and EXIT
127 CRELNT system service
128 DELLNT system service
129 IF...THEN...ELSE
130 Comments, labels, and GOTO
131 GOSUB and RETURN
132 CALL, SUBROUTINE, and ENDSUBROUTINE
133 ON, SET {NO}ON, and error handling
134 INQUIRE
135 SYS_WRITE Service
136 OPEN
137 CLOSE
138 DELLNM system service
139 READ
140 Command Recall
141 RECALL
142 RUN
143 LIB_RUN
144 The Data Stream Interface
145 Preparing for execution
146 EOJ and LOGOUT
147 SYS_DELPROC and LIB_GET_FOREIGN

CUSPs and utilities
148 The I/O Queue
149 Timers
150 Logging in, part one
151 Logging in, part 2
152 System configuration
153 SET NODE utility
154 UUI
155 SETTERM utility
156 SETTERM utility, part 2
157 SETTERM utility, part 3
158 AUTHORIZE utility
159 AUTHORIZE utility, UI
160 AUTHORIZE utility, Access Restrictions
161 AUTHORIZE utility, Part 4
162 AUTHORIZE utility, Reporting
163 AUTHORIZE utility, Part 6
164 Authentication
165 Hashlib
166 Authenticate, Part 7
167 Logging in, part 3
168 DAY_OF_WEEK, CVT_FROM_INTERNAL_TIME, and SPAWN
169 DAY_OF_WEEK and CVT_FROM_INTERNAL_TIME
170 LIB_SPAWN
171 CREPRC
172 CREPRC, Part 2
173 COPY
174 COPY, part 2
175 COPY, part 3
176 COPY, part 4
177 LIB_Get_Default_File_Protection and LIB_Substitute_Wildcards
178 CREATESTREAM, STREAMNAME, and Set_Contiguous
179 Help Files
180 LBR Services
181 LBR Services, Part 2
182 LIBRARY utility
183 LIBRARY utility, Part 2
184 FS Services
185 FS Services, Part 2
186 Implementing Help
187 HELP
188 HELP, Part 2
189 DMG_Get_Key and LIB_Put_Formatted_Output
190 LIBRARY utility, Part 3
191 Shutting Down UOS
192 SHUTDOWN
193 WAIT
194 SETIMR
195 WAITFR and Scheduling
196 REPLY, OPCOM, and Mailboxes
197 REPLY utility
198 Mailboxes
199 BRKTHRU
200 OPCOM
201 Mailbox Services
202 Mailboxes, Part 2
203 DEFINE
204 CRELNM
205 DISABLE
206 STOP
207 OPCCRASH and SHUTDOWN
208 APPEND

Glossary/Index


Downloads

SysUAF API

In the last few articles, we described the SysUAF module, which is the interface to the SysUAF.DAT file. We've covered the constants, structures, the TUser class, and one of the Module API functions. In this article, we will look at the rest of the API functions, and then we'll look at the USC methods that implement the forced login. A forced login is only called by the executive and doesn't provide any validation (that is assumed to be done before the function is called). It allows the executive to force a login without reguard to authentication, quotas, or flags. 

type TUAF_Header = packed record
                       Version : longint ;
                       Flags : longint ; // Reserved for future use
                       Encryption_Codec : TStringList_Ptr ;
                       Users : TList_Ptr ; // Users list
                       User_Names : TStringList_Ptr ; // lowercase user name list
                       Groups : TString_Ptr ; // Groups list
                       Group_Names : TStringList_Ptr ; // lowercase group name list
                   end ;

As a file heap, SysUAF needs a header that serves as the root structure for the heap. The header contains a version that we can check in case we ever change the SysUAF format. Flags are reserved for future use. The Encryption code (coder/decoder - or in this case encrypter/decrypter) indicates the codec that is used to has password values. We will discuss that in a later article. Users is the root of the list of users, and User_Names is a string list that contains lowercase versions of each user name. Using lowercase allows us to normalize the names so that we can do case-insensitive lookups of users by name. Groups and Group_Names will be discussed when we get to the topic of user groups in the future.

Here are the variables for the module: 

var SysUAF_File : TUOS_File = nil ;
var SysUAF_Store : TUOS_Native_File_Store = nil ;
var _SysUAF : TUOS_File_Heap = nil ;
var Users_List : TStore_List = nil ;
var Users_Names_List : TStore_String_List = nil ;
var _HAL : THAL = nil ;
var _SSC : TUOS_System_Services = nil ;
var UAF_Header : TUAF_Header ;

We will describe these variables as we look at the API functions. 

function Set_Sysuaf_File( F : TUOS_File ) : TUnified_Exception ;

begin
    Result := nil ;
    if( _SysUAF = nil ) then
    begin
        _SysUAF := TUOS_File_Heap.Create ;
        _SysUAF._File := F ;
    end ;
    if( _SysUAF.Origin = 0 ) then // Need to initialize SYSUAF.DAT
    begin
        fillchar( UAF_Header, sizeof( UAF_Header ), 0 ) ;
        Users_List := Create_Store_List( _SysUAF, 8 ) ;
        UAF_Header.Users := Users_List.Address ;
        Users_List.Count := 8 ;
        Users_List.Detach ;
        Users_Names_List := Create_Store_String_List( _SysUAF, 8 ) ;
        UAF_Header.User_Names := Users_Names_List.Address ;
        Users_Names_List.Count := 8 ;
        Users_Names_List.Detach ;
        _SysUAF.Origin := _SysUAF.Getmem( sizeof( UAF_Header ) ) ;
        _SysUAF.Write_Data( UAF_Header, _SysUAF.Origin, sizeof( UAF_Header ), Result ) ;
        if( Result <> nil ) then
        begin
            exit ;
        end ;
    end ;

    // Load header...
    _SysUAF.Read_Data( UAF_Header, _SysUAF.Origin, sizeof( UAF_Header ), Result ) ;
    if( Result <> nil ) then
    begin
        exit ;
    end ;
    Users_List := Get_Store_List( _SysUAF, UAF_Header.Users ) ;
    Users_Names_List := Get_Store_String_List( _SysUAF, UAF_Header.User_Names ) ;
end ; // Set_Sysuaf_File

Before any of the other API functions will work, the SysUAF module needs to be made aware of the SysUAF.DAT file. This is done via the Set_Sysuaf_File function. If an exception occurs, that exception will be returned. Otherwise we return nil. If our native file store instance has not yet been created, we create one. We set the file store instance's file to the passed file. If the file heap instance has not been created, we create it and assign the file to it.
If the file heap's Origin value is 0, then this is a newly created file. In such case, we need to construct a header record and write it out to the file. This involves clearing the values, then creating a store list and store string list and updating the header with the addresses for those structures.
Finally, we load the header from the file (whether it was just written or already existing) and construct the Users_List and Users_Names_List instances. The first is a list of user account records, and the latter is a list of lowercase user names. 

function Get_User( UIC : longint ) : TUser ;

var A : int64 ;
    Loop : longint ;

begin
    Result := nil ;
    if( ( UIC < 0 ) or ( UIC >= Users_List.Count ) ) then // Invalid UIC
    begin
        exit ;
    end ;

    // Check for cached user...
    for Loop := 0 to Cached_Users.Count - 1 do
    begin
        Result := TUser( Cached_Users[ Loop ] ) ;
        if( Result._UIC = UIC ) then
        begin
            exit ;
        end ;
    end ; // for

    // Load user from sysUAF...
    A := Users_List[ UIC ] ;

    // Create new user (cached) instance...
    Result := TUser.Create( UIC ) ;
    if( UIC = 1 ) then
    begin
        Result.Flags := UAF_DisCtlY or UAF_Disabled ; // No interrupts and can only be run by Kernel
        Result.Privileges := -1 ;
        Result.Auth_Privileges := -1 ;
    end else
    begin
        Result.Address := A ;
    end ;
end ; // Get_User

This function returns an instance of the TUser class, for the passed user. If the UIC is less than zero or greater than the maximum used UIC, we return nil and exit. Next we search the Cached_Users list for the passed UIC. If we find a match, we return the existing TUser instance. Otherwise, we create a new instance. If the UIC is 1 (Startup), we hard-code the values; all privileges are given and interrupts are disabled. We disable interrupts so that the user cannot abort the system startup or system setup. If it isn't UIC 1, we assign the address from Users_List. Then we return the instance. This demonstrates the way that the Cached_Users list is used to ensure only a single instance of each user exists at any time. 

function LC_Name_From_UIC( UIC : longint ) : string ;

begin
    if( UIC = 1 ) then
    begin
        Result := 'Startup' ;
        exit ;
    end ;
    Result := '' ;
    if( ( UIC < 1 ) or ( UIC >= Users_List.Count ) ) then // Invalid UIC
    begin
        exit ;
    end ;
    Result := Users_Names_List[ UIC ] ;
end ;


function Name_From_UIC( UIC : longint ) : string ;

var User : TUser ;

begin
    if( UIC = 1 ) then
    begin
        Result := 'Startup' ;
        exit ;
    end ;
    Result := '' ;
    if( ( UIC < 1 ) or ( UIC >= Users_List.Count ) ) then // Invalid UIC
    begin
        exit ;
    end ;
    User := Get_User( UIC ) ;
    User.Attach ;
    Result := User.Name ;
    User.Detach ;
end ;

These two functions return a user name for the UIC. LC_Name_From_UIC returns the lowercase version of the name from the User_Names_List list. Name_From_UIC returns the name from the user authorization record. 

function Add_User( Name : string ; Special : boolean ;
    UIC_Template : longint ) : longint ;

var I : int64 ;
    Loop : integer ;
    S : TUOS_String ;
    LName : string ;
    User, Template_User : TUser ;

begin
    // Sanity checks...
    Result := -1 ;
    Name := trim( Name ) ;
    if( Name = '' ) then
    begin
        exit ; // Null name not allowed
    end ;
    S := TUOS_String.Create ;
    S.Assign_From_String( Name, SF_UTF8 ) ;
    S.Lowercase ;
    LName := S.As_String ;
    S.Free ;
    if( Users_Names_List.IndexOf( LName, False ) > 0 ) then
    begin
        exit ; // Duplicate name not allowed
    end ;
    if( Users_List.Count < 8 ) then // Reserve space for special UICs
    begin
        Users_List.Count := 8 ;
        Users_Names_List.Count := 8 ;
    end ;

This function adds a new user to SysUAF. First we default the result to -1 to indicate an error condition (we will set it otherwise on success). Next we validate that the name isn't null. Then we create a lowercase version of the name, and use that to see if a user with this name already exists. If so, we exit. Finally, we ensure that there are at least 8 positions reserved for the system UICs. This should have already been done when SysUAF was initialized, but we check here just to be safe. 

    // Add user...
    I := _SysUAF.Getmem( sizeof( TUAF_User ) ) ;
    if( I = 0 ) then
    begin
        exit ;
    end ;
    if( Special ) then // Special user case...
    begin
        for Loop := 2 to 7 do
        begin
            if( Users_List[ Loop ] = 0 ) then
            begin
                Result := Loop ;
                Users_List[ Result ] := I ;
                Users_Names_List[ Result ] := LName ;
                break ;
            end ;
        end ;
    end ;
    if( Result <= 0 ) then // Not added yet
    begin
        Result := Users_Names_List.Add( LName ) ;
        Users_List.Add( I ) ;
    end ;

Next, the function allocates space for a new user record, and exits if that fails. If the caller requested a special (system) account, we check UICs 2 through 7 for an unused entry. If one is found, we update the Users_List and Users_Names_List with the new user's information., and we set the result to that UIC. If Special was false, or there were no available system UICs, we add the new user to the end of the Users_List and Users_Names_List, and set the result to the newly added index. 

    User := Get_User( Result ) ;
    try
        User.Attach ;
        User._UIC := Result ;
        User._Address := I ;
        User.Name := Name ;

        // Assign template values...
        if( ( UIC_Template >= 0 ) and ( UIC_Template < Users_List.Count ) ) then
        begin
            Template_User := Get_User( UIC_Template ) ;
            if( Template_User = nil ) then
            begin
                exit ; // Template not found
            end ;
            try
                Template_User.Attach ;
            finally
                Template_User.Detach ;
            end ;
            User.Flags := Template_User.Flags ;
            User.Shell := Template_User.Shell ;
            User.LGICMD := Template_User.LGICMD ;
            User.Privileges := Template_User.Privileges ;
            User.Auth_Privileges := Template_User.Auth_Privileges ;
            User.Expiration := Template_User.Expiration ;
            User.Owner := Template_User.Owner ;
            User.Priority := Template_User.Priority ;
            User.Quotas := Template_User.Quotas ;
            for Loop := 0 to Template_User.Authentication_Count - 1 do
            begin
                User.Add_Authentication( Template_User.Authentication[ Loop ] ) ;
            end ;
            for Loop := 0 to Template_User.Access_Count - 1 do
            begin
                User.Add_Access( Template_User.Access[ Loop ] ) ;
            end ;
            Template_User.Detach ;
        end ;
    finally
        User.Detach ;
    end ;
end ; // Add_User

Now that we have a TUser instance, we attach to it, set its UIC, address, and name. Note that we set the _Address instance data directly rather than using the property. This is because we don't want to trigger the side-effect of loading the header, since it isn't set up yet. Normally this would be pathlogical coupling, but this function is in the same module as the TUser class and these details are hidden from the outside world. Next, if a template UIC was passed, we load that user and then copy the information from the template to the new user (not including name or password). When done, we detach from the template user. Finally, we detach from the new user instance.

There is more to the SysUAF module that we will cover in the future. Now, let's move to the USC and look at the way it uses the SysUAF module. The Force_Login method is called directly by the Kernel startup. All other logins eventually make their way to this function as well: 

function TUSC.Force_Login( PID, UIC : cardinal ) : boolean ;

var F : TFile ;
    Loop : integer ;
    Home, S : string ;
    Str, Str1 : TString ;
    UEC : TUnified_Exception ;
    User : TUser ;

begin
    // Handle initial setup...
    if( not _Setup ) then
    begin
        // Open SysUAF.dat...
        F := FiP.Open_File( PID, '_sys0:\uos\sysuaf.dat', FM_RW or FM_Force ) ;
        if( F = nil ) then
        begin
            Set_Last_Error( FiP.Last_Error ) ;
            exit ;
        end ;
        UEC := Set_Sysuaf_File( F ) ;
        if( UEC <> nil ) then
        begin
            Set_Last_Error( UEC ) ;
            exit ;
        end ;

        // Open Accounting.dat...
        F := FiP.Open_File( PID, '_sys0:\uos\accounting.dat', FM_RW or FM_Force ) ;
        if( F = nil ) then
        begin
            Set_Last_Error( FiP.Last_Error ) ;
            exit ;
        end ;
        UEC := Set_Accounting_File( F ) ;
        if( UEC <> nil ) then
        begin
            Set_Last_Error( UEC ) ;
            exit ;
        end ;

        // Create logicals...
        Loop := 2 ;
        S := '' ;
        Str := TString.Create ;
        Str1 := TString.Create ;
        try
            while( Loop <= Max_UIC ) do
            begin
                S := LC_Name_From_UIC( Loop ) ;
                if( S <> '' ) then
                begin
                    User := Get_User( Loop ) ;
                    User.Attach ;
                    Str.P := Pchar( S ) ;
                    Str.Len := length( S ) ;
                    Home := User.Home ;
                    if( Home = '' ) then
                    begin
                        Home := 'sys:\Users\' + S ;
                    end ;
                    Str1.P := PChar( Home ) ;
                    Str1.Len := length( Home ) ;
                    SSC.Set_Symbol( LNM_System, 0, PChar( S ), Str1 ) ;
                    User.Detach ;
                end ;
                inc( Loop ) ;
            end ;
        finally
            Str.Free ;
            Str1.Free ;
        end ;

        _Setup := True ;
    end ; // if( not _Setup )

Force_Login is guaranteed to be called at the end of the executive startup and before the system startup. Thus, this is a good place to place deferred setup code - it is early, but not too early. So, when Force_Login is called the first time, the _Setup flag hasn't been set yet and the code executes the initial setup. The first order of business is to open the SysUAF.DAT file. We always use the file from the boot device. If there is a problem opening the file, we set the error and exit. Otherwise we call the Set_Sysuaf_File function in the SysUAF module. Then we repeat the process for the accounting file. As mentioned before, we will address accounting in future articles.
The next loop sets up a logical for each user. The logical name is the user name and its value is the user's home directory (which defaults to sys:\Users\). These logicals make it easier for people to locate files shared by a given user. We start at UIC 2 because we don't include the Startup account, which has no home directory. The loop skips blank user names (unused UICs). Note that we don't skip disabled user accounts as there may still be need to refer to the files in that user's home directory. We don't check for errors on logical creation since we don't want that to abort startup. Finally, we set the _Setup flag. 

    // Sanity checks..
    Result := False ; // Assume success
    if( ( PID = 0 ) or ( UIC = 0 ) ) then
    begin
        Result := True ;
        Set_Error( UOS_User_Security_Error_Invalid_Context ) ;
        exit ;
    end ;

    // Set on each login to ensure the latest values are set...
    Set_SSC( SSC ) ;
    Set_HAL( _HAL ) ;

    // Do the login
    User := Get_User( UIC ) ;
    if( User = nil ) then
    begin
        Result := True ;
        Set_Error( UOS_User_Security_Error_Unknown_User ) ;
        exit ;
    end ;
    User.Attach ;
    Logged_In_Users.Add( UIC ) ;
end ; // TUSC.Force_Login

We only return True if there is a problem, so we default the result to False (success). If the UIC or PID (Process ID) is invalid, we exit with an error. Next we set the SSC and HAL for the sysUAF module. We do this each time in case either have changed since the last call. Next we get the user. If nil is returned, the UIC is not valid so we set an error and exit. Otherwise, we attach to the TUser instance. Thus, each time this user logs in, the user instance for that UIC is attached to. Each time the user logs out, the instance will be detached from. Therefore, when all instances of this user log off, the user instance will self-destruct. Finally we add the UIC to our list of logged-in users.

We will address more about users and security as we go on, but in the next article, we will begin a discussion of the default UOS shell.

Copyright © 2017 by Alan Conroy. This article may be copied in whole or in part as long as this copyright is included.