Project 2 (MCP: Ghost-in-the-shell) CIS 415 - Operating systems Fall 2019 Prof. Allen Malony Due date: 11:59 pm, Tuesday, Nov 8th, 2023 Introduction: The British philosopher, Gilbert Ryle, in The Concept of Mind criticizes the classical Cartesian rationalism that the mind is distinct from the body (known as mind-body duality). He argues against the “dogma of the ghost in the machine” as an independent non-material entity, temporarily inhabiting and governing the body. In The Ghost in the Machine, Arthur Koestler furthers the position that mind and body are connected and that the personal experience of mind- body duality is emergent from the observation that everything in nature is both a whole and a part. The manga series by Masamune Shirow, The Ghost in the Shell, whose title is an homage to Koestler, explores the proposition that human consciousness and individuality IS the “ghost” and whether it can exist independently of a physical body, with all that implies. When I was going to UCLA for my B.S. and Master's degrees, I worked for 3 summers as an intern at Burroughs Corporation's Medium Systems Division in Pasadena, California. Sadly, Burroughs no longer exists, having merged with Sperry Corporation to form Unisys, but they were a major player in computing systems for a long time. During my last summer internship at Burroughs, my job was to write modules for the Master Control Program, affectionately known as the MCP: http://en.wikipedia.org/wiki/Burroughs MCP The MCP might be considered primitive in contrast to modern operating systems, but it was innovative in many respects, including: the first operating system to manage multiple processors, the first commercial implementation of virtual memory, and the first OS written exclusively in a high-level language. MCP executed “jobs” with each containing one or more tasks. Tasks within a job could run sequentially or in parallel. Logic could be implemented at the job level to control the flow of a job by writing in the MCP's workflow control language (WFL). Once all tasks in a job completed, the job itself was done. In some respects, we might regard these features of MCP as now being provided through shell scripts that access an operating system's services. I often wonder whether the ghosts of the modules I wrote for MCP continue to live in the shells being used today. Project Details: In this project, you will implement the MCP Ghost in the Shell (MCP for short) whose primary job is to launch a pool of sub-processes that will execute commands given in an input file. The MCP will read a list of commands (with arguments) to run from a file, it will then start up and run the process that will execute the command, and then schedule the processes to run concurrently in a time-sliced manner. In addition, the MCP will monitor the processes, keeping track of how the processes are using system resources. There are a total of 4 parts to the project, each building on the other. The objective is to give you a good introduction to processes, signals, signal handling, and scheduling. Each part is a complete program by itself and must be saved in a separate *.c file. Part 1: MCP Launches the Workload For part 1 of this project, you will be developing the first version of the MCP such that it can launch the workload and get all of the processes running together. Program Requirements: Your MCP v1.0 must perform the following steps: 1. Read the program workload from the specified input file (This should work just like the file mode from Project 1). Each line in the file contains the command and its arguments. 2. For each command, your MCP must launch the a separate process to run the command using some variant of the following system calls: a. fork(2): http://man7.org/linux/man-pages/man2/fork.2.html b. One of the exec() system calls (see exec): i. http://man7.org/linux/man-pages/man3/exec.3.html 3. Once all of the processes are running, your MCP must wait for each process to terminate using one of the wait family of system calls (wait/waitpid). a. wait(2): http://man7.org/linux/man-pages/man2/waitid.2.html 4. After all processes have terminated, your MCP must exit using the exit() system call. a. exit(): http://man7.org/linux/man-pages/man2/exit.2.html Fig. 1 presents the pseudocode for launching processes: for (int i = 0; i < line_number; i++) { pid_array[i] = fork(); if (pid_array[i] < 0) { //error handling } if (pid_array[i] == 0) { if (execvp (path, arg) : === -1) { error handling } // exit(-1); } } Remarks: Fig. 1: Pseudocode for launching processes To make things simpler, assume that the programs will run in the environment used to execute the MCP (i.e. the VM). While this may appear to be simple, there are many, many things that can go wrong. You should spend some time reading the entire man page for all of these system calls. Also, you will need to figure out how to read the commands and their arguments from the "workload" file and get them in the proper form to call exec. These commands can be anything that will run in the same environment as the MCP, meaning the environment in which the MCP is launched. A set of commands are provided to help to evaluate your work, but you should also construct your own. Lastely, the console output of your program has a heavy influence on your final grade so take some time to think about how your program represents what is going on at this stage. What is expected is that your output will be somewhat jumbled. This can be an excellent indicator that your code is working as intended. If you find that your messages are all synchronized then you know that something is going wrong. Part 2: MCP Controls the Workload Successful completion of Part 1 will give you a basic working MCP. However, if we just wanted to run all the workload programs at the same time, we might as well use a shell script. Rather, our ultimate goal is to schedule the programs in the workload to run in a time-shared manner. Part 2 will take the first step to allow the MCP to gain control for this purpose. This will be completed in two core steps: Step 1: Step 1 of Part 2 will implement a way for the MCP to stop all forked (MCC) child processes right before they call exec(). This gives the MCP back control before any of the workload programs are launched. The idea is to have each forked child process wait for a SIGUSR1 signal before calling exec() with its associated workload program. The sigwait() system call will be useful here. Note: that until the forked child process does the exec() call, it is running the same code as it's parent (i.e. MCP's code). Once Step 1 is working, the MCP is in a state where each child process is waiting on a SIGUSR1 signal. The first time a workload process is selected to run by the MCP scheduler, it will be started by the MCP sending the SIGUSR1 signal to it. Once a child process receives the SIGUSR1 signal, it launches the associated workload program with the exec() call. Step 2: Step 2 will implement the needed mechanism for the MCP to signal a running process to stop (using the SIGSTOP signal) and then to continue it again (using the SIGCONT signal). This is the mechanism that the MCP will use on a process after it has been started the first time. Sending a SIGSTOP signal to a running process is like running a program at the command line and typing Ctrl-Z to suspend (stop) it. Sending a suspended process a SIGCONT signal is like bringing a suspended job into the foreground at the command line. Program Requirements: Thus, in Part 2, you will take your MCP v1.0 and implement both Step 1 and 2 to create MCP v2.0. Your MCP v2.0 must do the following: 1. Immediately after each program is created using the fork() system call, the forked MCP child process waits until it receives a SIGUSR1 signal before calling exec(). 2. After all of the MCP child processes have been forked and are now waiting, the MCP parent process must send each child process a SIGUSR1 signal (at the same time) to wake them up. Each child process will then return from the sigwait() and call exec() to run the workload program. 3. After all of the workload programs have been launched and are now executing, the MCP must send each child process a SIGSTOP signal to suspend them. 4. After all of the child processes have been suspended, the MCP must send each child process a SIGCONT signal to wake them up. 5. Again, once all of the processes are back up and running, the MCP must wait for each child process to terminate. After all processes have terminated, the MCP will exit and free any allocated memory. Remarks: MCP 2.0 demonstrates that we can control the suspending and continuing of processes. You should test out that things are working properly. One way to do this is to create messages that indicate what steps the MCP is presently taking and for which child process. The console output of your program has a heavy influence on your final grade so take some time to think about how your program represents what is going on at this stage. Again, a set of programs will be provided to evaluate your work, but you should also construct your own. Handling asynchronous signaling is far more nuanced than described here. We recommend that you should spend some time studying to gain a better understanding of the system calls used in this project as well as signals and signal handling. Part 3: MCP Schedules Processes Now that the MCP can stop and continue workload processes, we want to implement a MCP scheduler that will run the processes according to some scheduling policy. The simplest policy is to equally share the processor by giving each process the same amount of time to run (e.g., 1 second). The general situation is that there is 1 workload process executing. After its time slice has completed, we want to stop that process and start up another ready process. The MCP decides which is the next workload process to run, starts the timer, and continues that process. Program Requirements: MCP 2.0 knows how to resume a process, but we need a way to have it run for only a certain amount of time. For Part 3 of this project, you will be upgrading your MCP v2.0 to include process scheduling. Note: if a child process is running, it is still the case that the MCP is running "concurrently" with it. Thus, one way to approach the problem of process scheduling is for the MCP to poll the system time to determine when the time slice is expended. This is inefficient and not as accurate as it can be. Thus, we will be using signal processing to implement a more intelligent way of process scheduling. In general, this is done by setting an alarm using the alarm(2) system call. This tells the operating system to deliver a SIGALRM signal after some specified time. In general, signal handling is done by registering a signal handling function with the operating system. When the signal is delivered, the MCP will be interrupted and the signal handling function will be executed. The following must happen after the alarm signal is received by the MCP: 1. The MCP will suspend the currently running workload process using SIGSTOP. 2. The MCP decides on the next workload process to run, and sends it a SIGCONT signal. 3. The MCP will reset the alarm, and continue with whatever else it was doing. Remarks: Your new and improved MCP v3.0 is now a working workload process scheduler./n