How A Shell Works

Walking through “ls -l” step by step

Co-authored with Kaleb Abiy

The?shell?is a command-line interpreter for your operating system. It can utilize a?command-line?interface?(CLI) or a?graphical user interface?(GUI). A CLI allows users of any operating system to type in a defined set of commands (e.g.?rm?to remove files,?cat?to combine text files, etc) and have the shell execute and print out the result of that command or an error message within less than a millisecond. It is slightly different from a GUI. Instead of typing in commands (e.g. to remove files) and seeing the terminal write it out in words (e.g. list all your files and you’ll see the unwanted one gone from the list) most people just use a mouse to click to open folders and delete files. GUI and CLI both have the same purpose to interact with the operating system but their input methods are different and some developers prefer using the CLI to interact with the shell because it’s faster. Most people just use a GUI.

There are many different versions of shells amongst different operating systems. For instance, in?Unix,?we have the “sh” shell (created as the first Unix shell in 1971 by Ken Thompson, also the author of the B language and co-creator of GoLang) to the most common “BASH” shell (the 1989 Bourne Again Shell is an improved version of the sh and Bourne shell). Each shell’s general structure is similar. Later shells handle memory leaks better and each different version might handle different commands differently (e.g.?echo “\n”?prints differently in the sh and BASH shells).

Shells in general use?system calls —?a function that requests services from the Kernel; which is a core component of the operating system that has complete control over the system. It provides an essential interface between a process and the operating system. It connects the machine to the operating system.

So what actually happens?

1. Shell activates:?Once you?open a terminal?on whatever operating system you’re on, you’ll activate the CLI.
2. Prompts user:?A prompt is always printed to the screen (standard output) with a blinking cursor to cue that it’s waiting on you to type something. (e.g.?$)
3. Stores your input:?getline()?is the function used to wait for you to type something in. It will read everything you wrote until you hit enter and store that command into a space in your computer’s memory (buffer).?getline?performs a few checks and acts accordingly. If it reads that you typed?exit or hit?Ctrl-Dor?Ctrl-C, the shell will immediately terminate the CLI. If it reads that a user typed in?entering for a new line, it will go back to step 2 and re-prompt you for a new command.
4. Tokenizes your input:?If you typed in something that doesn’t cue the termination of the shell or the re-prompting for a new command to be entered, the function?strtok()is used to break your stored command into separate strings (tokens.) It knows to separate tokens by the spaces (delimiter.) Any extra spaces put in by accident when the user types in the command will be ignored. In a simple command without pipes?|, the first token is the function to be performed and the rest of the tokens are options (detailing how to perform the function) and required arguments (e.g. filenames.)
5. Checks if alias:?The shell will first check to see if this first token is the result of one of your shortcut names (aliases.) If it is, the shell will then replace the first token with a string of tokens from your alias.
6. Checks if builtin:?The shell will then check if the first token is a?built-in?program specific to the shell and not an?executable. If it is built-in, it will execute that built-in, print the result on your terminal, and skip down to step 11, repeating the process to re-prompt you for more commands.
7. Checks if executable in PATH:?If it was not built-in, the first token is first matched to see if it exists. The directories that contain all the executable functions the shell can handle are in the?PATH?environmental variable. The?PATHis similarly tokenized like your command so each directory is separated by its special delimiters (e.g.?:) and compared. If no match is found, the shell will try again and append your command’s first token onto each?PATHdirectory token and will check for a match again. In essence, the command can only be executed using its absolute full path.?Note:?After we’ve checked it exists, we also have to check that it’s an?executable?command. If it’s not an executable command, steps 8–10 are skipped, the shell will print an error message pertaining to that specific situation, and you’ll be re-prompted with step 11.
8. Creates child process:?The shell performs a system called using fork()that creates a?child process?that works simultaneously along with the?parent process. Once the?fork?is created, the rest of the program is duplicated. The goal of this child process is to execute your command. The duplication of the program is necessary because the execution (discussed in step 9) will kill the program once it’s done executing the command. The shell makes sure to handle all command executions in the child process so only the duplicate gets killed. Meanwhile, the parent should?wait()?(another system call) until the child is finished executing the process.
9. Execute or print the error message:?Once the shell detects it’s inside the child process, a system call such as?execve()will execute your command and print the result of the command onto the standard output. After the execution, the system call kills the program. Since the fork duplicated the program, the child process is the one that is killed.
10. Parent finishes waiting and wakes:?After the child process fully terminates, the parent is done waiting and will begin to perform again. Since the shell has returned a result of your command to the standard output, it has done it’s job. Some versions of shells will take this time to free the memory allocated to certain things.
11. Reprompt and repeat:?After a result of your command is printed, the shell will re-prompt you (e.g.?$) The shell continuously repeats steps 2–11, reading the typed in command until you hit?enter and printing the result of that command onto your terminal until you cue it to?exit, press?CTRL-Cor?CTRL-D, or press “x” to close your screen!

Let’s go through these steps using a real example! ls -l!

1. Shell activates:?You search for “terminal” on your computer and open it.
2. Prompts user:?Depending on your shell, you see a prompt like?$and a blinking cursor, waiting on you to type something. You type?ls -land hit?enter?because you want to list all the files in your current directory in long format (with the file permissions, the owners of the files, the file size, the modification date, etc.)
3. Stores your input:?The shell reads what you typed before the enter tab and stores?ls -l?in a memory space/buffer.
4. Tokenizes:?The shell now sees the tokens?lsand?-l.
5. Checks if alias:?The shell first checks to see if the first token?lsis an alias. In the BASH shell,?ls?is an alias to?ls — -color=autoso your new tokens are?ls?,--color=auto?, and?-l.
6. Checks if builtin:?ls?is not a builtin

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/videos:/usr/local/videos        

7.?Checks if executable in PATH:?The shell then tokenizes and searches through the?PATHenvironmental variable to try to find a matching executable. The BASH shell has the above?PATH.It’ll go through these steps with your first token: Is “ls”?/usr/local/sbin? No. Is “ls”?/usr/local/bin? No. Is “ls”?/usr/sbin? No. Is “ls”?/usr/bin? No. Is “ls”?/sbin? No. Is “ls” /bin? No. Is “ls”?/usr/videos? No. Is “ls”?/usr/local/videos? No. Since it’s none, the shell will try again by appending the first token onto the?PATHdirectories and go through these steps: Does/usr/local/sbin/ls?exist? No. Does?/usr/local/bin/ls?exist? No. Does?/usr/sbin/ls?exist? No. Does?/usr/bin/ls?exist? No. Does?/sbin/ls?exist? No. Does?/bin/ls?exist? Yes! Is this executable? Yes!

8.?Creates child process:?The shell forks and creates a child process to execute this command.

9.?Execute:?The shell executes the command via the executable and absolute path?/bin/ls. It executes according to your next tokens. When it sees a?-, the?lscommand will use whatever letter comes after it and interpret it as an?option?that adjusts it’s standard output.?-lwill be interpreted as “long format.” The result of that is printed to the terminal.

10.?Parent finishes waiting and wakes:?Upon execution, the child process is killed. Your parent process, the shell is still running.

11.?Reprompt and repeat:?The shell is done with your command and will go back to the beginning of the steps and re-prompt you with?$waiting for another command! Cool huh?