ls-l command: understanding what happens in the shell

The main objective of this article is to try to understand what happens behind our shell when we type ls -l. However, it is necessary to emphasize that before getting into the subject it is essential to clarify some very specific details that are part of this whole process. First, we must understand what the structure of the Linux file system is.

What is the structure of the file system?

In Linux, the file system is designed through a hierarchy in which the different directories and files are located and expressed in a tree form. Also, as it states [1] the basis of the file system is the "/" directory. To better illustrate the above, below you can see the most important directories in the file system, which, as mentioned above, takes the directory "/" as its starting point, and from there the other directories follow:

Considering the above, it would be worth asking, where is all this file structure located on the Linux system? Basically you will be able to realize the different environments that the Linux system has when you press in your terminal the commands env or printenv, and one of the most important environment variables is the PATH, where the directories are, and as previously mentioned, that indicate to the shell where the executable files must be looked for once the user types the command inside the terminal.

PATH=/home/charles/bin:/home/charles/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/var/lib/snapd/snap/bin

Shell PATH example

Note that the file list starts after the word "PATH=" and that each directory is separated from the other by ":". This way, when the user types a command in his terminal, the shell looks in each of the directories for the command to be executed.

Now, in the case that the command you want to execute is not inside the PATH, it's important that the user provides the complete location of that command in order to execute. For example, suppose you want to execute a file by calling myshell, the user has two options:

• ./myshell, you can use this form if you are inside the current directory where the executable file is.
• /home/Documents/Programs/myshell, (may vary depending on where your executable file is located) if you are in a different directory than where the executable file is hosted.

We can condense what has been said so far, by saying that one of the most important environment variables of the Linux system is the PATH, and in it you can find all the file structure organized in a hierarchical way. In these directories you can find all of the executable programs.

Before examining what happens with the ls -l command, it is important to note that there are 4 types of commands as shown below:

In view of the above, it should be stressed that the ls -l command is of the "executable program" type. Knowing this, we ask ourselves first, what is the "ls -l" command?

What is ls -l?

Ls is the name of a Linux executable file in the /bin directory that when executed, lists the visible files in the directory specified as argument. However, if no directory is specified, ls lists the files in the current working directory. Ls, like most other executable files or commands, has a different behavior of execution when a flag is passed to it. But you may ask: “What is a flag?”. A flag (usually recognized by its first character “-”) is an argument passed to an executable command to change the specification of its output. So, to answer the main question; ls -l is a Linux command that will list all “visible” files in current directory in long format and sorted in descending and alphabetical order. The output of the command would look something like this:

Also, there are many other flags that change the output of the executable command ls and can be combined to acquire a more specific output. Some examples are:

● ls -la: List all files including hidden files                 

● ls -l --color: List file in long format with variations of color

● ls -n: List all “visible” files in long format and with numeric user and group IDs

● ls -nS: Like above but files are sorted in descending order by size

However, sometimes having to write out a command line that does a specific thing for you can take some time because you have to remember what argument to use and this is where aliases (one of programmers’ best friends) comes into play. First, you should know that aliasing is simply accessing a data location in memory with a symbolic name. To put in simple words, an alias is like an “a.k.a”; you can execute long and hard to remember command lines with a single word or even a letter. So, if you want to shorten a command line “ls -l” into a single word or letter you can use the syntax [alias <shortened name>=” command line”]. An example of this would be:

alias l=”ls -l”

In the terminal it would look like this:

So now that we know what happens when you type “ls -l” into the shell, let's take a look at what really happens behind the scenes for this command line to be executed.

How is ls -l executed?

1. Print prompt

First, the shell prints a prompt on the standard input. Once again, you may raise the question: “what is a prompt?”. A prompt is any symbol that a shell displays when receiving a command line; usually it is a ‘$’. 

2. Read input from standard input

Secondly, once the prompt ($) is printed, the shell waits for a command to be inputted. The command should be from standard input (ie. isatty (STDIN_FILENO)). When the command is typed in, the shell allocates memory for it and saves it in an array (contiguous block of memory) to be passed to the sparse function. If nothing is inputted, the shell will just wait and do nothing unless it receives an input. Also, if nothing is inputted and the enter key is pressed, it will simply print a new line with the prompt and wait for an input again.

3. Sparse input from standard input into tokens

Thirdly, the sparse function receives the array which holds the input from the standard input terminal and separates it into more than one array if possible. This is possible because the function recognizes that any space between words means that each word is a different array; hence separating them into what are called tokens. “How is this done?” you may ask, well there is a special function called strtok that separates a string according to a specified delimiter. If the specified delimiter is space (“ “), then strtok will make the first token equal to memory data up to a space character. The same thing will happen with the word following that first space and so on until it encounters the NULL character. The first token is always the command name or an executable file. For example, the command line “ls -l” will be separated into two arrays because of the space between the words making “ls” the first argument and “-l” the second. These separated arguments are then passed to the next function which will find the path of the first argument if it is an executable command.

4. Check for built-ins then PATH

Once the shell has its inputs separated into arguments, it checks to see if the first argument is the name of a built-in alias. If that is not true, then it will check to see if it is the name of a built-in script; by the way, scripts can also be aliased. Finally, after not finding any coincidence in the built-in commands, the shell moves on to the ‘PATH’, where it checks directory by directory to see if it can find an executable command that has the same name as argument one.

Okay, now this is where it gets interesting, once the coincidence is found, whether it be in the built-in commands or in the ‘PATH’, the shell checks to see if the ‘found’ file has permissions or access for execution. If it does, then it will return the complete path of the file’s location to the next process (fork). If it does not have permissions for execution, then the shell will just print something like “Permission denied”. If no file is found, then the shell will print something like “File or directory not found”.

5. Fork and execute

Finally, once the shell has the complete path or location of the executable file and access to execute it, it will execute the command with any other argument passed to it as “flags”. But this is not as simple as it sounds. Why? Because first, the shell creates a fork (child process), which is a copy of the ‘parent’ process, making both programs run simultaneously. And here comes the question: “How does that work?”. Well, the parent process waits on a signal from the child process indicating that it has terminated. On the other hand, the child process executes the command line that was inputted and once it is done, it sends the signal to the father. If the whole process is done successfully, then the program will print an output and then go back to step one. 

If you are a curious human being and you want to know what was meant by isatty when mentioned in the previous paragraph, then it is a good thing you are still reading this post.

If you are a curious human being and you want to know what was meant by isatty when mentioned in the previous paragraph, then it is a good thing you are still reading this post.

It is important to mention, that file descriptors, or also known as file handlers, according to [2] is a small non-negative integer that identifies a file with the kernel.

The following table shows how this is shared for the three specific cases that they are:

This process within our terminal can be expressed graphically as shown below:

In view of the above, we can understand that Isatty is simply a system call which when called, returns one as a true value for its purpose. Its purpose is to check if a file descriptor refers to a terminal or not. Therefore, if a file descriptor refers to terminal, it will return 1. If this fails, it will return a number other than 1. This syscall is in the library unistd.h. And the syntax for the system call is as followed:

int isatty(int fd);

Going back to the topic at hand and bearing in mind what was said before, we cannot finish this article without mentioning that the "ls -l" command provides us with very detailed information about each of the directories and files. Let us go a little deeper into this topic.

When talking about permissions, there are three entities or bodies directly associated with the files and directories in your system. They are as followed

● Owner: User who owns the file, directory, or symbolic link, usually also the one who created the file.

● Group: Group of users who have some level of access to the file, directory, or symbolic link

● Others: Other users who may or may not have any level of access to the file/directory.

Each user has three possible types of permission, also known as “-rwx” and are described as followed:

● r: Stands for read and it is the permission type that allows users to read a file, directory, or symbolic link.

● w: Stands for write and it is the permission type that allows users to edit a file, directory, or symbolic link.

● x: Stands for execute and it is the permission type that allows users to execute a file, directory, or symbolic link. 

● -: Stands for 0 and it is a permission type that prohibits users from reading, writing or executing a file, directory or symbolic link

 Permit types are structured in an octal system and their values are relative, defined in this way:

● Read (r): 4

● Write (w): 2

● Execute (x): 1

● - (none): 0

Also, it is important to consider these values when changing the permissions of our files and directories, shortly we will see some examples.

 There are also special flags that indicate whether the permission is for a file, directory, or symbolic link. These are usually written before the permission types and are represented as followed:

● -: file

● d: directory

● l: symbolic link

Therefore, putting all this information together we can show how all of this would be represented in a diagram followed by examples:

The permissions and ownership of a file, directory or symbolic link can be assigned, changed, or removed with several commands. For more information: man chmod and man chown.

As you can see in the image, we have a series of files and directories by typing in our terminal the command “ls- l”. Most of the files have the same permissions, let us go into detail with it:

● The owner has these permissions: rw-, i.e. he can read and write but not execute; this is represented by the number: 6. Why? Read (4) + write (2) + (0) = 6.

● The group and the others have these permissions: r--, that is, they can only read the file; this is represented for both cases with the number 4. Read (4) + 0 + 0 = 4.

In this way, we can conclude that the files whose permission representation is rw-r--r--, in the octal system is 644. The first digit corresponds to the owner, the second to the group and the third to the others.

 Meanwhile, something very curious happens when we analyze the directory that is in this folder. Let us analyze it:

 ● The owner has these permissions: rwx, he has all the permissions, that is, he can read, write, and execute, represented by the number: 7. Why? Read (4) + write (2) + execute (1) = 7.

● The group and the others have these permissions: r-x, that is, they can read the file and execute it, but they cannot write on it, represented for both cases with the number 4. Read (4) + execute (1) = 5. Read (4) + 0 + execute (1) = 5.

We can then conclude that the current directory whose permission representation is d

rwxr-xr-x, in the octal system is 755.

Summarizing, first we have covered how the file system is structured in Linux, we mention important aspects of the environment variables, specifically the PATH. After that, we understood how the commands were classified and in this way, we talked about the command "ls -l" specifically, what it is, what internal processes occur in the terminal when we type the command. Finally, we analyzed the information provided by this command referring to the structure of the permissions and applying some concrete examples in the terminal.

Annotations

This article is the product of a long project that consisted in making a simple shell programmed in C. For more details go to the repository.

Authors

Andrew Kalil

Carlos Barros

REFERENCES

[1]      V. Gedris, “An Introduction to Chemical Analysis for Beginners,” in Science, vol. 1.2, 2003, pp. 1–13.

[2]      J. Schaumann, “CS631 - Advanced Programming in the UNIX Environment File I / O , File Sharing,” 2019.

[3]      K. Sharma, M. Kabir, P. Norton, N. Good, and T. Steidler-Dennison, Professional Red Hat Enterprise Linux 3. 2005.