What happens when you type ls -l in the shell?
This time we are going to make a deeper analysis of the shell when you type a command, today we are going to check “ls -l”.
Let's go!
Environment
In this case, we are going to use:
$ uname -a
Linux vagrant-ubuntu-trusty-64 3.13.0-170-generic #220-Ubuntu SMP Thu May 9 12:40:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$But you can use any Linux distribution you want.
The command
Execute the command and see the output.
$ ls -l
total 4
drwxrwxr-x 2 vagrant vagrant 4096 Nov 27 20:29 dir_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_2
-rwxrwxr-x 1 vagrant vagrant 0 Nov 27 20:29 hsh
$This is what you are going to see when you type the command, but now are going deep.
The prompt && PS1
The first thing we are looking at is the PS1 variable, this is the variable that defines the prompt we are looking at.
In our case, if we try to see the value of the PS1 variable we are just going to get the value ‘$’ but in your terminal, you may get something like this.
username@hostname ~$ echo $PS1
\u@\H \W $
username@hostname ~$If you want to, the PS1 variable can be modified as you need to.
username@hostname ~$ PS1="$ "
$ echo $PS1
$
$If you have more interest in these variables you can read the man page of bash and read about PS1, PS2…, PS4.
One of the things we have to notice about the execution of “ls -l” is that after the command is executed the prompt is printed again.
Interactive mode
Now, we have to be aware that we are running an “interactive mode shell”, to understand this concept we can use other shells in the system, by default, most of the Linux distros will have sh available.
First, we are going to change our PS1 to differentiate the sh from our default shell.
$ PS1="> "
> And then we are going to call sh in interactive mode.
> sh
$The PS1 variable has changed again because we are now in the sh shell.
$ ls -l
total 4
drwxrwxr-x 2 vagrant vagrant 4096 Nov 27 20:29 dir_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_2
-rwxrwxr-x 1 vagrant vagrant 0 Nov 27 20:29 hsh
$In the interactive mode, we are consuming the standard input, executing the commands and printing the prompt again waiting for a new command.
Non-interactive mode
Knowing that the interactive mode uses the standard input, now we can figure out why this won't work.
> sh ls -l
sh: 0: Can't open ls
>Did you get it?
Yes! that's because the shell is reading ls has an argument, and whit our new knowledge we know we can redirect the standard output to the shell.
> echo "ls -l" | sh
total 8
drwxrwxr-x 2 vagrant vagrant 4096 Nov 27 20:29 dir_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_2
-rwxrwxr-x 1 vagrant vagrant 0 Nov 27 20:29 hsh
-rwxrw-r-- 1 vagrant vagrant 6 Nov 27 21:09 script
>Reading the command
So, now we know that, when we open a terminal, an interactive mode shell is open and is consuming the standard output.
The shell is expecting that the first argument is one of two types.
A script or a command, in this post we are only going to analyze the case of ls -l, so... let's focus on the shell waiting for a bin.
Once you press ENTER, the shell is going to break the input in tokens:
Aliases
And is going to check the aliases defined, looking for a match… yes… that means that no matter if the command exists in your system, the first thing that the shell is looking for is an alias.
Let's see a quick example.
$ alias
alias l='ls -CF'
alias la='ls -A'
$
$ alias cat='ls'
$
$ cat
total 8
drwxrwxr-x 2 vagrant vagrant 4096 Nov 27 20:29 dir_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_2
-rwxrwxr-x 1 vagrant vagrant 0 Nov 27 20:29 hsh
-rwxrw-r-- 1 vagrant vagrant 6 Nov 27 21:09 script
$ unalias cat
$
$ alias
alias l='ls -CF'
alias la='ls -A'
$
$ cat
_As you can see, even if the command cat exists, when the alias is set, it’s not going to look for a bin.
Returning with our original command of the shell, the most common scenario is that ls is not going to be found in the aliases.
Builtins
The next behavior of the shell is looking for functions that are build-in.
That's why you should always check that if you are making a bin you don't call it with the same name that a builtin in the most popular shell in your SO.
The PATH
When we are invoking a bin, we can do it in two ways.
The first one is giving the full path to the shell, in ls case it will be.
/bin/lsBut in this situation, the shell is going to check the aliases and after that is going to call the environment.
You can check you current environment variables whit this:
$ printenv
MAIL=/var/mail/vagrant
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PWD=/home/vagrant/github/hb4y/shell-lab
LANG=en_US.UTF-8
SHLVL=1
HOME=/home/vagrant
LOGNAME=vagrant
SSH_CONNECTION=10.0.2.2 5954 10.0.2.15 22
LESSOPEN=| /usr/bin/lesspipe %s
XDG_RUNTIME_DIR=/run/user/1000
LESSCLOSE=/usr/bin/lesspipe %s %s
OLDPWD=/home/vagrant/github/hb4y
$After this, the shell is going to call the env variable name PATH and search one by one of the directories looking for the bin named ls.
In this case, we already know that is going to be found in /bin.
Otherwise, the shell is going to return an error.
At this point, the shell has modified the tokens for.
And finally is going to make a call to the binary with the flags sent to the bin.
/bin/ls -lSyscalls
Here is where the magic happens, now that the shell has break the commands into tokens a fork is going to happen.
This means that a completely new process is going to be created for the command the shell just take.
For this is important to notice that the PATH must be sent to the new process, to execute directly on the “path” of the command.
The next syscall will be execve and this is the one in charge of executing the binary with the options.
All of this is happening in a child process, therefore, it is necessary to take a 3 syscall name wait.
This is telling the father process that he has to wait for the children to finish executing.
And after all this process, everything we need is set, and we see the output of the command in our STDOUT.
LS Command
The ls command is mean to “ls — list directory contents” according to the man page.
And the -l flag is going to print the results in a long listing format.
$ ls -l
total 8
drwxrwxr-x 2 vagrant vagrant 4096 Nov 27 20:29 dir_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_1
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_2
-rwxrwxr-x 1 vagrant vagrant 0 Nov 27 20:29 hsh
-rwxrw-r-- 1 vagrant vagrant 6 Nov 27 21:09 script
$ The first line that we are reading is the count of the number of files found on the directory.
If you are wondering why in this case the count is “total 8” you should check the “-a” option of ls.
After this, we are going to find:
drwxrwxr-xThe first letter indicates it is a directory (d), a link (l) or (-) for a file.
In the next spaces, you should read it in groups of 3 and those are the permissions of the owner of the file, the group and finally the rest of the users.
user group rest
--- --- ---And each of those has the permissions r (read), w (write), x (execute).
-rwxrw-r--In this case, we have a file and his owner has all permissions, the group can read and write and the rest of the users can read.
-rw-rw-r-- 1 vagrant vagrant 0 Nov 27 20:29 file_1The next number is going to tell us the number of links or files inside de element listed, in this case, a file will always be 1.
-rw-rw-r-- 1 user group 0 Nov 27 20:29 file_1Next, we find the owner of the file and his group.
-rw-rw-r-- 1 user group 1337 Nov 27 20:29 file_1The size in bytes.
-rw-rw-r-- 1 user group 1337 Nov 27 20:29 file_1Date of last modification.
-rw-rw-r-- 1 user group 1337 Nov 27 20:29 file_1And finally the name of the file.
And finally, we get the prompt again… ready for you.
We hope you enjoy this post and be amazed by all the magic that happens behind the most simple execution on your computer.
