How to use Linux Commands

It won’t be long after starting to use Linux that you ask a question and the answer begins with, “Open a terminal and…” At this point, you may be thrown into an alien environment with typed Linux commands instead of cheery-looking icons. But the terminal is not alien, it’s just different, and in this guide we’ll show you everything you need to know about how to use Linux commands.

While you can accomplish a lot in Linux using the graphical user interface, by being able to use Linux commands you’ll be able to do more complex things faster, and in many cases, automatically.

Everything you need to know about Linux Commands

Here is the GUI way of changing file permissions. You would need to do this for each file you wanted to change, and click a separate box for each permission

So what are the pros of using Linux commands?

It is consistent: The commands are generally the same on each distribution while desktops vary.

It is fast: When you know what you are doing, the shell is much faster for many tasks.

It is repeatable: Running the same task again is almost instant – no need to retrace all your steps.

There is more feedback: Error messages from the program are displayed in the terminal.

Help is available: Most Linux commands provide a summary of their options, while man pages go into more detail.

You can’t argue with the plus points, but what about the cons? Well, apart from not giving us pretty screenshots to brighten up the pages, the main disadvantage of the terminal is that you need to have an idea of the command you want to run, whereas you can browse the menus of a desktop system to find what you’re after.

On this page, we will look at the layout of the filesystem on Linux, and the various Linux commands that you can use to manipulate it. On the following pages we will cover several other aspects of administering and using a Linux system from the command line.

What goes where?

Users coming from Windows can be puzzled by the way Linux handles separate drives and partitions. Unlike the drive letter system used by Windows, Linux mounts everything in the same hierarchy.

Your root partition, containing the core system files, is mounted at /, the root of the filesystem tree. Other partitions or drives can be mounted elsewhere at what are called mount points.

For example, many distros use a separate partition for the home directory, where users’ files are kept, to make installing a new version easier. This is a completely separate partition, it can even be on a different hard drive, but it appears at /home just as though it were part of the root partition. This makes everything easier and transparent for the user.

There is another difference. Linux, in common with every operating system but MS-DOS, uses a forward slash to separate directories. The layout of directories is also different, organising files according to their type and use. The main directories in a Linux filesystem are as follows…

/ The root of the filesystem, which contains the most critical components.

/bin and /usr/bin General commands.

/sbin and /usr/sbin System administration commands for the root user.

/etc Where system configuration files are kept.

/usr Where most of the operating system lives. This is not for user files, although it was in the dim and distant past of Unix and the name has stuck.

/lib and /usr/lib The home of system libraries.

/var Where system programs store their data. Web servers keep their pages in /var/www and log files live in /var/log.

/home Where users’ data is kept. Each user has a home directory, generally at /home/username.

Moving around

Now that we know where everything is, let’s take a look at the common Linux commands used to navigate the filesystem. Before going anywhere, it helps to know where we are, which is what pwd does. Many Unix commands are short, often two to three characters; in this case, pwd is print working directory – it tells you where you are.

Many distros set up the terminal prompt to display the current directory, so you may not need this command often. Moving around is done with the cd (change directory) command. Run it with no arguments to return to your home directory.

Otherwise it takes one argument, the directory to change to. Directory paths can be either relative or absolute. An absolute path starts with / so cd /usr/local goes to the same place wherever you are starting from. A relative path starts at the current directory, so cd Documents goes to the Documents sub-directory of wherever you are, and gives an error if it is not there.

That sounds less than useful if you can only descend into sub-directories, but there are a couple of special directory names you can use. To go up a directory use cd .. – a single dot is the current directory. There is also a shortcut for your home directory: .

Let’s say you have directories called Photos and Music in your home directory and you are currently in Photos, either of these commands will move into Music:

cd ../Music

cd /Music

You can tell where you are with pwd, but how do you know what is in the current directory? With the ls command. Used on its own, it gives a list of files and directories in the current directory.

Add a path and it lists the contents of that directory. If you want to know more about the files, use the -l (–long) option, which tells you the size and date of the file, along with information about ownership and permissions, which we will look at later.

With your permission

Every file object (that is files, directories and device nodes in / dev) has a set of permissions associated with it, as shown in the screenshot of the output from ls -l. These are normally in the form rwxrwxrwx and shown by ls, or the numeric equivalents. The three letters stand for read, write and execute, and are shown three times for the file’s owner, the group it belongs to, and other users.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands
If you need help with a command, ask the command for it. Most commands give a brief summary of their options when run with –help

For example, rw-r–r– is a common set of permissions for files; it means the owner of the file can read from or write to it, all other users can only read it. Program files usually appear as rwxr-xr-x, the same permissions as before but also all users can execute the file.

If a program does not have execute permissions, you cannot run it. This is sometimes the case with system programs owned by the root user and only executable by root.

When applied to directories, the meanings are slightly different. Read means the same, but write refers to the ability to write into the directory, such as creating files. It also means that you can delete a file in a directory you have write permissions for, even if you don’t have write permissions on the file – it is the directory you are modifying.

You can’t execute a directory, so that permission flag is re-purposed to allow you to access the contents of the directory, which is slightly different from read, which only allows you to list the contents (that is, read the directory).

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

File permissions are displayed by using the -l option with ls and modified with chmod, which can be used in a number of different ways, best shown by example:

chmod u+w somefile

chmod o-r somefile

chmod a+x somefile

chmod u=rw somefile

chmod u=rwx,go=rx somefile

chmod 755 somefile

The string following chmod has three parts: the targets, the operation and the permissions. So the first example adds write permission for the user. The next one removes read permission for other users, while the third adds execute permission for all users. + and – add and remove permissions to whatever was already set, while = sets the given permissions and removes the others, so the next example sets read and write for the file’s owner and removes execute if it was previously set.

The next command shows how we can combine several settings into one, setting read, write and execute for the owner, and read and execute for the group and others. The final command does exactly the same, but using the numerical settings. Each permission has a number: 4 is read, 2 is write, and 1 is execute.

Add them together for each of the user types and you have a three-digit number that sets the permissions exactly (there is no equivalent to + or – with this method).

One of the biggest changes that catches Windows users moving to Linux is the way that software is installed. Instead of downloading an executable file from some website or other, running it and hoping it doesn’t clobber your existing library files (DLLs) or install some dubious adware or malware, Linux distributions maintain repositories of software, which are all packaged up for that distro and tested for compatibility with the rest of the distro.

On this page of our guide to everything you need to know about Linux Commands, we will look at how this is done by distros that use the Advanced Packaging Tool (apt) software management system, as developed by Debian and used by distros from Ubuntu to Raspbian on the Raspberry Pi.

Repositories

A repository is a collection of software packages for a distro. Each major release of a distro will have its own repositories, and the packages will have been built for and tested with that release, but a repository is more than a collection of files.

Each repo (as they are usually called) is indexed, making it easy to find what you want. It can also be quickly checked for updates for your package manager without any need to visit websites to check for updates, or the need for software to ‘phone home’ to check.

More importantly, each package in a repo is signed with the repository’s GPG (encryption) key, which is checked when installing packages. This means you can trust the software installed from there to be what it says it is, and not some infected trojan that’s been uploaded maliciously.

A repository also makes dependency handling simple. A dependency is a program that the program you want to install needs to run, such as a library. Instead of bundling everything in the package and ending up with multiple copies of the same library on your computer (which is what Windows does), a package simply lists its dependencies so that your package manager can check whether they are already installed, and grab them from the repo if not.

In addition to the default repositories provided by the distro, there are several third-party ones that can be added to your package manager. These are not guaranteed to be tested to the same standards as the official repos, but many of them are very good, and if you stick to the popularly recommended repos for your distro, you won’t go far wrong.

Ubuntu has also introduced the concept of the PPA, or Personal Package Archive, which are small repositories for individual projects. These may each be added individually to your package manager, but be careful about adding any untrusted sources.

Package management

We have used the term ‘package manager’ a few times now but what is it? Basically, this is a program that enables you to install, update and remove software, including taking care of dependencies. It also enables you to search for programs of interest, as well as performing other functions.

All distros will have command line package management tools. You can access them either by using your system’s search and looking for terminal or using [Ctrl]+[Alt]+[T] in Linux desktops such as Unity, Gnome or Xfce, even if they also provide a fancy graphical front end. The main Linux commands are:

apt-get: Installs, upgrades and uninstalls packages.

apt-cache: This works with the repository index files, such as searching for packages.

add-apt-repository: Adds extra repositories to the system.

dpkg: A lower level package manipulation command.

These commands generally require root (superuser) access, so should be run at the root user or with sudo – we will stick with the sudo approach here. We’ve already mentioned that repos are indexed, so the first thing to do is update your index files to match the current contents of the repositories with:

sudo apt-get update

Then you probably want to make sure that your system is up to date:

sudo apt-get upgrade

This will list the packages it wants to install, tell you how much space it needs for the download, and then get on with it when you tell it to. When you want to install some new software, unless you have been told the exact name to install, you may want to search for it first, like this:

apt-cache search gimp

This will spit out a long list of packages, because it searches both name and description, and lists anything mentioning gimp, and there are a lot of them. To search only the names, use the -n or –names-only option:

apt-cache search -n gimp

This often gives a more manageable output, but still a lot in this case, perhaps too much to fit in your terminal window. The solution to this is to pipe the output from this command to the program less:

apt-cache search -n gimp | less

The less command is a pager – it lets you read text page by page and scroll through it. It can be used with any program that generates lots of terminal output to make it easier to read (see the ‘Package management’ walkthrough below for more details). Once you have found the package you want, installation is as simple as:

sudo apt-get install gimp

You can install multiple programs by giving them all to aptget at once:

sudo apt-get install program1 program2…

Not every program you try will be what you want, so you can tidy up your hard drive by uninstalling it with:

sudo apt-get remove program1

Or you can use:

sudo apt-get purge program1

Both commands remove the program, but remove leaves its configuration files in place while purge deletes those, too.

There are a number of extra options you can use with aptget, the man page lists them all (type man apt-get in the terminal), but one of the most useful is –dry-run. This has apt-get show you what it would do without actually doing it, a useful chance to check that you are giving the right command. Remember, computers do what you tell them to, not what you want them to do!

Finally, you don’t normally need to use dpkg, but it is useful for listing everything you have installed with dpkg -L.

Package management

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

1. Install

Using apt-get install will check the dependencies of the packages you want and install any that are needed. Adding –dry-run to apt-get install enables you to see what would be done, without actually writing anything to your hard drive. If you are happy, run the command again without –dry-run.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

2. Search

Use apt-cache search to find what’s available. The –names-only option can give a more manageable set of results if you know the program’s name. Otherwise let apt-cache search go through the descriptions, too, and view the results in less. You don’t need to use sudo as search doesn’t write to your drive.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

3. Update

Run apt-get update to update all your package lists, followed by apt-get upgrade to update all your installed software to the latest versions. In our case, it’s well overdue. Then apt will show you what needs to be updated, and how much needs to be downloaded, before asking whether you want to proceed.

We store data in a variety of ways. On our hard disks, on optical discs (CDs and DVDs) and on removable devices such as USB sticks and external hard drives. In Windows, each would be given a drive letter, starting with C (A and B were reserved for floppy drives.

You may see words such as partition and filesystem bandied about, so let’s clarify them before we go any further. A disk is divided into partitions, separate physical areas each used to store data. Hard disks always have partitions, even when it appears you are using the whole disk, as in a Windows C drive, but there is a distinction.

Removable devices such as USB sticks may or may not have partitions – most do. The only types of media that are never partitioned are floppy and optical discs.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

Each partition contains one filesystem. A filesystem is a means of storing files, directories and their associated data – it organises the ones and zeros stored in the device’s memory into useful objects. It is important to understand the differences between these terms before we continue, even though they are often used interchangeably.

Types of filesystem

There are several different filesystems supported on Linux. A native filesystem is one that is designed for use with Linux or other Unix-like systems. The most commonplace of these is ext4 (along with its predecessors ext3 and ext2 that are still in use).

There are other native filesystems such as reiserfs, BTRFS and XFS. Why do we have these variants? Because not every system has the same needs; a mail server needs to store many thousands of small files, a database server works with a smaller number of much larger files, and requires access to random points in those files.

A laptop’s filesystem needs to be resilient to unplanned shutdowns, such as when a battery fails. So we end up with different filesystems for different purposes, although for general use ext4 does a fine job. In fact, the enhancements to ext4 over ext3 mean it works well in a wider variety of situations and is a sensible choice for a standard desktop or laptop system.

Non-native filesystems are those developed for use with other platforms, but supported in Linux. The most notable of these is FAT, the default Windows filesystem for many years and still the standard choice for USB sticks, camera memory cards and other removable media.

Also supported to a certain extent are the newer exFAT and NTFS filesystems from Microsoft and Apple’s HFS and HFS+ filesystems. The collection of supported filesystems is rounded out by the likes of ISO9660 and UDF, used by CD and DVD media. Most of the time, you don’t need to worry about the type of filesystem used on a media device, just plug it in and the Linux kernel recognises the filesystem in use.

Mounting a filesystem

Making the contents of a filesystem available to the OS and user is called mounting. In Linux, each filesystem is mounted at a particular point within the filesystem hierarchy, known as the mount point.

The root filesystem, the one containing the core OS components, is mounted at /, the root of the filesystem tree. Many distros use a separate partition and filesystem for the user’s home directories; it separates the OS from the user’s files so you can update, re-install or switch the OS without affecting your personal files.

Users’ home directories reside at /home/username, so a separate home filesystem would be mounted at /home, the mount point for the home filesystem. This directory must exist on the root filesystem but is usually empty – as soon as the home filesystem is mounted, its contents appear at /home.

Anything that was there before is no longer visible, but is still there and will reappear when home is unmounted. There are three main ways to mount a filesystem: at boot, manually or automatically. The system filesystems, such as / and /home, are mounted automatically at boot, using information stored in the filesystem table /etc/fstab – remember, /etc is where system settings are stored. Here is a typical fstab entry for mounting the a partition:

/dev/sda2 /home ext4 defaults 0 2

Each filesystem’s entry is on a single line, with six fields:

1. Device name: Linux disks are named sda, sdb and so on, with the partition’s number added to the disk’s device, so /dev/sda2 is the second partition on the first disk.

2. Mount point: Where the filesystem is to be mounted – it must exist.

3. Filesystem type: It is possible to use auto here and let the kernel figure it out for you, but with a fixed disk it makes sense to be specific.

4. Mount options: The keyword defaults means use whatever is standard for that filesystem. Specific settings can be given here, separated by commas.

5. Dump: Used for a particular type of backup, normally left at zero.

6. This is used by fsck to set the order in which filesystems are checked when booting. Set to 1 for the root partition and either 2 or 0 for others – 0 means do not check.

Mounting manually is done with the mount command:

sudo mount /dev/sda4 /mnt/backup -t ext4

The first two options are the device and mount point, -t specifies the filesystem type; if you omit this, auto is used. You can also use -o to add other options, such as: sudo mount /dev/sda4 /mnt/backup -t ext4 -o noatime

The noatime option is often used in fstab to improve performance; it reduces the number of writes to the disk by only recording when files are modified, not simply read. If a device is listed in fstab, you need only give either the device or mount point, and mount will take the rest from fstab:

sudo mount /dev/sda3

or

sudo mount /mnt/music

We have used sudo in all of these examples because mounting is generally only permitted by the root user. This behaviour can be changed for mounts listed in fstab by adding user or users to the options:

/dev/sda3 /mnt/music ext4 users,noatime 0 0

Now any user can mount this with:

mount /mnt/music

The difference between user and users is that with the former, only the user who mounted the filesystem can unmount it. Speaking of unmounting, that is done with the umount command, followed by either the mount point or device.

This command also accepts multiple paths to unmount several at once:

sudo umount /mnt/music /mnt/photos /dev/sda5

Creating partitions and filesystems

Partitioning a drive is normally done by a distro’s installer, but there are graphical and command-line programs to work with partitions and filesystems. At the command line, fdisk and gdisk are the standard Linux commands, the former working with old style MS-DOS partition tables, while gdisk works with the newer, more reliable and flexible GPT partitions.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands
Here we’re creating an ext4 filesystem using the standard defaults. Get the partition right, because mkfs doesn’t ask whether you are sure

They work in much the same way. See the step by step guide for more information. Once you have created partitions, you need to make filesystems on them, which is done with the mkfs range of commands. These all have names of the form mkfs.FSTYPE, such as: sudo mkfs.ext4 /dev/sdb1

There are different options you can use for each filesystem type, depending on that particular filesystem’s features, so you will have to consult the man page for that flavour of mkfs to see what you need, although they all have sensible defaults.

There is also a generic mkfs command that takes a -t option to specify the filesystem type, but the versions with the filesystem added save having to remember the exact name. The main ones you will need are mkfs.ext4, mkfs.ext3and mkfs.vfat, with the last one being used for OS-independent media, such as flash drives.

Step-by-step: Partition a disk

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

1. sudo gdisk /dev/sda

You need to use fdisk for hard drives partitioned with the old MBR system (gdisk will warn you if this is needed). You must give the name of the drive, then press [m] in fdisk or [?] in gdisk for a list of all the main commands.

Any changes you make here stay in memory and are only written to the disk when you press [w].

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

2. Create some partitions

Press [n] to create a partition – you normally can press [Enter] to accept the defaults for partition number and start point. The default end point is the end of the available space – if you want to use a specific size, enter + followed by the size you want, for example+50G.

For anything but a Linux partition, you should give its type; entering L lists them.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

3. Show the partition layout

Press [p] to list the current partition layout, either the existing layout from a freshly loaded disk or the layout you have created in fdisk/gdisk. Make sure everything is as you want before you use [w] to commit to the disk, as there is no going back from [w].

Until then, you can make further changes or press [q] to abort without writing anything to disk.

Most people reading this are probably pretty familiar with creating, sending or receiving ZIP files. Zip takes a collection of files and stores them in a ZIP archive file, compressing the data in the process. As well as storing the contents of the files, it also stores all their metadata, which is extra information associated with an object.

In the case of files, it includes the modification times, their owners and permissions, and, of course, the name of each file. When you unzip an archive, all of this information is extracted, which recreates the original set of files exactly as they were. This is pretty handy.

Archives have several uses, but the most popular include bundling up a set of files for download – a single file is easier to handle and the compression makes it faster to download – and creating backups. As Zip has been around for ages, everyone can use it and all OSes can handle it.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands

Nonetheless, Zip does have a number of drawbacks. The main issue is that its compression is poor by modern standards. Over the past 25 years, compression technology has moved on and even though there have been improvements to Zip, there are better alternatives available. Another disadvantage of Zip is that it is intended for archiving to a file, whereas sometimes we want to send the data to another device or service.

The standard archival program for Unix-like operating systems is Tar, so called because Tar was originally used to store backups on tape drives (Tape ARchive).

 

It works in a different way to Zip because it sends all the archived data to its standard output and doesn’t compress the data by default. This is because many tape drives already had hardware compression built in. The lack of compression code may seem like a disadvantage, but it’s actually a convenience.

As Tar is able to pipe its data via an external compression program, it can use any compressor it likes – even one that wasn’t in existence when Tar was developed.

Compression programs work on one file or stream of data and produce one compressed file or stream, so this splits the job into two parts: archival and compression. While this may seem more complex, Tar is capable of handling the details itself. Let’s say we have a directory that is called foo. We want to create an archive of it, which is often referred to as a tarball. We can use one of these options:

tar cf foo.tar foo

tar czf foo.tar.gz foo

tar cjf foo.tar.bz2 foo

tar cJf foo.tar.xz foo

The c option tells the Tar program that we are creating an archive, while f tells it that we are storing the archive in a file using the given name. Therefore, the first command creates an uncompressed archive that is called foo.tar.

The subsequent Linux commands add an extra option that tells Tar which particular type of compression to use: z uses gzip compression, j uses bzip2 compression and J uses xz compression. (Watch the capitalisation!)

There are also long versions of these arguments that make the commands more readable, but most of us are lazy and use the version that is shorter to type. However, we could also have used this command line if we wanted to:

tar –create –gzip –file foo.tar.gz foo

The file extension is not required, but it’s a convention that makes it easier for people to see what type of archive it is – the system itself needs no such help as it can work all this out for itself. Unpacking an archive is simply a matter of replacing cwith x, or –create with –extract.

However, you don’t need to give the compression type, as Tar figures it out:

tar xf foo.tar.gz

Another option you may want to add is v or –verbose, to show you what Tar is doing. If you have been given a tarball, you may want to see what is inside it without unpacking it. If you have created an archive, particularly a backup, you may want to check it’s correct before relying on it.

The test option checks the integrity and lists the contents of the archive.

tar tvf foo.tar.gz

Those are the main Tar options, but the program has many more, such as A or –concatenate to add files to an existing archive instead of creating a new one.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands
Most environments can view the archive contents. Shown above is the result of double-clicking a tarball in LXDE (the preferred desktop for Raspberry Pi)

Future proofing

We mentioned that Tar can handle any new compression format that comes along because it passes compression to another program. There are command line options to do this automatically for gzip, bzip2 and xz, but what if someone comes up with a new compressor? Say something like sdc – super-duper compressor?

You could create an uncompressed tarball and then use sdc to compress it, but that’s wasteful and slow, so instead use a pipe:

tar c foo | sdc >foo.tar.sdc

unsdc foo.tar.sdc | tar xv

Here, we use only the –create option with Tar. The lack of destination causes Tar to send the archive data to standard output, which is then piped to the sdc compressor program.

The second command reverses the process, decompressing the archive and sending it to Tar for extraction.

There are thousands of Linux commands, from the commonplace to the arcane, but you need only a handful of key commands to get started in Linux.

Here we will look at some of the core workhorse commands, giving a brief description of what each one is for. As always, the man pages give far more detail on how to use them.

Many of these produce more output than can fit in your terminal display, so consider piping them through Less (as described in our introduction to apt-get).

File handling

Central to any terminal activity is working with files, creating, removing, listing and otherwise examining them. Here are the main commands for this.

  • ls: lists the contents of the current or given directory.
  • ls -l: as for ls, but gives more information about each item. Add human-readable file sizes with -h
    ls -lh MyPhotos
  • rm: deletes a file. Use the -i option for confirmation before each removal or -f to blitz everything. With -rit deletes directories and their contents too.
  • rmdir: deletes a directory, which must be empty.
  • df: Shows free disk space on all filesystems or just those given on the command line.
  • du: Shows the amount of space used by individual files or directories.
    df -h /home
    du -sh /home/user/*
  • file: identifies the type of a file. Unlike Windows, which uses the file name extension, this command looks inside the file to see what it really contains.
  • find: searches the current or given directory for files matching certain criteria. For example you could find all LibreOffice spreadsheets with
    find Documents -name ‘*.ods’
  • locate: This also looks for files but using a much faster system. The locate database is automatically rebuilt each day by updatedb, and locate then searches this. It’s fast, but doesn’t know about very recent changes.

Text handling

How to master the Linux terminal with core commands

How to master the Linux terminal with core commands
Stay out of the Vi vs Emacs arguments by using the lightweight Nano to edit config files. The instructions at the bottom of the window make it almost impossible to get lost.

Text files are all around us, from emails to configuration files, and there are plenty of commands to deal with them. If you want to edit a text file, there are a number of choices, with the two big ones beingEmacs and Vi. Both are overkill if you just want to tweak a configuration file; in this instance, try nanoinstead:

nano -w somefile.txt

The -w option turns off word wrapping, which you certainly don’t want when editing configuration files. The status bar at the bottom shows the main commands – for example, press Ctrl + X to save your file and exit.

This assumes you know which file you want, but what if you know what you’re looking for but not the name of the file? In that case, use grep. This searches text files for a string or regular expression.

grep sometext *.txt

This will search all .txt files in the current directory and show any lines containing the matching text from each file, along with the name of the file. You can even search an entire directory hierarchy with -r (or –recursive):

grep -r -I sometext somedir

Be careful when you’re searching large directory trees: it can be slow and return strange results from any non-text files it searches. The -I option tells grep to skip such binary files.

Text is also the preferred way of passing data between many programs, using the pipes we looked at previously. Sometimes you want to pass data straight from one program to the next, but other times you may want to modify it first. You could send the text to a file, edit it and then send the new file to the next program, or you could pass it though a pipe and modify it on-the-fly.

Nano edits files interactively, grep searches them automatically, so we just need a program to edit automatically; it’s called sed (Stream EDitor). Sed takes a stream of text, either from a file or a pipe, and makes the changes you tell it to. The most common uses are deletion and substitution. Normally, sedsends its output to stdout, but the -i option modifies files in place:

sed -i ‘s/oldtext/newtext/g’ somefile.txt
sed -i ‘/oldtext/d’ somefile.txt

The second example deletes all lines containing “oldtext”.

Another useful program is awk, which can be used to print specific items from a text file or stream.

awk ‘{print $1}’ somefile.txt
cat *.txt | awk ‘/Hello/ {print $2}’

The first example prints the first word from each line of the file. The second takes the contents of all files ending in .txt, filters the lines starting with “Hello” (the string between the slashes is a pattern to match) and then prints the second word from each matching line.

Networking

We normally picture big graphical programs like Chromium and Thunderbird when we think of networked software, but there are many command line programs for setting up, testing and using your network or internet connection.

  • Ping sends a small packet to a remote server and times the response, which is useful if you want to check whether a site is available, or if your network is working.
    ping -c 5 http://www.google.com
  • wget: downloads files. The only argument it needs is a URL, although it has a huge range of options that you will not normally need.
  • hostname: shows your computer’s host name, or its IP address with -i.
  • lynx: a text mode web browser. While not as intuitive as Chromium or Firefox, it is worth knowing about in case you ever suffer graphics problems.
How to master the Linux terminal with core commands

How to master the Linux terminal with core commands
Lynx is a web browser than runs in a terminal window. It may not be the best choice for watching YouTube, but it’s fast and always there, even if you have no desktop available.

Commands, options and arguments

You’ll often see references to command arguments and options, but what exactly are they?

Options and arguments are the things that tell a program what to do. Put simply, arguments tell a command what to do, while options tell it how to do it – although the lines can get a little blurred.

Take the ls command as an example: this lists the contents of a directory. With no options or arguments, it lists the current directory using the standard format:

ls
Desktop Downloads Music Public Videos
Documents examples.desktop Pictures Templates

If you want to list a different directory, give that as an argument:

ls Pictures

or

ls Desktop Downloads

Arguments are given as just the names you want listed, but options are marked as such by starting with a dash. The standard convention among GNU programs, and used by most others, it to have long and short options.

A short option is a single dash and one letter, such as ls -l, which tells ls to list in its long format, giving more detail about each file. The long options are two dashes followed by a word, such as ls –reverse, which lists entries in reverse order, as is pretty apparent from the name. The form ls -r does the same thing but it is not so obvious what it does.

Many options, like this one, have long and short versions, but there are only 26 letters in the alphabet, so less popular options are often available only in the long version. The short options are easier to type but the long ones are more understandable. Compare:

ls -l –reverse –time

with

ls -l -r -t

or even

ls -lrt

Each gives a long listing in reverse time/date order. Notice how multiple short options can be combined with a single dash.

While we’re talking about ls, this is a good time to mention so-called ‘hidden’ files. In Linux, any files or directories beginning with a dot are considered hidden and do not show up in the output from ls or in most file managers by default.

These are usually configuration files that would only clutter up your display – but if you want to see them, simply add the -A option to ls.

Linux core commands

Linux core commands
By piping the output from du through sort, and adding extra options to both commands, we can see which directories use the most space.

Getting help

The command line may appear a little unfriendly at first, but there’s plenty of help available if you know where to look. Most commands have a –help option that tells you what the options are. The man and info pages are the main source of information about anything. To learn all the options for a program and what they do, run the following:

man progname

The man pages are divided into numbered sections. The ones that are most applicable to using the system are:

  • 1 User commands
  • 5 File formats and conventions
  • 8 System Administration tools

If you don’t specify the number, man will pick the first available, which usually works.

But man pages are not limited to programs; they also cover configuration files. As an example, passwords are managed by the passwd command, and information is stored in the /etc/passwd file, so you could use:

man passwd

man 1 passwd

man 5 passwd

The first two would tell you about the passwd command while the third would explain the content and format of the /etc/passwd file.

Man pages have all the information on a single page but info pages are a collection of hypertext linked pages contained in a single file.

How to master the Linux terminal with core commands

How to master the Linux terminal with core commands
Info pages, when available, provide more detail than man pages, but you will need to read info’s own info page first to learn to use it.

They often provide more detail but aren’t intuitive to read – try info info to see how to use them. It’s often easier to use a search engine to find the online version of info pages, which contain the same information in the more familiar HTML format.

Linux is a multi-user operating system, even if you are the only person using your computer. The most basic of systems has two users: you and the superuser, also called root. Every file or directory is owned by a user and has settings, called permissions, that specify who can read or write to it.

This safeguards your files from being overwritten by another user, or possibly even read by them if so set. It also safeguards system files because they are owned by root and can only be changed by root. This includes writing to system directories, so only the root user can install new software there.

So how do you install software? See the ‘Becoming root’ box opposite for the answer. One word of caution when using data on multiple computers, such as with an external hard disk. While you may see your username as a name, say johnny99, the computer sees and stores it as a number, a UID or user ID.

During installation, your distro will have created a root user, who always has a UID of 0, and a normal user. Most distros start at 1000 for the first general user, but some start at 500. The point is, it’s that number stored on the disk as the owner of a file, so the same username may not own the file when you move the disk to another computer.

Create a user

Every user has a home directory. This is usually /home/username but it can actually be anywhere – the user created to run a web server will have a home directory somewhere like /var/www. In addition to users, Linux also has groups.

A group is basically a collection of users. For example if you have a USB scanner on your computer, you often need to be a member of the scanner group to be able to use it. Now that we understand usernames, groups, UIDs and home directories, we can create a user:

sudo useradd -m -c “John Smith” -g users -G scanner,audio john

We use sudo here because only the root user can create users. The -m option creates a home directory at /home/john, -c specifies a comment to store for the user, which is usually the user’s full name, -g sets the primary group of the user, while -G adds secondary groups.

Finally, we give the username. Not all of these options are necessary – if you omit -g, a default group will be used. Some distros use a single group called users for all non-system users, while others create a separate group for each user. The groupaddcommand works in a similar way, as do both of their counterparts – userdel and groupdel.

Add a password

We have created a user but he cannot log in yet because we haven’t given him a password.

sudo passwd john

will ask you for the password twice. The passwd command can also be used to change the password of an existing account. If you run it without sudo or a username, however, it will change your own user’s password, because only root can set passwords for anyone else.

It is considered good practice to change passwords regularly, and you can enforce this by using passwd:

sudo passwd –maxdays 60 -warndays 7 john

This password will become invalid after 60 days and john will be warned about the expiry a week before. The usual rules about passwords – make them long and mixed case, preferably with numbers – apply here, and doubly so to the root password, the key to the kingdom.

User details are stored in /etc/passwd, which may be edited should you wish to change them. However, making a mistake could prevent you from logging in, so use vipw to edit it. This loads a copy of /etc/passwd into your preferred editor (as defined in $EDITOR) and checks its validity when you save it, before replacing the existing file. The format of /etc/passwd is explained fully with:

man 5 passwd

Transfer ownership

If you want to change the owner of a file, you need chown:

chown john somefile

chown john:users someotherfile

chown john: someotherfile

chown -R john: somedir

The first makes john the owner of a single file. The second command also changes the group. If you do not give a group after the colon, as in the third example, the group is changed to the user’s default group. When applied to a directory, the -Roption also changes all files and sub-directories in that directory.

You can change just the group with chgrp. These commands must be run as root. Changing file permissions is done with chmod.

Becoming root

Almost all of the user management commands require root access. The superuser, often referred to as root, can do anything anywhere, regardless of any permissions. So how does a humble user do anything that requires root privileges? There are two ways of doing this.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands
With sudo, each user can have all, some or no administrative rights. This is a typical definition that allows members of the admin group to do anything

The traditional way is to use the su (for switch user) command, which allows you to become another user, provided you know their password. Run su in a terminal (it switches to root if you don’t specify a user), and it will ask for the root password. Once you have entered this, you are root in that terminal session, and can do anything you like – muhahaha!

Of course, this is a bit risky, especially if you don’t log straight out after doing what you need. It also means that you have to give the root password, and therefore full access, to any user who needs to run a root command – so sudo was invented. Sudo is used to run individual commands as root (or another user) but does not require you to know the root password. Instead, the root user must have set things up to allow you to run some or all commands as root, so it asks for your password. The normal installation of most distros allows the first user created to have sudo rights to everything, so you can install software, or do anything else, without ever having to log in as root, or spread the root password.

In fact, some distros have no root password by default, and sudo is the only way to run administrative commands. Sudo can be used to give individual users the right to execute only specific commands, plus it records every command it runs in the system log, along with who ran it, giving a far more secure and accountable system.

RTFM has long been considered the battle cry of the supposed Linux experts, and is claimed to scare new users away. If you haven’t come across it before, it stands for something like ‘Read the f***ing manual’. It is perhaps easy to appreciate the frustration some individuals feel when asked a question for the umpteenth time when the information is clearly covered in the manual.

However, it’s only possible for a user to read a program’s documentation if they know where to find it. Happily, there are some sources of help within Linux, so let’s look at each of them.

Before you even look for the manual, remember that many programs have built-in help. Run the command from a terminal with the –help option to see a synopsis of its options. This applies to GUI programs, as well as shell commands. For example:

firefox –help

If you need more detail, then it may be time to RTFM, which on Linux systems usually means the man page. Man pages document just about everything on your system and are viewed by keying in the man command. If you want to know how man itself works, the classic recursive example is to open a terminal and run:

man man

A man page is a single page containing the reference documentation for its topic. The man command renders this document as readable text and displays it in your default pager, usually less. This means you use the same keyboard controls as less for navigating the page: cursor keys to scroll up and down, [Spacebar] to page down, and so on. Some man pages can be very long, so try man bash to enable you to search. In less, press [/] to start searching (or [?] if you want to search backwards), followed by your search term.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands
This is the man page for ls, and it helpfully lists the various options that you can use in alphabetical order

Then use [n] to jump to the next match or [N] for the previous one. The man pages are divided into sections:

1. User commands

2. System calls

3. C library functions

4. Devices and special files

5. File formats and conventions

6. Games et al

7. Miscellany

8. System administration tools and daemons

As a normal user, you would normally only use sections 1, 5 and 8 (and possibly 6). If you use the section number with the man command, it will only look in that section, otherwise it will show the first match it finds. This is necessary because you can have more than one page with the same name.

The passwd Linux command is used to set user passwords, which are stored in the file /etc/passwd. Try:

man passwd

man 1 passwd

man 5 passwd

The first two document the passwd command from section 1, while the third shows the man page for the passwd file. This is one of the strengths of the man page system – it documents everything: commands, configuration files, library functions and more. It’s not limited to specific commands or files, and section 7 contains man pages for all sorts of things.

Ever wondered how symbolic links work, or what happens when you turn on your computer? Try:

man 7 symlink

man 7 boot

Quick help

There is more to man than a bunch of formatted text pages and a program to display them in a pager. Man maintains a searchable database of pages, which is automatically updated by Cron, and has some other programs for working with it.

Everything you need to know about Linux Commands

Everything you need to know about Linux Commands
The ls info page goes into more detail and groups options according to their function. Info pages generally provide more detail than man pages

Each man page has a NAME section, which includes a short description of the page’s topic. The whatis command gives the description – it tells you what the program (or file) is, without going into details of options. Here is the classic geeky, recursive example:

whatis whatis

whatis (1) – search the whatis database for complete words

This is a faster way to see what commands do, especially if you want to check more than one:

whatis grep sed symlink

The whatis command searches only the name, and only matches on whole words, so it assumes you know the name of the command and just want to know what it does. For a more wide-ranging search, use apropos, which does a similar job but searches the descriptions as well and returns all matches – compare these two commands:

whatis png

apropos png

There is another form of documentation that’s favoured by the GNU project: info pages. While a man page is basically one very long text file with a bit of formatting markup, an info document contains a tree of linked pages in a single file. It’s more like HTML than plain text, but is designed for reading in a text console and all the ‘pages’ are contained in a single file.

Unsurprisingly, the Linux command used to read info pages is this:

info info

This time, the self-referencing is not gratuitous. Man pages are usually quite intuitive to navigate – it’s just like reading any other text in a pager. Info uses a different set of key commands, so reading its own info page is a good place to start. You move around individual pages as you would normally, but if you press [Enter] when the cursor is on a link (noted by an asterisk), you descend into that node, getting further information on the topic at hand.

To go back up to the parent node, press [u]. You can also navigate within a level of the documentation tree with [n] and [p], which take you to the next and previous nodes. If you have ever looked at any GNU documentation online you will recognise this layout. It’s simpler than HTML, with the navigation commands generally moving around the same level of the tree or up and down one level.

You may be asking yourself what the point of this structure is. Well, if you’ve ever tried to find information in a long man page, such as man bash or man mplayer, you’ll know how frustrating and time-consuming the ‘everything on one page’ approach can be.

Info documents are divided into sections and chapters, enabling clearer and more succinct presentation. The majority of GNU programs have fairly brief man pages, but have more detail in their info pages. Splitting the document up into pages alters the way that searching is carried out.

Press [s] followed by a search string and then [Enter]. Info will jump to the next occurrence of the string, even if it’s in a different node. Continue to press [s] then [Enter], with no search string, to jump to subsequent occurrences of the same string. Those keys, plus [q] to quit, should enable you to navigate info pages with ease. You might be wondering why we’re not using HTML.

The main reasons are that info pre-dates HTML, and that info documents are contained within a single file. Conceptually, they are similar – so much so that info2html can be used to convert an info document to a series of HTML pages.

Printing manuals

There may be times you want a hard copy. As man pages are stored in a markup format and converted for display by the man program, you can use -t to convert them to a printable format, Postscript by default, like this:

man -t somecommand | lpr

The Postscript is output to stdout; piping it to lpr like this sends it straight to the printer. You could also convert the Postscript to PDF, and therefore create versions of your man pages suitable for carrying around on a tablet or e-reader:

man -t somecommand | ps2pdf –

somecommand.pdf

Print info documents by passing them through the col command:

info somecommand | col -b | lpr

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s