Globbing Basics

07 Feb 2021

Globbing is how the Linux shell carries out “filename expansion”. Basically, when the shell encounters a special character (called wildcard characters), it will expand the command to include all filenames that match that specific character. The most common wildcard characters to know are * and ?.

The * is used to represent any number of characters (including 0 matches). For example, here I’m telling the “ls” command to list any file names in the current directory that start with “hello”. The files can have any number of characters after “hello”.

$ ls -l hello*
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello1
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello2
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello3
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello4
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:26 hello_a
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:26 hello_b
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:26 hello_c
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:27 hello_d
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:27 hello_x
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:27 hello_y

The ? is used to represent any single character. In the following example, I will only find the files that have “hello” and then one just single character.

$ ls -l hello?
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello1
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello2
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello3
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello4

Ranges - []

You can use [] to specify a range of characters to match. The shell will match files whose next character is in the range (i.e. enclosed in the square brackets).

$ ls -l hello_[xy]
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:27 hello_x
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:27 hello_y

$ ls -l hello_[axy]
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:26 hello_a
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:27 hello_x
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:27 hello_y

If you put a ! before the range, this becomes a logical NOT and the shell will not match files whose next character is in the range.

$ ls -l hello_[!axy]
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:26 hello_b
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:26 hello_c
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:27 hello_d

Curly Brackets - {}

If you put a list of terms inside curly brackets, the shell will try to match any filenames that fit any of the terms. This is an OR relationship.

$ ls -l {ab*,hello?}
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:23 abcrowl
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:22 abdoul
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:22 abfoul
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:22 absoul
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello1
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello2
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello3
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 20:24 hello4

The above is saying: “Match anything that has ‘ab’ in the beginning and whatever characters after OR anything that has ‘hello’ and exactly one character after.”

Note that you cannot have spaces after commas in the curly brackets.

Backslash - \

Use backslash to escape any of the above special characters. For example, if you have a file named hello*, you would do this ls -l hello\*. Warning: it is not recommended to actually name files with these special characters.

$ ls -l hello\*
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello*

If you did not use \, globbing would take effect and you would see any filename that has “hello” in it.

$ ls -l hello*
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello*
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello1
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello2
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello3
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello4
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello_a
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello_b
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello_c
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello_d
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello_x
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello_y

Single quotes also work for the same purpose.

$ ls -l 'hello*'
-rw-rw-r--. 1 vagrant vagrant 0 Feb  7 21:17 hello*

$ stat 'hello*'
  File: ‘hello*’
  Size: 0               Blocks: 0          IO Block: 4096   regular empty file
Device: fd02h/64770d    Inode: 33583764    Links: 1
Access: (0664/-rw-rw-r--)  Uid: ( 1000/ vagrant)   Gid: ( 1000/ vagrant)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2021-02-07 21:17:49.252343751 +0000
Modify: 2021-02-07 21:17:49.252343751 +0000
Change: 2021-02-07 21:17:49.252343751 +0000
 Birth: -

Archiving and Compressing Files

30 Jan 2021

tar - Archiving

To archive files you will use tar. tar combines many files and directories into one archive file that usually ends in “tar” however this is only by convention. This archive can be a regular file or a device, such as a tape archive. This is where the name of tar comes from: tape archive.

####################
# Basic tar options
####################
c - Create

v - Verbose

f <file_name> - File option, next argument after 'f' flag should be the name of the archive file. Use with the rest of the options to specify a specific archive file.

t - Table of contents mode, use to list contents of archive file 

x - Extract mode

p - Preserve permissions, tar sets the permissions **after** checking the entire archive

z - Filter the archive through gzip, when used with Extract it will decompress, with Create, it will compress

-C - Specify different directory to extract to

--same-owner - Keep the same ownership as exists in the archive, other wise it defaults to the user running the tar command

Examples

Creating a tar archive $ tar cvf archive.tar file1 file2

Extracting an archive $ tar xvf archive.tar

Extracting an archive to a different directory $ tar xvf archive.tar -C /path/to/dir

See what is inside of archive $ tar tvf archive.tar

gzip - Compressing

Standard Unix compression program is gzip (GNU Zip). Files ending in .gz are GNU Zip archives.

To use:

• Compress by running gzip file
• Uncompress by running gunzip file.gz

Dealing with .tar.gz files

This is a common filetype that you will encounter on Linux systems. You will first have to uncompress (get rid of the .gz) and then extract the archive (the .tar part).

This is the long way and you should build up the muscle memory of this first:

$ gunzip file.tar.gz

$ tar xvf file.tar

After getting tired of the above, shortcuts can be done numerous ways.

zcat is the same as using gunzip -dc.

$ gunzip -dc archive.tar.gz | tar xvf - -C random_dir/

$ zcat archive.tar.gz | tar xvf - -C random_dir/

tar comes with a shortcut for zcat. You can use the “z” flag.

Shortcuts for creating+compressing and extracting+decompressing:

$ tar zcvf archive.tar.gz file1 file2

$ tar zxcf archive.tar.gz -C /path/to/dir/

Quick note on zip/unzip

To interact with .zip files, you will need zip and unzip on the Linux machine.

Create a zip file

$ zip archive file1 file2

Unzip a zip file $ unzip archive.zip

Unzip a zip file to a specific directory $ unzip archive.zip -d /path/to/dir

List contents of a zip file $ zip -l archive.zip