Command-line Basics: How to Loop Through Files in a Directory

joshtronic

Looping is one of the most powerful things you can do in programming. It allows you to apply the same logic over and over to a group of items with minimal code. If done properly, infinite loops can be a blessing instead of a curse. Unfortunately though, not all command-line utilities allow you to run a command against multiple files, but with the power of shell scripting and the for loop, we can super charge any command we choose!

Getting Started

We are going to be using the for shell scripting loop. This particular control flow method is baked into your command-line shell, but it’s recommended that you use either Bash or Z shell for the greatest compatibility with the syntax below.

The following commands can be applied to any directory of your choosing, but for the sake of example, and in case you want to get crazy and try to manipulate your files without a backup, let’s go ahead and create a directory and some files to play around with:

$ mkdir /tmp/alliloop
$ cd /tmp/alliloop
$ touch file-{1..5}.txt

Looping Through Files

Okay, so this example will basically be the same as using ls in a directory, but trust me, we’re going to build upon this. To loop through a directory, and then echo the name of the file you can run:

$ for FILE in *; do echo $FILE; done
file-1.txt
file-2.txt
file-3.txt
file-4.txt
file-5.txt

Not much to it. You probably noticed we’re using the wild card character, *, in there. That tells the for loop to grab every single file in the directory. The wild card could just as easily be changed to file-* to target all of the files that started with file- or *.txt to grab just the text files.

Applying a Command

Now that we know how to loop through the files in a directory and spit out the names, let’s try something a bit more advanced. The files we created in the getting started section were all created with the touch command and are empty, so showing out how to cat each file wouldn’t be very eventful.

Fortunately, using a similar for loop, we can insert text into each of our files with ease:

$ for FILE in *; do echo "$FILE\nAlligator's Rule\!" > $FILE; done

$ for FILE in *; do cat $FILE; done
file-1.txt
Alligator's Rule!
file-2.txt
Alligator's Rule!
file-3.txt
Alligator's Rule!
file-4.txt
Alligator's Rule!
file-5.txt
Alligator's Rule!

Now each of our files contains the name of the file on the first line, and a universal truth about alligator’s on the second line.

To take it a step further, we could also combine these examples to first write to the file, then cat it’s contents in a single loop:

$ for FILE in *; do echo "$FILE\nAlligator's Rule\!" > $FILE; cat $FILE; done
file-1.txt
Alligator's Rule!
file-2.txt
Alligator's Rule!
file-3.txt
Alligator's Rule!
file-4.txt
Alligator's Rule!
file-5.txt
Alligator's Rule!

By separating our commands with a semi-colon, ;, we’re able to string together whichever commands we’d like to.

Making backups

Now that we know the ropes of how the for loop works, let’s try something a bit more real world by taking a directory of files, and making backups that are suffixed with the .bak extension:

$ for FILE in *; do cp $FILE "$FILE.bak"; done;

$ ls -l
total 40K
-rw-r--r-- 1 josh josh 29 Nov  7 18:34 file-1.txt
-rw-r--r-- 1 josh josh 29 Nov  7 18:41 file-1.txt.bak
-rw-r--r-- 1 josh josh 29 Nov  7 18:34 file-2.txt
-rw-r--r-- 1 josh josh 29 Nov  7 18:41 file-2.txt.bak
-rw-r--r-- 1 josh josh 29 Nov  7 18:34 file-3.txt
-rw-r--r-- 1 josh josh 29 Nov  7 18:41 file-3.txt.bak
-rw-r--r-- 1 josh josh 29 Nov  7 18:34 file-4.txt
-rw-r--r-- 1 josh josh 29 Nov  7 18:41 file-4.txt.bak
-rw-r--r-- 1 josh josh 29 Nov  7 18:34 file-5.txt
-rw-r--r-- 1 josh josh 29 Nov  7 18:41 file-5.txt.bak

We now have exact copies of each of our files, with an extension that indicates that they are backup files!

You’re not limited to making copies in the same directory either. You could just as easily specify a new path for your backup files:

$ for FILE in *; do cp $FILE "/tmp/my-backups/$FILE.bak"; done;
  Tweet It

🕵 Search Results

🔎 Searching...

Sponsored by #native_company# — Learn More
#native_title# #native_desc#
#native_cta#