🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:
Response:
Calculated Shard: 95 (from laksa168)

2. Crawled Status Check

Query:
Response:

3. Robots.txt Check

Query:
Response:

4. Spam/Ban Check

Query:
Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

đź“„
INDEXABLE
âś…
CRAWLED
12 hours ago
🤖
ROBOTS ALLOWED

Page Info Filters

FilterStatusConditionDetails
HTTP statusPASSdownload_http_code = 200HTTP 200
Age cutoffPASSdownload_stamp > now() - 6 MONTH0 months ago
History dropPASSisNull(history_drop_reason)No drop reason
Spam/banPASSfh_dont_index != 1 AND ml_spam_score = 0ml_spam_score=0
CanonicalPASSmeta_canonical IS NULL OR = '' OR = src_unparsedNot set

Page Details

PropertyValue
URLhttps://opensource.com/article/19/6/how-write-loop-bash
Last Crawled2026-04-18 22:18:49 (12 hours ago)
First Indexed2019-06-12 07:05:45 (6 years ago)
HTTP Status Code200
Meta TitleHow to write a loop in Bash | Opensource.com
Meta DescriptionAutomatically perform a set of actions on multiple files with for loops and find commands.
Meta Canonicalnull
Boilerpipe Text
A common reason people want to learn the Unix shell is to unlock the power of batch processing. If you want to perform some set of actions on many files, one of the ways to do that is by constructing a command that iterates over those files. In programming terminology, this is called execution control, and one of the most common examples of it is the for loop. A for loop is a recipe detailing what actions you want your computer to take for each data object (such as a file) you specify. The classic for loop An easy loop to try is one that analyzes a collection of files. This probably isn't a useful loop on its own, but it's a safe way to prove to yourself that you have the ability to handle each file in a directory individually. First, create a simple test environment by creating a directory and placing some copies of some files into it. Any file will do initially, but later examples require graphic files (such as JPEG, PNG, or similar). You can create the folder and copy files into it using a file manager or in the terminal: $ mkdir example $ cp ~/Pictures/vacation/*.{png,jpg} example Change directory to your new folder, then list the files in it to confirm that your test environment is what you expect: $ cd example $ ls -1 cat.jpg design_maori.png otago.jpg waterfall.png The syntax to loop through each file individually in a loop is: create a variable ( f for file, for example). Then define the data set you want the variable to cycle through. In this case, cycle through all files in the current directory using the * wildcard character (the * wildcard matches everything ). Then terminate this introductory clause with a semicolon ( ; ). $ for f in * ; Depending on your preference, you can choose to press Return here. The shell won't try to execute the loop until it is syntactically complete. Next, define what you want to happen with each iteration of the loop. For simplicity, use the file command to get a little bit of data about each file, represented by the f variable (but prepended with a $ to tell the shell to swap out the value of the variable for whatever the variable currently contains): do file $f ; Terminate the clause with another semi-colon and close the loop: done Press Return to start the shell cycling through everything in the current directory. The for loop assigns each file, one by one, to the variable f and runs your command: $ for f in * ; do > file $f ; > done cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced You can also write it this way: $ for f in *; do file $f; done cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced Both the multi-line and single-line formats are the same to your shell and produce the exact same results. A practical example Here's a practical example of how a loop can be useful for everyday computing. Assume you have a collection of vacation photos you want to send to friends. Your photo files are huge, making them too large to email and inconvenient to upload to your photo-sharing service . You want to create smaller web-versions of your photos, but you have 100 photos and don't want to spend the time reducing each photo, one by one. First, install the ImageMagick command using your package manager on Linux, BSD, or Mac. For instance, on Fedora and RHEL: $ sudo dnf install ImageMagick On Ubuntu or Debian: $ sudo apt install ImageMagick On BSD, use ports or pkgsrc . On Mac, use Homebrew or MacPorts . Once you install ImageMagick, you have a set of new commands to operate on photos. Create a destination directory for the files you're about to create: $ mkdir tmp To reduce each photo to 33% of its original size, try this loop: $ for f in * ; do convert $f -scale 33% tmp/$f ; done Then look in the tmp folder to see your scaled photos. You can use any number of commands within a loop, so if you need to perform complex actions on a batch of files, you can place your whole workflow between the do and done statements of a for loop. For example, suppose you want to copy each processed photo straight to a shared photo directory on your web host and remove the photo file from your local system: $ for f in * ; do convert $f -scale 33% tmp/$f scp -i seth_web tmp/$f seth@example.com:~/public_html trash tmp/$f ; done For each file processed by the for loop, your computer automatically runs three commands. This means if you process just 10 photos this way, you save yourself 30 commands and probably at least as many minutes. Limiting your loop A loop doesn't always have to look at every file. You might want to process only the JPEG files in your example directory: $ for f in *.jpg ; do convert $f -scale 33% tmp/$f ; done $ ls -m tmp cat.jpg, otago.jpg Or, instead of processing files, you may need to repeat an action a specific number of times. A for loop's variable is defined by whatever data you provide it, so you can create a loop that iterates over numbers instead of files: $ for n in {0..4}; do echo $n ; done 0 1 2 3 4 More looping You now know enough to create your own loops. Until you're comfortable with looping, use them on copies of the files you want to process and, as often as possible, use commands with built-in safeguards to prevent you from clobbering your data and making irreparable mistakes, like accidentally renaming an entire directory of files to the same name, each overwriting the other. For advanced for loop topics, read on. Not all shells are Bash The for keyword is built into the Bash shell. Many similar shells use the same keyword and syntax, but some shells, like tcsh , use a different keyword, like foreach , instead. In tcsh, the syntax is similar in spirit but more strict than Bash. In the following code sample, do not type the string foreach? in lines 2 and 3. It is a secondary prompt alerting you that you are still in the process of building your loop. $ foreach f (*) foreach? file $f foreach? end cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced In tcsh, both foreach and end must appear alone on separate lines, so you cannot create a for loop on one line as you can with Bash and similar shells. For loops with the find command In theory, you could find a shell that doesn't provide a for loop function, or you may just prefer to use a different command with added features. The find command is another way to implement the functionality of a for loop, as it offers several ways to define the scope of which files to include in your loop as well as options for Parallel processing. The find command is meant to help you find files on your hard drives. Its syntax is simple: you provide the path of the location you want to search, and find finds all files and directories: $ find . . ./cat.jpg ./design_maori.png ./otago.jpg ./waterfall.png You can filter the search results by adding some portion of the name: $ find . -name "*jpg" ./cat.jpg ./otago.jpg The great thing about find is that each file it finds can be fed into a loop using the -exec flag. For instance, to scale down only the PNG photos in your example directory: $ find . -name "*png" -exec convert {} -scale 33% tmp/{} \; $ ls -m tmp design_maori.png, waterfall.png In the -exec clause, the bracket characters {} stand in for whatever item find is processing (in other words, any file ending in PNG that has been located, one at a time). The -exec clause must be terminated with a semicolon, but Bash usually tries to use the semicolon for itself. You "escape" the semicolon with a backslash ( \; ) so that find knows to treat that semicolon as its terminating character. The find command is very good at what it does, and it can be too good sometimes. For instance, if you reuse it to find PNG files for another photo process, you will get a few errors: $ find . -name "*png" -exec convert {} -flip -flop tmp/{} \; convert: unable to open image `tmp/./tmp/design_maori.png': No such file or directory @ error/blob.c/OpenBlob/2643. ... It seems that find has located all the PNG files—not only the ones in your current directory ( . ) but also those that you processed before and placed in your tmp subdirectory. In some cases, you may want find to search the current directory plus all other directories within it (and all directories in those ). It can be a powerful recursive processing tool, especially in complex file structures (like directories of music artists containing directories of albums filled with music files), but you can limit this with the -maxdepth option. To find only PNG files in the current directory (excluding subdirectories): $ find . -maxdepth 1 -name "*png" To find and process files in the current directory plus an additional level of subdirectories, increment the maximum depth by 1: $ find . -maxdepth 2 -name "*png" Its default is to descend into all subdirectories. Looping for fun and profit The more you use loops, the more time and effort you save, and the bigger the tasks you can tackle. You're just one user, but with a well-thought-out loop, you can make your computer do the hard work. You can and should treat looping like any other command, keeping it close at hand for when you need to repeat a single action or two on several files. However, it's also a legitimate gateway to serious programming, so if you have to accomplish a complex task on any number of files, take a moment out of your day to plan out your workflow. If you can achieve your goal on one file, then wrapping that repeatable process in a for loop is relatively simple, and the only "programming" required is an understanding of how variables work and enough organization to separate unprocessed from processed files. With a little practice, you can move from a Linux user to a Linux user who knows how to write a loop, so get out there and make your computer work for you!
Markdown
[Skip to main content](https://opensource.com/article/19/6/how-write-loop-bash#main-content) [![Supported by Red Hat](https://opensource.com/themes/osdc/assets/img/l_supported-by-redhat-white.svg)](https://www.redhat.com/en?intcmp=701600000011l7VAAQ) ## User account menu - [Log in](https://opensource.com/user/login?current=/article/19/6/how-write-loop-bash) - [RSS](https://opensource.com/feed "RSS") [![Home](https://opensource.com/themes/osdc/logo.svg)](https://opensource.com/) [![Supported by Red Hat](https://opensource.com/themes/osdc/assets/img/l_supported-by-redhat-white.svg)](https://www.redhat.com/en?intcmp=701600000011l7VAAQ) ## Main navigation - [Articles](https://opensource.com/) - [Resources](https://opensource.com/resources) - [What is open source?](https://opensource.com/resources/what-open-source) - [The open source way](https://opensource.com/open-source-way) - [Projects and applications](https://opensource.com/resources/projects-and-applications) - [Organizations](https://opensource.com/resources/organizations) - [Open source alternatives](https://opensource.com/alternatives) - [Alternatives to Acrobat](https://opensource.com/alternatives/adobe-acrobat) - [Alternatives to AutoCAD](https://opensource.com/alternatives/autocad) - [Alternatives to Dreamweaver](https://opensource.com/alternatives/dreamweaver) - [Alternatives to Gmail](https://opensource.com/alternatives/gmail) - [Alternatives to MATLAB](https://opensource.com/alternatives/matlab) - [Alternatives to Minecraft](https://opensource.com/alternatives/minecraft) - [Alternatives to Google Photos](https://opensource.com/alternatives/google-photos) - [Alternatives to Photoshop](https://opensource.com/life/12/6/design-without-debt-five-tools-for-designers) - [Alternatives to Skype](https://opensource.com/alternatives/skype) - [Alternatives to Slack](https://opensource.com/alternatives/slack) - [Alternatives to Trello](https://opensource.com/alternatives/trello) - [More...](https://opensource.com/alternatives) - [Linux](https://opensource.com/resources/linux) - [Downloads](https://opensource.com/downloads) - [Frequently Asked Questions](https://opensource.com/faq) # How to write a loop in Bash Automatically perform a set of actions on multiple files with for loops and find commands. By [Seth Kenlon](https://opensource.com/users/seth) (Team, Red Hat) June 12, 2019 \| [10 Comments](https://opensource.com/article/19/6/how-write-loop-bash#comments) \| %t min read ![bash logo on green background](https://opensource.com/sites/default/files/lead-images/bash_command_line.png) Image by: Opensource.com A common reason people want to learn the Unix shell is to unlock the power of batch processing. If you want to perform some set of actions on many files, one of the ways to do that is by constructing a command that iterates over those files. In programming terminology, this is called *execution control,* and one of the most common examples of it is the **for** loop. A **for** loop is a recipe detailing what actions you want your computer to take *for* each data object (such as a file) you specify. ## The classic for loop The Linux Terminal - [Top 7 terminal emulators for Linux](https://opensource.com/life/17/10/top-terminal-emulators?intcmp=7016000000127cYAAQ) - [10 command-line tools for data analysis in Linux](https://opensource.com/article/17/2/command-line-tools-data-analysis-linux?intcmp=7016000000127cYAAQ) - [Download Now: SSH cheat sheet](https://opensource.com/downloads/advanced-ssh-cheat-sheet?intcmp=7016000000127cYAAQ) - [Advanced Linux commands cheat sheet](https://developers.redhat.com/cheat-sheets/advanced-linux-commands/?intcmp=7016000000127cYAAQ) - [Linux command line tutorials](https://opensource.com/tags/command-line?intcmp=7016000000127cYAAQ) An easy loop to try is one that analyzes a collection of files. This probably isn't a useful loop on its own, but it's a safe way to prove to yourself that you have the ability to handle each file in a directory individually. First, create a simple test environment by creating a directory and placing some copies of some files into it. Any file will do initially, but later examples require graphic files (such as JPEG, PNG, or similar). You can create the folder and copy files into it using a file manager or in the terminal: ``` $ mkdir example $ cp ~/Pictures/vacation/*.{png,jpg} example ``` Change directory to your new folder, then list the files in it to confirm that your test environment is what you expect: ``` $ cd example $ ls -1 cat.jpg design_maori.png otago.jpg waterfall.png ``` The syntax to loop through each file individually in a loop is: create a variable (**f** for file, for example). Then define the data set you want the variable to cycle through. In this case, cycle through all files in the current directory using the **\*** wildcard character (the **\*** wildcard matches *everything*). Then terminate this introductory clause with a semicolon (**;**). ``` $ for f in * ; ``` Depending on your preference, you can choose to press **Return** here. The shell won't try to execute the loop until it is syntactically complete. Next, define what you want to happen with each iteration of the loop. For simplicity, use the **file** command to get a little bit of data about each file, represented by the **f** variable (but prepended with a **\$** to tell the shell to swap out the value of the variable for whatever the variable currently contains): ``` do file $f ; ``` Terminate the clause with another semi-colon and close the loop: ``` done ``` Press **Return** to start the shell cycling through *everything* in the current directory. The **for** loop assigns each file, one by one, to the variable **f** and runs your command: ``` $ for f in * ; do > file $f ; > done cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced ``` You can also write it this way: ``` $ for f in *; do file $f; done cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced ``` Both the multi-line and single-line formats are the same to your shell and produce the exact same results. ## A practical example Here's a practical example of how a loop can be useful for everyday computing. Assume you have a collection of vacation photos you want to send to friends. Your photo files are huge, making them too large to email and inconvenient to upload to your [photo-sharing service](https://nextcloud.com/). You want to create smaller web-versions of your photos, but you have 100 photos and don't want to spend the time reducing each photo, one by one. First, install the **ImageMagick** command using your package manager on Linux, BSD, or Mac. For instance, on Fedora and RHEL: ``` $ sudo dnf install ImageMagick ``` On Ubuntu or Debian: ``` $ sudo apt install ImageMagick ``` On BSD, use **ports** or [pkgsrc](https://pkgsrc.org/). On Mac, use [Homebrew](https://brew.sh/) or [MacPorts](https://www.macports.org/). Once you install ImageMagick, you have a set of new commands to operate on photos. Create a destination directory for the files you're about to create: ``` $ mkdir tmp ``` To reduce each photo to 33% of its original size, try this loop: ``` $ for f in * ; do convert $f -scale 33% tmp/$f ; done ``` Then look in the **tmp** folder to see your scaled photos. You can use any number of commands within a loop, so if you need to perform complex actions on a batch of files, you can place your whole workflow between the **do** and **done** statements of a **for** loop. For example, suppose you want to copy each processed photo straight to a shared photo directory on your web host and remove the photo file from your local system: ``` $ for f in * ; do convert $f -scale 33% tmp/$f scp -i seth_web tmp/$f seth@example.com:~/public_html trash tmp/$f ; done ``` For each file processed by the **for** loop, your computer automatically runs three commands. This means if you process just 10 photos this way, you save yourself 30 commands and probably at least as many minutes. ## Limiting your loop A loop doesn't always have to look at every file. You might want to process only the JPEG files in your example directory: ``` $ for f in *.jpg ; do convert $f -scale 33% tmp/$f ; done $ ls -m tmp cat.jpg, otago.jpg ``` Or, instead of processing files, you may need to repeat an action a specific number of times. A **for** loop's variable is defined by whatever data you provide it, so you can create a loop that iterates over numbers instead of files: ``` $ for n in {0..4}; do echo $n ; done 0 1 2 3 4 ``` ## More looping You now know enough to create your own loops. Until you're comfortable with looping, use them on *copies* of the files you want to process and, as often as possible, use commands with built-in safeguards to prevent you from clobbering your data and making irreparable mistakes, like accidentally renaming an entire directory of files to the same name, each overwriting the other. For advanced **for** loop topics, read on. ## Not all shells are Bash The **for** keyword is built into the Bash shell. Many similar shells use the same keyword and syntax, but some shells, like [tcsh](https://en.wikipedia.org/wiki/Tcsh), use a different keyword, like **foreach**, instead. In tcsh, the syntax is similar in spirit but more strict than Bash. In the following code sample, do not type the string **foreach?** in lines 2 and 3. It is a secondary prompt alerting you that you are still in the process of building your loop. ``` $ foreach f (*) foreach? file $f foreach? end cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced ``` In tcsh, both **foreach** and **end** must appear alone on separate lines, so you cannot create a **for** loop on one line as you can with Bash and similar shells. ## For loops with the find command In theory, you could find a shell that doesn't provide a **for** loop function, or you may just prefer to use a different command with added features. The **find** command is another way to implement the functionality of a **for** loop, as it offers several ways to define the scope of which files to include in your loop as well as options for [Parallel](https://opensource.com/article/18/5/gnu-parallel) processing. The **find** command is meant to help you find files on your hard drives. Its syntax is simple: you provide the path of the location you want to search, and **find** finds all files and directories: ``` $ find . . ./cat.jpg ./design_maori.png ./otago.jpg ./waterfall.png ``` You can filter the search results by adding some portion of the name: ``` $ find . -name "*jpg" ./cat.jpg ./otago.jpg ``` The great thing about **find** is that each file it finds can be fed into a loop using the **\-exec** flag. For instance, to scale down only the PNG photos in your example directory: ``` $ find . -name "*png" -exec convert {} -scale 33% tmp/{} \; $ ls -m tmp design_maori.png, waterfall.png ``` In the **\-exec** clause, the bracket characters **{}** stand in for whatever item **find** is processing (in other words, any file ending in PNG that has been located, one at a time). The **\-exec** clause must be terminated with a semicolon, but Bash usually tries to use the semicolon for itself. You "escape" the semicolon with a backslash (**\\;**) so that **find** knows to treat that semicolon as its terminating character. The **find** command is very good at what it does, and it can be too good sometimes. For instance, if you reuse it to find PNG files for another photo process, you will get a few errors: ``` $ find . -name "*png" -exec convert {} -flip -flop tmp/{} \; convert: unable to open image `tmp/./tmp/design_maori.png': No such file or directory @ error/blob.c/OpenBlob/2643. ... ``` It seems that **find** has located all the PNG files—not only the ones in your current directory (**.**) but also those that you processed before and placed in your **tmp** subdirectory. In some cases, you may want **find** to search the current directory plus all other directories within it (and all directories in *those*). It can be a powerful recursive processing tool, especially in complex file structures (like directories of music artists containing directories of albums filled with music files), but you can limit this with the **\-maxdepth** option. To find only PNG files in the current directory (excluding subdirectories): ``` $ find . -maxdepth 1 -name "*png" ``` To find and process files in the current directory plus an additional level of subdirectories, increment the maximum depth by 1: ``` $ find . -maxdepth 2 -name "*png" ``` Its default is to descend into all subdirectories. ## Looping for fun and profit The more you use loops, the more time and effort you save, and the bigger the tasks you can tackle. You're just one user, but with a well-thought-out loop, you can make your computer do the hard work. You can and should treat looping like any other command, keeping it close at hand for when you need to repeat a single action or two on several files. However, it's also a legitimate gateway to serious programming, so if you have to accomplish a complex task on any number of files, take a moment out of your day to plan out your workflow. If you can achieve your goal on one file, then wrapping that repeatable process in a **for** loop is relatively simple, and the only "programming" required is an understanding of how variables work and enough organization to separate unprocessed from processed files. With a little practice, you can move from a Linux user to a Linux user who knows how to write a loop, so get out there and make your computer work for you\! What to read next [![arrows cycle symbol for failing faster](https://opensource.com/sites/default/files/styles/article_teaser/public/lead-images/fail_progress_cycle_momentum_arrow.png?itok=QQ2M_rP5)](https://opensource.com/article/19/10/programming-bash-loops) ## [How to program with Bash: Loops](https://opensource.com/article/19/10/programming-bash-loops) Learn how to use loops for performing iterative operations, in the final article in this three-part series on programming with Bash. [David Both](https://opensource.com/users/dboth) (Correspondent) October 23, 2019 Tags [Bash](https://opensource.com/tags/bash) [Command line](https://opensource.com/tags/command-line) [Linux](https://opensource.com/tags/linux) [Seth Kenlon](https://opensource.com/users/seth) ![Seth Kenlon](https://opensource.com/sites/default/files/styles/150x150/public/pictures/seth_headshot-lawrence_0.jpg?itok=jZUHBHx4) Seth Kenlon is a UNIX geek, free culture advocate, independent multimedia artist, and D\&D nerd. He has worked in the film and computing industry, often at the same time. [More about me](https://opensource.com/users/seth) ## 10 Comments These comments are closed. ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [Gonca Sousa](https://opensource.com/users/goncasousa) \| June 12, 2019 great ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [howtopam](https://opensource.com/users/howtopamm) \| June 12, 2019 Great little tutorial. People enjoy small bits of valuable information. This "for loop" statement can be very helpful to increase productivity when used in correct coding scenarios. Thanks for the lesson Gonca Sousa. ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [howtopam](https://opensource.com/users/howtopamm) \| June 12, 2019 My mistake and apology @Seth Kenlon. You are the author and I quoted Gonca Sousa in my thanks for the lesson statement in the above comment. Sorry Seth Kenlon and to Gonca Sousa as well for the incorrect credit. howtopam In reply to [Great little tutorial. People](https://opensource.com/article/19/6/how-write-loop-bash#comment-178126) by [howtopam](https://opensource.com/users/howtopamm "View user profile.") ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [Seth Kenlon](https://opensource.com/users/seth) \| June 12, 2019 Thanks for reading my article and for the comment\! In reply to [My mistake and apology @Seth](https://opensource.com/article/19/6/how-write-loop-bash#comment-178131) by [howtopam](https://opensource.com/users/howtopamm "View user profile.") ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [JJ](https://opensource.com/users/wavesailor) \| June 12, 2019 Great article - Nice to learn about the \`file\` command and the \`covert\` in addition to learning about looping. Thx ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [Seth Kenlon](https://opensource.com/users/seth) \| June 12, 2019 The \``convert\`` command from ImageMagick is one of those things I use a heck of a lot more than I think I realise. I think I'd just give up on computers if I had to open every image I resize or convert from one format to another in a GUI, one at a time. Thanks for reading, JJ, and thanks for the comment\! In reply to [Great article - Nice to learn](https://opensource.com/article/19/6/how-write-loop-bash#comment-178146) by [JJ](https://opensource.com/users/wavesailor "View user profile.") ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [Apostolos Kritikos](https://opensource.com/users/akritiko) \| June 19, 2019 Excellent work with the article. ++ To the author for the idea and execution. ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [Seth Kenlon](https://opensource.com/users/seth) \| June 19, 2019 In Bash, that would have to be (( ++ )) ;-) In reply to [Excellent work with the](https://opensource.com/article/19/6/how-write-loop-bash#comment-178466) by [akritiko](https://opensource.com/users/akritiko "View user profile.") ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [Carl T. Miller](https://opensource.com/users/carltm) \| June 25, 2019 Great article! I use the type of looping you describe. I notice that you didn't cover the case where you have hundreds or even thousands of files to process, and you want them run sequentially instead of in parallel. The following command can be rewritten easily. find . -name "\*png" -exec convert {} -scale 33% tmp/{} \\; This is the new version. find . -name "\*png" \|\\ while IFS= read -r file; do convert "\$file" -scale 33% tmp/ done ![Avatar](https://opensource.com/sites/default/files/styles/medium/public/osdc_default_avatar_1.png?itok=G0WcUo3c) [Seth Kenlon](https://opensource.com/users/seth) \| June 25, 2019 Great point, and a great topic. I do feel that IFS, and parsing the output of find for sequential processing is complex enough for an article all its own. I didn't want to rush past the intricacies of analyzing find's output and what you need to set IFS to (or what IFS is to begin with), and how read needs to be set, for the input to be sane. Great idea for a separate article, though. I've taken note of it, but if you'd like to write it, I'll just casually slide this URL over to you... [https://opensource.com/how-submit-article](https://opensource.com/how-submit-article) In reply to [Great article! I use the](https://opensource.com/article/19/6/how-write-loop-bash#comment-178831) by [carltm](https://opensource.com/users/carltm "View user profile.") ## Related Content [![What sets Krita apart from other open source digital painting tools](https://opensource.com/sites/default/files/styles/222x125/public/lead-images/paint_brushes_design_creative.jpg?itok=UAbMxkfW)](https://opensource.com/article/23/4/web-safe-color-guide-bash) [Make a web-safe color guide with Bash](https://opensource.com/article/23/4/web-safe-color-guide-bash) [![Person using a laptop](https://opensource.com/sites/default/files/styles/222x125/public/lead-images/laptop_screen_desk_work_chat_text.png?itok=0OxFOFsJ)](https://opensource.com/article/22/2/configure-vim-default-editor) [How I configure Vim as my default editor on Linux](https://opensource.com/article/22/2/configure-vim-default-editor) [![Digital creative of a browser on the internet](https://opensource.com/sites/default/files/styles/222x125/public/lead-images/browser_web_internet_website.png?itok=mTAT99U3)](https://opensource.com/article/21/11/jekyll-config-files) [How I dynamically generate Jekyll config files](https://opensource.com/article/21/11/jekyll-config-files) [![Creative Commons License](https://opensource.com/themes/osdc/assets/img/cc-by-sa-4.png)](https://creativecommons.org/licenses/by-sa/4.0/)This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. ## About This Site The opinions expressed on this website are those of each author, not of the author's employer or of Red Hat. [Opensource.com](https://opensource.com/) aspires to publish all content under a [Creative Commons license](https://creativecommons.org/licenses/) but may not be able to do so in all cases. You are responsible for ensuring that you have the necessary permission to reuse any work on this site. Red Hat and the Red Hat logo are trademarks of Red Hat, LLC, registered in the United States and other countries. A note on advertising: Opensource.com does not sell advertising on the site or in any of its newsletters. [![Home](https://opensource.com/themes/osdc/logo.svg)](https://opensource.com/) Copyright ©2021 Red Hat, LLC ## Legal - [Privacy Policy](https://opensource.com/privacy-policy) - [Terms of use](https://opensource.com/legal)
Readable Markdown
A common reason people want to learn the Unix shell is to unlock the power of batch processing. If you want to perform some set of actions on many files, one of the ways to do that is by constructing a command that iterates over those files. In programming terminology, this is called *execution control,* and one of the most common examples of it is the **for** loop. A **for** loop is a recipe detailing what actions you want your computer to take *for* each data object (such as a file) you specify. ## The classic for loop An easy loop to try is one that analyzes a collection of files. This probably isn't a useful loop on its own, but it's a safe way to prove to yourself that you have the ability to handle each file in a directory individually. First, create a simple test environment by creating a directory and placing some copies of some files into it. Any file will do initially, but later examples require graphic files (such as JPEG, PNG, or similar). You can create the folder and copy files into it using a file manager or in the terminal: ``` $ mkdir example $ cp ~/Pictures/vacation/*.{png,jpg} example ``` Change directory to your new folder, then list the files in it to confirm that your test environment is what you expect: ``` $ cd example $ ls -1 cat.jpg design_maori.png otago.jpg waterfall.png ``` The syntax to loop through each file individually in a loop is: create a variable (**f** for file, for example). Then define the data set you want the variable to cycle through. In this case, cycle through all files in the current directory using the **\*** wildcard character (the **\*** wildcard matches *everything*). Then terminate this introductory clause with a semicolon (**;**). ``` $ for f in * ; ``` Depending on your preference, you can choose to press **Return** here. The shell won't try to execute the loop until it is syntactically complete. Next, define what you want to happen with each iteration of the loop. For simplicity, use the **file** command to get a little bit of data about each file, represented by the **f** variable (but prepended with a **\$** to tell the shell to swap out the value of the variable for whatever the variable currently contains): ``` do file $f ; ``` Terminate the clause with another semi-colon and close the loop: ``` done ``` Press **Return** to start the shell cycling through *everything* in the current directory. The **for** loop assigns each file, one by one, to the variable **f** and runs your command: ``` $ for f in * ; do > file $f ; > done cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced ``` You can also write it this way: ``` $ for f in *; do file $f; done cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced ``` Both the multi-line and single-line formats are the same to your shell and produce the exact same results. ## A practical example Here's a practical example of how a loop can be useful for everyday computing. Assume you have a collection of vacation photos you want to send to friends. Your photo files are huge, making them too large to email and inconvenient to upload to your [photo-sharing service](https://nextcloud.com/). You want to create smaller web-versions of your photos, but you have 100 photos and don't want to spend the time reducing each photo, one by one. First, install the **ImageMagick** command using your package manager on Linux, BSD, or Mac. For instance, on Fedora and RHEL: ``` $ sudo dnf install ImageMagick ``` On Ubuntu or Debian: ``` $ sudo apt install ImageMagick ``` On BSD, use **ports** or [pkgsrc](https://pkgsrc.org/). On Mac, use [Homebrew](https://brew.sh/) or [MacPorts](https://www.macports.org/). Once you install ImageMagick, you have a set of new commands to operate on photos. Create a destination directory for the files you're about to create: ``` $ mkdir tmp ``` To reduce each photo to 33% of its original size, try this loop: ``` $ for f in * ; do convert $f -scale 33% tmp/$f ; done ``` Then look in the **tmp** folder to see your scaled photos. You can use any number of commands within a loop, so if you need to perform complex actions on a batch of files, you can place your whole workflow between the **do** and **done** statements of a **for** loop. For example, suppose you want to copy each processed photo straight to a shared photo directory on your web host and remove the photo file from your local system: ``` $ for f in * ; do convert $f -scale 33% tmp/$f scp -i seth_web tmp/$f seth@example.com:~/public_html trash tmp/$f ; done ``` For each file processed by the **for** loop, your computer automatically runs three commands. This means if you process just 10 photos this way, you save yourself 30 commands and probably at least as many minutes. ## Limiting your loop A loop doesn't always have to look at every file. You might want to process only the JPEG files in your example directory: ``` $ for f in *.jpg ; do convert $f -scale 33% tmp/$f ; done $ ls -m tmp cat.jpg, otago.jpg ``` Or, instead of processing files, you may need to repeat an action a specific number of times. A **for** loop's variable is defined by whatever data you provide it, so you can create a loop that iterates over numbers instead of files: ``` $ for n in {0..4}; do echo $n ; done 0 1 2 3 4 ``` ## More looping You now know enough to create your own loops. Until you're comfortable with looping, use them on *copies* of the files you want to process and, as often as possible, use commands with built-in safeguards to prevent you from clobbering your data and making irreparable mistakes, like accidentally renaming an entire directory of files to the same name, each overwriting the other. For advanced **for** loop topics, read on. ## Not all shells are Bash The **for** keyword is built into the Bash shell. Many similar shells use the same keyword and syntax, but some shells, like [tcsh](https://en.wikipedia.org/wiki/Tcsh), use a different keyword, like **foreach**, instead. In tcsh, the syntax is similar in spirit but more strict than Bash. In the following code sample, do not type the string **foreach?** in lines 2 and 3. It is a secondary prompt alerting you that you are still in the process of building your loop. ``` $ foreach f (*) foreach? file $f foreach? end cat.jpg: JPEG image data, EXIF standard 2.2 design_maori.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced otago.jpg: JPEG image data, EXIF standard 2.2 waterfall.png: PNG image data, 4608 x 2592, 8-bit/color RGB, non-interlaced ``` In tcsh, both **foreach** and **end** must appear alone on separate lines, so you cannot create a **for** loop on one line as you can with Bash and similar shells. ## For loops with the find command In theory, you could find a shell that doesn't provide a **for** loop function, or you may just prefer to use a different command with added features. The **find** command is another way to implement the functionality of a **for** loop, as it offers several ways to define the scope of which files to include in your loop as well as options for [Parallel](https://opensource.com/article/18/5/gnu-parallel) processing. The **find** command is meant to help you find files on your hard drives. Its syntax is simple: you provide the path of the location you want to search, and **find** finds all files and directories: ``` $ find . . ./cat.jpg ./design_maori.png ./otago.jpg ./waterfall.png ``` You can filter the search results by adding some portion of the name: ``` $ find . -name "*jpg" ./cat.jpg ./otago.jpg ``` The great thing about **find** is that each file it finds can be fed into a loop using the **\-exec** flag. For instance, to scale down only the PNG photos in your example directory: ``` $ find . -name "*png" -exec convert {} -scale 33% tmp/{} \; $ ls -m tmp design_maori.png, waterfall.png ``` In the **\-exec** clause, the bracket characters **{}** stand in for whatever item **find** is processing (in other words, any file ending in PNG that has been located, one at a time). The **\-exec** clause must be terminated with a semicolon, but Bash usually tries to use the semicolon for itself. You "escape" the semicolon with a backslash (**\\;**) so that **find** knows to treat that semicolon as its terminating character. The **find** command is very good at what it does, and it can be too good sometimes. For instance, if you reuse it to find PNG files for another photo process, you will get a few errors: ``` $ find . -name "*png" -exec convert {} -flip -flop tmp/{} \; convert: unable to open image `tmp/./tmp/design_maori.png': No such file or directory @ error/blob.c/OpenBlob/2643. ... ``` It seems that **find** has located all the PNG files—not only the ones in your current directory (**.**) but also those that you processed before and placed in your **tmp** subdirectory. In some cases, you may want **find** to search the current directory plus all other directories within it (and all directories in *those*). It can be a powerful recursive processing tool, especially in complex file structures (like directories of music artists containing directories of albums filled with music files), but you can limit this with the **\-maxdepth** option. To find only PNG files in the current directory (excluding subdirectories): ``` $ find . -maxdepth 1 -name "*png" ``` To find and process files in the current directory plus an additional level of subdirectories, increment the maximum depth by 1: ``` $ find . -maxdepth 2 -name "*png" ``` Its default is to descend into all subdirectories. ## Looping for fun and profit The more you use loops, the more time and effort you save, and the bigger the tasks you can tackle. You're just one user, but with a well-thought-out loop, you can make your computer do the hard work. You can and should treat looping like any other command, keeping it close at hand for when you need to repeat a single action or two on several files. However, it's also a legitimate gateway to serious programming, so if you have to accomplish a complex task on any number of files, take a moment out of your day to plan out your workflow. If you can achieve your goal on one file, then wrapping that repeatable process in a **for** loop is relatively simple, and the only "programming" required is an understanding of how variables work and enough organization to separate unprocessed from processed files. With a little practice, you can move from a Linux user to a Linux user who knows how to write a loop, so get out there and make your computer work for you\!
Shard95 (laksa)
Root Hash12209221891525358495
Unparsed URLcom,opensource!/article/19/6/how-write-loop-bash s443