🕷️ Crawler Inspector

[](https://www.cyberciti.biz/faq/) [nixCraft](https://www.cyberciti.biz/) → [Howto](https://www.cyberciti.biz/faq/) → [BASH Shell](https://www.cyberciti.biz/faq/category/bash-shell/) → How To Write Bash Shell Loop Over Set of Files # [How To Write Bash Shell Loop Over Set of Files](https://www.cyberciti.biz/faq/bash-loop-over-file/) Author: Vivek Gite Last updated: March 19, 2024 [51 comments](https://www.cyberciti.biz/faq/bash-loop-over-file/#comments) [](https://www.cyberciti.biz/faq/category/bash-shell/ "See all Bash/Shell scripting related FAQ") How do I run shell loop over set of files stored in a current directory or specified directory? You can use for loop easily over a set of shell file under bash or any other UNIX shell using wild card character. | Tutorial details | | |---|---| | Difficulty level | [Easy](https://www.cyberciti.biz/faq/tag/easy/ "See all Easy Linux / Unix System Administrator Tutorials") | | Root privileges | No | | Requirements | Linux or Unix terminal | | Category | [Linux shell scripting](https://bash.cyberciti.biz/guide/Main_Page "See Linux Bash Shell Scripting Tutorial") | | OS compatibility | BSD • [Linux](https://www.cyberciti.biz/faq/category/linux/ "See all Linux distributions tutorials") • [macOS](https://www.cyberciti.biz/faq/category/mac-os-x/ "See all macOS (OS X) tutorials") • [Unix](https://www.cyberciti.biz/faq/category/unix/ "See all Unix tutorials") • WSL | | Est. reading time | 6 minutes | ## Syntax The general syntax is as follows: ``` for f in file1 file2 file3 file5 do echo "Processing $f" # always double quote "$f" filename # do something on $f done ``` You can also use shell variables: ``` FILES="file1 /path/to/file2 /etc/resolv.conf" for f in $FILES do echo "Processing $f" done ``` You can loop through all files such as \*.c, enter: `$ for f in ./*.c; do echo "Processing $f file..."; done` ### Sample Shell Script To Loop Through All Files Try the following script: ``` #!/bin/bash # NOTE : Quote it else use array to avoid problems # FILES="/path/to/*" for f in $FILES do echo "Processing $f file..." # take action on each file. $f store current file name cat "$f" done ```  ### [How to check if file does not exist in Bash](https://www.cyberciti.biz/faq/bash-check-if-file-does-not-exist-linux-unix/) We can [find out if a file exists with conditional expressions in a Bash shell](https://www.cyberciti.biz/tips/find-out-if-file-exists-with-conditional-expressions.html) itself. In other words, [bash for loop](https://www.cyberciti.biz/faq/bash-for-loop/) will not work blindly on files. Think it as failsafe: ``` #!/bin/bash FILES="/etc/sys*.conf" for f in $FILES do # FAILSAFE # # Check if "$f" FILE exists and is a regular file and then only copy it # if [ -f "$f" ] then echo "Processing $f file..." cp -f "$f" /dest/dir else echo "Warning: Some problem with \"$f\"" fi done ``` ### Filename Expansion You can do filename expansion in loop such as work on all pdf files in current directory: ``` for f in /path/to/*.pdf do   echo "Removing password for pdf file - $f" # always "double quote" $f to avoid problems /path/to/command --option "$f" done ``` However, there is one problem with the above syntax. If there are no pdf files in current directory it will expand to \*.pdf (i.e. f will be set to \*.pdf”). To avoid this problem add the following statement before the for loop: ``` #!/bin/bash # Usage: remove all utility bills pdf file password shopt -s nullglob for f in *.pdf do echo "Removing password for pdf file - $f" pdftk "$f" output "output.$f" user_pw "YOURPASSWORD-HERE" done # unset it now shopt -u nullglob   # rest of the script below ``` If `nullglob` is set using the [shopt command](https://bash.cyberciti.biz/guide/Shopt_command "Shopt command - Linux Bash Shell Scripting Tutorial Wiki"), bash allows patterns which match no files to expand to a null string, rather than themselves. This is useful to avoid errors in loops. ### Dealing with POSIX issue Use the following syntax when POSIX is your concern and when you are not using Bash with `nullglob`. For example: ``` for filename in /path/to/*.pdf do # failsafe, does filename exists? # if not continue [ -e "$filename" ] || continue _run_commands_on "$filename" done ``` ## Using A Shell Variable And [Bash While Loop](https://www.cyberciti.biz/faq/bash-while-loop/ "Bash While Loop Examples") You can read list of files from a text file. For example, create a text file called /tmp/data.txt as follows: ``` file1 file2 file3 ``` Now you can use the [bash while loop](https://www.cyberciti.biz/faq/bash-while-loop/ "Bash While Loop Examples") as follows to read and process each by one by one using the combination of [\$IFS](https://bash.cyberciti.biz/guide/$IFS) and the read command: ``` #!/bin/bash while IFS= read -r file do [ -f "$file" ] && rm -f "$file" done < "/tmp/data.txt" ``` Here is another example which removes all unwanted files from chrooted lighttpd / nginx or Apache webserver: ``` #!/bin/bash _LIGHTTPD_ETC_DEL_CHROOT_FILES="/usr/local/nixcraft/conf/apache/secure/db/dir.etc.list" secureEtcDir(){ local d="$1" local _d="/jails/apache/$d/etc" local __d="" [ -f "$_LIGHTTPD_ETC_DEL_CHROOT_FILES" ] || { echo "Warning: $_LIGHTTPD_ETC_DEL_CHROOT_FILES file not found. Cannot secure files in jail etc directory."; return; } echo "* Cleaning etc FILES at: \"$_d\" ..." while IFS= read -r file do __d="$_d/$file" [ -f "$__d" ] && rm -f "$__d" done < "$_LIGHTTPD_ETC_DEL_CHROOT_FILES" }   secureEtcDir "nixcraft.net.in" ``` One can search for files using the find command and then feed that as input to [bash while loop](https://www.cyberciti.biz/faq/bash-while-loop/ "Bash While Loop Examples") as follows: ``` # Safety feature in read command # The '-r' option is used not allow backslashes to escape any characters # The "-d ''" option is used to continue until the first character of DELIM (which is set to '') is read rather than newline while IFS= read -r -d '' filename do _command_one_on "$filename" /path/to/command_two_on "$filename" done < <(find /path/to/search/dir/ -type f -name '*.pdf' -print0) ``` ## Processing Command Line Arguments Here is another simple example: ``` #!/bin/bash # make sure you always put $f in double quotes to avoid any nasty surprises i.e. "$f" # we use "$@" (with double quotes) instead of $* or $@ because # we want to prevent whitespace problems with filenames. for f in "$@" do echo "Processing $f file..." # rm "$f" done ``` Run it as follows: `$ ./script.sh file1 file2 fil3` Outputs: ``` Processing file1 file Processing file2 file Processing file3 file ``` The following **will not work** as we double quoted `"$*"`. In other words it will not word split, and the loop will only run once. Hence, use the above syntax: ``` #!/bin/bash # make sure you always put $f in double quotes to avoid any nasty surprises i.e. "$f" for f in "$*" do echo "Processing $f file..." # rm "$f" done ``` Try it: `$ ./script.sh file1 file2 fil3` Outputs: ``` Processing file1 file2 file3 file ``` Please note that [\$@](https://bash.cyberciti.biz/guide/$@) expanded as “\$1” “\$2” “\$3” … “\$n” and [\$\*](https://bash.cyberciti.biz/guide/$*) expanded as “\$1y\$2y\$3y…\$n”, where y is the value of IFS variable i.e. “`$*`” is one long string and [\$IFS](https://bash.cyberciti.biz/guide/$IFS) act as an [separator or token delimiters](https://bash.cyberciti.biz/guide/$IFS). ## Bash Shell Loop Over Set of Files The following example use shell variables to store actual path names and then files are processed using the for loop: ``` #!/bin/bash _base="/jail/.conf" _dfiles="${_base}/nginx/etc/conf/*.conf"   for f in $_dfiles do lb2file="/tmp/${f##*/}.$$" #tmp file sed 's/Load_Balancer-1/Load_Balancer-2/' "$f" > "${lb2file}" # update signature scp "${lb2file}" nginx@lb2.nixcraft.net.in:${f} # scp updated file to lb2 rm -f "${lb2file}" done ``` ## Summing up You learned how to write or set up bash for loop-over files. It is easy, but care must be taken when dealing with non-existing files or files with special characters such as white spaces. To overcome those two issues we can combine find and xargs command as follows: ``` # Search all pdf files copy it safely # # Tested on GNU/Linux, macOS and FreeBSD (BSD/find/xargs) # find /dir/to/search -type f -name "*.pdf" -print0 | xargs -0 -I {} echo "Working on {}" find /dir/to/search -type f -name "*.pdf" -print0 | xargs -0 -I {} cp "{}" /dest/dir   # # Find all "*.c" files in ~/project/ and move to /nas/oldproject/ dir # Note -iname is same as -name but the match is case insensitive. # find ~/projects/ -type f -iname "*.c" |\ xargs -0 -I {} mv -v "{}" /nas/oldprojects/ ``` Where find command options are: 1. **`-type f`** : Only search files 2. **`-name "*.pdf"`** OR **`-iname "*.c"`** : Search for all pdf files. The `-iname` option enables case insensitive match for all C files. 3. **`-print0`** : Useful to deal with spacial file names and xargs. It will print the full file name on the standard output (screen), followed by a null character (instead of the newline character that `-print` uses). This allows file names that contain newlines or other types of white space to be correctly interpreted by programs that process the find output. This option corresponds to the `-0` option of xargs. The output of find command is piped out to the xargs command, and xargs options are: 1. **`-0`** : Input items are terminated by a null character instead of by whitespace, and the quotes and backslash are not special (every character is taken literally). Disables the end of file string, which is treated like any other argument. Useful when input items might contain white space, quote marks, or backslashes. The GNU find -print0 option produces input suitable for this mode. 2. **`-I {}`** : Replace occurrences of `{}` in the initial-arguments with names read from standard input i.e. find command output. 3. **`echo`** OR **`cp`** : Shell command to run over this file denoted by `{}` See man pages using the [man command](https://bash.cyberciti.biz/guide/Man_command "Man command - Linux Bash Shell Scripting Tutorial Wiki") and [help command](https://bash.cyberciti.biz/guide/Help_command "help command - Linux Bash Shell Scripting Tutorial Wiki"): 🥺 Was this helpful? Please add [a comment to show your appreciation or feedback](https://www.cyberciti.biz/faq/bash-loop-over-file/#respond "Please add your comment below ↓ to show your appreciation or feedback to the author"). [Vivek Gite](https://www.vivekgite.com/) is an expert IT Consultant with over 25 years of experience, specializing in Linux and open source solutions. He writes about Linux, macOS, Unix, IT, programming, infosec, and open source. Follow his work via [RSS feed](https://www.cyberciti.com/atom/atom.xml "Get nixCraft updates using RSS feed") or [email newsletter](https://newsletter.cyberciti.com/subscription?f=1ojtmiv8892KQzyMsTF4YPr1pPSAhX2rq7Qfe5DiHMgXwKo892di4MTWyOdd976343rcNR6LhdG1f7k9H8929kMNMdWu3g "Get nixCraft updates using Email"). 51 comments… [add one](https://www.cyberciti.biz/faq/bash-loop-over-file/#respond) - DAY May 11, 2008 @ 12:35 The scripts are wrong, use \`\$FILES\` (without quotes) in the loop instead of \`”\$FILES”\`. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37884) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37884 "permalink to this comment") - 👮🛡️  [nixCraft](https://www.vivekgite.com/) May 11, 2008 @ 13:57 DAY, I don’t think so it is wrong. Do you have any problem running script? [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37885) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37885 "permalink to this comment") - Max May 28, 2016 @ 7:09 What if you have 5 or 6 files, specific files .biz in a directory and you want to store them one at the time in a variable \$mirage, and want to process \$mirage until no more .biz file are found. How will you modify the script accordingly [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-850768) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-850768 "permalink to this comment") - DAY May 11, 2008 @ 15:14 Yes, the script does not work for me. Both the one directly typed in the cmd line and the one from a script file. They output of the first one is: Processing \*.c file.. The output of the second is: Processing \* file… In fact, I just start learn the shell script. Here is another one which can do the same job: \#!/bin/sh for f in \`ls\` do echo “Processing \$f file …” done [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37886) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37886 "permalink to this comment") - Brock Noland May 11, 2008 @ 19:36 Hmm…not sure why you wouldn’t use for file in \* or for file in \*.c [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37887) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37887 "permalink to this comment") - Max May 28, 2016 @ 7:08 What if you have 5 or 6 files, specific files .biz in a directory and you want to store them one at the time in a variable \$mirage, and want to process \$mirage until no more .biz file are found. How will you modify the script accordingly [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-850767) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-850767 "permalink to this comment") - 👮🛡️  [nixCraft](https://www.vivekgite.com/) May 11, 2008 @ 20:54 DAY, Hmm, it should work (I hope you have \*.c files in current directory) or try as suggested by brock. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37889) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37889 "permalink to this comment") - DAY May 12, 2008 @ 2:47 Thanks, vivek and Brock. Replacing “\$FILES” with \* or \*.c works as expected. But I am just curious why the original one does not work for me. It seems that the all items following \`in’ should not be enclosed in a pair of quotes, otherwise, all of them will be interpreted as one item, the string. I did try the following example, the output is as what I expected: [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37891) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37891 "permalink to this comment") - DAY May 12, 2008 @ 4:13 OK, read sth. in the bash man. page. Quoting is used to remove the special meaning of certain characters or words to the shell. Quoting can be used to disable special treatment for special characters, to prevent reserved words from being recognized as such, and to prevent parameter expansion. I think that’s why the original script doesn’t work. “\$FILES” is treated as “\*”, which does disable special treatment for the special char \`\*’. It’s similar when you type \`echo “\*”\`. \`echo’ would not print out everything in the \`pwd\`. Instead, it simply prints out \`\*’. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37893) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37893 "permalink to this comment") - Jeff Schroeder May 12, 2008 @ 17:08 Actually… if \$FILES + the contents of /proc/\$pid/environ are together \> the output of “getconf ARG\_MAX” this will fail. The proper way to do this that always works is in the “useless use of cat awards” page: <http://partmaps.org/era/unix/award.html#backticks> The for is easier to read, but it is really annoying when your scripts fail with the dreaded “argument list too long” errors. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37901) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37901 "permalink to this comment") - Jeff Schroeder May 12, 2008 @ 17:10 Forgot to share a little bashism that makes it easy to determine a good guess of the ARG\_MAX: MAXARGS=\$(( \$(getconf ARG\_MAX) – \$(env \| wc -c) )) [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37902) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37902 "permalink to this comment") - David Thompson May 16, 2008 @ 4:46 I use loops like this a lot, for x in \* ; do test -f “\$x” \|\| continue COMMAND “\$x” done helps to easily ignore subdirectories [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37930) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-37930 "permalink to this comment") - Baz Jun 8, 2008 @ 0:12 Double quotes disable the special meaning of most enclosed characters. They do not disable the interpretation of variables with a leading \$. To do this yopu need single quotes. FILES=”\*” is wrong unless you want the value of \$FILES to be \*. The same is true of “\*.c”. Lose the quotes to get what you want. I have always just used – for F in \* do … etc for F in \`ls\` is OK except that for F in ls -1 (one) is better, but both are more cumbersome and less elegant that for F in \* (or \*.c and so on) [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-38060) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-38060 "permalink to this comment") - Chris Dec 11, 2008 @ 1:00 \> FILES=”\*” is wrong unless you want the value of \$FILES to be \*. The same is true of “\*.c”. Lose the quotes to get what you want. I dont think his wrong. He just gave those of us who are new to Bash scripting a placeholder for other commands or values. In fairness, the script did what it said it would do. Thanks for explaining the difference with using quotes and doing away with them. For others reading this. Dont just take our words for it. Use the script – test it for yourself. Play with it. Thanks and congratulations to nixcraft for sharing. Peace\! [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-39426) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-39426 "permalink to this comment") - Foo Feb 11, 2009 @ 21:47 FILES=\*; for f in \$FILES; do… is WRONG. for f in \`ls\`; do… is even WORSE. Both break with filenames including whitespaces, newlines etc. Since this is about bash use array if you want files in variables: files=(\*.c) for f in “\${files\[@\]}”; do cmd “\$f”; done Or just use glob: for f in \*.c; do cmd “\$f”; done [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-40241) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-40241 "permalink to this comment") - Michael Smith Feb 11, 2009 @ 22:00 I was initially a little confused by the thread. It’s not useful to assign \* to a variable if you only intend to use it for a loop. Furthermore, as others have stated, putting quotes around the variable name prevent the glob, in this case \*, from expanding. \* Vivek is not correct that \$FILES should be quoted. \* DAY’s initial response that \$FILES should be unquoted is not wrong, but using the variable at all is not useful. \* DAY’s second idea of looping over the output of \`ls\` is a very common mistake. It’s wrong because of [wordsplitting](http://bash-hackers.org/wiki/doku.php/syntax/words "Wordsplitting"). \* Brock Noland’s instinct to use `for file in *.c...` is spot-on. \* Jeff Schroeder is right to avoid ARG\_MAX in general, but it only applies when you call exec\*() via the kernel. Since for is a shell builtin, ARG\_MAX doesn’t apply here. \* David Thompson and Baz’s comments are OK, but to Baz I would reiterate to avoid using the `ls` command for anything except human-readable output. \* As for Chris’ comment: `FILES="*"` and `FILES=*` are equivalent since sh-compliant shells don’t expand globs during variable assignment. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-40242) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-40242 "permalink to this comment") - rduke15 Apr 12, 2009 @ 14:40 One curious problem. If there are NO files matching the glob, you get this: the file variable is now ‘\*.jpg’ I would have expected that the contents of the for loop would not be executed at all, since there was no jpg file. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-41162) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-41162 "permalink to this comment") - rduke15 Apr 12, 2009 @ 14:53 Found the problem with my previous example. The nullglob shell option needs to be set: `shopt -s nullglob; for file in *.jpg; do echo " the file variable is now '$file' " ; done` produces no output, as expected. As opposed to: [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-41163) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-41163 "permalink to this comment") - TheBonsai May 5, 2009 @ 4:57 @Jeff Schroeder: The for is easier to read, but it is really annoying when your scripts fail with the dreaded “argument list too long” errors. This won’t happen on a for-loop statement, since no exec() is done for the loop itself. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-41473) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-41473 "permalink to this comment") - gurpur Dec 17, 2009 @ 3:29 I need some in writing a script that will delete a specific line from a bunch of file in a directory. your help is appreciated [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-45203) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-45203 "permalink to this comment") - 👮🛡️  [nixCraft](https://www.vivekgite.com/) Dec 17, 2009 @ 6:29 @gurpur Again, offtopic, go to our forum at nixcraft.com to post all your shell scripting related queries. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-45206) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-45206 "permalink to this comment") - Xa Dec 19, 2010 @ 4:14 for i in ‘ls’ did NOT work for me. What does work is this: for i in \$(ls) [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-52886) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-52886 "permalink to this comment") - mwcz Apr 5, 2013 @ 17:56 That’s because you used single quotes instead of backticks. It’s not this: for i in ‘ls’ It’s this: for i in \`ls\` [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-85444) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-85444 "permalink to this comment") - she Mar 7, 2011 @ 16:21 can somebody help me develop a program that has the option whether to overwrite a document or not. when making a file, it will ask the user if he would want to create a new file on the same file name or overwrite the said filename.. please i need this badly. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-56174) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-56174 "permalink to this comment") - Benjamin Jun 24, 2011 @ 20:12 Nice compilation, thanks! One suggestion, though: I think many people would be looking for still another thing: how to get file extensions and how to get filenames without extensions. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-60217) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-60217 "permalink to this comment") - TheBonsai Jun 25, 2011 @ 14:46 <http://wiki.bash-hackers.org/syntax/pe#common_use> [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-60225) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-60225 "permalink to this comment") - 👮🛡️  [nixCraft](https://www.vivekgite.com/) Jun 25, 2011 @ 15:29 Also, see [HowTo: Use Bash Parameter Substitution Like A Pro](https://www.cyberciti.biz/tips/bash-shell-parameter-substitution-2.html) [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-60226) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-60226 "permalink to this comment") - monr4 Jul 15, 2011 @ 21:09 hello, a have a problem, i need to do a script por a rutine something like for i in \$(cat list1) but with 2 list for i in \$(cat list1) AND b in \$(cat list2);do mknod /dev/\$i/group c64\[\$b\];done somebody helpme with that please [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-60779) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-60779 "permalink to this comment") - Hari Dec 7, 2011 @ 12:21 Hello I have a problem .i want to run a program N times by give N number of different .txt files from a directory. i want to give all the files in single time by using loops can any one help me to script . [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-65268) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-65268 "permalink to this comment") - Saad Jan 10, 2012 @ 6:48 Is it possible to know how many files will this loop run on, e.g. in the following loop for f in \*.pdf do done; can i know how many files are present in \*.pdf glob, and can i use that to know how many files have been processed out of the total number [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-66581) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-66581 "permalink to this comment") - RDuke Jan 10, 2012 @ 15:32 @saad: maybe something like count=\$( ls -1 \*.pdf \| wc -l ) [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-66603) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-66603 "permalink to this comment") - 👮🛡️  [nixCraft](https://www.vivekgite.com/) Jan 11, 2012 @ 8:27 Pure bash code and nothing else :) ``` filecount=(*.pdf) echo "Total files in $PWD: ${#filecount[@]}" ``` [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-66628) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-66628 "permalink to this comment") - Potato Jan 11, 2012 @ 0:26 try COUNT=\`ls -1 \*.pdf \| wc -l\` [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-66619) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-66619 "permalink to this comment") - ziming May 12, 2012 @ 10:41 Very good tutorials. Helps me lot. Thx mate. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-69342) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-69342 "permalink to this comment") - Manuel Rodriguez May 30, 2012 @ 16:42 good stuff [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-69599) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-69599 "permalink to this comment") - Mike Feb 14, 2013 @ 7:53 thanks this was helpfull (still!!) ;) and I have been scripting for 20 years\! [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-79499) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-79499 "permalink to this comment") - Inukaze Oct 11, 2013 @ 19:51 Hi there im trying to say to Bash , wait for a Process Start / Begin im trying by this way for example ``` notepad=`pidof notepad.exe` until [ $notepad > 0 ] do taskset -p 03 $notepad renince -n 5 -p $notepad done ``` well i have 2 questions: First -\> why this generate a file called “0” (the file are empty) i dont wanna make a new file , just wait for the PID to check execution. Second -\> This is a Loop , but if the 2 commands are execute correclty , 1 time , how i can continue to done ??? [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-99206) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-99206 "permalink to this comment") - Aaron Burns Apr 1, 2014 @ 21:12 I keep seeing “for f in “…. what is the f for. I don’t see it defined anywhere. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-126866) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-126866 "permalink to this comment") - [Nix Craft](https://www.vivekgite.com/) Apr 2, 2014 @ 7:51 f is a shell variable see – <https://www.cyberciti.biz/faq/bash-for-loop/> or <http://bash.cyberciti.biz/guide/Main_Page> [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-126889) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-126889 "permalink to this comment") - Giri Mar 19, 2015 @ 9:31 Hi All, I want to write a shell script for processing the input files(file.csv) from one path and redirect to the other mount or same mount for required query based on the acknowledgment file name (same as input file but file.done) .Please help me [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-609866) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-609866 "permalink to this comment") - JKT Apr 20, 2015 @ 20:07 It should be noted that none of the above examples work if you want to explicitly specify a directory to iterate through (instead of the current directory, aka \*) that has a space in it. I still can’t figure out how to do that. If I set a variable to the explicit path, neither quotes nor delimiters allow iteration through that directory; the script’s attempt to navigate to that directory stops at the space as if a new argument started after it. And since this isn’t that unusual a situation, it’s unfortunate nobody covered it above. Everyone got stuck on using the current directory, which I would think is actually a less-versatile construct. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-646642) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-646642 "permalink to this comment") - The Calibre Jun 9, 2015 @ 16:11 Very good tutorial, Thanks. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-683836) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-683836 "permalink to this comment") - MiaS Jul 14, 2015 @ 15:22 It seems to be a nice thread. I’m having one similar issue and I guess Unix Loop might help. I need a write a cleanup script in unix shell to delete files and folders older than 3 days but the problem is that I need to exclude .snp files/folders. And that can be present any where in the directory. I tried with normal “find” command but that is not excluding the file from nested directory… It seems only loop can help… Any idea? [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-708967) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-708967 "permalink to this comment") - Captain Obvious Jan 19, 2016 @ 15:57 Continue to use your find command. You can match the inverse of patterns with grep. The following will return everything-but what is specified in the pattern. `grep -v somepatternIwanttoexclude` so for example, `find blah blah | grep -v .snp | grep -v .snpFolder` [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-776104) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-776104 "permalink to this comment") - Alan Santos Jul 17, 2015 @ 13:52 Very usefull\! Thank you\! att, [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-711345) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-711345 "permalink to this comment") - Arun Feb 17, 2016 @ 15:59 shopt -s nullglob helped me. Thanks [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-781439) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-781439 "permalink to this comment") - Old Unix Dude Apr 4, 2016 @ 19:35 It is worth noting that none of these examples that use \* will scale. It is much more reliable to use the find command. To “loop” over every file in /dir and run some\_command : find /dir -exec some\_command {} \\; HINT: find replaces {} with the current filename and \\; marks the end of the command. [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-816596) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-816596 "permalink to this comment") - Marek Feb 12, 2021 @ 8:20 I would definitely enrich the loops with a check if file exists: ``` FILES=/path/to/* for f in $FILES do [ -e "$f" ] || continue echo "Processing $f file..." # take action on each file. $f store current file name cat $f done ``` [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-933517) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-933517 "permalink to this comment") - KYY May 18, 2021 @ 4:31 in this example: there is a typo, it should be \_base, not base thanks\! [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-934281) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-934281 "permalink to this comment") - 👮🛡️  [Vivek Gite](https://www.vivekgite.com/) May 18, 2021 @ 20:25 Fixed it. Thanks\! [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-934284) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-934284 "permalink to this comment") - Richard G. Aug 24, 2022 @ 10:32 can also get a list of text from a file using this syntax… ``` for f in $(cat myfiles.txt); do echo "file actions on $f" done ``` [↩](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-943034) [∞](https://www.cyberciti.biz/faq/bash-loop-over-file/#comment-943034 "permalink to this comment") Leave a Reply[Cancel reply](https://www.cyberciti.biz/faq/bash-loop-over-file/#respond) Use HTML \<pre\>...\</pre\> for code samples. Your comment will appear only after approval by the site admin. Next FAQ: [How To Show or Hide Line Numbers In vi / vim Text Editor](https://www.cyberciti.biz/faq/vi-show-line-numbers/) Previous FAQ: [FreeBSD Accounting: Install and Configure System Activity Reporter (SAR)](https://www.cyberciti.biz/faq/freebsd-bsdsar-installation-configuration/) 🔥 FEATURED ARTICLES - 1 [30 Cool Open Source Software I Discovered in 2013](https://www.cyberciti.biz/open-source/30-cool-best-open-source-softwares-of-2013/) - 2 [30 Handy Bash Shell Aliases For Linux / Unix / Mac OS X](https://www.cyberciti.biz/tips/bash-aliases-mac-centos-linux-unix.html) - 3 [Top 32 Nmap Command Examples For Linux Sys/Network Admins](https://www.cyberciti.biz/networking/nmap-command-examples-tutorials/) - 4 [25 PHP Security Best Practices For Linux Sys Admins](https://www.cyberciti.biz/tips/php-security-best-practices-tutorial.html) - 5 [30 Linux System Monitoring Tools Every SysAdmin Should Know](https://www.cyberciti.biz/tips/top-linux-monitoring-tools.html) - 6 [40 Linux Server Hardening Security Tips](https://www.cyberciti.biz/tips/linux-security.html) - 7 [Linux: 25 Iptables Netfilter Firewall Examples For New SysAdmins](https://www.cyberciti.biz/tips/linux-iptables-examples.html) - 8 [Top 20 OpenSSH Server Best Security Practices](https://www.cyberciti.biz/tips/linux-unix-bsd-openssh-server-best-practices.html) - 9 [Top 25 Nginx Web Server Best Security Practices](https://www.cyberciti.biz/tips/linux-unix-bsd-nginx-webserver-security.html) - 10 [My 10 UNIX Command Line Mistakes](https://www.cyberciti.biz/tips/my-10-unix-command-line-mistakes.html) 👀 /etc - ➔ [Linux shell scripting tutorial](https://bash.cyberciti.biz/guide/Main_Page "Linux Shell Scripting Tutorial Wiki") - ➔ [RSS/Feed](https://www.cyberciti.com/atom/atom.xml "Get updates using RSS feed") - ➔ [About nixCraft](https://www.cyberciti.biz/tips/about-us "About nixCraft") ©2002-2025 nixCraft • [Privacy](https://www.cyberciti.biz/tips/privacy "Privacy policy") • [ToS](https://www.cyberciti.biz/tips/disclaimer "Term of Service") • [Contact/Email](https://www.cyberciti.biz/tips/contact-us "Contact us via Email") • Corporate patron [Cloudflare](https://www.cyberciti.biz/tips/nixcraft-sponsors "Corporate patron")