🕷️ Crawler Inspector

#  By clicking “Sign up”, you agree to our [terms of service](https://unix.stackexchange.com/legal/terms-of-service/public) and acknowledge you have read our [privacy policy](https://unix.stackexchange.com/legal/privacy-policy). # OR Already have an account? [Log in](https://unix.stackexchange.com/users/login) [Skip to main content](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#content) #### Stack Exchange Network Stack Exchange network consists of 183 Q\&A communities including [Stack Overflow](https://stackoverflow.com/), the largest, most trusted online community for developers to learn, share their knowledge, and build their careers. [Visit Stack Exchange](https://stackexchange.com/) 1. - [Tour Start here for a quick overview of the site](https://unix.stackexchange.com/tour) - [Help Center Detailed answers to any questions you might have](https://unix.stackexchange.com/help) - [Meta Discuss the workings and policies of this site](https://unix.meta.stackexchange.com/) - [About Us Learn more about Stack Overflow the company, and our products](https://stackoverflow.co/) 2. ### [current community](https://unix.stackexchange.com/) - [Unix & Linux](https://unix.stackexchange.com/) [help](https://unix.stackexchange.com/help) [chat](https://chat.stackexchange.com/?tab=site&host=unix.stackexchange.com) - [Unix & Linux Meta](https://unix.meta.stackexchange.com/) ### your communities [Sign up](https://unix.stackexchange.com/users/signup?ssrc=site_switcher&returnurl=https%3A%2F%2Funix.stackexchange.com%2Fquestions%2F7011%2Fhow-to-loop-over-the-lines-of-a-file) or [log in](https://unix.stackexchange.com/users/login?ssrc=site_switcher&returnurl=https%3A%2F%2Funix.stackexchange.com%2Fquestions%2F7011%2Fhow-to-loop-over-the-lines-of-a-file) to customize your list. ### [more stack exchange communities](https://stackexchange.com/sites) [company blog](https://stackoverflow.blog/) 3. [Log in](https://unix.stackexchange.com/users/login?ssrc=head&returnurl=https%3A%2F%2Funix.stackexchange.com%2Fquestions%2F7011%2Fhow-to-loop-over-the-lines-of-a-file) 4. [Sign up](https://unix.stackexchange.com/users/signup?ssrc=head&returnurl=https%3A%2F%2Funix.stackexchange.com%2Fquestions%2F7011%2Fhow-to-loop-over-the-lines-of-a-file) [](https://unix.stackexchange.com/) 1. 1. [Home](https://unix.stackexchange.com/) 2. [Questions](https://unix.stackexchange.com/questions) 3. [Unanswered](https://unix.stackexchange.com/unanswered) 4. [AI Assist](https://stackoverflow.com/ai-assist) 5. [Tags](https://unix.stackexchange.com/tags) 6. [Chat](https://chat.stackexchange.com/) 7. [Users](https://unix.stackexchange.com/users) 8. [Companies](https://stackoverflow.com/jobs/companies?so_medium=unix&so_source=SiteNav) 2. Stack Internal Stack Overflow for Teams is now called **Stack Internal**. Bring the best of human thought and AI automation together at your work. [Try for free](https://stackoverflowteams.com/teams/create/free/?utm_medium=referral&utm_source=unix-community&utm_campaign=side-bar&utm_content=explore-teams) [Learn more](https://stackoverflow.co/internal/?utm_medium=referral&utm_source=unix-community&utm_campaign=side-bar&utm_content=explore-teams) 3. [Stack Internal]() 4. Bring the best of human thought and AI automation together at your work. [Learn more](https://stackoverflow.co/internal/?utm_medium=referral&utm_source=unix-community&utm_campaign=side-bar&utm_content=explore-teams-compact) **Stack Internal** Knowledge at work Bring the best of human thought and AI automation together at your work. [Explore Stack Internal](https://stackoverflow.co/internal/?utm_medium=referral&utm_source=unix-community&utm_campaign=side-bar&utm_content=explore-teams-compact-popover) # [How to loop over the lines of a file?](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file) [Ask Question](https://unix.stackexchange.com/questions/ask) Asked 15 years, 1 month ago Modified [4 years, 3 months ago](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file?lastactivity "2021-11-21 21:34:14Z") Viewed 340k times This question shows research effort; it is useful and clear 93 Save this question. Show activity on this post. Say I have this file: ``` hello world hello world ``` This program ``` #!/bin/bash for i in $(cat $1); do echo "tester: $i" done ``` outputs ``` tester: hello tester: world tester: hello tester: world ``` I'd like to have the `for` iterate over each line individually ignoring whitespaces though, i.e. the last two lines should be replaced by ``` tester: hello world ``` Using quotes `for i in "$(cat $1)";` results in `i` being assigned the whole file at once. What should I change? - [shell](https://unix.stackexchange.com/questions/tagged/shell "show questions tagged 'shell'") - [control-flow](https://unix.stackexchange.com/questions/tagged/control-flow "show questions tagged 'control-flow'") [Share](https://unix.stackexchange.com/q/7011 "Short permalink to this question") Share a link to this question Copy link [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/ "The current license for this post: CC BY-SA 3.0") [Improve this question](https://unix.stackexchange.com/posts/7011/edit) Follow Follow this question to receive notifications [edited Apr 17, 2016 at 22:52](https://unix.stackexchange.com/posts/7011/revisions "show all edits to this post") [](https://unix.stackexchange.com/users/4671/faheem-mitha) [Faheem Mitha](https://unix.stackexchange.com/users/4671/faheem-mitha) 36\.2k3333 gold badges133133 silver badges192192 bronze badges asked Feb 7, 2011 at 10:28 [](https://unix.stackexchange.com/users/863/tobias-kienzler) [Tobias Kienzler](https://unix.stackexchange.com/users/863/tobias-kienzler) 9,6121717 gold badges7070 silver badges112112 bronze badges [Add a comment](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file "Use comments to ask for more information or suggest improvements. Avoid answering questions in comments.") \| ## 6 Answers 6 Sorted by: [Reset to default](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file?answertab=scoredesc#tab-top) This answer is useful 100 Save this answer. Show activity on this post. With `for` and [IFS](https://unix.stackexchange.com/questions/16192/what-is-ifs-in-context-of-for-looping): ``` #!/bin/bash IFS=$'\n' # make newlines the only separator set -f # disable globbing for i in $(cat < "$1"); do echo "tester: $i" done ``` Note however that it will skip empty lines as *newline* being an IFS-white-space character, sequences of it count as 1 and the leading and trailing ones are ignored. With `zsh` and `ksh93` (not `bash`), you can change it to `IFS=$'\n\n'` for newline not to be treated specially, however note that all *trailing* newline characters (so that includes trailing empty lines) will always be removed by the command substitution. Or [with `read`](https://unix.stackexchange.com/questions/18886/why-is-while-ifs-read-used-so-often-instead-of-ifs-while-read) (no more `cat`): ``` #!/bin/bash while IFS= read -r line; do echo "tester: $line" done < "$1" ``` There, empty lines are preserved, but note that it would skip the last line if it was not properly delimited by a newline character. [Share](https://unix.stackexchange.com/a/7012 "Short permalink to this answer") Share a link to this answer Copy link [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/ "The current license for this post: CC BY-SA 3.0") [Improve this answer](https://unix.stackexchange.com/posts/7012/edit) Follow Follow this answer to receive notifications [edited Apr 13, 2017 at 12:37](https://unix.stackexchange.com/posts/7012/revisions "show all edits to this post") [](https://unix.stackexchange.com/users/-1/community) [Community](https://unix.stackexchange.com/users/-1/community)Bot 1 answered Feb 7, 2011 at 10:39 [](https://unix.stackexchange.com/users/3069/wag) [wag](https://unix.stackexchange.com/users/3069/wag) 37k1313 gold badges6969 silver badges5151 bronze badges 6 - 7 thanks, I didn't know one could `<` into a whole loop. Although it makes perfectly sense now I saw it Tobias Kienzler – [Tobias Kienzler](https://unix.stackexchange.com/users/863/tobias-kienzler "9,612 reputation") 2011-02-07 11:06:31 +00:00 [Commented Feb 7, 2011 at 11:06](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment8715_7012) - 2 I see `IFS \ read -r line' in second example. Is really`IFS=\` needed ? IMHO it enough to say : `while read -r line; do echo "tester: $line"; done < "$1"` Grzegorz Wierzowiecki – [Grzegorz Wierzowiecki](https://unix.stackexchange.com/users/9689/grzegorz-wierzowiecki "14,880 reputation") 2012-03-19 16:45:42 +00:00 [Commented Mar 19, 2012 at 16:45](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment46873_7012) - 5 @GrzegorzWierzowiecki `IFS=` turns off the stripping of leading and trailing whitespace. See [In `while IFS= read..`, why does IFS have no effect?](http://unix.stackexchange.com/questions/18922/in-while-ifs-read-why-does-ifs-have-no-effect) Gilles 'SO- stop being evil' – [Gilles 'SO- stop being evil'](https://unix.stackexchange.com/users/885/gilles-so-stop-being-evil "868,726 reputation") 2014-01-12 00:29:39 +00:00 [Commented Jan 12, 2014 at 0:29](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment167742_7012) - 1 @BenMares To prevent globbing expressions possibly appearing in the text we are reading from being expanded to matching file names. Try, for instance, `printf '%s\n' '*o' 'bar' >afile; touch foo; IFS=$'\n'; for i in $(cat afile); do echo "$i"; done`. fra-san – [fra-san](https://unix.stackexchange.com/users/315749/fra-san "10,865 reputation") 2020-03-04 08:00:25 +00:00 [Commented Mar 4, 2020 at 8:00](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment1062307_7012) - 3 A `while IFS= read -r line || [ "$line" ]; do` will process a trailing line not properly delimited by a newline character (but it will be added back). user232326 – user232326 2020-03-04 16:15:44 +00:00 [Commented Mar 4, 2020 at 16:15](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment1062498_7012) \| [Show **1** more comment](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file "Expand to show all comments on this post") This answer is useful 29 Save this answer. Show activity on this post. (9 years later:) Both provided answers would fail on files without a newline at the end, this will effectively skip the last line, produce no errors, would lead to disaster (learned hard way:). The best concise solution I found so far that "Just Works" (in both bash and sh): ``` while IFS='' read -r LINE || [ -n "${LINE}" ]; do echo "processing line: ${LINE}" done < /path/to/input/file.txt ``` For more in-depth discussion see this StackOverflow discussion: [How to use "while read" (Bash) to read the last line in a file if there’s no newline at the end of the file?](http://stackoverflow.com/questions/4165135/how-to-use-while-read-bash-to-read-the-last-line-in-a-file-if-theres-no-new) Beware: this approach adds an additional newline to the last line if there is none already. [Share](https://unix.stackexchange.com/a/580545 "Short permalink to this answer") Share a link to this answer Copy link [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/ "The current license for this post: CC BY-SA 4.0") [Improve this answer](https://unix.stackexchange.com/posts/580545/edit) Follow Follow this answer to receive notifications [edited Apr 21, 2020 at 13:45](https://unix.stackexchange.com/posts/580545/revisions "show all edits to this post") answered Apr 16, 2020 at 20:02 [](https://unix.stackexchange.com/users/227338/dmitry-shevkoplyas) [Dmitry Shevkoplyas](https://unix.stackexchange.com/users/227338/dmitry-shevkoplyas) 41244 silver badges66 bronze badges 2 - files with characters after the last newline are not text files, and those characters don't constitute a line. In many cases those bogus characters are better left ignored or removed, though there are cases where you may want to treat it as a extra line, so it's good you show how. Stéphane Chazelas – [Stéphane Chazelas](https://unix.stackexchange.com/users/22565/st%C3%A9phane-chazelas "592,297 reputation") 2023-03-10 07:53:48 +00:00 [Commented Mar 10, 2023 at 7:53](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment1403700_580545) - Note that files with NUL characters in them are also not text files, and except in zsh that loop would also *fail* if the intention was to keep them as if they were allowed in text files. Stéphane Chazelas – [Stéphane Chazelas](https://unix.stackexchange.com/users/22565/st%C3%A9phane-chazelas "592,297 reputation") 2023-03-10 07:59:27 +00:00 [Commented Mar 10, 2023 at 7:59](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment1403701_580545) [Add a comment](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file "Use comments to ask for more information or suggest improvements. Avoid comments like “+1” or “thanks”.") \| This answer is useful 5 Save this answer. Show activity on this post. If you can avoid it, don't especially if it's to process text. Most text utilities are already designed to process text one line at a time, and, at least for the GNU implementations, do it efficiently, correctly and handle error conditions nicely. Piping one to another which runs them in parallel also means you can leverage more than one processor to do the job. Here: ``` <input.txt sed 's/^/tester /' > output.txt ``` Or: ``` <input.txt awk '{print "tester", $0}' > output.txt ``` More on that at: [Why is using a shell loop to process text considered bad practice?](https://unix.stackexchange.com/q/169716) If it's not about text processing and you do need to run some command per line of a file, also note GNU `xargs` where you can do: ``` xargs -rd'\n' -I@ -a input.txt cp -- @ @.back ``` for instance. With the bash shell, you can get each line of a file into an array with the `readarray` builtin: ``` readarray -t lines < input.txt && for line in "${lines[@]}"; do do-some-non-text-processing-you-cannot-easily-do-with-xargs "$line" || break done ``` POSIXly, you can use `IFS= read -r line` to read one line off some input, but beware that if you redirect the whole `while read` loop with the input file on stdin, then commands inside the loop will also have their stdin redirected to the file, so best is to use a different fd which you close inside the loop: ``` while IFS= read -r line <&3 || [ -n "$line" ] # to cover for an unterminated last line. do { do-some-non-text-processing-you-cannot-easily-do-with-xargs "$line" || break # abort upon failure if relevant } 3<&- done 3< input.txt > output.txt ``` `read -r line` removes leading and trailing whitespace characters from the line that it reads provided they are in the `$IFS` variable, though only the `yash` shell honours that POSIX requirement. With most shells, that's limited to space and tab. ksh93 and recent versions of `bash` do it for all single-byte characters considered as whitespace in the locale. So to read a line and also strip leading and trailing blanks, you can do: `IFS=$' \t' read -r line`. With ksh93, yash¹ or recent versions of `bash`. `IFS=$' \t\r'` would also strip the trailing CR character found in text files from the Microsoft world. *** ¹ though `yash` doesn't support the `$'...'` syntax yet, you'd need `IFS=$(printf ' \t\r')` there. [Share](https://unix.stackexchange.com/a/670789 "Short permalink to this answer") Share a link to this answer Copy link [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/ "The current license for this post: CC BY-SA 4.0") [Improve this answer](https://unix.stackexchange.com/posts/670789/edit) Follow Follow this answer to receive notifications [edited Sep 27, 2021 at 6:55](https://unix.stackexchange.com/posts/670789/revisions "show all edits to this post") answered Sep 27, 2021 at 5:52 [](https://unix.stackexchange.com/users/22565/st%C3%A9phane-chazelas) [Stéphane Chazelas](https://unix.stackexchange.com/users/22565/st%C3%A9phane-chazelas) 592k9797 gold badges1\.1k1\.1k silver badges1\.7k1\.7k bronze badges 0 [Add a comment](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file "Use comments to ask for more information or suggest improvements. Avoid comments like “+1” or “thanks”.") \| This answer is useful 2 Save this answer. Show activity on this post. For what it is worth, I need to do that quite often, and can never remember the exact way of using `while IFS= read...`, so I defined the following function in my bash profile: ``` # iterate the line of a file and call input function iterlines() { (( $# < 2 )) && { echo "Usage: iterlines <File> <Callback>"; return; } local File=$1 local Func=$2 n=$(cat "$File" | wc -l) for (( i=1; i<=n; i++ )); do "$Func" "$(sed "${i}q;d" "$File")" done } ``` This function first determines the number of lines in the file, then uses `sed` to extract line after line, and passes each line as a single string argument to any given function. I suppose this might get really inefficient with large files, but that hasn't been a problem for me so far (suggestions on how to improve this welcome of course). The usage is pretty sweet IMO: ``` >> cat example.txt # note the use of spaces, whitespace, etc. a/path This is a sentence. "wi\th quotes" $End >> iterlines example.txt echo # preserves quotes, $ and whitespace a/path This is a sentence. "wi\th quotes" $End >> x() { echo "$#"; }; iterlines example.txt x # line always passed as single input string 1 1 1 1 1 ``` [Share](https://unix.stackexchange.com/a/487978 "Short permalink to this answer") Share a link to this answer Copy link [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/ "The current license for this post: CC BY-SA 4.0") [Improve this answer](https://unix.stackexchange.com/posts/487978/edit) Follow Follow this answer to receive notifications [edited Dec 14, 2018 at 15:13](https://unix.stackexchange.com/posts/487978/revisions "show all edits to this post") answered Dec 14, 2018 at 13:46 [](https://unix.stackexchange.com/users/45354/jonathan-h) [Jonathan H](https://unix.stackexchange.com/users/45354/jonathan-h) 2,56333 gold badges2424 silver badges2929 bronze badges [Add a comment](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file "Use comments to ask for more information or suggest improvements. Avoid comments like “+1” or “thanks”.") \| This answer is useful 1 Save this answer. Show activity on this post. To read all the lines, regardless of whether they are ended with a new line or not: ``` cat "somefile" | { cat ; echo ; } | while read line; do echo $line; done ``` Source : My open source project <https://sourceforge.net/projects/command-output-to-html-table/> [Share](https://unix.stackexchange.com/a/670764 "Short permalink to this answer") Share a link to this answer Copy link [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/ "The current license for this post: CC BY-SA 4.0") [Improve this answer](https://unix.stackexchange.com/posts/670764/edit) Follow Follow this answer to receive notifications [edited Oct 10, 2021 at 6:39](https://unix.stackexchange.com/posts/670764/revisions "show all edits to this post") answered Sep 27, 2021 at 0:07 [](https://unix.stackexchange.com/users/492508/nathan-s-r) [Nathan S.R.](https://unix.stackexchange.com/users/492508/nathan-s-r) 3344 bronze badges 2 - [It is not acceptable to post identical answers to multiple questions.](https://unix.meta.stackexchange.com/a/5849/108618) Kamil Maciorowski – [Kamil Maciorowski](https://unix.stackexchange.com/users/108618/kamil-maciorowski "24,549 reputation") 2021-10-10 08:06:34 +00:00 [Commented Oct 10, 2021 at 8:06](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment1269124_670764) - Hi Kamil Maciorowski, Answers are identical because Questions are identical too \! Nathan S.R. – [Nathan S.R.](https://unix.stackexchange.com/users/492508/nathan-s-r "33 reputation") 2021-10-10 15:36:06 +00:00 [Commented Oct 10, 2021 at 15:36](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment1269177_670764) [Add a comment](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file "Use comments to ask for more information or suggest improvements. Avoid comments like “+1” or “thanks”.") \| This answer is useful \-1 Save this answer. Show activity on this post. There is an answer here that may be useful to people who this doesn't work for [loop through file by row in tcsh](https://unix.stackexchange.com/questions/678487/loop-through-file-by-row-in-tcsh/678531#678531) [Share](https://unix.stackexchange.com/a/678532 "Short permalink to this answer") Share a link to this answer Copy link [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/ "The current license for this post: CC BY-SA 4.0") [Improve this answer](https://unix.stackexchange.com/posts/678532/edit) Follow Follow this answer to receive notifications answered Nov 21, 2021 at 21:34 [](https://unix.stackexchange.com/users/314584/zaks) [ZakS](https://unix.stackexchange.com/users/314584/zaks) 31511 gold badge55 silver badges1313 bronze badges 1 - Not an answer as these usually contain some sort of explanation and/or a procedure to follow. A comment at best but not even a good one? muthuh – [muthuh](https://unix.stackexchange.com/users/100189/muthuh "350 reputation") 2022-01-26 12:27:37 +00:00 [Commented Jan 26, 2022 at 12:27](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file#comment1300882_678532) [Add a comment](https://unix.stackexchange.com/questions/7011/how-to-loop-over-the-lines-of-a-file "Use comments to ask for more information or suggest improvements. Avoid comments like “+1” or “thanks”.") \| ## You must [log in](https://unix.stackexchange.com/users/login?ssrc=question_page&returnurl=https%3A%2F%2Funix.stackexchange.com%2Fquestions%2F7011) to answer this question. Start asking to get answers Find the answer to your question by asking. [Ask question](https://unix.stackexchange.com/questions/ask) Explore related questions - [shell](https://unix.stackexchange.com/questions/tagged/shell "show questions tagged 'shell'") - [control-flow](https://unix.stackexchange.com/questions/tagged/control-flow "show questions tagged 'control-flow'") See similar questions with these tags. - The Overflow Blog - [Open source for awkward robots](https://stackoverflow.blog/2026/03/13/open-source-for-awkward-robots/?cb=1) - Featured on Meta - [Logo updates to Stack Overflow's visual identity](https://meta.stackexchange.com/questions/417394/logo-updates-to-stack-overflows-visual-identity?cb=1) #### Linked [5](https://unix.stackexchange.com/questions/103963/cycle-through-one-commands-output-and-use-in-another-command?lq=1 "Question score (upvotes - downvotes)") [cycle through one command's output and use in another command](https://unix.stackexchange.com/questions/103963/cycle-through-one-commands-output-and-use-in-another-command?noredirect=1&lq=1) [4](https://unix.stackexchange.com/questions/58040/what-is-the-fastest-way-to-process-line-by-line-in-bash?lq=1 "Question score (upvotes - downvotes)") [What is the fastest way to process line by line in bash?](https://unix.stackexchange.com/questions/58040/what-is-the-fastest-way-to-process-line-by-line-in-bash?noredirect=1&lq=1) [4](https://unix.stackexchange.com/questions/437127/bash-is-stripping-leading-whitespace-from-variables?lq=1 "Question score (upvotes - downvotes)") [bash is stripping leading whitespace from variables](https://unix.stackexchange.com/questions/437127/bash-is-stripping-leading-whitespace-from-variables?noredirect=1&lq=1) [\-1](https://unix.stackexchange.com/questions/448493/why-does-using-array-and-for-loop-breaks-line-into-2?lq=1 "Question score (upvotes - downvotes)") [Why does using array and for loop breaks line into 2?](https://unix.stackexchange.com/questions/448493/why-does-using-array-and-for-loop-breaks-line-into-2?noredirect=1&lq=1) [0](https://unix.stackexchange.com/questions/636306/how-to-iterate-over-matches-which-contain-spaces?lq=1 "Question score (upvotes - downvotes)") [How to iterate over matches which contain spaces](https://unix.stackexchange.com/questions/636306/how-to-iterate-over-matches-which-contain-spaces?noredirect=1&lq=1) [288](https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice?lq=1 "Question score (upvotes - downvotes)") [Why is using a shell loop to process text considered bad practice?](https://unix.stackexchange.com/questions/169716/why-is-using-a-shell-loop-to-process-text-considered-bad-practice?noredirect=1&lq=1) [107](https://unix.stackexchange.com/questions/18886/why-is-while-ifs-read-used-so-often-instead-of-ifs-while-read?lq=1 "Question score (upvotes - downvotes)") [Why is \`while IFS= read\` used so often, instead of \`IFS=; while read..\`?](https://unix.stackexchange.com/questions/18886/why-is-while-ifs-read-used-so-often-instead-of-ifs-while-read?noredirect=1&lq=1) [93](https://unix.stackexchange.com/questions/26784/understanding-ifs?lq=1 "Question score (upvotes - downvotes)") [Understanding IFS](https://unix.stackexchange.com/questions/26784/understanding-ifs?noredirect=1&lq=1) [42](https://unix.stackexchange.com/questions/27382/how-do-i-reverse-a-for-loop?lq=1 "Question score (upvotes - downvotes)") [How do I reverse a for loop?](https://unix.stackexchange.com/questions/27382/how-do-i-reverse-a-for-loop?noredirect=1&lq=1) [35](https://unix.stackexchange.com/questions/16192/what-is-the-ifs-variable?lq=1 "Question score (upvotes - downvotes)") [What is the "IFS" variable?](https://unix.stackexchange.com/questions/16192/what-is-the-ifs-variable?noredirect=1&lq=1) [See more linked questions](https://unix.stackexchange.com/questions/linked/7011?lq=1) #### Related [3](https://unix.stackexchange.com/questions/67175/bash-while-loop-that-breaks-at-a-given-file-size?rq=1 "Question score (upvotes - downvotes)") [bash while loop that breaks at a given file size](https://unix.stackexchange.com/questions/67175/bash-while-loop-that-breaks-at-a-given-file-size?rq=1) [3](https://unix.stackexchange.com/questions/84826/looping-over-a-folder-enter-anyways-the-loop?rq=1 "Question score (upvotes - downvotes)") [Looping over a folder enter anyways the loop](https://unix.stackexchange.com/questions/84826/looping-over-a-folder-enter-anyways-the-loop?rq=1) [1](https://unix.stackexchange.com/questions/213466/for-loop-for-a-set-of-variables?rq=1 "Question score (upvotes - downvotes)") [For loop for a set of variables](https://unix.stackexchange.com/questions/213466/for-loop-for-a-set-of-variables?rq=1) [2](https://unix.stackexchange.com/questions/225219/cron-only-occasionally-sends-e-mail-on-output-and-errors?rq=1 "Question score (upvotes - downvotes)") [Cron only occasionally sends e-mail on output and errors](https://unix.stackexchange.com/questions/225219/cron-only-occasionally-sends-e-mail-on-output-and-errors?rq=1) [4](https://unix.stackexchange.com/questions/299727/maintain-filenames-as-separate-arguments-in-successive-commands?rq=1 "Question score (upvotes - downvotes)") [Maintain filenames as separate arguments in successive commands](https://unix.stackexchange.com/questions/299727/maintain-filenames-as-separate-arguments-in-successive-commands?rq=1) [3](https://unix.stackexchange.com/questions/639859/how-to-loop-through-file-and-execute-the-lines?rq=1 "Question score (upvotes - downvotes)") [How to loop through file and execute the lines](https://unix.stackexchange.com/questions/639859/how-to-loop-through-file-and-execute-the-lines?rq=1) [1](https://unix.stackexchange.com/questions/794990/issue-with-bash-lineno-in-bash-function-which-get-debugging-information?rq=1 "Question score (upvotes - downvotes)") [Issue with \`BASH\_LINENO\` in bash function which get debugging information](https://unix.stackexchange.com/questions/794990/issue-with-bash-lineno-in-bash-function-which-get-debugging-information?rq=1) #### [Hot Network Questions](https://stackexchange.com/questions?tab=hot) - [What's the difference between these two versions of Little Nightmares: Enhanced Edition?](https://gaming.stackexchange.com/questions/418159/whats-the-difference-between-these-two-versions-of-little-nightmares-enhanced) - [How random is RandomInteger\[{0,B}\]?](https://mathematica.stackexchange.com/questions/319020/how-random-is-randominteger0-b) - [Pi Day: estimating pi using probability](https://mathoverflow.net/questions/509101/pi-day-estimating-pi-using-probability) - [How should I compare two survival prediction models when a predictor’s definition drifts over time?](https://stats.stackexchange.com/questions/675174/how-should-i-compare-two-survival-prediction-models-when-a-predictor-s-definitio) - [How does Validation work for Time-Series Forecasting?](https://datascience.stackexchange.com/questions/137847/how-does-validation-work-for-time-series-forecasting) - [How does user-configurable SRAM/flash size work?](https://electronics.stackexchange.com/questions/766946/how-does-user-configurable-sram-flash-size-work) - [What would the Latin phrase be for "Stellar Wanderer"?](https://latin.stackexchange.com/questions/27121/what-would-the-latin-phrase-be-for-stellar-wanderer) - [Automatically include -converted version of images in LaTeX if it exists](https://tex.stackexchange.com/questions/760850/automatically-include-converted-version-of-images-in-latex-if-it-exists) - [Why do archaic colonial‑era laws remain in effect long after independence in many Asian and African countries?](https://politics.stackexchange.com/questions/94374/why-do-archaic-colonial-era-laws-remain-in-effect-long-after-independence-in-man) - [A question on a type of doxastic logic](https://philosophy.stackexchange.com/questions/137012/a-question-on-a-type-of-doxastic-logic) - [Beamer: Align content to top when having text and bullet list](https://tex.stackexchange.com/questions/760906/beamer-align-content-to-top-when-having-text-and-bullet-list) - [Definition of n-huge cardinals in Harvey Friedman's works](https://mathoverflow.net/questions/509093/definition-of-n-huge-cardinals-in-harvey-friedmans-works) - [When is a theory effective?](https://physics.stackexchange.com/questions/870082/when-is-a-theory-effective) - ["Out of malice"?](https://ell.stackexchange.com/questions/374290/out-of-malice) - [LTSpice model of IRF520 body diode](https://electronics.stackexchange.com/questions/766936/ltspice-model-of-irf520-body-diode) - [Will modeling frameworks like JuMP become the dominant interface for complex algorithmic research?](https://or.stackexchange.com/questions/13525/will-modeling-frameworks-like-jump-become-the-dominant-interface-for-complex-alg) - [Vertical asymptote](https://mathematica.stackexchange.com/questions/319015/vertical-asymptote) - [How to calculate the total actual positives given known precision, recall and false omission rate and number of predicted positives/negatives?](https://stats.stackexchange.com/questions/675159/how-to-calculate-the-total-actual-positives-given-known-precision-recall-and-fa) - [About the Dwarapalakas of other deities](https://hinduism.stackexchange.com/questions/69741/about-the-dwarapalakas-of-other-deities) - [Where can I find video of someone struggling to get into the Lunar Module with a rope?](https://space.stackexchange.com/questions/70366/where-can-i-find-video-of-someone-struggling-to-get-into-the-lunar-module-with-a) - [How to get list of installed packages of previous TeX Live installation?](https://tex.stackexchange.com/questions/760883/how-to-get-list-of-installed-packages-of-previous-tex-live-installation) - [Cardinality of the set of isomorphism classes of 2-generated groups](https://math.stackexchange.com/questions/5128742/cardinality-of-the-set-of-isomorphism-classes-of-2-generated-groups) - [Accounts for changing Kinetic energy of electrons in a circuit](https://physics.stackexchange.com/questions/870146/accounts-for-changing-kinetic-energy-of-electrons-in-a-circuit) - [What is the best folder structure for custom KiCad symbol and footprint libraries (Best Practices)?](https://electronics.stackexchange.com/questions/766950/what-is-the-best-folder-structure-for-custom-kicad-symbol-and-footprint-librarie) [Question feed](https://unix.stackexchange.com/feeds/question/7011 "Feed of this question and its answers") # Subscribe to RSS Question feed To subscribe to this RSS feed, copy and paste this URL into your RSS reader.  lang-bash # Why are you flagging this comment? It contains harassment, bigotry or abuse. This comment attacks a person or group. Learn more in our [Abusive behavior policy](https://unix.stackexchange.com/conduct/abusive-behavior). It's unfriendly or unkind. This comment is rude or condescending. Learn more in our [Code of Conduct](https://unix.stackexchange.com/conduct/abusive-behavior). Not needed. This comment is not relevant to the post. ``` ``` Enter at least 6 characters Something else. A problem not listed above. Try to be as specific as possible. ``` ``` Enter at least 6 characters Flag comment Cancel You have 0 flags left today #  # Hang on, you can't upvote just yet. You'll need to complete a few actions and gain 15 reputation points before being able to upvote. **Upvoting** indicates when questions and answers are useful. [What's reputation and how do I get it?](https://stackoverflow.com/help/whats-reputation) Instead, you can save this post to reference later. Save this post for later Not now ##### [Unix & Linux](https://unix.stackexchange.com/) - [Tour](https://unix.stackexchange.com/tour) - [Help](https://unix.stackexchange.com/help) - [Chat](https://chat.stackexchange.com/?tab=site&host=unix.stackexchange.com) - [Contact](https://unix.stackexchange.com/contact) - [Feedback](https://unix.meta.stackexchange.com/) ##### [Company](https://stackoverflow.co/) - [Stack Overflow](https://stackoverflow.com/) - [Stack Internal](https://stackoverflow.co/internal/) - [Stack Data Licensing](https://stackoverflow.co/data-licensing/) - [Stack Ads](https://stackoverflow.co/advertising/) - [About](https://stackoverflow.co/) - [Press](https://stackoverflow.co/company/press/) - [Legal](https://stackoverflow.com/legal) - [Privacy Policy](https://stackoverflow.com/legal/privacy-policy) - [Terms of Service](https://stackoverflow.com/legal/terms-of-service/public) - Cookie Settings - [Cookie Policy](https://policies.stackoverflow.co/stack-overflow/cookie-policy) ##### [Stack Exchange Network](https://stackexchange.com/) - [Technology](https://stackexchange.com/sites#technology) - [Culture & recreation](https://stackexchange.com/sites#culturerecreation) - [Life & arts](https://stackexchange.com/sites#lifearts) - [Science](https://stackexchange.com/sites#science) - [Professional](https://stackexchange.com/sites#professional) - [Business](https://stackexchange.com/sites#business) - [API](https://api.stackexchange.com/) - [Data](https://data.stackexchange.com/) - [Blog](https://stackoverflow.blog/?blb=1) - [Facebook](https://www.facebook.com/officialstackoverflow/) - [Twitter](https://twitter.com/stackoverflow) - [LinkedIn](https://linkedin.com/company/stack-overflow) - [Instagram](https://www.instagram.com/thestackoverflow) Site design / logo © 2026 Stack Exchange Inc; user contributions licensed under [CC BY-SA](https://stackoverflow.com/help/licensing) . rev 2026.3.13.41005 Linux is a registered trademark of Linus Torvalds. UNIX is a registered trademark of The Open Group. This site is not affiliated with Linus Torvalds or The Open Group in any way.

If you can avoid it, don't especially if it's to process text. Most text utilities are already designed to process text one line at a time, and, at least for the GNU implementations, do it efficiently, correctly and handle error conditions nicely. Piping one to another which runs them in parallel also means you can leverage more than one processor to do the job. Here: ``` <input.txt sed 's/^/tester /' > output.txt ``` Or: ``` <input.txt awk '{print "tester", $0}' > output.txt ``` More on that at: [Why is using a shell loop to process text considered bad practice?](https://unix.stackexchange.com/q/169716) If it's not about text processing and you do need to run some command per line of a file, also note GNU `xargs` where you can do: ``` xargs -rd'\n' -I@ -a input.txt cp -- @ @.back ``` for instance. With the bash shell, you can get each line of a file into an array with the `readarray` builtin: ``` readarray -t lines < input.txt && for line in "${lines[@]}"; do do-some-non-text-processing-you-cannot-easily-do-with-xargs "$line" || break done ``` POSIXly, you can use `IFS= read -r line` to read one line off some input, but beware that if you redirect the whole `while read` loop with the input file on stdin, then commands inside the loop will also have their stdin redirected to the file, so best is to use a different fd which you close inside the loop: ``` while IFS= read -r line <&3 || [ -n "$line" ] # to cover for an unterminated last line. do { do-some-non-text-processing-you-cannot-easily-do-with-xargs "$line" || break # abort upon failure if relevant } 3<&- done 3< input.txt > output.txt ``` `read -r line` removes leading and trailing whitespace characters from the line that it reads provided they are in the `$IFS` variable, though only the `yash` shell honours that POSIX requirement. With most shells, that's limited to space and tab. ksh93 and recent versions of `bash` do it for all single-byte characters considered as whitespace in the locale. So to read a line and also strip leading and trailing blanks, you can do: `IFS=$' \t' read -r line`. With ksh93, yash¹ or recent versions of `bash`. `IFS=$' \t\r'` would also strip the trailing CR character found in text files from the Microsoft world. *** ¹ though `yash` doesn't support the `$'...'` syntax yet, you'd need `IFS=$(printf ' \t\r')` there.