Concatenating a list of files using For loop

bellicosemagpie

I'm new to unix/bash/code in general, and I'm trying to merge multiple sample files with the same name in different directories into a new file in its own directory (the outputs of multiple data collections).

To do this, I'm trying to create a script which uses two arrays - listA, the name of each sample file, and list B, the names for the newly merged sample files.

My code looks a little like this:

#!/bin/sh

listA=( old1 old2 old3 etc.)
listB=( new1 new2 new3 etc.)

i=0

for i in $listA $listB

do

cp ./folder1/$listA ./merged/$listB
cat ./folder2/$listA >> ./merged/$listB
cat ./folder3/$listA >> ./merged/$listB
cat ./folder4/$listA >> ./merged/$listB

((i=+1))

done
echo "Done stitching"

As is, it seems to merge the files for the first entry in listA into the first file in listB perfectly, but it won't repeat the process for the subsequent entries on the list.

Any advice to make this work as intended? Apologies for my ignorance, I'm very new to all of this and enjoying the learning process immensely - just a bit stumped.

Sparhawk

Short answer

while IFS=$'\t' read -r old new; do cat folder*/"$old" > merged/"$new"; done <list.tsv

This assumes that list.tsv contains a tab-delimited list of source (old) and destination (new) files.

Explanation

  • while IFS=$'\t' read -r old new; FOO; done <list.tsv: this is a common shell contruct to read each line of list.tsv, then do FOO on each line. In this case, the input file separator IFS is a tab $'\t', i.e. the first field of the line is saved to $old, and the second field (after the tab) is saved to $new.
  • cat folder*/"$old" > merged/"$new": concatenate the files in folder*/"$old", and write this to merged/"$new". This assumes that the source files are all in folder*, e.g. named folder1, folder2, etc.
  • As with any script, I would certainly test this before running it. You can do that by copying a subset of your files to another directory, then using a short version of your input list list.tsv. Check the results, and if okay, then run it for real (but keep a backup!).

Notes on your suggested solution

I just thought I'd write a few comments on your proposed solution, in order to clarify how the various commands work. Hopefully this is helpful to your understanding!

  • listA=( old1 old2 old3 etc.). This creates an array. The way to access parts of it is shown here. Using for i in $listA… will just access the first entry, as you have (kinda) observed.
  • for i in a b c. This iterates over a, b, and c, putting each into $i. For example, try running for i in a b c; do echo $i; done. N.B. it makes no sense to refer to $listA within the do…done block. Similarly, i=0 and ((i=+1)) are inconsistent with for usage.
  • ./folder1/$listA. ./ is unnecessary. . is the current folder, so it doesn't do anything here.
  • cat FOO >>. cat will concatenate by itself (hence the name). There is no need to write separate cat statements that append >> to a file. You can just list multiple files after cat, e.g. cat file1 file2 file3 > output_file.

この記事はインターネットから収集されたものであり、転載の際にはソースを示してください。

侵害の場合は、連絡してください[email protected]

編集
0

コメントを追加

0

関連記事

分類Dev

Concatenating files from a directory using cat gives "No such file or directory" error

分類Dev

Loop Through List of Files in Bash

分類Dev

Loop through list of files in Makefile

分類Dev

Using a For Loop to Sort and Save Files Using Grep

分類Dev

converting list into dictionary using for loop

分類Dev

Using files in directory that are matching with a list

分類Dev

Concatenating and Minifying HTML/CSS/JS files with npm?

分類Dev

Iterate all files in a directory using a 'for' loop

分類Dev

Python - Renaming all files in a directory using a loop

分類Dev

compare two files using loop to see what strings are not in both files

分類Dev

Better way for concatenating two sorted list of integers

分類Dev

Concatenating all list values to an array in python

分類Dev

Concatenating a dictionary item with a string inside a list

分類Dev

Using List Stream filter vs for loop

分類Dev

Loop through a list using puppeteer and click a button

分類Dev

Adding Elements to List using For Loop in R

分類Dev

how to make list comprehension using while in loop

分類Dev

How to list all subdirectories and files using nodeJS?

分類Dev

Display list of files in webpage using servlet

分類Dev

For-loop over list of txt.files with if conditions in R

分類Dev

How to loop over ever-increasing list of files in bash?

分類Dev

Concatenating LINQ query results when using loops

分類Dev

using sql concat() in django for concatenating two columns

分類Dev

combine 2 similars list of data frame into one list using a loop

分類Dev

Create list of csv files, open one, loop through each row, then open the next and loop through each row?

分類Dev

How to loop through multiple pdf files using R libraries

分類Dev

Using st_write in a for loop to export multiple shape files

分類Dev

How to apply an awk script to multiple files in one folder using a loop

分類Dev

Using an initializer list of lambdas in a range-based loop

Related 関連記事

  1. 1

    Concatenating files from a directory using cat gives "No such file or directory" error

  2. 2

    Loop Through List of Files in Bash

  3. 3

    Loop through list of files in Makefile

  4. 4

    Using a For Loop to Sort and Save Files Using Grep

  5. 5

    converting list into dictionary using for loop

  6. 6

    Using files in directory that are matching with a list

  7. 7

    Concatenating and Minifying HTML/CSS/JS files with npm?

  8. 8

    Iterate all files in a directory using a 'for' loop

  9. 9

    Python - Renaming all files in a directory using a loop

  10. 10

    compare two files using loop to see what strings are not in both files

  11. 11

    Better way for concatenating two sorted list of integers

  12. 12

    Concatenating all list values to an array in python

  13. 13

    Concatenating a dictionary item with a string inside a list

  14. 14

    Using List Stream filter vs for loop

  15. 15

    Loop through a list using puppeteer and click a button

  16. 16

    Adding Elements to List using For Loop in R

  17. 17

    how to make list comprehension using while in loop

  18. 18

    How to list all subdirectories and files using nodeJS?

  19. 19

    Display list of files in webpage using servlet

  20. 20

    For-loop over list of txt.files with if conditions in R

  21. 21

    How to loop over ever-increasing list of files in bash?

  22. 22

    Concatenating LINQ query results when using loops

  23. 23

    using sql concat() in django for concatenating two columns

  24. 24

    combine 2 similars list of data frame into one list using a loop

  25. 25

    Create list of csv files, open one, loop through each row, then open the next and loop through each row?

  26. 26

    How to loop through multiple pdf files using R libraries

  27. 27

    Using st_write in a for loop to export multiple shape files

  28. 28

    How to apply an awk script to multiple files in one folder using a loop

  29. 29

    Using an initializer list of lambdas in a range-based loop

ホットタグ

アーカイブ