Thoughts on life, liberty, and information technology

Using the FOR command to copy files listed in a text file

A coworker asked me for a script. Here’s the request:

… would want to copy all files on this list [an attached text document] to another location (doesn’t really matter where for now). All are currently located in \\server\share\folder. The path in the new location should begin with the part after “folder”…

In a nutshell, here is the high-level description of what the script must do:

Given a text file which provides a list of files, copy the files from a fixed source to a fixed destination, recreating the directory trees on the destination. A simple file copy won’t work because there may be files in the source folders which should not be copied.

Sample content of the text file is:

client\CD120\Samarai Legends\Drafts\drafts folder.txt
client\CD120\Samarai Legends\Inbox\Legends.doc
client\CD120\Bushido Warriors\Inbox\Warrior Code.doc

The solution to this is to make a simple batch file that parses the content of the text file, generating the appropriate xcopy command to copy the file. We’ll call the batch file xcopylist.bat; its one line of content is below. Change c:\temp\ to whatever path you want the files copied to. (I used c:\temp\ for testing.) Change \\server\share\folder to the root folder of the files to copy.

for /f "delims=" %%i in (filelist.txt) do echo D|xcopy "\\server\share\folder\%%i" "c:\temp\%%i" /i /z /y

Put the file list in the same directory as the batch file and name the file list filelist.txt. Then run the batch file and viola! You’ve got to love the for command, which lets you (among other things) parse text files and use the line-by-line output. Another trick in here is the output parser pipe, which allows us to automatically press the “D” key with each xcopy command.

Note: The batch file overwrites files in the destination automatically. To turn this off (have it prompt you), change /y to /y- in the batch file. However, if you’re using Windows NT 4.0, just delete the /y switch altogether – it’s only supported in Windows XP and Windows 2000.

146 responses to “Using the FOR command to copy files listed in a text file”

  1. Adeya Avatar
    Adeya

    Hello, Brian!

    By executing the batch I started to get prompt:
    “Does C:\temp_ad\output\test2.xlsx specify a file name or directory name on the target
    <F = file, D = directory)?"

    I think it started after MS Office 2013 was installed. My OS is WINDOWS 7.

    How to prevent this prompt or how to select 'F' automatically?

    Thank you.
    Adeya

    Like

  2. brian Avatar
    brian

    Can you send me the command that your batch file is generating that is causing this error?

    Like

  3. Adeya Avatar
    Adeya

    for /f “delims=” %%i in (filelist.txt) do xcopy “C:\Test1\%%i” “C:\Test1\Test2\%%i”

    Batch file itself and filelist.txt are in Test1 folder along with files .xlsx that are copied from Test1 into Test2.
    Prompt forces to enter ‘F’ as many times as number of files to be copied.

    Thank you for your time.
    Adeya

    Like

  4. brian Avatar
    brian

    Try using the XCOPY command line switches /I /Y.

    for /f “delims=” %%i in (filelist.txt) do xcopy “C:\Test1\%%i” “C:\Test1\Test2\%%i” /I /Y

    Like

  5. Adeya Avatar
    Adeya

    Brian,

    xcopy switch /I says: “If destination does not exist and copying more than one file, assumes that destination must be a directory.”

    So, the prompt is displaying no matter destination exists or not, copying one file or more.

    xcopy switch /Y says: “Suppresses prompting to confirm you want to overwrite an existing destination file.”

    So, by first run switch /Y does not help: prompt requiring ‘F’ or ‘D’ is displayed.
    If files are already copied into destination file, then by the second run neither the prompt in question nor the confirmation overwriting prompt are displayed.

    I experimented with Win XP & MS Office 2007, Win XP & MS Office 2010, and Windows 7 & MS Office 2013. The behavior in all cases are the same: prompt is displayed.
    I was absolutely surprised to see it on my Win XP machines. Since April 30 (see my msg above) I was occupied with other assignments, I new that batch exists and did not try it until recently.
    I think this prompt is caused by some updates that change some security settings. I googled and people mention it, too.
    Maybe you can figure this out.
    Thank you.
    Adeya

    Like

  6. Adeya Avatar
    Adeya

    If destination folder is created before batch execution, than prompt has preceding phrase:
    “A subdirectory or file Test2 already exists.” and further the known question ‘F’ or ‘D’.
    Thanks a lot.

    Like

  7. Marcos Avatar
    Marcos

    Hi brian, great job.

    I have this issue, i wonder if you could help me out..

    I have several pdf files that i need to relocate, the list.txt has the directory names of the location of the pdf files.

    What i need is that the command use the list.txt where the locations of the files are…

    I apologize for my bad english… and thanks in advance

    list.txt

    folder1
    folder2
    folder321

    This coommand works for what i need to do

    for /R f:\temp\folder1 %f in (*.pdf) do copy %f F:\temp\folder1 /y

    Like

  8. Adeya Avatar
    Adeya

    Hello, Brian!
    Googling brought the solution. Though this solution is not universal, but for my case it is the solution.
    If by copying the file name is not changing (what is my case), then there is no need to provide file name at destination (in my case %%i at the end of the destination path can be skipped) and “Does C:... specify a file name or directory name on the target
    <F = file, D = directory)?" prompt will not displayed.
    So, my command will look as follows:
    for /f “delims=” %%i in (filelist.txt) do xcopy “C:\Test1\%%i” “C:\Test1\Test2\”
    Thank you.
    Adeya

    Like

  9. brian Avatar
    brian

    @Marcos — if your list file has folders and not files, you can do something like this:

    for /f "delims=" %%i in (filelist.txt) do xcopy "%%i" "c:\temp\destination\" /s /i /z /y

    Like

    1. Marcos Avatar
      Marcos

      Hi brian, thanks for the answer, perhaps, i failed on the explanation of my problem.

      I have several pdf files on subfolders, and I need to move these files to an upper level directory. for example

      I have
      c:\FolderA\SubfolderA\1.pdf
      c:\FolderA\SubfolderA\SubSubFolderA\2.pdf
      c:\FolderB\SubfolderB\3.pdf
      c:\FolderB\SubFolderB\SubSubfolderB\4.pdf

      And I need:
      C:\FolderA\1.pdf
      c:\FolderA\2.pdf
      c:\FolderB\3.pdf
      c:\FolderB\4.pdf

      I Have this list.txt where i have all the folders listed

      FolderA
      FolderB
      FolderC
      FolderD

      So long, i´ve managed that the script “reads” all directory but it wont copy

      If I do This

      for /R C:\FolderA %f in (*.pdf) do copy %f c:\FolderA\ /y

      The Result is what i Expect, al pdf files in c:\FolderA , and it’s subfolders copies to the directory c:\FolderA

      But if i do this

      for /f “delims=” %i in (list.txt) do echo D|for /R “c:\%i” %f in (*.pdf) do copy %f “c:\%i\” /y

      The script “reads” all directory but it wont copy any files

      Like

      1. brian Avatar
        brian

        @Marcos — in the second FOR loop, don’t re-use %i, try a different letter (%q).

        Like

      2. Nikos Avatar
        Nikos

        Hello Brian,

        Thanks for all these posts here. I have a similar problem I have a folder with all the files there and need to split them in different folders and subfolders

        So in the destination I want to create also the folders like:

        folder/subfolder/name_of_file.txt

        Can you please help me?

        Nikos

        Like

  10. Alex David Avatar
    Alex David

    Sir,
    I have two logs called chknew.log and new.log
    chknew.log contains the source filepath as C:\ILProject\DateFormat.vb

    new.log contains the destination file path as C:\ILProject\BUILD_MANDATORY\DateFormat.vb

    I want to xcopy files in log please help

    Like

    1. brian Avatar
      brian

      I’m assuming that each line in chknew.log (source) has a corresponding line in new.log (destination), so each row matches in each file… If that’s the case, let me think… you can loop through one file but not two files.

      This was a cool one, worthy of its own blog post… but in short:

      source.txt
      source1
      source2
      source3

      dest.txt
      dest1
      dest2
      dest3

      Batch file to copy files where the source is the value in source.txt and the destination is the value in dest.txt:

      @echo off
      setlocal ENABLEDELAYEDEXPANSION
      set /a line=0
      set dest=
      for /f %%i in (source.txt) do (
      call :getdest !line!
      set /a “line+=1”
      echo %%i, !dest!
      )
      endlocal
      goto :eof

      :getdest
      if “%1″==”0” (
      for /f %%j in (dest.txt) do (
      set dest=%%j
      exit /b
      )
      )
      for /f “skip=%1” %%j in (dest.txt) do (
      set dest=%%j
      exit /b
      )

      Like

  11. Kal Avatar
    Kal

    I have two directory on my D Drive. D:\1 and D:\2 Directory 1 and 2 and subdirectories have a bunch of files .

    I need a batch script that will Read each file in Directory 1 and checks, if same file exists in Directory 2, it should avoid duplicates files. Move only those files which are not present in directories 2 to a different Directory C:\3 with the same folder sturcture.

    Thanks in advance

    Like

  12. brian Avatar
    brian

    @Kal — I would recommend using a tool such as WinMerge, which can compare folders of files, show you what’s the same, different, and missing on one side or the other, and move them accordingly. It’s a great tool, and it’s free. http://winmerge.org/

    Like

  13. Kal Avatar
    Kal

    Thanks Brian. I have already tried that!!! But no success as the folder structure are diff on both Directories. I required script that search for file in A and B folders and compare the filename if same file exist on both folder it should avoid it. If Unique file exist on Dir A it should copy the file with same folder structure of DirA to DirC.

    Thanks in advance.

    Like

  14. Kal Avatar
    Kal

    Hi Brian;

    I am using below script for the above requirement, but no success.
    @echo off
    set d1=”D:\Login*”
    set d2=”D:\LOGIN2*”
    set dest=”D:\RESULT”
    for %%f in (%d1%*) DO (
    if not exist %d2% move %%f %dest%
    )

    Thanks

    Like

  15. Alex David Avatar
    Alex David

    Thanks for your solution but its not working…please do needful help

    Like

  16. Alex David Avatar
    Alex David

    Hi Brian,

    I want to delimit this statement $/MIGRATION_UAT/Integration/NSureIntegration/NSure_event1/dbutility/BIN/Interop.MSXML2.dll

    as \172.21.24.34\c$\inetpub\wwwroot\NSureIntegration\NSure_event1\dbutility\BIN\Interop.MSXML2.dll
    using batch script

    Like

  17. Alex Avatar
    Alex

    I have a list(“MissPDF.txt”) that contains path and file name like below:
    EDI 2011 – 1\184160184160594.pdf
    EDI 2011 – 3b\184497184497738.pdf
    EDI 2011 – 3b\184497184497739.pdf
    EDI 2011 – 3e – Copy\184473184473289.pdf

    I want to copy those files to this location:
    E:\PRGX\Data\MissedPDF

    I tried creating this line of code but it is not working for me, can you help?

    for /f “delims=” %%i in (MissPDF.txt) do echo D|xcopy “E:\PRGX\Data\Carrefour\%%i” “E:\PRGX\Data\MissedPDF\%%i” /i /z /y /s

    I put the bat file and misspdf.txt in this directory:
    E:\PRGX\Data\Carrefour

    Like

  18. Alex Avatar
    Alex

    The text file has this data. There was a slash missing on my example earlier

    EDI 2011 – 1\184160184160594.pdf
    EDI 2011 – 2c\184307184307471.pdf
    EDI 2011 – 3b\184531184531849.pdf

    Like

  19. Alex Avatar
    Alex

    There should be a slash after the 6th number in hte pdf file name. ie 184160 “\” 184160594. for some reason it disappears when I post this msg.

    EDI 2011 – 1\184160184160594.pdf
    should be:
    EDI 2011 – 1\184160 “\” 184160594.pdf (witohout the quotes and space)

    Like

  20. brian Avatar
    brian

    When I copied your text into a sample file, the following from a command line works fine. I think you need to better explain what “it is not working for me” means.

    for /f “delims=” %i in (c:\temp\MissPDF.txt) do echo xcopy “e:\prgx\data\carrefour\%i” “e:\prgx\data\missedpdf\%i” /i /z /y /s

    Like

    1. Alex Avatar
      Alex

      Any ideas?

      Like

  21. Alex Avatar
    Alex

    The files are not being copied I right click on my bat file and click open and it opens and closes right away…nothing happens. Could it be the spaces in the directory name “EDI 2011 – 1…” ?

    Like

  22. Alex Avatar
    Alex

    This is the full path of the first file in my example (184160184160594.pdf)
    E:\PRGX\Data\Carrefour\EDI 2011 – 1\184160\184160\184160594.pdf

    The MissPDF.txt file has this on the first line:
    EDI 2011 – 1\184160\184160594.pdf

    Like

  23. Alex Avatar
    Alex

    I found the issue. I had these double quotes (”) instead of (“). Once i changed it, it started working, kinda. Now I get an error msg for each file asking to specify if a file name or directory name on the target <F = file, D = Directory)? D 0 File(s) Copied

    what to do?

    Like

  24. Alex Avatar
    Alex

    for /f “delims=” %%i in (MissPDF.txt) do echo D|xcopy “E:\PRGX\Data\Carrefour\%%i” “E:\PRGX\Data\MissedPDF\%%i” /I /Z /Y /S

    This worked. Just needed the \ at the end of the destination

    Like

  25. Alex Avatar
    Alex

    Only thing is it is creating the file in a folder with the file name. how do I prevent it from doing that?

    Like

  26. brian Avatar
    brian

    You have backslashes in the file names in your text file. This will cause problems with file copies, as Windows will think it’s a directory path. Try replacing the slashes with a valid character (dash or space).

    Like

  27. Alex Avatar
    Alex

    The files are in different subfolders…that’s why I have the backslashes.

    Like

  28. brian Avatar
    brian

    You should use the various “tilde” functions to transform a full path to a file to just the file name, to avoid the source having a full path and the destination using just the file name. See https://en.wikibooks.org/wiki/Windows_Batch_Scripting#Percent_tilde.

    Like

  29. Alex Avatar
    Alex

    ok..started over and I put all my pdf’s in 1 folder (E:\PRGX\Data\Miss2Left)
    I have a text file called Miss2.txt which has the file names in it
    ie
    0184155473.pdf
    0184155930.pdf
    0184155931.pdf
    0184155932.pdf
    0184155933.pdf

    I want to copy the files from Miss2Left folder that exist in the text file to this folder (E:\PRGX\Data\Miss3Left). I have this command that works but that creates a folder with the filename and then the file inside that folder:

    for /f “delims=” %%i in (Miss2.txt) do echo D|xcopy “E:\PRGX\Data\Miss2Left\%%i” “E:\PRGX\Data\Miss3Left\%%i” /i /z /y

    What am I missing?

    Like

  30. brian Avatar
    brian

    Try taking out the /i switch. Or, to debug the copy command, use the /l switch to see what would happen without actually doing the file copies.

    Like

  31. Alex Avatar
    Alex

    I got it to work but it is going very slow. I am putting it into an “ALL” folder. There are about a million pdf files and about 120,000 file names in Miss2.txt file. It is copying about 10 files a minute.

    for /f “delims=” %%i in (Miss2.txt) do echo D|xcopy “E:\PRGX\Data\Miss2Left\%%i” “E:\PRGX\Data\Miss3Left\%i:All%” /i /z /y

    I was experimenting with this below. Would something like this go faster? If so, what would i need to edit below? It is not copying the files.

    for /f “tokens=” %%A in (Miss3Path.txt) DO (If Exist %%A (XCOPY “%%A*.pdf” “E:\PRGX\Data\Miss4Left” /D /K /Y /R ) )

    Like

  32. Alex Avatar
    Alex

    FYI…the Miss3path.txt file has this data in it:
    E:\PRGX\Data\Miss2Left184155473.pdf
    E:\PRGX\Data\Miss2Left184155930.pdf
    E:\PRGX\Data\Miss2Left184155931.pdf
    E:\PRGX\Data\Miss2Left184155932.pdf
    E:\PRGX\Data\Miss2Left184155933.pdf

    Like

  33. brian Avatar
    brian

    Do you have “about a million pdf files” all in the same folder in the source? If so, I’d say that is part of your problem. I’ve seen Windows have problems with folders of 10,000 or more files. A million is a LOT of files, much less all in the same directory.

    Like

  34. Paul Avatar
    Paul

    Why am I getting the error?

    for /f “delims=” %%i in (filelist.txt) do copy “%%i” “”c:\folder1\%%i”
    ” was unexpected at this time.

    Like

  35. Tristen Avatar
    Tristen

    As an extension, how would you go about copying a group of files listed in “FILEPATH_FROM.CSV” to the location in the corresponding “FILEPATH_TO.CSV”? e.g the ith line of both files correspond to the from and to arguments of xcopy respectively.

    Like

  36. Tristen Avatar
    Tristen

    Sorry, I now see the question had been asked, my mistake.

    Like

  37. Art Avatar
    Art

    Hi,

    The batch file works fine, is it possible to make it generate a list (txt) file of the ones that it couldn’t find?

    Like

  38. leslie Avatar
    leslie

    Using the FOR command to delete files listed in a text file from a specific directory?

    for /f “delims=” %%i in (imagelist.txt) do echo D|del “\c:\imagestore\%%i” /i

    nothing happens, help?

    Like

  39. sourabh Avatar
    sourabh

    You can use the following command in case you are stuck with folders getting created with file name:
    for /f “delims=” %%i in (filelist.txt) do echo F|xcopy “\server\share\folder\%%i” “c:\temp\%%i” /i /z /y

    the key here is to tell cmd that destination is a file instead of a directory.

    Like

  40. Nikos Avatar
    Nikos

    Hello Brian,

    I have a folder with files that I do need to copy to multiple folders and subfolders. The folders and subfolders are not created but have them in txt list together with the name of the specific file.

    For example I want to move

    test1.txt to folder1/subfolder1 test2.doc to folder1/subfolder2 test3.txt to folder2/subfolder1 …………..
    • test1.txt to folder1/subfolder1
    • test2.doc to folder1/subfolder2
    • test3.txt to folder2/subfolder1

    Can I do it in from command line copying the files and creating the folders at same time if I have each destination path in a text list?

    Like

  41. Nikos Avatar
    Nikos

    I have a folder with files that I do need to copy in multiple folders and subfolders. The folders and subfolders are not created.

    For example I want to move

    • test1.txt to folder1/subfolder1
    • test2.doc to folder1/subfolder2
    • test3.txt to folder2/subfolder1

    Can I do it in from command line copying the files and creating the folders at same time if I have each destination path in a text list?

    Like

  42. Dei Wills Avatar
    Dei Wills

    Hello Brian.

    I am very new to this forum and process. I am trying to create a batch file that will move employee files into it’s own employee folder. For example…

    move from – O:\Payroll\Exempt Time Off\2014\1 Extract\Andersen Julie -TAFW 07.31.14 19.pdf
    move to – O:\Payroll\Exempt Time Off\2014\Andersen Julie

    Is there a way to do this, as it is for multiple employees?

    Thanks for kindly.

    Like

  43. Bill Avatar
    Bill

    the first *.tif file is being dropped and not copied to new folder? Does anyone know why?

    for /f “delims=” %%i in (CD200.txt) do echo f|xcopy “X:\oldCDFolder\CD200\%%i” “X:\newCDfolder\CD200\%%i” /i /z/y/f

    CD200.txt contains:
    C0045971.tif
    C0045972.tif
    C0045973.tif
    C0045974.tif
    C0045975.tif
    C0045976.tif
    C0045977.tif
    C0045978.tif
    C0045979.tif
    C0045980.tif
    C0045981.tif
    C0045982.tif
    C0045983.tif
    C0045984.tif
    C0045985.tif

    Like

  44. Ryan Avatar
    Ryan

    Brian,

    Thanks so much for this. I realize it is an old post but I have a question regarding my results. Here is what I’m running:

    for /f “delims=” %%I in (TEST.txt) do echo F| xcopy “Source\%%I” “Destination\%%I”

    I ran 28,000 files from various folders/levels of folders and it worked great (27990 files copied)! However, about 50 seemingly random files do not get moved and result in a “0 File(s) copied” message.

    I can see nothing odd about the files that are not getting copied. For example, one folder contains 100 .tif images, all filenames are numbers in ascending order. Only 2 of the images were copied. If I alter my TEST.txt and only run a subset of those 100 through (10 say), DIFFERENT images get copied/don’t get copied.

    Any reason it’s randomly not copying certain files? If not, no worries and thanks for the batch file!

    Like

  45. Ryan Avatar
    Ryan

    I’m thinking maybe my TEST.txt file contains hidden characters or something. I guess I’ll have to clean it somehow.

    Like

  46. prakash Avatar
    prakash

    Hi Brain

    I have used this script to copy files , but found some files got skipped , is it possible to create an log of skipped files.

    Like

  47. Zach Avatar
    Zach

    I’m getting “xxi was unexpected at this time.” Thoughts?

    Like

    1. Tomas Avatar
      Tomas

      You need to run it from .bat file, not from commandline itself…

      Like

  48. Andrea Avatar
    Andrea

    Hi guys,

    i’m sorry for my english.
    how can i edit the file batch so that it gives me a final result??
    For example, i have this list with 250 files and i want to know, in a new txt file if some of this aren’t copied.

    thanks to all

    Like

  49. Chintan Avatar
    Chintan

    I am not programming guy but you made it so simple it saved me hours….Thanks a heaps

    Like

  50. mansh Avatar
    mansh

    how can i open one new excel sheet inside which is located folder in desktop using with cmd prompt.

    Like

Leave a comment