This is a place for Systems Administrators and IT Professionals to find and share ideas, solutions and templates. If you have something that helps you solve a problem, chances are it will help someone else too. So pay it forward and send an email to TheAgreeableCow at gmail. Full mudos to you!

Wednesday 6 June 2012

Using multiple copy streams with Robocopy

This is a quick post to outline a technique I've used to migrate file servers using robocopy - or to be specific, lots of parallel robocopy streams.

The script works best when you have multiple 2nd level sub-directories contained in the root directory. The VBS file below parses the directory names into seperate robocopy command line statements inside a new batch file. When the batch file(s) are run, a seperate robocopy stream is processed for every sub-directory at the same time.

All of the usual robocopy command switches can be used of course. For one off jobs, these multiple parallel streams are much faster than a single stream. It's also great for syncronising file stores if run as a sheduled task.

Here is the code.

         (oo)  ok
   /------\/  /
  / |    ||
 *  /\---/\
    ^^   ^^


  1. An idea using ThreadPool and seeing errorlevel status for each task.

  2. Hi thank you for sharing. Great Post.
    Very clean and detailed
    I would like to ask for some flavours :)
    how to do this by adding another variable like every destination or source I just want to copy a specific kind of file (exemple: *.101 form source1/destination, *.102 from source2/destination2)
    And how to make a log file for each day resuming all.


    1. The file type and logging could be done in robocopy command line string at line 39. For example

      txtOutput.Write "start robocopy *.101 " & Source & "\" & objFolder.Name & " " & Destination & "\" & objFolder.Name & " /w:1 /r:1 /MIR >> c:\temp\copy101.log" & vbCrLf

      If you wanted to match a specific file type just to a specific source, then you would just create multiple "createscript" functions, with the appropriate command line string. These would be then be called accordingly. eg

      CreateScript101 Source1,Destination1,BatchFile1
      CreateScript102 Source2,Destination2,BatchFile2

  3. Hi and thank you for sharing your script.... I use it to copy my files til BitCasa.
    I have a question you maybe can help me with.

    For every output I make a logfile like this:

    txtOutput.Write "start robocopy *.* """ & Source & "\" & objFolder.Name & """ """ & Destination & "\" & objFolder.Name & """ /MIR /log+:\\Freja\Faelles\Faelles_filer\SkyDrive_MAB\Logfiles\""" & objFolder.Name & """\Bitcasa_" & LogFile & "_""" & objFolder.Name & """_%date%.txt /np /r:3 /w:1 /XF" & vbCrLf

    And that gives me a lot of logfiles.

    What I would like is to store the logfiles in seperate directories. for each objFolder.Name

    How to do that...
    I have tried with this code:
    IF NOT EXIST "" & Source & "\" & objFolder.Name & "" mkdir "" & Source & "\" & objFolder.Name & ""

    But it does not Work.

    Thanks in advance.

    Best regards

    1. I found a solution:

      Dim objNetwork
      Dim FSO
      Dim Folder

      Set FSO = CreateObject("Scripting.FileSystemObject")

      Set objNetwork = CreateObject("WScript.Network")

      If NOT (FSO.FolderExists("\\Freja\Faelles\Faelles_filer\SkyDrive_MAB\Logfiles\" + LogFile)) Then
      FSO.CreateFolder("\\Freja\Faelles\Faelles_filer\SkyDrive_MAB\Logfiles\" + LogFile)
      End If

      This is put in just after:

      For Each objFolder in colFolders

      What I would like now is to remplace: