While exploring the Office 365 Import Service in my test lab I encountered repeated timeout errors trying to upload PST files to Azure using AzCopy.exe.

[2015/07/26 23:26:36.206+10:00] >>>>>>>>>>>>>>>>
[2015/07/26 23:26:36.221+10:00][VERBOSE] 3.1.0 : AzCopy /source:\mgmtpst /dest:https://uploadurl.blob.core.windows.net/ingestiondata/ /DestKey:****** /S /V:C:AdminPSTupload.log
[2015/07/26 23:26:39.680+10:00][VERBOSE] Start transfer: Alan Reid.pst
[2015/07/26 23:26:40.352+10:00][VERBOSE] Start transfer: Dawn Evans.pst
[2015/07/26 23:26:40.352+10:00][VERBOSE] Start transfer: Elaine West.pst
[2015/07/26 23:26:40.352+10:00][VERBOSE] Start transfer: Vik Kirby.pst
[2015/07/26 23:41:41.552+10:00][VERBOSE] Transfer FAILED: Alan Reid.pst.
[2015/07/26 23:41:41.568+10:00][ERROR] Alan Reid.pst: The client could not finish the operation within specified timeout.
The client could not finish the operation within specified timeout.
[2015/07/26 23:52:41.450+10:00][VERBOSE] Transfer FAILED: Elaine West.pst.
[2015/07/26 23:52:41.450+10:00][ERROR] Elaine West.pst: The client could not finish the operation within specified timeout.

[2015/07/26 23:52:41.450+10:00][VERBOSE] Transfer FAILED: Vik Kirby.pst.
[2015/07/26 23:52:41.450+10:00][ERROR] Vik Kirby.pst: The client could not finish the operation within specified timeout.

[2015/07/26 23:53:23.429+10:00][VERBOSE] Transfer FAILED: Dawn Evans.pst.
[2015/07/26 23:53:23.429+10:00][ERROR] Dawn Evans.pst: The client could not finish the operation within specified timeout.
The client could not finish the operation within specified timeout.
[2015/07/26 23:53:23.445+10:00] Transfer summary:
-----------------
Total files transferred: 4
Transfer successfully:   0
Transfer skipped:        0
Transfer failed:         4
Elapsed time:            00.00:26:47

I was trying to upload using my home internet connection, which doesn’t have high upload speed. The upload process was also showing a transfer speed of 0 KB/s. Clearly it was not making any progress.

Reviewing the AzCopy command line options I saw that there is a /NC parameter that can be used to control the number of concurrent operations.

AzCopy by default starts a certain number of concurrent operations to increase the data transfer throughput. Note that large number of concurrent operations in a low-bandwidth environment may overwhelm the network connection and prevent the operations from fully completing. Throttle concurrent operations based on actual available network bandwidth.

Lowering the thread count to 2 (by specifying /NC:2) seems to have resolved the issue for me, and the upload was able to finish on the next attempt.

About the Author

Paul Cunningham

Paul is a former Microsoft MVP for Office Apps and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul no longer writes for Practical365.com.

Comments

  1. Claire

    It is easy now, there are many tools that can do this job easily and quickly like Shatregate, Gs Richcopy 360, and Syncback

  2. SRIJITH

    Thanks a lot for the switch /NC.

    It worked for me too.

  3. RichardS

    This has 100% just saved my bacon, after a couple of days trying to upload a PST – the M$ documentation for this is pretty poor.

    Thank you Chumrade!

  4. Zach

    Still a problem in September of 2021. I was only trying to upload one 8GB PST file. Thanks for this solution!

  5. Lawrie Boarer

    Well, I thought this can’t help my case as I only have one file to upload. But it did

  6. Val

    Thanks – Great find. /NC:2 worked a treat.

  7. Abubakr

    This fix was extremely helpful. I was having failures exporting the initial large files and almost got frustrated. I have successfully transfer over 650GB of data thanks to the /NC:2 switch.

    Thank you Paul. May your toughest challenge have the easiest solution.

    1. Avanish

      what was the final command that you ran to do this ?

      1. Andrew Hitchcock

        yes I would love to see a full example — the /NC:2 option does not seem to be supported anywhere in my command.

  8. Kostas Nikoloudis

    i am very frustruting knowing that in 2020 MS hasn’t a proper tool for PST migration to O365 especially when AzCopy is a pain in the ass and never works!!!
    thanks a lot for your post!

  9. Laz

    A HUGE thanks to Paul. It is 2019 and STILL an issue. The /NC:2 switch saved the day. Well, Paul did πŸ˜‰

  10. JD

    This seems to work! I couldn’t even get started before but with this switch it started uploading…albeit very slow at 525 KB/s which translates to around 4.2 Mb/s. Meh, as long as I can keep uploading steadily for the next 36 hours I’ll definitely make progress on the project. Thanks!

  11. Peter Cap

    Worked a treat πŸ™‚ MSFT should add this to the instructions for Azcopy.

  12. Dinesh

    To resume the transfer, just run the same .\azCopy.exe command or use a script like:
    [int]$Failed = $Result[5].Split(“:”)[1].Trim() $Journal = “$env:LocalAppData\Microsoft\Azure\AzCopy” $i=1 while ($Failed -gt 0) { $i++ [int]$Failed = $Result[5].Split(“:”)[1].Trim() $Result = .\AzCopy.exe /Z:$Journal $Result $i }
    This script will continue to run the same .azCopy.exe command until “Transfer Failed” output is down to zero. azcopy tool stores two journal files under %\AppData%\Local\Microsoft\Azure\AzCopy like C:\Users\samb\AppData\Local\Microsoft\Azure\AzCopy folder. It looks them up upon starting to detect if this is a transfer that did not complete and resumes it.

    Ref : https://social.technet.microsoft.com/wiki/contents/articles/26528.resuming-timed-out-uploadsdownloads-tofrom-azure-blob-storage-with-azcopy.aspx

  13. Dave

    Beautiful post – fixed my issue; thank you.
    I have a 40mbp/s upload speed, but AzCopy just stuck at 0kb the whole time. When it eventually bombed out with the error above, it had somehow managed to copy one 4MB file, but left a 4GB and a 50MB PST. I think there are serious issues with AzCopy. Having to search Google to fix the most basic of problems is a pain in the ass. Perhaps Microsoft might even manage to make a GUI to do this job one day, and just maybe with friendly error messages as well, and hell, even a “retry” option??? Just a thought…

  14. SimonCPX

    Maybe someone knows what is maximal size of pst file?
    There is no official info about that. I’ve read about 10GB, but nothing confirmed. I’m preparing to migration via pst and have users with even 70GG files.

  15. Peter

    Thanks Paul,
    its so frustrating searching relevant information for azcopy.
    Dokumentation and advice for importing pst to office 365 varies in every post. I first stuck at the commandline, since i never read sth that you dont need a key as parameter anymore – just use the dest url.
    When it finally started i had the files on an attached usb3 drive and started without your nc:2 switch – broken connection every 15 minutes, average speed 50kb/s.
    Moving files on hd and using your switch works fine and leads to 700kb/s – and no loss of connections. Thanks again

  16. Rob L

    After about 10 calls to Microsoft Technical Support and hours of frustration, I finally found this post and it has fixed my problem…..Thank you!! Thank you!! Thank you!!

  17. David Z

    Thanks Paul! 2 years later and your post is still helping. It’s going to take about 24 hours to complete but it has already gone far beyond where it was failing before I added the /NC switch.

  18. Mat

    I have a 100Mbps connection, and was getting transfer speeds of 1k. This resolved the problem for me, the transfer speed shot up. (Although still not exactly fast, it was 200k-1MBps)

  19. Bernie H

    Would be nice if there was an RETRY option in the command.

  20. Chupacabras

    Each time you issue a command to AzCopy, it checks whether a journal file exists in the default folder, or whether it exists in a folder that you specified via this option. If the journal file does not exist in either place, AzCopy treats the operation as new and generates a new journal file.

  21. Ariel

    For a Lower connections, NC:2 It’s worked to me. Many thnks!

  22. Gary

    Thanks – we had the same problem when trying to upload from any on-prem server in any physical location – adding this parameter worked. Any idea what the ‘certain number of concurrent operations’ that AZCopy starts by default is? It would be nice if they would tell us so we have something to gauge it by. FYI – we uploaded our test PST file (1.3 GB in size) to an azure VM and ran the tool and it worked perfectly. The upload was very fast – 2 minutes for a 1.3 GB file.

    1. Avatar photo
      Paul Cunningham

      I think it may be a default of 8, but I couldn’t find a definite answer when I was looking into it.

      1. Gary

        I actually got a reply from a MS guy – you are kind of correct – the default is 8 per core. I quote from the MS reply “The default thread count is 8*core count, such on a machine with 8 CPU cores, the thread count would be 8*8=64.”

    2. larry ponciano

      Thank you for the guidance, this worked perfectly!

Leave a Reply