rsync on Windows not transferring large files

Dominic P

I'm noticing a strange issue where rsync isn't transferring large (approx >3GB) files. I'm running MSYS rsync and I'm transferring on my LAN from my Windows box to my Linux based NAS over ssh. Most of the files transfer without issue, but with large files I see the file name on the destination, but it is listed as 0KB in size and the files won't open.

I don't see any errors when I enable verbose output except a note about a couple long file paths for unrelated files. Here's my command (added line breaks for legibility):

rsync -avv -e 'ssh' --hard-links --inplace --no-inc-recursive
    --modify-window=2 --delete --delete-excluded --exclude=".svn*"
    "/d/All Files" user@local_ip:"/mnt/All Files"

Has anyone seen something like this before? What can I do to debug this?

Update: Here's the verbose output from rsync for the transfer with --progress and --stats enabled. The file "Uncompressed 1080 vs 720.avi" is the problem file. It is almost 7GB. The strange thing to me is the negative file size rsync is reporting. What could be causing that?

building file list ...
3 files to consider
delta-transmission enabled
Uncompressed 1080 24p vs 24pa 29 97.avi is uptodate
Uncompressed 1080 vs 720.avi
 -1546369996 100%    0.00kB/s    0:00:00 (xfer#1, to-check=0/3)
total: matches=0  hash_hits=0  false_alarms=0 data=-1546369996

Number of files: 3
Number of files transferred: 1
Total file size: 4868647526 bytes
Total transferred file size: 2748597300 bytes
Literal data: -1546369996 bytes
Matched data: 0 bytes
File list size: 124
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 180
Total bytes received: 37

sent 180 bytes  received 37 bytes  39.45 bytes/sec
total size is 4868647526  speedup is 22436163.71
jdigital

It sounds like rsync is using a signed 32 bit integer for storing the file size, and your file is so large that the value looks negative.

If you are on a 64bit box, see if you can find a 64bit version of rsync. If not, try other rsync implementations (the two that come to mind are DeltaCopy and cwRsync). I suspect all these rsync implementations are just ports of the same code, but it's worth a try. The company that provides DeltaCopy has a supported commercial product that might solve your problem.

There are lots of file copy programs out there, both free and commercial, so there has to be one that can solve your problem. One example is SyncBack (available in free and commercial versions).

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Rsync - not transferring empty files

From Dev

Rsync - not transferring empty files

From Dev

Reasons for rsync NOT transferring all files?

From Dev

Reasons for rsync NOT transferring all files?

From Dev

Transferring large files between an Ubuntu machine and a windows machine

From Dev

Transferring large files over TCP in Qt

From Dev

Transferring large (8 GB) files over ssh

From Dev

inotify and rsync on large number of files

From Dev

Transferring big files from local hard drive to computing cluster by rsync

From Dev

transferring files from vmware ubuntu to windows ubuntu

From Dev

Rsync not transferring the changes

From Dev

Encrypting/Decrypting(Using AES) Large files and transferring over HTTP

From Dev

Transferring large files: combining streamed transfer and content-length

From Dev

Transferring large files using scp with CPU and memory considerations

From Dev

rsync - why is it transferring whole file

From Dev

rsync comparing directories of large binary files: is it comparing the content of the files?

From Dev

Windows Server network card transferring files at 100 Mb/s on the LAN

From Dev

What is the effect of rsync --block-size on large files?

From Dev

Why is rsync taking a long time on large files that already exist?

From Dev

Memory problems when compressing and transferring a large number of small files (1TB in total)

From Dev

Windows: Disable caching of large files

From Dev

MQ Websphere - Transferring files

From Dev

Transferring files using Telnet

From Dev

Transferring files through SFTP

From Dev

MQ Websphere - Transferring files

From Dev

Which is the most secure and fast way of transferring large amount of data (SQL Server backup files) to Azure Blob or Azure VM hosting SQL

From Dev

Transferring files over TCP with CopyTo()

From Dev

Transferring files through wifi network

From Dev

transferring audio files via WebSocket

Related Related

  1. 1

    Rsync - not transferring empty files

  2. 2

    Rsync - not transferring empty files

  3. 3

    Reasons for rsync NOT transferring all files?

  4. 4

    Reasons for rsync NOT transferring all files?

  5. 5

    Transferring large files between an Ubuntu machine and a windows machine

  6. 6

    Transferring large files over TCP in Qt

  7. 7

    Transferring large (8 GB) files over ssh

  8. 8

    inotify and rsync on large number of files

  9. 9

    Transferring big files from local hard drive to computing cluster by rsync

  10. 10

    transferring files from vmware ubuntu to windows ubuntu

  11. 11

    Rsync not transferring the changes

  12. 12

    Encrypting/Decrypting(Using AES) Large files and transferring over HTTP

  13. 13

    Transferring large files: combining streamed transfer and content-length

  14. 14

    Transferring large files using scp with CPU and memory considerations

  15. 15

    rsync - why is it transferring whole file

  16. 16

    rsync comparing directories of large binary files: is it comparing the content of the files?

  17. 17

    Windows Server network card transferring files at 100 Mb/s on the LAN

  18. 18

    What is the effect of rsync --block-size on large files?

  19. 19

    Why is rsync taking a long time on large files that already exist?

  20. 20

    Memory problems when compressing and transferring a large number of small files (1TB in total)

  21. 21

    Windows: Disable caching of large files

  22. 22

    MQ Websphere - Transferring files

  23. 23

    Transferring files using Telnet

  24. 24

    Transferring files through SFTP

  25. 25

    MQ Websphere - Transferring files

  26. 26

    Which is the most secure and fast way of transferring large amount of data (SQL Server backup files) to Azure Blob or Azure VM hosting SQL

  27. 27

    Transferring files over TCP with CopyTo()

  28. 28

    Transferring files through wifi network

  29. 29

    transferring audio files via WebSocket

HotTag

Archive