Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Mausschubser]

Folks!

After a few days spent on backup and restore 1.5TB of data *), I made some observation which I do not want to withhold from you.
A bunch of write tests with different file size may explain why you sometimes see very slow transfer rates.
So I set up files from 0 Bytes, a few bytes, some kB, some MB to 1 GB and measured how long it takes to transfer this from my PC via a Gigabit Ethernet to my NAS (Raid 1 with 2x3TB WD Red).

The results in short (write to NAS):

• micro size files: Even transfering 0 Bytes takes 22 sec for 1000 files! Maximum is about 45 files/sec
• mini size files: Up to 100kB file size, the transfer time is almost independent of size, only dependent on number of files.
• medium size files: Between 500kB up to 5MB (typical size of Photos & Music) it is a mixture of size and number of files, and definitely not at maximum speed.
• large size files: From 20MB to 1GB (and larger) the speed is limited by the maximum write data rate, which is ~28MByte/sec in my case.

And here the details:
Time needed to write 1000 files of given size to NAS.

file size	| MB/s	| files/s	| time for 1000 files [h:m:s]
0 Bytes	| 0.00	| 45.5	| 0:00:22
10 Byte	| 0.00	| 43.5	| 0:00:23
1000 Byte	| 0.04	| 41.7	| 0:00:24
4000 Byte	| 0.15	| 38.5	| 0:00:26
10kB	| 0.37	| 38.5	| 0:00:26
20kB	| 0.70	| 37.0	| 0:00:27
50kB	| 1.68	| 34.5	| 0:00:29
100kB	| 2.94	| 30.1	| 0:00:33
500kB	| 9.09	| 18.6	| 0:00:54
1MB	| 13.13	| 13.1	| 0:01:16
5MB	| 23.81	| 4.8	| 0:03:30
20MB	| 27.21	| 1.4	| 0:12:11
100MB	| 27.58	| 0.3	| 1:00:00
1GB	| 27.58	| 0.0	| 10:00:00

How long does it take to write 1GB to NAS, depending on single file size.

file size	| number of files	| duration for 1GB [min:sec]
10kB	| 104,858	| 46:43
20kB	| 52,429	| 24:15
50kB	| 20,972	| 10:09
100kB	| 10,486	| 05:48
500kB	| 2,097	| 01:53
1MB	| 1,024	| 01:18
5MB	| 205	| 00:43
20MB	| 51	| 00:38
100MB	| 10	| 00:37
1GB	| 1	| 00:37

*) Following JavaLawyer's hint to backup data: you need time! A lot of time!
For a half full NAS (1.5TB), you need a full night to write the backup from the NAS and about 1 day to restore it. Wow!


Looking forward for somebody making a fancy graph displaying the odd numbers above...
Mausschubser

[sbrbot]

First of all everybody should know that an action of writing a file on disk assumes at least two different write operations. One operation is, of course, to store file's content somewhere on available disk space, and another write operation is to write the info into file system table about the file name and its location on disk. That's why writing of 0 byte file also takes some time (although effective writing is zero in that extreme case). Writing file name and physical location where this file resides on HDD takes approximately the same time for small and big files. So relatively, the smaller file is the bigger relative part of its writing time goes to writing info into file system table. For example, spending 1 second on writing file system info for 1 byte file and 1 GB file is not the same thing (looking at relative time for writing of fs info and file content).

If one looks at figures from your experiment, he/she can see that as bigger file is as less effect of this writing to file system table there is. So writing speed for files bigger than 100 MB (or 50 MB) is steady 27.58 MB/s. For smaller files this speed is lower because bigger and bigger part of writing time is used for writing file system info.

According to mechanisms used inside different file systems for writing info about file name, its location on disc (and other things) various file systems have different performances compared to file size. Some file systems work better with smaller file sizes some file systems better with big ones.

On DNS-320L Ext3 file system is used. This is very good file system with journaling feature capable of taking care about avoiding fragmentation on its own. (Contrary to newer DNS-320L, older DNS-320 boxes use Ext2 file system).

[sbrbot]

I couldn't help myself and I created graph out of your figures:



This graph shows effective speed (effective writing where you count only file content size but NAS writes some additional data needed for file system operation).

Conclusion, from graph above one can see that for:

• files smaller than cca 50kB - writing time is mostly spent for writing file system table data - low effective data writing speed
• files between cca 50kB and 10MB - time used for writing file system things is comparable to time for writing effective data
• files bigger than cca 10MB - writing time is mostly spent on writing file content on disk - steady effective data writing speed

P.S. X-axis (file size in bytes) is in logarithmic scale

[Mausschubser]

Hi sbrbot!

Thanks for analyzing the results! The graph clearly shows what we sometimes read in this forum: "My NAS is so slow, only xxxkB per second. Why?"
I made some more tests with realistic data and came to a nice equation:

formula to estimate the time needed to write a bunch of data

time [sec] = (files+folders)/45.5 + totalsize[MB]/27.6

• files+folders: enter number of files and folder to be copied. This is the total number of "objects" needed to be entered in the "file location table" - seems to be the important number. You explained nicely how they will be stored in a kind of "address book" on the disk.
• totalsize: The total size of all files in MBytes. This is the amount of data needed to be transfereed through the network into the NAS
• "45.5": The maximum number of files per second which can be copied to the NAS, even if they have zero size. See my tests in the 1st post for the "micro size files".
• "27.6": the transfer rate [MB/sec] for large files, where size matters but not number. See my test in the 1st post for "large size files".

Of course, your system may get different numbers instead 45.5 files/s and 27.6MB/s, but you can measure them by copying a bunch of very small files and 1 large file.


and for all number crunchers here the results of my test (extrapolated to 10GByte of total data)

Time needed and speed to write different type of data to NAS:

type of data	| size [GB]	| files	| folders	| objects	| speed [MB/s]	| time [mm:ss]	| estd [mm:ss]	| error
Video	| 10.0	| 2	| 0	| 2	| 28.7	| 05:57	| 06:11	| 3.9%
Music	| 10.0	| 2,336	| 148	| 2,484	| 24.8	| 06:53	| 07:06	| 3.1%
Photos	| 10.0	| 2,777	| 10	| 2,787	| 23.4	| 07:17	| 07:12	| 1.1%
Game	| 10.0	| 22,142	| 22,916	| 45,058	| 7.9	| 21:41	| 22:43	| 4.8%
Program	| 10.0	| 84,707	| 23,405	| 108,112	| 4.0	| 42:32	| 45:52	| 7.9%

time: time measured for write from PC to NAS
estd: estimated time using the above equation
error: error of the above equation

Cheers, have fun & take your time to backup your valuable data...!
Mausschubser

[Ardalista]

Hey all,

Thanks for taking the time to do these tests and sharing the information in an understandable way.

I recently performed a local nas to local nas backup of approx 3.8TB of data, with various file sizes and numerous folders, sub folders etc... it took 4 days and 10 minutes to perform the initial backup, now the rest is incremental and ticking away nicely each morning.

now thanks to your efforts I can see why it took me so long.