• February 25, 2025, 01:36:45 AM
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

News:

This Forum Beta is ONLY for registered owners of D-Link products in the USA for which we have created boards at this time.

Author Topic: Very slow writing with small files - any way to improve?  (Read 15201 times)

RicRoller

  • Level 2 Member
  • **
  • Posts: 45
Very slow writing with small files - any way to improve?
« on: July 31, 2012, 08:41:45 AM »

Trying out my DNS-320 again (H/W rev. A1, F/W 2.02), fitted a single WD30EZRX drive.

NAS and Windows 7 PC connected via gigabit switch built into Linksys WAG320N router; mapped drive on PC and tried copying a load of files onto the NAS.

For folders containing large numbers of small files, the writing rate was truly dreadful - dropping to around 350kbyte/sec - however for large files things seemed "about right" at approx. 20-25Mbyte/sec. And the NASTester tool (using its default 400MByte file size) shows an average write speed around 22MByte/sec and average read speed around 48MByte/sec. So doesn't seem to be much wrong with the network...

So the issue seems to be with small files - one thing I suspect is perhaps the DNS-320 firmware isn't making use of the large cache in the hard drive (the WD30EZRX has 64MB) - for example not caching writes at all (either in the DNS-320's own memory or in the hard drive's buffer), or using only a very small write cache.

Is there any user adjustment possible to improve the performance? Or, if the issue is down to not doing any write caching, should I add it to the "firmware feature request" thread?

(I am aware that enabling write cache could potentially increase the risk of data loss if there's a power outage - however IMO having the option to enable the write cache in the drive (but not having additional delayed-write caching in the firmware) might be a good compromise between performance and safety...)

Regards,
Richard
Logged

JavaLawyer

  • BETA Tester
  • Level 15 Member
  • *
  • Posts: 12190
  • D-Link Global Forum Moderator
    • FoundFootageCritic
Re: Very slow writing with small files - any way to improve?
« Reply #1 on: July 31, 2012, 08:50:03 AM »

Did you try connecting the DNS-320 and PC to a dedicated switch (apart from the router) to see if the throughput issues persist? This could rule out the router as a potential suspect. Also, you should test data transfer speeds using another PC as well to rule out the NIC card (or NIC settings) on the client PC.
Logged
Find answers here: D-Link ShareCenter FAQ I D-Link Network Camera FAQ
There's no such thing as too many backups FFC

RicRoller

  • Level 2 Member
  • **
  • Posts: 45
Re: Very slow writing with small files - any way to improve?
« Reply #2 on: July 31, 2012, 09:30:07 AM »

Did you try connecting the DNS-320 and PC to a dedicated switch (apart from the router) to see if the throughput issues persist?
Yes - tried Trendnet TEG-S5g gigabit switch instead of the router's built-in switch. No change in performance.
Quote
Also, you should test data transfer speeds using another PC as well to rule out the NIC card (or NIC settings) on the client PC.
Tried with laptop PC; slight improvement as laptop has a solid state drive, but only got up to around 450kbytes/sec for the same set of files/folders

FYI the "slow" set of files/folders I used for the comparative test are from a Pinnacle Studio 14 video project - there were 1412 files and 181 folders, 26.6Mbytes total.
Copying that lot from either PC to the NAS took around 90 seconds, copying from one PC into the public folder on the other took around 30 seconds.

Regards,
Richard
Logged

JavaLawyer

  • BETA Tester
  • Level 15 Member
  • *
  • Posts: 12190
  • D-Link Global Forum Moderator
    • FoundFootageCritic
Re: Very slow writing with small files - any way to improve?
« Reply #3 on: July 31, 2012, 09:54:42 AM »

Did you ever test DNS-320  throughput using another make/model HDD? Were the transfer speeds similar?
Logged
Find answers here: D-Link ShareCenter FAQ I D-Link Network Camera FAQ
There's no such thing as too many backups FFC

RicRoller

  • Level 2 Member
  • **
  • Posts: 45
Re: Very slow writing with small files - any way to improve?
« Reply #4 on: July 31, 2012, 10:28:09 AM »

Did you ever test DNS-320  throughput using another make/model HDD? Were the transfer speeds similar?
When I first got the DNS-320 I was evaluating its potential for replacing an old decrepit Linux file server at work; at the time it had two Seagate Barracuda Green ST2000DL003 drives in RAID1. But I was never able to get above 2.5MByte/sec writing - partly because the work network is 100baseT and not gigabit. So unfortunately not a fair comparison...

But if it would be constructive I could borrow one of the Seagate drives from work and try it out again using my PCs and gigabit switch.

Regards,
Richard
Logged

JavaLawyer

  • BETA Tester
  • Level 15 Member
  • *
  • Posts: 12190
  • D-Link Global Forum Moderator
    • FoundFootageCritic
Re: Very slow writing with small files - any way to improve?
« Reply #5 on: July 31, 2012, 11:17:42 AM »

When I first got the DNS-320 I was evaluating its potential for replacing an old decrepit Linux file server at work; at the time it had two Seagate Barracuda Green ST2000DL003 drives in RAID1. But I was never able to get above 2.5MByte/sec writing - partly because the work network is 100baseT and not gigabit. So unfortunately not a fair comparison...

But if it would be constructive I could borrow one of the Seagate drives from work and try it out again using my PCs and gigabit switch.

Regards,
Richard

I'm wondering more about the kbps write speed for the small files, which would not have been capped by the 100 MB network.  For kicks, I performed a write test from my PC to DNS-343 (2TB Seagate, non-AFT) using a folder of 44kb files, and average write speed was 0.93 MB/s.

Here is an article benchmarking WD and Seagate HDD transfer rates with different for different file sizes: http://www.behardware.com/art/imprimer/797/
Logged
Find answers here: D-Link ShareCenter FAQ I D-Link Network Camera FAQ
There's no such thing as too many backups FFC

RicRoller

  • Level 2 Member
  • **
  • Posts: 45
Re: Very slow writing with small files - any way to improve?
« Reply #6 on: July 31, 2012, 12:58:24 PM »

I'm wondering more about the kbps write speed for the small files, which would not have been capped by the 100 MB network.  For kicks, I performed a write test from my PC to DNS-343 (2TB Seagate, non-AFT) using a folder of 44kb files, and average write speed was 0.93 MB/s.

Here is an article benchmarking WD and Seagate HDD transfer rates with different for different file sizes: http://www.behardware.com/art/imprimer/797/
Interesting article...especially the dire performance turned in by one of the older WD20EARS drives.

So, to eliminate drive performance issue I removed it from the DNS-320, plugged into an eSATA hard drive dock and attached directly to one of the PCs, booted from a Fedora Linux live CD (so I could get at the EXT3 filesystem without having to reformat the drive).

Used Gnome file manager to copy the same bunch of files/folders onto the WD30EZRX drive - operation was completed in just under 10 seconds.

I then did another test using the DNS-320; I put in an old Seagate Barracuda 7200.10 1TB drive that had been in a USB enclosure; the writing performance to that drive was a little better than the WD - took 1 minute 3 seconds to write the set of small files (~650kbytes/sec).

I am now pretty sure that the slowness is not due to the hard drive itself, but is down to the way it is being accessed by the DNS-320 firmware - so there may be scope for improvement in a future version.

I will add a comment to the "firmware wishlist contributions" thread and reference this one.

Regards,
Richard
Logged

JavaLawyer

  • BETA Tester
  • Level 15 Member
  • *
  • Posts: 12190
  • D-Link Global Forum Moderator
    • FoundFootageCritic
Re: Very slow writing with small files - any way to improve?
« Reply #7 on: July 31, 2012, 01:30:32 PM »

Interesting dilemma. I think the one HDD with the really poor write speed has 4 platters instead of 3, which compounded the poor performance. Nonetheless, there is a performance hit for transferring small files, although 350 kbps seems too far outside the norm considering the 650 kbps and 930 kbps observed with Seagate. Too many variables here to get a conclusive answer.

Are there any other DNS-320 owners who can provide average write speeds for a large volume transfer of small kb sized files?

Logged
Find answers here: D-Link ShareCenter FAQ I D-Link Network Camera FAQ
There's no such thing as too many backups FFC

RicRoller

  • Level 2 Member
  • **
  • Posts: 45
Re: Very slow writing with small files - any way to improve?
« Reply #8 on: July 31, 2012, 03:54:28 PM »

It may be that the Pinnacle Studio project is a particularly extreme example of a bunch of folders and numerous small files (most are only a few kilobytes). I have packed one such project into a 7zip archive that is just under 1MB; I'll find somewhere to upload it and post the link.

The old Seagate ST31000340AS does consistently outperform the WD30EZRX when I copy the set of small files - probably only because it is a 7200rpm drive so rotational latency etc. is lower.

Regards,
Richard
Logged

RicRoller

  • Level 2 Member
  • **
  • Posts: 45
Re: Very slow writing with small files - any way to improve?
« Reply #9 on: July 31, 2012, 04:35:19 PM »

I have put the just-under-1MB 7zip archive containing approx. 26MB of small files/folders from a Pinnacle Studio project on Mediafire (link here http://www.mediafire.com/?o9l7zc1avbc6b6k. (Sorry - it's a free trial account so full of ads around the download button)

Unpack the archive into a temporary folder then copy from the PC into a network share on the DNS-320 and see how long it takes.

Regards,
Richard
Logged

albert

  • Level 5 Member
  • *****
  • Posts: 510
    • SoHo NAS Forum
Re: Very slow writing with small files - any way to improve?
« Reply #10 on: July 31, 2012, 08:43:51 PM »

Try turning off your antivirus for the duration of the copy process, I'm able to achieve 1.5MBps as compared with 450KBps using the small files provided by TS. NAS firmware is 2.03b3, HDD used are from Samsung (both AFT and non-AFT model) and network is only FE aka 100Mbps.

I believed more tuning can still be done on the SMB/CIFS (samba) protocols to improve its performance.
Logged
D-Link DNS-320 rev A1 (FW: 2.05) [FFP-0.7]
PCI NAS-01G (FW: Encore ENNHD-1000 4.10)
PCI NAS-01G (FW: OpenNAS 1.9]

RicRoller

  • Level 2 Member
  • **
  • Posts: 45
Re: Very slow writing with small files - any way to improve?
« Reply #11 on: August 01, 2012, 02:14:14 AM »

Try turning off your antivirus for the duration of the copy process,
Tried - made no noticeable improvement

Quote
I'm able to achieve 1.5MBps as compared with 450KBps using the small files provided by TS. NAS firmware is 2.03b3
So, that unofficial firmware may be significantly improved over the 2.02 release my NAS shipped with (and also the 2.03b1 official beta?). I do have a copy of that unofficial beta firmware but reluctant to try it as there's no way back to 2.02 if I find something gets broken...

Also, what do you mean by "small files provided by TS"? If not my set of files I put on Mediafire, please let me know what/where they are so I can repeat my test using the same files.

Regards,
Richard
Logged

albert

  • Level 5 Member
  • *****
  • Posts: 510
    • SoHo NAS Forum
Re: Very slow writing with small files - any way to improve?
« Reply #12 on: August 01, 2012, 05:22:24 AM »

Tried - made no noticeable improvement
So, that unofficial firmware may be significantly improved over the 2.02 release my NAS shipped with (and also the 2.03b1 official beta?). I do have a copy of that unofficial beta firmware but reluctant to try it as there's no way back to 2.02 if I find something gets broken...

Also, what do you mean by "small files provided by TS"? If not my set of files I put on Mediafire, please let me know what/where they are so I can repeat my test using the same files.

Regards,
Richard

TS simply meant Thread/Topic Starter which in this case is you.  ;)

Only firmware higher than 2.00 cannot be downgraded back to 2.00 or 1.00. I have downgraded from 2.03b3 to 2.02 without problem. BTW, I have came across 2 copy of 2.03b1 dated 17/01/12 and 21/02/12.

Logged
D-Link DNS-320 rev A1 (FW: 2.05) [FFP-0.7]
PCI NAS-01G (FW: Encore ENNHD-1000 4.10)
PCI NAS-01G (FW: OpenNAS 1.9]

RicRoller

  • Level 2 Member
  • **
  • Posts: 45
Re: Very slow writing with small files - any way to improve?
« Reply #13 on: August 01, 2012, 05:38:48 AM »

TS simply meant Thread/Topic Starter
Thanks - I was more familier with the term OP (Original Poster) so was confused, thinking that perhaps TS stood for Tech Support  ???

Quote
Only firmware higher than 2.00 cannot be downgraded back to 2.00 or 1.00. I have downgraded from 2.03b3 to 2.02 without problem. BTW, I have came across 2 copy of 2.03b1 dated 17/01/12 and 21/02/12.
Thanks, in that case (assuming that the official release 2.02b1 on the D-link UK FTP site is the same beast as the 2.02 currently in my NAS) I can try the 2.03 betas (b01 or b03 and get back to 2.02 if I don't like what I see. I had misunderstood what I had read in anothrer thread and thought that downgrading was not possible at all.

Regards,
Richard
« Last Edit: August 01, 2012, 05:49:39 AM by RicRoller »
Logged