Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[rjensen]

I am running into problems with my NAS to NASback-up. Specifically, I have a thoroughput of 3.5 MBs.

Configuration:
 NAS-343 with 6 TB RAID 5 and 1.04 firmware. (Primary backup, receives data from the computers.)
 NAS-323 with 4 TB JBOD and 1.09 firmware. (Secondary backup, backs up the NAS-343.)
 WRT310N router. Both NASs have static IPs and are connected directly to this router with Cat 5 cables.
 The 323 uses ftp to pull files from the 343 -- all firmware functions.

I back up the 343 every few months. At the current data transfer rate, it will take 5 days to back-up 1.5 TB to the 323.

I am looking for options to improve the data transfer rate. Surfing around, I discovered Alt-F (http://sourceforge.net/projects/alt-f/) and Fonz fun_plug. I am not familiar with either and would appreciate recommendations from those with experience.

Thanks!

[fordem]

Configure for gigabit if you don't already have that done, make sure ftp rate limiting is disabled in both the 323 & 343 (I don't remember exactly what D-Link calls it and my 323 died a month or two back so I can't go look and see) - if you don't have jumbo frame enabled (I don't know if your router supports it), try it both enabled & disabled - jumbo frame has the potential to improve throughput but the actual improvement (if any) depends on file size - I have seen situations where throughput decreased - I'm not certain how much more can be done - the processors in the D-Link NAS can only handle so much and I think you're hitting that limit

In the six years I used a 323 (primarily as backup storage) I learned that throughput is limited by the processing power, that throughput is highest when transferring large files and lowest when transferring small files - you can hit as much as 35MB/sec under ideal conditions, say transferring a single 2GB file, but transferring 1000 2MB files (the same total volume of data) would be significantly slower.

I initially bought my DNS-323 with hopes of installing a few as servers in SOHO applications (and I mean SOHO not SMB), but after a year or so of tweaking and experimenting I just accepted that the unit had it's limitations, and more importantly that although D-Link could improve the unit, they really had no interest in doing so and it was relegated to backup duties until it failed.

Alt-F is an alternative firmware, you would need to find out more about that from their web page, a fun_plug is a script that can be run on the NAS at power up and used to add functionality through other scripts and applications compiled to run on the NAS, Fonz fun_plug is a fun_plug created by someone called Fonz, you can create your own if you have the necessary knowledge - at one point I did have one that allowed me to connect a USB external drive as an attempt at backing up the 323 but it required too much CLI manipulation to suit my needs.

I don't see either of these as a solution to your problem because I see the problem as a limitation of the hardware - and let me now contradict myself by saying that you could, if running one of these, possibly try backing up using rsync instead of ftp - it won't improve the throughput, but I believe it can be set to just transfer the part of the file which has changed, rather than copying every file in it's entirety, which potentially could reduce the amount of data to be transferred considerably.

[rjensen]

you can hit as much as 35MB/sec under ideal conditions, say transferring a single 2GB file, but transferring 1000 2MB files (the same total volume of data) would be significantly slower.

I am getting 3.5 MB/s transferring multiple 4.5 GB files. The backup is now into a plethora of smaller files (photos and music). If there is no change in transfer rate, would that help identify the problem?

I have heard about rsync, which is why I am interested in either Alt-F or fun_plug. Do these applications work together, or does fun_plug need to be installed on both devices?

Another observation: deleting files over a network connection takes a long time, with the time-to-delete a file increasing with file size. It takes 10-15 seconds to delete a 4.5 GB file. Any idea why? 

Thanks!

[JavaLawyer]

Another observation: deleting files over a network connection takes a long time, with the time-to-delete a file increasing with file size. It takes 10-15 seconds to delete a 4.5 GB file. Any idea why?  

Please correct me if I'm wrong, but very large files take longer to delete because they span many blocks which must be individually freed-up. Each block takes time to free-up, so the more blocks a file consumes, the more time it takes to delete.

[fordem]

If I recall correctly, with 4.5GB files you should be able to see transfer rates approaching 20 MByte/sec - at least I could do that from an IBM xSeries server to my 323.

A thought has just occurred to me - you're running RAID5 on the 343, and the RAID processing is being handled by the "fleapower" processor, which may limit the throughput - the numbers I mentioned were achieved with a RAID1 array in the 323 and an IBM xSeries server also with a RAID1 array - along the way I upgraded the IBM to RAID5 with an Adaptec RAID card powered by it's own PowerPC (I think) processor, and even with that throughput went down, once the cache memory on the RAID card filled.

[JavaLawyer]

Configure for gigabit if you don't already have that done, make sure ftp rate limiting is disabled in both the 323 & 343 (I don't remember exactly what D-Link calls it and my 323 died a month or two back so I can't go look and see). . .

D-Link still maintains a DNS-323 FW emulator here: DNS-323 Emulator (at least for the sake of nostalgia).

I am running into problems with my NAS to NASback-up. Specifically, I have a thoroughput of 3.5 MBs.

Configuration:
 NAS-343 with 6 TB RAID 5 and 1.04 firmware. (Primary backup, receives data from the computers.)
 NAS-323 with 4 TB JBOD and 1.09 firmware. (Secondary backup, backs up the NAS-343.)
 WRT310N router. Both NASs have static IPs and are connected directly to this router with Cat 5 cables.
 The 323 uses ftp to pull files from the 343 -- all firmware functions.

I back up the 343 every few months. At the current data transfer rate, it will take 5 days to back-up 1.5 TB to the 323.

Do you have a switch available? To facilitate troubleshooting, you may consider testing the DNS-323 and DNS-343 on a dedicated switch to eliminate potential variables, such as your router and other networked devices.

[fordem]

Thanks JL, flow control is what D-Link calls it - it's in the ftp server under the advanced tab.

[dosborne]

You don't need alt-f for rsync. Installing FFP0.5 (fun_plug) takes about 10 minutes. Installing and configuring rsyc takes another 10 minutes

[rjensen]

Okay...the backup is done and I have done some testing.

To confirm settings:
 FTP flow control: unlimited
 Both are reporting speeds of "1000 Mbps"
 The computer (below) is limited to 100 Mbps because its that old.
 The router (WRT310N router) does not support Jumbo Frames
 I do not have a switch I can sub in. I can say that I had the same issue with the previous router: WRT56G

I tested the transfer on both small files: 4.1 MB mp3 files (total 28.8 GB) and large files: 4.55 GB back-up files (total 47.7 GB).

FTP transfer from 343 to 323: small files 13.3 MB/s; large files: 4.6 MB/s
Computer to 343 (through router): small files 7.4 MB/s; large files: 6.0 MB/s
Computer to 323 (through router): small files 10.7 MB/s; large files: 7.7 MB/s

rsync will update backup's faster because it only revises the changed information.

The 323 and 343 appear well-built. I didn't realize they were processor limited -- "fleapowered" -- a humorous but apparently apt statement. Unfortunate. @D-Link: I am not impressed.

[fordem]

Let me put it this way - it's a consumer grade NAS appliance, meant to be left running 24/7 and to consume very little power whilst doing so.

You could, if you needed something with a few more hp under the hood, cobble together a full blown PC and then load something like FreeNAS or NASLite as the operating system, and whilst you might get the throughput you think you need (more on that later), you'd also find it takes more space, consumes more power, and runs hotter & noisier.

Honestly speaking - I was somewhat disappointed with the throughput on my DNS-323 and spent countless hours testing & tweaking before concluding that the limitation was the available processing power - actual network throughput tops out at 400 mbps but it won't/can't write to the disks at that rate.  In hindsight however, once I accepted that the unit could only do so much, it has sat in the corner behind me, and done everything I've actually needed it to do, mostly backup the data on my IBM server (and trust me, backing up to an online NAS is way easier & quicker than managing a tape subsystem) and server the occasional file across the web via ftp.

It never had the throughput I wanted, but in reality, it had the throughput I needed

The consumer NAS market has grown since the DNS-323 first appeared, and when my DNS-323 failed after roughly six years, I was not looking forward to identifying a replacement (Small Network Builder is a good resource if you want reviews) - I ended up buying a Netgear ReadyNAS, quite by accident - I went to Staples looking for a drafting stool, wandered past the display, feature set looked right, price looked right - and the store was out of stock.

At this point three weeks later, I have no idea what the throughput on the new NAS is like, it's already sitting in the same spot the old D-Link was, doing pretty much the same job.

To go back to the original issue - your 343 to 323 backup - from your tests I would say the 343 is limiting the transfer rate, possibly because of the RAID5 processing I mentioned earlier - I find it interesting that your throughput drops on large file transfers, my tests showed dramatic increases - you may have some degree of fragmentation, especially if the NASs have been in use for a while.

[rjensen]

My frustration is that home networks and devices are marketed with 1 Gbps throughputs but then lack the hardware to achieve 1/10th of that.  Many people have spent hundreds of hours trouble-shooting their networks because they weren't achieving the expected (based on advertising) functionality.

[fordem]

I hear you and I understand where you're coming from, and having said that - it's a matter of expectation, and as far as I can tell what you expected was not what you got, but was it a reasonable expectation?

I've never seen the 343 packaging, and I've long disposed of my 323 packaging - but - as far as I can recall, it never claimed gigabit throughput, although it does have a gigabit Ethernet interface.  It did specify "up to xxx MB/s read & yyy MB/s write" and whilst I no longer remember the numbers, a) they were easily achievable out of the box with no tweaking and b) they were well over what could be achieved with a 100mb/s interface.

It's important that you understand what happens when you transfer data from storage location A to storage location B - the transfer is limited by the slowest link through which the data passes - so assuming A & B are disks (no RAID, for simplicity), that's the slowest of - disk A transfer speed, disk A interface speed, bus A speed, network interface A speed, network infrastructure speed, network interface B speed, bus B speed, disk interface B speed, disk B transfer speed - and this is if you're transferring a continuous uninterrupted stream of data - completely ignoring the need to update directories, etc..

A disk with an interface speed of 1.5Gb/s (SATA-I) does not actually transfer data at that speed, mechanical limitations such as spindle rpm impose physical limits.  The network hardware may have a gigabit interface, but, it's ability to transfer data is going to be limited by the source of that data.

Data protection systems, such as RAID, impose their own limitations - having the system processor perform the RAID calculations is the least expensive way to do it, and the slowest, installing a RAID card with a dedicated processor improves performance and pushes the price up, but then will you be pricing the product "out of the target market"?

Make no mistake - I got the read/write speeds that D-Link advertised for the DNS-323 and they were gigabit speeds (technically, anything over 100mb/s is gigabit) - yes - I wanted more, but, was I willing to pay for it - not really.