• February 26, 2025, 11:00:26 AM
  • Welcome, Guest
Please login or register.

Login with username, password and session length
Advanced search  

News:

This Forum Beta is ONLY for registered owners of D-Link products in the USA for which we have created boards at this time.

Author Topic: Large Data Transfer starts "fast", then exponentially slows  (Read 6925 times)

Moose_On_Roids

  • Level 1 Member
  • *
  • Posts: 11
Large Data Transfer starts "fast", then exponentially slows
« on: January 12, 2011, 06:09:34 PM »

Disclaimer:  newb to DNS-323 / NAS in general...

I have used "LAN Speed Test" to determine my gigabit network speed.  I average around 18~22 MB/S when I try packet sizes ranging from 100MB to 2GB.  The link for this freeware from CNET is below...

(http://download.cnet.com/LAN-Speed-Test/3000-2085_4-10908738.html)

Here is a typical report summary (this was a 2GB transfer test)
Folder       : \\DLINK-7376B1\Volume_1

                           ---Writing---    ---Reading---
Packet length   :     2,097,152,000    2,097,152,000
Time to complete:     91.0324849       59.4027673
Bytes per second:     23,037,403  :)      35,303,944  :)
Bits per second :     184,299,224      282,431,552
                         -------------     -------------
Mbps:                 175.7614365      269.3477173

I am somewhat happy to see this based on speed postings in this forum.  I've performed "real-world" single file transfers with similar results.  However i've got a big problem performing "mirrored" backups to the DNS.  

Specifically, I am trying to mirror 160GB of files/folders to the DNS from my local HDD.  Based on the above, it should take about 2.25 hours (160GB/20MBsec/60sec/60min = 2.25hrs).  My backup software reports that it will take just over two hours at first, then real slooowly creeps up (over hours of time) to 4 hours, then 8 hours, then 12 hours, then 24 hours, and so on up to 37 hours (i let it run for 24hrs by this point).  After 24 hours, it still reports only 75% progress.  >:(

The software allows for mirroring without compression, which is how I have it configured and verified by browsing the DNS.  So i know it is not compression causing the slow down.  I've tried "Backup4All" & "FBackup4", both demonstrating the same exponential slow down.

Has anyone experienced this or can offer insight?  ???

My setup:
Gigibit NIC on PC
Gigibit switch
CAT 6 hardwire LAN throughout (from PC-->Switch-->DNS)
DNS-323 set to...
 - Auto LAN speed
 - Jumbo Frame Disabled
 - Single 1TB SAmsung F3 HD103SK HDD
 - Standard volume type (not RAID)

 BTW - One final note, I tried to change Lan speed on DNS and PC to 1000 fixed, instead of auto, and enabled jumbo on DNS to 8000.  This actually tested slower (around 13MB/S)  ???  Restored back to auto no jumbo, back to 20GB/S.  ::)
« Last Edit: January 12, 2011, 07:39:29 PM by Moose_On_Roids »
Logged

fordem

  • Level 10 Member
  • *****
  • Posts: 2168
Re: Large Data Transfer starts "fast", then exponentially slows
« Reply #1 on: January 13, 2011, 03:21:53 AM »

If your data is comprised of many small files, you'll experience the scenario you describe - the DNS-323 has an internal copy feature (quite primitive), which I use to tell it which folders on my server I want backed up, it seems to run significantly faster backing up the same data.
Logged
RAID1 is for disk redundancy - NOT data backup - don't confuse the two.

Moose_On_Roids

  • Level 1 Member
  • *
  • Posts: 11
Re: Large Data Transfer starts "fast", then exponentially slows
« Reply #2 on: January 13, 2011, 05:55:20 AM »

It is comprised of 310K files, a good mix of large and small files.  After I posted this subject, I found your reply to another similar subject...

If I could just add one single thing to jmonfar's excellent post ...

File size makes a significant difference.

For every file transferred, a certain amount of disk "housekeeping" must occur, storage blocks (sectors if you like) have to be marked as used and directory entries have to be created - these activities take place at what are essentially fixed locations that are often not adjacent to the location where the data is being written and the disks' read/write heads must swing backward & forward, and whilst this is happening, no data is being read or written.

The time taken to transfer a single 2GB file will be significantly less than that required to transfer 1000 x 2MB files, even though the volume of data can be considered to remain the same.


Sounds like this is the problem then.  Can you explain how to implement the DNS internal copy command?
« Last Edit: January 13, 2011, 06:04:54 AM by Moose_On_Roids »
Logged

fordem

  • Level 10 Member
  • *****
  • Posts: 2168
Re: Large Data Transfer starts "fast", then exponentially slows
« Reply #3 on: January 13, 2011, 09:27:52 AM »

When you go to the admin login webpage there are two buttons, configuration & application - enter your credentials & select application - the scheduled download application should be there by default.

Create a user on the system you wish to backup and make sure they have access to the data files that you wish backed up and then enter the relevant details on the DNS-323.
Logged
RAID1 is for disk redundancy - NOT data backup - don't confuse the two.