Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[tentimes]

** IMPORTANT: This is the first posting of this guide and I recommend only advanced users with some linux knowledge use it until it gets a bit more community feedback. This information is provided by me (tentimes) as a user and I am NOT a member of dlink staff, nor is this an official support document from dlink - use at your own risk! **

Please note this from Stan:

Tentimes,

Thank you so much for the helpful guide.  I bought the 321 on sale at Tiger Direct and based on the success stories documented in this thread got a couple 1.5TB EARS drives to go in it.

I had exactly the same problem as Avenger when following the instructions, and the solution proposed by grixx was the answer.  Maybe the guide should be updated to say that if you are going to be formatting two brand new drives you have to insert both and let the DLink format them otherwise it does not expect to be using two separate drives.  When done exactly according to the instructions in the guide, I only got Volume 1 to show up.

Introduction

This guide shows how to install a 4k sector "advanced format" drive in the DNS-323 (for example the new WD EARS drives). It is intended for standalone drives; i.e. both drives separate, not JBOD or RAID. I will amend later for JBOD if someone can tell me how the 323 sets up the partitions for this. In theory if I had the partition info for RAID it could be updated for that too, but given that in the case of WD drives the TLER bit is disabled there seems little point at the moment. There is a guide on doing it with RAID though here: http://cellushun.co.tv/swapon-device-or-resource-busy

You don't really need to know linux, but I would not recommend this procedure for the totally clueless - just in case. It requires formating the drive(s) you are adding, so if you already have an advanced drive with stuff on it you will need to back that up and replace the data after "fixing" the drive. This method can be used to add one "4k sector" drive, or adding a set of drives.

This is definitely not all my own work, a lot is based on Consumed Consumer's blog, which you can read here: http://www.consumedconsumer.org/2010/06/upgrading-my-dns-323-to-2-x-2-tb-ii_08.html
I have used a different method to format the drives that I hope should work regardless of the size of the drive being added.

Before you start

Warning! Make sure you check and double check which drive you (a) Physically put into the drive bay, and (b) are writing commands to when formatting. I can't stress this enough - I have wiped a drive before by formatting the wrong one by mistake (got my left and right mixed up!) It is also possible to format the wrong drive from software with a typo. Here is a guide to which drive is which, as you look at the front of the NAS where the purdy blue lights are:

LEFT disk 2                                            RIGHT disk 1

4k Sect. Drive
/Advanced Format

HD_b1/2/4                                            HD_a1/2/4
sdb1/2/4                                               sda1/2/4

                                                            Funplug Bootstrap here!

Format this                                            DON'T Format this!!!!


Check which drive is in which slot by using the status section of the DNS-323 web interface. 1 & 2 correspond to the diagram above. I would also recommend physically marking the drives with a sticker or something.

Firmware: Make sure you have the NAS updated to firmware version 1.08. There is a guide for this on the dlink support site.

Changing drives: The NAS must be powered down to change a drive. You can do this via the web interface or by pressing the front button.

Funplug: Funplug is a small program which gives you access to the linux command line on the DNS-323. You will need to install it to use this method of formatting your 4k sector drive(s). Once this procedure is over you can either forget about it (having secured it) or use it to play about with your 323. It's great fun, doesn't alter the BIOS/Firmware and is a good introduction to linux if you haven't played with this stuff before. There are addons for allowing a USB drive, soundcard and all other sorts of stuff if you are so inclined.

Putty: You will need the windows SSH client "Putty" which you can download here: http://the.earth.li/~sgtatham/putty/latest/x86/putty-0.60-installer.exe

FDISK update: Download an updated FDISK, which is compatible with 4k sector drives, and store it somewhere for now: http://forum.dsmg600.info/attachment.php?item=393

Code sections: I've used the forums code formating to show the bits where you need to type stuff. Anything in square brackets is just me commenting on that line you are typing in.

4k Sectors: (For nerds only ) Although the physical sectors on the disk are 4k, the drive reports them as 512 Bytes in length, which means that any numbers you see when looking at the partition table in FDISK might not seem to make sense if you try and convert them to physical numbers. It took much wailing and gnashing of teeth to work out the right logical partition sizes. I don't think the NAS care's too much, but I tried to get them as close to the original as possible for the swap and mystery partition once scaled to 4k sectors.
 
The procedure

( 1 ) If you are adding TWO new 4k sector drives: remove any existing drives and hide so you can't format them by  mistake! Add the first of the new drives to the RIGHT bay, boot up the NAS and use the web interface to format drive 1 (standalone, not JBOD or RAID). At this point it will be formatted with the wrong cylinder boundaries, but it will do for now.

( 2 ) If you are adding ONE drive then make sure the drive being added is going to the LEFT bay. This may mean moving the drive you are keeping to the RIGHT bay and clearing the left slot for our new drive.

( 3 ) Add the new drive to the left bay (or add the next drive to the left bay if you are installing two new 4k sector drives). Boot up the NAS. If the new drive is unformatted then you will be prompted to format it. If so, hit SKIP - we do not want to format this drive using the web interface. At this point click on the status section of the web interface and check the drives are in the right place.

( 4 ) We now need to add Funplug. Rather than me typing it all out again, please go to the following link, install and follow the instructions all the way to the point where you have a working SSH login from windows. NB: It says CH3NAS - don't worry, it's the same process for the 323. You want to install it on Volume 1, the RIGHT side drive - we will be formatting the LEFT drive and running linux commands from the right drive: Here's the link: http://nas-tweaks.net/CH3SNAS:Tutorials/fun_plug


( 5 ) Unzip the fdisk file you downloaded earlier to the root directory of Volume 1 (RIGHT drive). We will rename the file fdisk in the /ffp/sbin folder to fdisk.old, add the new one, then make it executable.

Connect via SSH to the NAS, by running putty on your windows machine, and login as root with the password you made in the step above. Then use these commands:

Code: [Select]

cd /mnt/HD_a2/ffp/sbin
mv fdisk fdisk.old
mv ../../fdisk .
chmod +x fdisk
That third line: it's fdisk then a space then a dot. Note that we are working with HD_a2, which is the second (also the main and visible) partition of drive a (drive 1) which is in the right hand bay, where we will be running our linux commands from. You could just copy these lines over by copying each line from here and then a right click in putty will paste whats in the clipboard. Typing 'ls' is the same as 'dir' in dos. You can also use 'cd ..' to go up a directory level or '../..' to get to the root directory in the syntax. Just throwing that in in case you want to look around the directories a bit and see what's going on

( 6 ) Next we need to un-mount any partitions on the new drive we just inserted, drive 2 (on the left hand side of the NAS as we look at it) and also stop the samba file system so nothing on the network can interfere with the drive we are reformatting. You don't need to type the 'umount' lines if the drive you are adding is unformatted:

Code: [Select]

smb stop
nfs stop
umount /mnt/HD_b2
umount /mnt/HD_b4
umount /mnt/HD_b1
swapoff /dev/sdb1
These commands might give errors if any of those partitions aren't mounted. If so, don't worry: if they aren't mounted, they aren't mounted

( 7 ) If there are any partitions on Disk 2 then we need to delete them. Firstly start fdisk and list the partitions:

Code: [Select]

fdisk -H 224 -S 56 /dev/sdb
p
At this point if it shows no partitions press CTRL+C to exit FDISK and go to the next step. Otherwise, you need to delete the partitions. For each partition use the 'd' command, for example to erase partition 2 type:

Code: [Select]

d
2
When you have finished deleting we need to write the partition table, just type:

Code: [Select]

w
Check again the partitions are gone by typing p to print (list) the partition list. If they are all gone you can exit the FDISK program by pressing CTRL+C

( 8 ) Now we partition the drive so that it is useable by the NAS but also is aligned to 4k sectors. The partitions are numbered 1,2 and 4 (binary) Partition 1 is the swap, needed in case this will be Drive 1 in the NAS, as linux will need a swap partition. Partition 2 is the one that is visible and stores all your files, partition 4... well I have no idea what this is for, but the NAS needs it for something! After we set the partitions we will list them to check they are correct, then finalise. Keep in mind that the order on disk of the partitions is: 1,4,2. Make sure you partition sdb NOT sda!

Code: [Select]

fdisk -H 224 -S 56 /dev/sdb
n
p   [for primary partition]
1   [in response to choose partition]
1   [when asked for first cylinder]
85 [when asked for last cylinder]
n
p
4
86
169
n
p
2
170
[Press enter to accept the default value = last cylinder available]
t
1
82
p
The last bit marks partition 1 as swap. The partitions should be as follows: 1 = start 1, end 85; 4= start 86, end 169; 2 = Start 170, end = [big number depending on drive size}. Now check the readout of partitions to make sure there are no typos and just enter the last command:

Code: [Select]

w
to commit these partitions and write to the drive. If you make a mistake just enter 'd' to delete a partition, p to show what you have got, and enter it again correctly. Then press CTRL+C to exit FDISK. Next we need to format these three partitions, starting with the swap:

Code: [Select]

mkswap /dev/sdb1
mke2fs -j -m 0 -T largefile4 /dev/sdb4
mke2fs -j -m 0 -T largefile4 /dev/sdb2
mkswap formats a partition for swap use and mke2fs formats for filesystem use. Important! the '-j' argument makes it an EXT3 partition (it adds journalling). If you just want EXT2 then leave this argument out. Google EXT2 vs EXT3 for more information on the pro's and con's for this. I don't believe EXT3 will be any slower on the 323, but it is better for repairing errors. The character after 'm' is a zero (no need for super user reserved blocks as only ever root user stuff going on in the NAS). -T specifies the filesystem format, in this case largefile4 is what we need for our 4k sector blocks.

Next we finalise the filesystem, turn it on, and set the swap partition (setting swap in case we are on the second run through this procedure for people adding two drives)

Code: [Select]

hd_verify -w
smb start
nfs start
swapon /dev/sdb1
Now reboot the NAS using the web interface or by power cycling with the button on the front panel. Remember it takes a couple of minutes for the NAS to come back up

( 9 ) If you have a second drive to add then power down Mr NAS, move the drive in the left bay (2) to the right bay (1), and use steps 3 to 8 to setup this second drive.

( 10 ) When the system has rebooted, use the NAS web interface to check both drives are as they should be, then try some file transfers over your network to each drive to make sure it is working.

And there we have it! Your drive(s) will now be aligned to 4k sectors, meaning it will not suffer the terrible read/write penalties that it would if it was installed using the web interface. This problem is not Dlink's fault though, as they could not have anticipated that HD manufacturers would produce a drive that had a new sector size but reported the wrong sector size to the operating system.

Please keep in mind that this is the first posting of this guide and I will endeavour to update it regularly as people reply.

tentimes

EDIT: Added in the command to set partition 1 to swap (oops!)
EDIT2: Added "swapoff /dev/sdb1" to Step 6, as per Derek Lee, ProfM2 et al

[jeffers.r]

Great guide tentimes, much appreciated!

As a novice, I was hoping you might be able to clarify a few points for me:

(1) When choosing the number of heads for the fdisk -H option, what made you pick 224? Could this be any number divisible by 8? Fdisk suggests reasonable values are 255 and 16, so I wasn't sure what made 224 your desired number.

(2) Similarily, when choosing the number of sections for the fdisk -S option, what made you pick 56? Fdisk suggests a reasonable value is 63, so I'm curious as to your choice aside from picking a number divisible by 8.

Having reviewed the guide (http://wdc.custhelp.com/cgi-bin/wdc.cfg/php/enduser/std_adp.php?p_faqid=5655) from WD about formatting AF drives in linux, what came to my attention was:

Make sure that all partitions start on a multiple of 8 sectors (8x 512B = 4KB) and that partition sizes are multiples of 8 sectors. Make sure that there is space left at the start of partitions as required. For example on a boot drive, do not start at sector 0 as there needs to be space for the boot code.

(3) With this in mind, your start and end sectors (and resulting partition sizes) don't seem to be divisible by 8. Is this taken care of by the head and sector specifications you use within the fdisk command instead?

(4) In terms of the starting sector, I'm guessing the need for boot space isn't critical in this usage, nonetheless, should it be there anyway?

(5) How did you determine the size for the 1st and 4th partitions?

Last but not least, why not just activate the jumper setting on the WD EARS drives to force a hardware-level '+1' on every OS sector request? This doesn't appear to account for the partition size itself, but was curious if other's had input on this. (This article http://www.bit-tech.net/hardware/storage/2010/04/01/the-facts-4k-advanced-format-hard-disks/2 refers to it's use for Windows XP, but the default format by the DNS-323 also starts at sector 63.)

Many thanks for your help!

[tentimes]

Hi,

The first thing to remember is that H and S are logical values, but splitting them this way means that it is impossible to map to anything other than 4k sectors (8k actually). It is an arbitrary choice but one which I read about in an IBM white paper on the subject. This simply means the logical boundaries line up with the physical boundaries. It's been fairly well tested.

The choice of start and end points for the partitions wasn't quite arbitrary though. I worked out the actual sizes of the partitions that the 323 makes and then divided that by the new logical cylinder size we are using to get them as close to (but not smaller than) the previous partitions.

Many years ago I used to write disk copying programs for floppy disks (look up procopy by Kevin McCaughey for the Amstrad 464 ) so I have a bit of experience messing around with disks.

About the jumper.. don't do it! It causes massive slowdowns in random small writes and slowdowns in other areas as well. Look up Toms hardware etc guide on 4k sector drives - jumper = really bad idea. Actually I think they really should have made the drives proper 4k sector drives (and not report 512k sectors) and things would have been even easier. That's the reason the logical cyclinder size calculation might seem not to work Took me a while to figure that one out.

Remember the start and end sectors are logical not physical!

[m2k3423]

For drives with zone-bit recording, which are the case for practically all modern drives, the cylinder, head, sectors (CHS) settings are arbitrary. The drive electronics tell stories when the BIOS ask for identification, and the BIOS also tell stories when the operating system ask for drive parameters.  For Linux, hdparm has two options -I and -i that will give respective version of the stories.

In fact, specifying head and cylinder manually may actually be bad for some less intelligent boot loader, such as NTLDR. 

Modern operating system only work on LBA, logical block address. So, 8 x 512 byte sector makes 4096 bytes. So, to prevent an operating level cluster split between two disk-surface 4K-byte-sector, we just have to make sure that the start sector of a partition is multiples of 8 512-byte-sector.  For most file system type, the actual file-system organize allocation units in cluster.  This cluster size is more important for performance, and it should be set to 4K bytes (for mke2fs -b4096 -m0 -j /dev/sdX).  This instruct the operating system to always read/write in unit of 4K-bytes.  However, whether this will give you the performance boost expected depends on the kernel version - most recent Linux kernel versions are aware of logical and physical sector size. This is important since the LBA which the kernel places file system allocation information is important for real-application read/write performance. Those are the ones actually get accessed most frequently. 

For kernel version which are not aware of physical and logical sector sizes, then fdisk with option -b 4096 is what is takes to force sector size to 4096.

For Western Digital 4K sector, it is very important for the partition to start on multiples of 8.

[scaramanga]

Since you're a tad bit more informed about this than I am, can you take a look at the info before I let me know if I got it right or botched it. The HDD is a 1Gig WD Caviar Green AFD drive model number WD10EARS-00Y5B1.

Code: [Select]

/ # fdisk -ul /dev/sdb

Disk /dev/sdb: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders, total 1953525168 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdb1              64     1060295      530116  82 Linux swap
Partition 1 does not end on cylinder boundary
/dev/sdb2         2088456  1953525167   975718356  83 Linux
/dev/sdb4         1060296     2088455      514080  83 Linux
Partition 4 does not end on cylinder boundary

Partition table entries are not in disk order


Code: [Select]

/ # tune2fs -l /dev/sdb2
tune2fs 1.41.0 (10-Jul-2008)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          ae258f63-2d61-41b7-b74e-3b6be1fc298e
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags:         unsigned_directory_hash
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              238240
Block count:              243929589
Reserved block count:     0
Free blocks:              198973030
Free inodes:              229398
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      965
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         32
Inode blocks per group:   2
Filesystem created:       Tue Aug 10 18:20:45 2010
Last mount time:          Mon Aug 16 16:01:00 2010
Last write time:          Mon Aug 16 16:01:00 2010
Mount count:              10
Maximum mount count:      30
Last checked:             Tue Aug 10 18:20:45 2010
Check interval:           15552000 (6 months)
Next check after:         Sun Feb  6 17:20:45 2011
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      13266c80-20d6-43d0-a9da-0f05bf654ff2
Journal backup:           inode blocks

It's a bit after the fact but at least I'll learn something.
Thank you!

[tentimes]

Unfortunately that looks bad. The key line is the text that says it does not start on a cyclinder boundary. Using the method "fdisk -H 224 -S 54" it is impossible to have this happen. As far as I can tell, I reckon that even if you did not use my numbers for the size or placement of the partitions, but kept the fdisk settings, it would still work as the partitions would not cross cyclinder boundaries.

Make sure you have downloaded and written the updated Fdisk though - the old one won't work properly

[scaramanga]

I've done some reading (http://en.wikipedia.org/wiki/Logical_block_addressing) and am now convinced that it's a non-issue.

[jeffers.r]

Thanks to everyone for your replies and input.

@scaramanga, you might want to check out this article (http://www.osnews.com/story/22872/Linux_Not_Fully_Prepared_for_4096-Byte_Sector_Hard_Drives) as well, as performance wise, it might convince you that it could very well be an issue.

[scaramanga]

Thanks to everyone for your replies and input.

@scaramanga, you might want to check out this article (http://www.osnews.com/story/22872/Linux_Not_Fully_Prepared_for_4096-Byte_Sector_Hard_Drives) as well, as performance wise, it might convince you that it could very well be an issue.

Interesting. What surprised me here was that writing a large flie suffered from misaligned partition, not just small files. But this is different from what we're discussing above. I have, in fact, manually aligned my HDD(s) partitions on a 4KiB (= 8 * 512KiB) boundary. I posted the output from fdisk is sector units and everything should be divisible by 8.
The discussion above centered around the fact that we should also take into account that partitions should end on a cylinder boundary. You can read more about LBA and Cylinder/Head/Sector addressing on wikipedia. If I'm not mistaken, with today's modern drives where Cylinders contain a different amount of sectors depending on their location (outer cylinders contain more sectors) and all this complexity is hidden from the Kernel/Driver which uses LBA it is practically impossible to do that, and the warning message that fdisk shows is based on some bogus geometry reported by the Driver/HDD.

[wilburyan]

At the end when you say go into the web interface and restart it that way.... or press the power button.

There is a third option... seeing as you already have the telnet window up... just type: reboot

[daj01jka]

Hi Tentimes,

Before I launch of into your process of doing the formating inside the DNS - is there a way to do this easier in an external hard drive case/chassi and after formating inserting the disk into the DNS?

Thanks for your guide!

Warm regards,
Jonas

p.s. maybe a tool such as GParted (http://gparted.sourceforge.net/) could be used.

[tentimes]

Hi Tentimes,

Before I launch of into your process of doing the formating inside the DNS - is there a way to do this easier in an external hard drive case/chassi and after formating inserting the disk into the DNS?

Thanks for your guide!

Warm regards,
Jonas

p.s. maybe a tool such as GParted (http://gparted.sourceforge.net/) could be used.

The short answer is no. Possiblt using a linux box, but it's not worth the trouble tbh

[Tank_Killer]

BAM!

[ddcheng]

Have to ask a question here.  My case pertains to a DNS-321, but in terms of formatting, I think it's the same.  If it's not, I'll follow-up in the DNS-321 forum.

Anyway, I have just installed 2 Samsung HD204UI (2TB drives with 4k sectors) in my DNS-321 with RAID1.  I went ahead and used the web formatting to install the drives, and it didn't seem to have a problem.  However, I knew there were some issues with 4k sectors, so I did a little more research, and came across this link, so I thought I'd ask.  So far, I'm getting 133Mbps writes and 75Mbps reads, and I'm wondering if this will degrade due to the 4k sectors.  It's a new install, so I'm not worried about the data at this point.

This is my output from my drives.  Should I follow the procedure outline above?  Or is it ok since I'm not getting a message about the not starting or ending on a cylinder boundary?

Code: [Select]

root@dcNAS:/etc# fdisk -ul /dev/sda

Disk /dev/sda: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sda1              63     1060289      530113+ 82 Linux swap
/dev/sda2         2088450  3904903484  1951407517+ 83 Linux
/dev/sda4         1060290     2088449      514080  83 Linux

Partition table entries are not in disk order
root@dcNAS:/etc# fdisk -ul /dev/sdb

Disk /dev/sdb: 2000.3 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes

   Device Boot      Start         End      Blocks  Id System
/dev/sdb1              63     1060289      530113+ 82 Linux swap
/dev/sdb2         2088450  3904903484  1951407517+ 83 Linux
/dev/sdb4         1060290     2088449      514080  83 Linux

Partition table entries are not in disk order
root@dcNAS:/etc# tune2fs -l /dev/sda2
tune2fs 1.41.0 (10-Jul-2008)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          0da8ca80-4a21-4c6c-81f9-593b080774d6
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file
Filesystem flags:         unsigned_directory_hash
Default mount options:    (none)
Filesystem state:         not clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              121970688
Block count:              487851856
Reserved block count:     0
Free blocks:              458479878
Free inodes:              121925477
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      907
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Filesystem created:       Wed Nov  3 14:02:59 2010
Last mount time:          Wed Nov  3 22:53:54 2010
Last write time:          Thu Nov  4 16:45:24 2010
Mount count:              6
Maximum mount count:      33
Last checked:             Wed Nov  3 14:02:59 2010
Check interval:           15552000 (6 months)
Next check after:         Mon May  2 14:02:59 2011
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:           256
Required extra isize:     28
Desired extra isize:      28
Default directory hash:   half_md4
Directory Hash Seed:      46a82cf3-6f9a-469c-b801-2655fa861d5e


[Avenger]

Hoping for some help.  I have installed 1 2tb Samsung with no issue - it works in either slot with my 1.5 and 1tb drives.  I then swapped it to format another properly, and it works fine with the smaller drives.  Whenever I put BOTH 2tb drives in together though, it doesnt recognize them and asks to format it on logging in.  How can I get both 2tb drives working if they are seen properly one at a time with another drive but not together?

EDIT: To clarify, I am trying to make them both individual disks.  They will only work in combination with an old drive I had in.  If I put both, they ask me to format it.