D-Link Forums

The Graveyard - Products No Longer Supported => DNS-323 => D-Link Storage => Beta code! => Topic started by: drick on May 16, 2009, 08:18:43 AM

Title: EXT2 to EXT3 conversion?
Post by: drick on May 16, 2009, 08:18:43 AM
From the release notes:

6. Re-added EXT3 Support.

so it looks like EXT3 has made it back into the 323, but i can't find it anywhere. i assume this option is only available upon new HDD initialization, right?

i did a search for EXT2 to EXT3 conversion on the D-Link forums, but got no hits. doing a web search however implies that this can be done as there are multiple FAQ's and directions on how to do so.

is this something that D Link has looked into, or perhaps could be accomplished via ffp?



Title: Re: EXT2 to EXT3 conversion?
Post by: ttmcmurry on May 16, 2009, 08:43:53 AM
ECF made this (http://forums.dlink.com/index.php?topic=4586.msg26555#msg26555) post over in the 321 forum and it makes sense for upgrading drive size.  But I have some questions relating to RAID upgrades since I can't try it out at the moment:

1)  Can you format a single drive as RAID-1 / EXT3?
2)  If not, Can you format a single drive as EXT3, then "upgrade" to RAID-1 without destroying data?

Title: Re: EXT2 to EXT3 conversion?
Post by: drick on May 16, 2009, 09:59:54 AM
both valid points, and good questions.

i'm not sure how applicable the other post is though as it doesn't mention anything around ext3
Title: Re: EXT2 to EXT3 conversion?
Post by: Arvald on May 16, 2009, 10:15:28 AM
I'd like to see being able to create a degraded RAID set.
Most enterprise class systems have this feature.  but that may be the answer in saying "enterprise class"

Why you ask?  well in such a limitted unit as the 323 (meaning 2 bays) having the ability to degrade, put in another drive to copy data then put the old drive back in and rebuild is very handy.

I'd also like to not have to muck around to move data off the drive I have when I upgrade to 2 new disks.
My WD Caviar Black was not meant for this unit but is stuck in there now till I can migrate.

Title: Re: EXT2 to EXT3 conversion?
Post by: fordem on May 16, 2009, 01:43:40 PM
I'd like to see being able to create a degraded RAID set.
Most enterprise class systems have this feature.  but that may be the answer in saying "enterprise class"

Why you ask?  well in such a limitted unit as the 323 (meaning 2 bays) having the ability to degrade, put in another drive to copy data then put the old drive back in and rebuild is very handy.

I'd also like to not have to muck around to move data off the drive I have when I upgrade to 2 new disks.
My WD Caviar Black was not meant for this unit but is stuck in there now till I can migrate.



This is not how RAID was intended to be used.
Title: Re: EXT2 to EXT3 conversion?
Post by: Arvald on May 16, 2009, 05:22:01 PM
RAID is intended to be usable in a degraded state which can include system maintenance.

in higher end hardware raid solutions the disks are not tied to each other and can be separated... the controller does all the work.

I work with SANs (and prior to SAN large scale raid arrays ie 48 disk raid 1+0 arrays) for a living and have plenty experience with them and how they are to be used.
Title: Re: EXT2 to EXT3 conversion?
Post by: ttmcmurry on May 16, 2009, 09:45:26 PM
The more I think about my question especially in regard to some of the storage solutions used at my workplace, the more I began to realize what a stupid question it was that I asked.

I can't honestly think of an enteprise RAID controller that allows an array to be created with "missing" drives.  Be it a PERC 3/4/5/6 or something by EMC (Cx Series) or Equallogic (5xxx & 6xxx) series.

So my presumption of being able to take a drive out and create a raid-1 array with only one drive is fundamentally flawed. 

However, the possibility of converting EXT2 to EXT3 could still exist.  Remember, the 323 has flash memory onboard and could boot into a "conversion" mode where Volume_1 wouldn't be mounted and a conversion process began.

Seems the easiest thing is to borrow a hard drive of equal size to the 323's current storage, copy everything over with the fixed FTP server, create the array, and format with EXT3. 
Title: Re: EXT2 to EXT3 conversion?
Post by: Arvald on May 16, 2009, 11:56:40 PM
another option would be to boot a linux live CD on another machine and run the conversion in linux.

I agree with you ttmcmury about wishing for enterprise level features... though we are starting to get close to having them in our homes.  maybe in a few more years.
Title: Re: EXT2 to EXT3 conversion?
Post by: ttmcmurry on May 17, 2009, 03:19:52 PM
Oh no, I don't want enterprise features in the 323.  It's nice to pay less than $150 for this kind of hardware.

I'm perfectly satisfied with what the 323 does at its price point.
Title: Re: EXT2 to EXT3 conversion?
Post by: fordem on May 17, 2009, 06:13:29 PM
ECF made this (http://forums.dlink.com/index.php?topic=4586.msg26555#msg26555) post over in the 321 forum and it makes sense for upgrading drive size.  But I have some questions relating to RAID upgrades since I can't try it out at the moment:

1)  Can you format a single drive as RAID-1 / EXT3?
2)  If not, Can you format a single drive as EXT3, then "upgrade" to RAID-1 without destroying data?



I haven't tested this yet, but I will most likely take a look at it during this coming week.

In fw 1.05 the ability to migrate a single standard volume to a RAID1 array was added, presumably this still exists, so you should be able to create a single drive ext3 volume, and then, by adding a second drive, migrate to a RAID1, ext3 pair.
Title: Re: EXT2 to EXT3 conversion?
Post by: drick on May 17, 2009, 08:19:28 PM
Oh no, I don't want enterprise features in the 323.  It's nice to pay less than $150 for this kind of hardware.

I'm perfectly satisfied with what the 323 does at its price point.

agreed.

think of it this way. 2 years ago, maybe a year ago you couldn't get this much functionality at 3x the price.

IMHO this is very similiar to Moore's law. 2 years from now, we will all have RAID10 / RAID6 devices with 2TB+ HDD's at the same cost as this unit, and that is goodness.

if you want all the rest of the features you mentioned, go with EMC as you mentioned above, but bring your checkbook.
Title: Re: EXT2 to EXT3 conversion?
Post by: drick on May 17, 2009, 08:20:43 PM
back to the subject at hand, aside from drive swapping idea is there any other way to do this?

does anyone think that i could run the linux ext3 convert commands from ffp successfully?
Title: Re: EXT2 to EXT3 conversion?
Post by: ttmcmurry on May 17, 2009, 09:28:08 PM
drick--

You bring up a good point.  While I had no trouble with ffp being installed during an upgrade from 1.06 -> 1.08, is everyone else seeing the same?  Does D-Link test out upgrades with fun_plugs installed?  I'm sure the official answer is "that's unsupported"

When I upgraded, I made sure to chmod -x several .sh scripts that might interfere with an upgrade-- namely starttwonky.sh, cleanboot.sh, lighttpd.sh, and mysqld.sh.

I got to thinking and checked for tune2fs and it is installed by default with this firmware (at least I didn't see where it was part of ffp 0.5) and you can look it up on the web (http://lmgtfy.com/?q=linux+convert+ext2+ext3) for some how-tos regarding filesystem conversion.  I'm a bit worried about playing around with it for the time being.  Fordem-- perhaps this is up your alley. :)

On a separate note, what did I do to get TWO -1s today?   ???
Title: Re: EXT2 to EXT3 conversion?
Post by: ttmcmurry on May 17, 2009, 10:02:56 PM
I came across this (http://www.archlinux.it/wiki/index.php?title=Convert_a_single_drive_system_to_RAID) article.  So, for those of us with ffp, we can borrow tenets from the idea ECF had in another post and if someone is willing to test it out:

1)  Label both drives so you know what bay they came from
2)  Test out each drive individually to ensure there is data on both
3)  (hazy part) put the right drive back in (could be left?)
4)  Use the methods in the above article to create a new raid-1 with a missing drive and format it ext3
5)  Make sure auto rebuild is disabled
6)  Put the left drive back in & mount it
7)  (hazy part) Copy the data to the new volume
8)  Use the methods in the article to join the 2nd drive to the array and format that ext3
9)  Allow RAID-1 replication to propogate

What I don't know would happen when an ext2 volume that's set to have its replication partner in the opposite bay suddenly appears.. would it mount.. what would the ext3 volume do since it was created with a missing disk?  Guessing that's where /dev/md0 /dev/md1 etc comes into play..

If I had two spare sata drives laying around I'd try it out myself. Perhaps I can go hunting for spare parts at work. :-D
Title: Re: EXT2 to EXT3 conversion?
Post by: drick on May 18, 2009, 10:00:32 AM

On a separate note, what did I do to get TWO -1s today?   ???

wasn't me, i don't even know what those are or how to "assign" them.
Title: Re: EXT2 to EXT3 conversion?
Post by: drick on May 18, 2009, 02:34:27 PM
just scrounged up 2 more HDD's, and started the install (saving the upgrade trial for a rainy day).

ext3 shows up after the RAID selection as was suggested.
Title: Re: EXT2 to EXT3 conversion?
Post by: Fatman on May 18, 2009, 04:24:21 PM
It would be interesting to have someone mount their drives as soft raid drives on a Linux PC and do the manual conversion, it takes like 30 seconds to do the conversion in Linux, and since it is not going to be a system partition there are no weird scenarios.

http://www.troubleshooters.com/linux/ext2toext3.htm

P.S. This should be done at your own risk, don't mess around with data you can't replace.


*** Modified by Fatman upon realizing we live in a litigious society 2 seconds after posting.
Title: Re: EXT2 to EXT3 conversion?
Post by: ttmcmurry on May 18, 2009, 04:47:57 PM
I was trying to do this with two spare 40GB drives and my 323, but ffp won't install on 1.08. 

Did you guys remove that architecture?


I should be smited for that.  After taking out my contacts and replacing them with my glasses, I realized the error between my ears--

Anyhow I'm working on a EXT2->EXT3 migrate path with fallback plan. 
Title: Re: EXT2 to EXT3 conversion?
Post by: drick on May 18, 2009, 04:49:03 PM
It would be interesting to have someone mount their drives as soft raid drives on a Linux PC and do the manual conversion, it takes like 30 seconds to do the conversion in Linux, and since it is not going to be a system partition there are no weird scenarios.

http://www.troubleshooters.com/linux/ext2toext3.htm

P.S. This should be done at your own risk, don't mess around with data you can't replace.

i just keep thinking that someone has already done this for ffp which would eliminate the need to move drives in and out of the 323 which would be ideal.

i'll post it over there and see what they say.
Title: Re: EXT2 to EXT3 conversion?
Post by: drick on May 18, 2009, 04:50:26 PM
I was trying to do this with two spare 40GB drives and my 323, but ffp won't install on 1.08. 

Did you guys remove that architecture?

it's still there, i just did it 30m ago with 1.08

make sure you save the ffp script & binaries directly to the 323 not anywhere else (i.e. to a windows machine) and then copy/move them to the 323.

windows steps on them and corrupts them, figured that one out the hard way
Title: Re: EXT2 to EXT3 conversion?
Post by: ttmcmurry on May 18, 2009, 08:36:58 PM
I was able to successfully upgrade from ext2->ext3 on my 323.  Here's what I did:

1)  Have ffp 0.5 installed & configured with telnet access & the root account
2)  Disable *all* ffp & startup scripts with the exception of telnetd.sh
3)  Shutdown and take out the RIGHT drive
4)  Run tune2fs against /dev/md0 & reboot
5)  Run tune2fs against /dev/sdb2 (or whatever it is on your 323) & reboot
6)  Verify your (test) data is intact and that the mount still works with RW privileges
7)  If successful, you can shutdown and reinsert the RIGHT drive
8)  Depending on your settings, it will either auto rebuild or wait for you to activate the rebuild

Then wait for it to rebuild the array.  There is no need to mess around with /etc/fstab.

The only real issue I noticed was when i deliberately forced a drive to fail (mdadm /dev/md0 --fail /dev/sda2) and checked it (mdadm --detail /dev/md0), it showed failed status as it should have.  The GUI also showed degraded status.  HOWEVER -- the drive lights remained blue and showed no visible sign of failure. 

PS:  don't try tune2fs againse /dev/root  .. just a piece of advice :)
Title: Re: EXT2 to EXT3 conversion?
Post by: drick on May 19, 2009, 06:57:18 AM
nicely done!

i wish i could have tried this, but i went the easy route and just used new drives.  ;D
Title: Re: EXT2 to EXT3 conversion?
Post by: ttmcmurry on May 19, 2009, 07:53:04 PM
Yeah I have mixed feelings about converting.... whoopty-doo it can be done.. but until thousands of these have been successfully performed and have proven to be stable through multiple raid degredations (is that a word?) as well as through various filesystem issues & repairs... THEN I'd do it with my real data.

So-- I don't see D-Link going down this path even though it is possible.  It doesn't make much sense for them to introduce a "risky" feature since it hasn't been thoroughly tested and they would catch a lot of flak for every failed conversion. 

This is not as easy/supported as on Windows a-la convert.exe /fs:xxxx - which i've used hundreds of times over the years going from every format it supports. 
Title: Re: EXT2 to EXT3 conversion?
Post by: amichel on June 01, 2009, 09:31:31 AM
Hi all,
so I migrated my data following the instructions by ttmcmurry adn all went fine.
However when looking at the details using dmesg I get this message.

 ext2_fill_super: mounting ext3 filesystem as ext2

Which showed me this solution http://forum.dsmg600.info/t3202-WIERD-ext3-issue.html
Is there a still a need to mess around with fstab in firmware 1.08?

EDIT:
Actually after implementing the tip in the above article I was able to mount the Array with ext3.
fonz fschk works fine on the array

amichel