Plugging into the switch is a better choice in my house. The router ports are not GBit.
Thanks for posting your specs. Unfortunately I cannot lend any further assistance since I assume you would prefer good help. I run a RAID1 and like you have a basic NAS setup. I have NFS disabled.
For some reason, and it could be a false memory, but I thought I saw folks having problems with NFS and SMB running at the same time. I'm probably wrong. I'm sure that since I added this statement, someone will chime in and lay down some facts.
Also, did you format in EXT2 or EXT3. Doubt it makes a difference but you never know.
Good luck...
-Joe
P.S. 5 people on this NAS? is it working fast enough?