[dipstick]

Hey guys. I just finished wiring up the whole house to take advantage of a gigabit network. Everything is on Cat6 cables and running off a D-Link 4300 and a Netgear gigabit switch. There are a couple of 100 foot cables and I'm wondering if they are affecting speed throughput.
Anyways, I can't seem to break 28MB/sec transfer speeds. Windows task manager shows usage hovering at around 25-29% during heavy transfers. Does this sound ok? Seems like an awful waste of network utilization but I wont argue that its not a sh*tload better than the old 100mbps setup.
Any ideas?
Thanks

[mkk]

The harddrives at both ends need to be modern and free from doing other chores at the same time in order to be able to deliver much better performance than that, so perhaps the answer lies there. It would also take a RAID setup to deliver speeds up to the Gbit limit.

[Zardon]

yeah depends on whether you mean 28mbit or 28 MB/s if its 28MB/s its probably being limited by the Hard drive and unless you are copying across the network from raid to raid it seems about right. ive a few USB2 drives and they limit at around 30-35mb/s. I would expect internal sata drive copies to be higher than 28 however, perhaps around 50mb/s. I think you need to play with HDtach and make sure that the HD limits aren't coming into play before you look at the network.

If you have 3 or 4 pcs you could try several copies at the same time to see how the network handles that.

[Judas]

I don't think you should need to worry about the footage at all

[dipstick]

Thanks guys, that totally makes sense. I feel like a fool lol. Even though all the PC's are running raid most transfers are happening off of usb2 external storage drives and HD tach confirms why my transfer speeds are what they are. Many thanks


PS. It is definitely MB/s and it seems pretty darn good to me now. Also good to hear cable length doesn't matter much

[Judas]

i think the fastest i've obtained via 100mbit pipe is something like 9.2MB/s sustained..... and with my raid 0 arrays, 1000mbit pipe, i managed 36MB/s occasionally seeing high 40s.... i think 90-100MB/s would be accurate for the highest obtainable

btw, anyone come across the 10gbit cards?

[dipstick]

yea I wasn't complaining about speeds, they're great. I was just wondering if they seemed ok since I've never been a gigabit network before. 10 gigabit would be just insane and probably totally useless for any home PC lol. I notice that every 1% of network usage is 1MB/s so it seems you are spot on with the 90-100MB/s max figure.

[Judas]

it's unfortunate that a newer TCP/IP... or something that it a new generation altogether that would further improve network utilization hasn't arrived.....

[Ferret_Style]

Ip6 came out but even if you try it i have not seen any better performance.

On the distance of CAT 6 on gigabit i think you can go up to 100m without much of an issue. Any further then that and you need to ensure good shielding on the cable as interference will really mess it up..... had a line in a house i was working on that ran behind the microwave... Was funny when they cooked dinner and the network started to go funky.

the other issue on network trafic is general windows overhead. It does not allow as full of transfers as you would think. Google limitations and i know you will find a few tweeks.

[H3X4D3C1M4L]

I recently broke this issue in half. The keyword: MTU

See... I did something similar but I have the cables in place (by the way you got ripped off with the Cat 6, Cat 5e works the same). I was worried about length, shielding, how close the wires ran to AC power etc. but it ended up being that it was the machines.

The first thing to do is find out how big the frames can be on your switch/router (I think your DLink will take 9600K frames). Then you want to enable jumbo frames on everything.

After that, get this tool DrTCP windows tcp tuning and tweaking - dslreports.com and set the MTU to something big. Example: Before my MTU was just blank meaning it used the Windows default or whatever it found. I had jumbo frames on and I was still getting garbage performance (200MBits barely at best, at worst 10Mbits!). After turning off Path MTU discovery (PMTU), changing my receive window to 256960, and making sure my receive and send buffers were as large as possible, and making my MTU 9000 well... I saw 950 MBits to every machine solid

Note that Vista is unaffected by this, during initial testing I found that I could receive really fast from that machine but not send to it very fast, changing the MTU fixed that though. I also found that Linux and BSD are largely unaffected by this... I just enabled larger buffers and fixed it.

Anyway once you do that it should spike the performance into space. Then you'll find out that your disks are much more of a limit than you ever imagined

I can give you a more full story and tell you which tool I used to measure bandwidth if you like but the end result is just what this post describes. Lucky for you I did all the grunt work

[dipstick]

Thanks a lot for the info; I will definitely look into it

[Judas]

H3X, in vista, what did you change?

[PandaZ3]

When I asked in other forums a year and a half ago I was told it couldn't be done. Kinda steams me now! I will have to go back thru my network. I am already mostly Cat 6 with a couple of Cat 5e's and maybe one or two Cat 5's. Cat 5e and Cat 6 are just a dollar apart so I but Cat 6. I was think in of getting a dual Wan router anyway, as I have both T1 input and cable

[Judas]

xinxom...... speedy bastard of a router..

plug it in, then use a gigabit hub

the xincom 702 is suspected of showing up soon which is reported to have "gigabyte wan/lan"

[H3X4D3C1M4L]

Originally Posted by Judas

H3X, in vista, what did you change?

Nothing. Forcing an MTU size or changing anything else did little to nothing. Vista's networking stack "learns" to adapt to the network(s) it is connected to. Its XP you gotta worry about. But I'd take a well configured XP over Vista any day.

[Judas]

really?

Since moving to vista... i've noticed that my throughput has been exceptional..... i've experiencing less slowdowns or oddities ..... it's like going from networking on windows 98 to xp..... lol

[Tipstaff]

Something of interest to all this (and this is for people thinking of switching over to a Gigabit setup): for the best Gigabit performance get a full Gigabit setup. By this I mean don't add a Gigabit switch to a router that is only 10/100. Get yourself a router that has a Gigabit switch built in. You would think that routers wouldn't interfere with data being transfered between 2 computers on a Gigabit switch. Wrong. The reason it does is because the computer makes calls to the Internet, and because of this it switches back to 100 in order to talk to the router, but it won't stay there. Back and fourth it will switch, from 100 to a gig and back again all during the transfer to the other computer.

Here's what I mean: I was transferring a 6.7GB file between 2 Gigabit machines, and I tried different setups all on good quality Cat 5e cable. On the 10/100 router setup it would take 22 minutes. On the Gig switch/ 10/100 router setup it dropped to 11 minutes. However, on a full gig setup using the D-Link DIR-655 router (with firmware 1.11) that time has dropped to 6 minutes. Of interest, I redid these tests using Cat 6. Difference in speed? 1 minute on the Gigabit setups. 1 freaking minute. So, again, if you are going to go Gigabit, spend the money, and get the right setup.

Last thing: if you are going to do your own cable (run your own cable, hookup your own wall outlets, make your own ends, and so fourth) read up on how to do it properly. Most people that think they know how to do networking make 3 serious mistakes every single time (on top of using cheap cable that is): they run cable right beside electrical lines the entire length of the run, they cut more than an inch of shielding off from the ends of the cable, and they untwist too much of the wires to make the connectors. The last thing is really important. Those twists, they are their for a reason. On 10/100 network setup those twists become a huge factor in performance on longer distances, but on a Gigabit network, those twists are essential in short and long runs. Each pairs of wire is twisted (4 pairs total), and are twisted 3 times per inch to reduce not only crosstalk, but to reduce the effects of outside interference. So what does this have to do with the connectors? Well, most people put wall outlets near other ones, usually near electrical outlets. If the ends are untwisted too much you will get interference from that electrical receptacle, which in turn will impact performance. So, when you do the connectors and wall outlets remember this: you want to maintain those twists as close to the connectors as possible, and keep those wall outlets away from electrical outlets.

[Judas]

It's REALLY router specific.. and switch specific.

I currently use the xincom dpg-502 twin wan router which has a lan network speed of 100mbps .. but i got a few machines on a 1gbps switch..

Yes, transfering over and through the router it slows to 100mbps.... however, a direct connection from the machine, to the switch to the machine..... in my case, remains fully 1gbps... This is while transfering nearly 500gb of information... The Time difference is HOURS from the 100mbps connection adn the 1gbps.....

I also tried a direct connection from PC-PC avoiding the switch altogether (this is where the real test is imo)....

1gbps, the time difference was meer minutes, but obviously a little bit faster..... probably on a scale of 10 minutes....

[H3X4D3C1M4L]

Its also chipset. From what I've learned and experienced setting up OpenBSD routers, Realtek or nVidia ethernet chipsets are usually poor performers, Intel ones are usually pretty good, Marvell falls somewhere in the middle, etc. A quick Google usually yields all the info fairly well.

Remember that its not brand that counts. DLink for example will use anything from Marvell chipsets to Sundance. Again, Google will usually give you the details.

[Tipstaff]

Originally Posted by H3X4D3C1M4L

Its also chipset. From what I've learned and experienced setting up OpenBSD routers, Realtek or nVidia ethernet chipsets are usually poor performers, Intel ones are usually pretty good, Marvell falls somewhere in the middle, etc. A quick Google usually yields all the info fairly well.

Remember that its not brand that counts. DLink for example will use anything from Marvell chipsets to Sundance. Again, Google will usually give you the details.

On top of that, a lot of companies will use good chips, but in order to put out a cheap card, or a line of cards offering cheap to pricier cards, will make their cheap card with the bare minimum components.. just enough to get it to work. In this case you get a card that, while it is faster than 100mb speeds, gets nowhere close to Gigabit speeds, AND puts more load on the CPU btw. I've seen some cards that use the Realtek chip, built to Realteks specs, perform amazingly, and I've seen that same chip on cheap ass boards run like absolute crap for the exact reason I mentioned: they were built to minimum specs.

[dipstick]

Hey guys, sorry to bump this again but I'd like to say that you guys were right about the HD limiting my speeds. I removed them from their external enclosures and speeds have increased dramatically. I'm now hitting 80%+ network utilization and speeds hover at 45-50MB/s. All I can say is that its smoking fast. CPU usage is also surprisingly low at under 5%. Thanks again for all the help and tips.

[dj_stick]

the CPU usage from the external enclosures is due to the USB interface - massive CPU overhead - firewire is a better option, no-where near as high a CPU usage, and e-sata is even better

[dipstick]

Yeah usb is definitely a little heavy on the cpu. Its one of the things I hope they improve on with the next version. I was a little disappointed to find out the drives are pata but its still much better than usb
Thanks

[Tipstaff]

Originally Posted by dipstick

Yeah usb is definitely a little heavy on the cpu. Its one of the things I hope they improve on with the next version. I was a little disappointed to find out the drives are pata but its still much better than usb
Thanks

Intel kept side stepping that very question during their Intel Development Forum. They were unwilling to say if USB 3.0 would reduce or increase CPU usage, and since they say it draws less power, they wouldn't say by how much.

In any case we'll be waiting 2 more years before we get our hands on it.

[H3X4D3C1M4L]

Considering though that these days people have typically 1 - 3 cores sitting idle while they're shunting files around.... I don't think CPU overhead is as big a deal as once it was. No excuse for designing a data bus in an inefficient way (as I hope they don't do with USB3 and not that I think USB2 is all that bad) but still.... not like we gotta worry about overloading our 500MHz CPUs here

[Judas]

They thought the same thing before UDMA was introduced..... meh, who needs to reduce the cpu usage durring such transfers or writes..... it doesn't take that long..

it would help considerably if they introduced a ultra usb mode or something.

[H3X4D3C1M4L]

I hate to bump an old thread but I've got some more good information on gigabit speeds.

If you've not got any hardware limitations (PCI card sitting on a full PCI bus for example), I've found that installing the drivers for your device and leaving everything but the jumbo buffers option (enabling that if it isn't already) at the default and then enabling Window Scaling with Dr. TCP and disabling Selective Acknowledgements, I was able to get 850 - 950 Mbits/sec consistantly testing with Iperf using only a 256K window size.

I'll also retract my previous statement where I said to make the MTU big with Dr. TCP. That won't do anything - leave the box blank and just enable Window Scaling.

In any case using Iperf I can test with any machine on my network no matter the platform. It also escapes any hard drive limits by transferring data purely from memory and not off of any storage medium, so its a great way to test the theoretical maximum of your network.

[PandaZ3]

Well.... as long as you have bumped it ! ......... I have discovered that Intel can/does make good nics and here is one worthy of mention because it does not use a, precious to some, PCI slot. It uses PCI-E 1X (Those little short slots nobody made anything for... but all newer boards have) So even if your on-board Nic is not Gigabit and you can't bear to loose your sound-card... there is still hope.