Another 10/100/1000 Nic Question.... (1 Viewer)

[Mark Giles]

I have two computers that are connected by a crossover cable. They also have each another nic card that are used to access the internet thru a router and cable modem.

The question I have is, both of my computers have a 10/100/1000 nic card that have the crossover cable plugged into. But yet it takes a 700mb file over a minute to transfer.

Doing my math. With 1000 megabits transfer rate, shouldnt the file only take 5 seconds or so (1000mbits/8 = 125mbytes and 700mb file/125mbytes/s=5.6 seconds) ? Well, I know it also may come down to the speed of the hard drive on both computers...so is this now my bottleneck? The hard drives? isnt the transfer speed for the average HD like 60mb/s (megabytes/second)? If so, that would still be like 11 seconds to transfer a 700 mb file. Yet it's still taking over a minute to transfer that.

How to I take advantage of the 1000 megabit transfer rate??? What is causing it to be so slow?

 

[Tekara]

Just like everything, no ethernet card runs to spec, heck it's pretty impressive if you get over half spec.

The biggest killer is the overhead imposed by the OS, there's a lot of extra traffic that will be generated that will slow things down a fair bit.

Your network is running good and there's nothing you should be worried about.

--------------

Hardrives have a very high burst speed (transfer buffered data from cache), but their sustaned transfer rate is actually very low. They will almost always be the bottleneck for this reason.

 

[rob-h]

There is not rally any overhead imposed by the OS unless he is running services that would require it. I doubt that because he only has 2 and he is connecting via a crossover.

Are these the same type network cards? Are they set to auto negotiate? You may want to set it manually if so as auto negotiate can sometimes cause overhead itself when the cards are constantly changing settings. If you have set it manually, try setting it to auto. What grade crossover are you using?

 

[Mark Giles]

Rob-h

Yeah, I that's a good idea. I havnt checked the settings on either nic card. And, I dont remember the type of crossover cable I have. Perhaps that's the problem. I forgot for the last week about the 10/100/1000 cables. I will go pick one up later today. The xover cable I have now is probably 3 years old (or whenever the Xbox first came out). So more than likely I bought just a 10/100 cable.

Both of my computers are custom built, one has the 10/100/1000 nic card built into it. The MSI K8N Neo2 Platinum Sli board. And the other is a Abit AT7 Max2. I just had to buy a fast nic card for the Abit board cause its built in nic card was just 10/100.

At first I thought it was using the internet connection to transfer files to each other but after unplugging the other cat5 cables, I saw I was still able to transfer files.

I transfer a lot of large video files that are several GB's in size that normally take several 10's of minutes to transfer. I'm just trying to cut down on this rate.

 

[rob-h]

It's not uncommon to see 250 mb/s with gig ethernet (using 5e or better cable).

 

[Tekara]

Actually windows file sharing sends out announcments to establish itself on a network periodically. You can always tell your on a network with a lot of windows clients by all the broadcasts that you can pick up with a packet sniffer. This is the primary reason why windows file sharing protocols are blocked at the switches in most large networks.

But I am still in agreement with the others that the network looks to be running pretty good. You may want to try using a different protocol for transmitting the file though, you should have some better throughput by using a simple ftp setup instead of file sharing.

 

[rob-h]

Rob.M.G. Port 445 traffic is what you are referring to. It only occurs if you have a mapped drive or an open UNC connection to a share. Even then its only a packet every 3 to 5 minutes. Standard windows PC's DO NOT send large amounts of broadcasts. As I said, if he were running services he might have that traffic. I seriously doubt that he is running anything with that configuration.


Paul Simoneau, you can drastically reduce throughput if the cards are constantly renegotiating. Even cards of the same MFG sometimes have trouble autonegotiating with each other. Its best practice to manually set the speed and duplex. If you are using a switch, its best to set it manually there as well. This is not always the case though. I have had it where auto negotiate was significantly faster then manually setting it to the exact same speed as was autonegotiated.

Paul Simoneau, if he is using cat 5e, it tops at 200mb/s, its recommended that he use cat 6 to exploit the full capability. Even though his link may say 1000mb/s, he is not likely getting on a consistant basis, which is why the cards also may be autonegotiating.

 

[rob-h]

Paul, I by no means am saying he has an issue, I am giving him things to try. And as I said, anything less then cat6 cable is going to cause slow down regardless. You are introducing errors at the physical layer and froceing retries. Thats needless traffic and will force instantaneous autoegotiation problems. You will never see it on the PC itself. FYI, I have been doing this along time myself

 

[Mark Giles]

Is there a program or anything I can install that will measure the speed of a "test" file being transferred to determine the speed of different configurations I try? I havnt had a chance to purchase a cat6 xover yet or get to my puters to change the setting rob mentioned but this weekend, I will and post the results.

 

[Kimmo Jaskari]

At home I've seen speeds approach about 25 megabytes per second on my gigabit ethernet setup. Two machines, hooked into the same 3com gigabit switch. The file server is running fairly slow (ie affordable ) storage drives which probably drags me down a bit.

Now, one gigabit means around 100 megabytes/second maximum speed, but in practice you never get that. Transferring small files it can be well below 10 megabytes/second due to drive seek issues etc.

One fragmented drive, either on the sending or receiving machine, will bring even a big file transfer to a crawl.

At work, with a bit more serious machinery (Unix servers) I've seen speeds reach the 40s and 50s (megabytes/sec) but nothing higher than that, even with jumbo frames on.

 

[Ken Chan]

Both Windows XP and Mac OS X have network monitors (in the Task Manager and Activity Monitor respectively) that show throughput.

 

[Rommel_L]

Chuck that 3 year old cable. It's only good for stable 10Mbps connections. I would suggest to acquire a CAT6 rated crossover cable. CAT6 is designed for 1000E connections. Might cause you a pretty penny though...

 

[rob-h]

Kimmo Jaskari, you have it backwards. Large files transfer much slower then small files. Thats why speed transfers tests are done with files of at 10 megs.

 

[Kimmo Jaskari]

Actually, I'm quite certain I'm correct, at the very least in practice if not theory.

If you have one unfragmented drive in computer A, a 1 GB file on said drive, a gigabit network link to computer B which has one unfragmented drive in it, transferring that 1GB file will stream nicely from one drive to the other. It is quite likely that the drives themselves will be the bottleneck.

If, instead, you have many small files on computer A, you'll be waiting for that harddrive to reposition its seek heads continuously when going from file to file. This will bring transfer speed overall to a crawl, comparatively speaking.

Rommel_L's advice is also good. New cables are needed for gigabit. At the very least, CAT5e, but CAT6 is good.

 

[Christian Behrens]

That is definitely NOT my experience. When transferring files between remote computers at work (say the equivalent to a CD image of a product), ftp'ing the folder with all the files was A LOT slower (painfully slow in fact) than the same folder tarred into one large file.

This was from the West Coast to Boulder, CO or the East Coast on the company intranet.

-Christian

 

[rob-h]

T

 

[Mark Giles]

Ok, I'll write more about my findings this past weekend when I return from lunch. But I bought a cat6 xover cable and pulled up the network moniter (cntr-alt-delete) to see the speed. I wish I took screen shots but the transfer speed kept going up and down. It looked like a lie detector. It kept going from 1% of(1GB) to 17-25%. Just up and down repeatedly. My test file was 7.5 GB's in size. Pretty big.

Also, I tested it on the 10/100 connection thru the net. And the graph stayed steady at 70% of (100MB). Still took 7 minutes longer than the 1GB line.

 

[Mark Giles]

It took the 7.5GB file about 10 to 11 minutes to transfer over the new cat6 xover cable. Which is pretty ding close to the rate of the 700mb file I tested with the cat5 xover. I even defragged both hard drives before testing! I chose this large of a file because unfortunately this is the type of transferring I'm going to be doing (videos).

However..going back to the rate fluctuating up and down, theres another small problem I ran into. The settings for one of the gig cards would not allow me to change it to 1000 duplex. Just auto, 100 duplex and settings under. This is the card that is built into the MSI board. The gig card I purchased on the other computer did allow me to set it to 1000 duplex. I updated the drivers on the problem gig card and verified the change. It added new settings in other areas but no 1000 duplex option.

So, I just went and bought another gig card (the exact same one that's in the other pc I'm having no problems with). I'm going to install that one and see if the fluctuating problem goes away and if I can get a higher transfer rate by setting it to 1000 duplex.

 

[Mark Giles]

One last thing. One thing I noticed was that when I went to transfer the files and while observing the Networking monitor in the Task Manager, it always defaulted to the card that is connected thru my router and the net. Not my xover cable. I had to disable the internet connection to even have it use the connection I wanted.

Is there to use a specific connection method by the card installed? I have my shared folders from the opposite computers in 'my network places' if that helps any ;-)

 

[ShaneD]

Ok, there's a lot of ground for me to cover here so i'll try to work my way down this thread, add where i can. As expected, all warnings about altering your system settings apply

Mark, given the speeds that you're reporting, it's sounding likely that you're running into HDD limitations (simply, it's transmitting the data faster than it can pull it from the HDD). Of course, i don't know your physical config (how are your drives attached, what north/southbridge, etc...) so i can't rule on that definitively. However, there are still a couple of things that you can check.

First off, what OS are you running? If it's WindowsXP, then you're automatically losing 20% of your total theoretical bandwidth due to the Windows Registry QoS reservation. You can remove QoS from your network settings but the registry still maintains the 20% reserve. TO change it, here's how.

Note: This following tip will not work on XP Home Edition.
Make sure you're logged on as actually "Administrator". Do not log on with any account that just has administrator privileges.
-Start > Run > type gpedit.msc (not available in home version).
-Expand the Local Computer Policy branch.
-Expand the Administrative Templates branch.
-Expand the Network branch.
-Highlight the "QoS Packet Scheduler" in left window.
-In right window double click the "limit reservable bandwidth" setting.
-On setting tab check the ENABLED item.
-Where it says "Bandwidth limit %" change it to read 0 (ZERO).
-Close gpedit.msc.
Effect is immediate on some systems, some need to re-boot.

Second, what type of network connections are you using? For the Abit system, which NIC are you using? i'm presuming that it's PCI-based so my next question is how many other devices in that PC are using the PCI bus? I ask because a PCI-GigE card can saturate the entire PCI bus all on its own. and any other devices that are on the bus are going to just eat up any available bandwidth.
For the MSI system, which NIC interface are you using? the Marvell PCIe or the Nvidia interface? If it's the Nvidia one, do you have the hardware acceleration (and appropriate software) enabled? if you do, there's your problem. (It's ~40% off the top if it's on. Nvidia really buggered it up)

Finally, if your systems are multi-honed (which they sound like they are) then if you want it to prefer one NIC orver the other, you're going to need to change or specify the interface metrics on the NIC's (assuming WinXP: Control Panel -> Network Connections -> Right-click adapter and choose properties -> double click 'Internet Protocol (TCP/IP) -> Advanced -> uncheck 'automatic metric' (right-click it to see what it does if you're curious) and specify your adapters in order of priority. This is going to be a consideration because XP doesn't handle multihoning too well and it gets a little confused when you want it to use a specific interface for a particular function.
I'd also like to point out that just wiring everything into a GigE-enabled home gateway (like the D-Link DGL-4100) is also an option to this process, but also involved spending a little more money.

Lastly, some other things you can check are things like:
- driver updates on both PC's
- BIOS updates for the MSI board
- There's also some other things to ensure that you're getting as direct a point-to-point connection (protocol-wise) as possible by doing things like specifying entries in your HOSTS file and whatnot, but i'd say start with this first.

As far as some of the suggestions made here, It's certainly possible to use Cat5e cabling for GigE traffic. In fact, i'm using it on my home network right now and seeing full transfer rates, so as long as we're talking cabling that's not older than dirt, it should be fine (and yes, i've tested with Cat6 as well. over short [less than 50m] i've measured no discernable diference in speeds.)

Also, it IS faster to transfer 1 big file than many small files (why? your HDD isn't performing multiple Seek/read/copy operations. it can just find its file and send)

You can really get into the nitty-gritty (like specifying NIC transmission settings and whatnot, but i'd say this is a good start for now. Remember that this is only going to be as fast as your slowest link, so if you've got an ancient drive or an oversaturated system bus, you're not going to see much improvement from this stuff.

Hope it helps.