Re: Company network slowdown

In the Usenet newsgroup alt.internet.wireless, in article
<gr6fi1tofbn8oii6uggb7sjd3hnb7eulmj@xxxxxxx>, Jeff Liebermann wrote:

>Well, if the 802.3 Ethernet packets were well formed and contained MAC
>addresses, tracing the problem back to the source would have been
>trivial. Instead, what I was seeing was bursts of garbage that I
>couldn't decode. I tired Ethereal, a Network General Sniffer, NT
>Netmon, and a bunch of demo sniffers I downloaded just to see if they
>could make sense of the traffic.

Oh, _that_ kind of garbage. Yeah, had that with old 10Base5 tranceivers
with later model Sun SS5 and SS20. Drove us absolutely bananas till
we caught one in the lab. We were using a NetGen sniffer, and I
forget what it was that we were finally able to spot - vaguely it was
a fraction of the Ethernet header, but that was years ago.

>I could see the garbage very lightly flashing the lights on the hubs,
>but could not decode anything. I spend two days with a logic analyzer
>trying to capture useful data and decode the contents manually, but
>even that didn't produce anything useful.

We had a Tektronix 535 scope on a platform, with another guy with the
probe in the overhead ceiling. Total waste of time. We did see there
was an occasional fractional packet (wasn't long enough to be a
collision), and actually had people log into each box on the subnet
and look at the ifconfig -a stuff. No joy.

>Just to make it interesting, I made a rather stupid series of
>mistakes. This was in the days when hubs were in fashion and switches
>were expensive and scarce (approx 1997)

1997 - we had just completed installing Kalpana Etherswitches to break
our 750 foot lengths of 10Base5 into smaller segments, and to get the
routers and busy servers onto their own ports. I didn't ask how
expensive the Etherswitches were, but they made a significant
improvement - and they had (some) smarts!!!

>Nobody every deduced that the network running slow was caused by running
>out of paper because there was always someone around to replace the paper
>that was not directly involved in using the computahs. Running out of
>paper was a very uncommon experience, so the time of slow downs were not
>easy to predict.

Yeah, our users are "trained" to reload paper bins. They'd manage to
screw something else up, but paper usually got loaded as soon as
someone came to pick up their printouts, and found nada. Some of
the "smarter" ones learned how to cancel and re-run print jobs on
alternative printers. Why they wouldn't reload the paper? Who knows.

>I had wrongly decided that the various 16 port Linksys 10baseT hubs
>were the likely culprits and convinced management to go for an HP
>Procurve 4000 switch, mostly on the basis of speeding things up to
>100baseTX-FDX.

We had two buildings with twisted pair - I swear it was Cat 1/2 - and
one section of the main building with Cat 5. Everything else was coax.

>The nice thing about switches is that garbaged and trashed packets do
>not go through a store and forward switch. Everything that was plugged
>into the Procurve switch worked without a slowdown. Everything that was
>still on the hub slowed to a crawl whenever the HP LJ4 ran out of paper.

Similar with the old tranceivers, except we had them on three of the
16 ports. That narrows it down, but doesn't get the exact answer.

>Of course, I didn't bother labeling the cables so I didn't have an
>immediate clue as to where the junk was coming from.

Boy does that sound familiar ;-)

>If there had been anything decoded by a sniffer, I would have found
>the source almost immediately. Instead, it was a painful 6 month
>ordeal, with lots of bad guesswork, and a substantial amount of luck
>in finding the problem. What I consider the most important lesson
>from the aformentioned exercise was that I could not have figured it
>out without the statistics and diagnostics from the managed switch.

Coax is just as bad if not worse - the blinky lights are on the
tranceiver up in the ceiling (and under floor in the server rooms).
Until we broke things up with the Etherswitches, our coax runs were
up to 750 feet long, and had up to 400 systems on that one wire.
Slightly out of spec, but it worked.

Old guy
.

Relevant Pages

  • Re: Catalyst 4000 - Ciscos Response
    ... on a variety of factors such as Switch load and traffic patterns. ... Flooding packets ... database on the switch containing switch ports and the MAC addresses sourced ... Sniffer is on a different port than the workstation and servers. ...
    (Bugtraq)
  • Re: Experts Help Please Settle Arguement - Hub or Switch if ISP offers several IPs
    ... Hubs and switches will both work perfectly in this scenario. ... Switches send the minimal amount of traffic to each port. ... you don't use a switch in this kind of arrangement. ... >because it doesn't know where to route packets. ...
    (alt.internet.wireless)
  • Re: Configure switch for sniffing
    ... switch port so it recives the packets. ... sniffer, I really need to know who is collapsing my network. ... Netflow does, but if you had a netflow capable switch, you'd most likely ...
    (comp.dcom.sys.cisco)
  • Re: Duplicate Echo Replies with Channel Bonding
    ... In this mode both interfaces receive packets, ... >When both eth0 and eth1 are up and I ping from Host C to Host A I get ... >The destination network 192.168.120.0/24 exists on both Router A and ... Switch B does not have the MAC address in its MAC address table ...
    (RedHat)
  • Re: Another 4WD question -- 97 Mazda B4000
    ... When I turn the switch on the dash to 4WD HI or 4WD LO, ... corresponding to the switch position but the truck clearly IS NOT ... that activates the automatic hubs but the only thing I see going to ... you probably have a vacuum problem. ...
    (rec.autos.4x4)