Re: High EIGRP Pending routes


"Stephen" <stephen_hope@xxxxxxxxxxxx> wrote in message
news:q6gkm4d803r3jbnl3a3fvocic4v3gqam80@xxxxxxxxxx
On Sun, 11 Jan 2009 07:59:05 -0800 (PST), sonic31ss
<sonic31ss@xxxxxxxxxxx> wrote:

On Jan 11, 3:17 am, "Thrill5" <nos...@xxxxxxxxxxxxx> wrote:
"sonic31ss" <sonic3...@xxxxxxxxxxx> wrote in message

news:199828b8-905e-4a41-bdcb-30c6d515583c@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx



Most of the documentation I can find on "Pending routes" from show ip
eigrp interfaces comes directly from Cisco and is not that helpful.
Their definition is "Number of routes in the packets sitting in the
transmit queue waiting to be sent."
The reason for this question is that we have an EIGRP meltdown on our
core routers (6500s - Sup720) about every three months. We created
scripts to collect more data from the router on a 5 minute interval
and the early indication that EIGRP was in trouble was that there were
52 out of approximately 300 EIGRP interfaces that had as many as 8000
pending routes. The remainder of the interfaces had zero pending
routes. The IP routing table is approximately 4000 routes.
The CPU utilization was 34% at 5 sec with 11% consumed by EIGRP PDM.
Five minutes later the CPU was at 100% and Pending routes were as
high as 100,000 on some of the EIGRP interfaces. All interfaces with
EIGRP neighbors had tens of thousands of pending routes.
This core router (Core_01) has another core router attached (Core_02)
and it did not record a high number of Pending routes.
There was no indication in the syslog of any significant topology
change leading up to or during this event.
So my questions to the group are;
1. Does anyone have a better definition of Pending routes?
2. How often are the counters from show ip eigrp interface updated by
the IOS?
3. Does 8000 Pending routes seem to high considering that there are
only 4000 routes in the IP routing table?

Thanks in advance

What is your topology that you have 300 interfaces running EIGRP? That
seems very excessive!!! If you have two 6500's for redundancy, each with
same 300 VLANs on them, running EIGRP on all of the interfaces is a very
bad practice. You should only have 1 or 2 neighbor relationship between
the same two routers. In a configuration like that on a 6500, the
best-practice is enable "passive-interface default" under the EIGRP
process,
and then enable EIGRP on a one or two of the VLAN's using "no
passive-interface vlan 100", "no passive-interface vlan 101", etc. Each
time a route goes away, it sends a query to each of its neighbors, and
then
has to wait for a response from each one. If you have 300 neighbor
connections to the same router, it will send a 300 queries to it, and
wait
for 300 responses. Since all the queries are going to th same router,
it's
going to get back the same answer 300 times.

I'm sure your "melt-downs" are due to SIA's (stuck in actives) which is
directly related to the above issue.

Hi Thrill5,

No, the 300 interfaces are mostly GRE tunnels to field routers. And
the SIAs only begin after the CPU reaches 100%. And we all know that
EIGRP does not behave well under CPU load.

FWIW if you manage to have a partially stable EIGRP with 300
adjacencies, you have proved in "conditionally" stable under high
load......

GRE tunnels can be a pain with any routing protocol, since typically
the CPU has to maintain state for each tunnel as well as EIGRP, and
when key interfaces get hit with congestion.

You also need to see if there can be contention for GRE tunnel
bandwidth - since EIGRP will eat up to 50% of bandwidth on an
interface, if lots of GRE tunnels contend with each other, you may be
generating your own congestion storm.......

Thrill5 is going the right way - what you need to do is
1. reduce the number of adjacenies if possible,
2. cut down the info that flows across them,
3. cut down the number of peers that get probed for alternate routes
when the topology changes.

i would add:
4. control the EIGRP traffic - reduce EIGRP allowed bandwidth, and
maybe tweak the timers.

You might want to grab one of the cisco books like "large scale IP
network solutions" - this has a good chapter on EIGRP for big hub
routers.

Thanks,
Jeff
--
Regards

If all of the field offices need to go through this central site then an
easy way to fix this is to send only a summary-default to the remotes. If
all the remotes need to go through this site, then they don't need a full
routing table, only a default route. By doing this, you will solve the
pending routes problem because you will be sending only one route instead of
4000. It will also solve the "melt-downs" because the summary default route
will also prevent the router from sending a query when a route goes away.
(If the route that goes away is within the summary route on the interface,
then a query is not sent.) Basically what is happening is that you have
300 adjacencies, and every time a route changes, every router is queried.
The router must wait for a response from each router. This will drive up
the CPU, causing received queries to be lost. If that happens, the router
will then reset the neighbor and need to send the full routing table to that
router, again driving up the CPU, causing more replies to be lost. You then
have what is commonly referred to as a "cascading failure" and all hell
breaks loose.

To enable the summaries, on the interface to the remote router add the
commad:
ip summary-address eigrp <eigrp process number> 0.0.0.0 0.0.0.0 255

For a default summary you must set the admin-distance to 255 or weird things
will happen if your default route goes away for some reason. For summaries
other than a default you should set the admin-distance to 5.

Another thing you could do is change the remotes to a "stub". You can only
do this if the remote is not a transit point for other routing destinations.



.

Relevant Pages

  • Re: High EIGRP Pending routes
    ... eigrp interfaces comes directly from Cisco and is not that helpful. ... Their definition is "Number of routes in the packets sitting in the ... The reason for this question is that we have an EIGRP meltdown on our ... Five minutes later the  CPU was at 100% and Pending routes were as ...
    (comp.dcom.sys.cisco)
  • Re: High EIGRP Pending routes
    ... eigrp interfaces comes directly from Cisco and is not that helpful. ... Their definition is "Number of routes in the packets sitting in the ... The reason for this question is that we have an EIGRP meltdown on our ... Five minutes later the  CPU was at 100% and Pending routes were as ...
    (comp.dcom.sys.cisco)
  • Re: High EIGRP Pending routes
    ... eigrp interfaces comes directly from Cisco and is not that helpful. ... Their definition is "Number of routes in the packets sitting in the ... The reason for this question is that we have an EIGRP meltdown on our ... This core router has another core router attached ...
    (comp.dcom.sys.cisco)
  • Re: EIGRP and its limits
    ... the spokes end eigrp stub. ... neighbors will cause the neighborships and routes to flap, ... > We had to tune the broadcast queues on the T3 interfaces because of the ... > routes need to move to a different router. ...
    (comp.dcom.sys.cisco)
  • Re: Adjusting EIGRP metrics
    ... I would recommend you to keep all metrics numbers the same (as it was ... If you need to send router over one link, ... I am trying to adjust the EIGRP metric of routes so a certain path is ...
    (comp.dcom.sys.cisco)