Showing posts with label cisco. Show all posts
Showing posts with label cisco. Show all posts

Wednesday, April 6, 2011

Cisco 7600-ES+20G3C

Im not sure how to word this post.

The cisco  7600-ES+20G3C  modules running on c7600rsp72043-advipservicesk9-mz.122-33.SRD4.bin have been misbehaving on me. Here's a short list of issues I've had:

One started spewing out the following:
Feb 10 13:37:11.536: %C7600_ES-DFC9-5-BRIDGE_ASIC_INTR: The Bridge-ASIC-AR[0] interrupt asserted. Addr[0x0200]=0x00000004
Feb 10 13:37:11.544: %C7600_ES-DFC9-5-BRIDGE_ASIC_INTR: The Bridge-ASIC-AR[0] interrupt asserted. Addr[0x0200]=0x00000004
Feb 10 13:37:13.524: %C7600_ES-DFC9-5-BRIDGE_ASIC_INTR: The Bridge-ASIC-AR[0] interrupt asserted. Addr[0x0200]=0x00000004
Feb 10 13:37:13.532: %C7600_ES-DFC9-5-BRIDGE_ASIC_INTR: The Bridge-ASIC-AR[0] interrupt asserted. Addr[0x0200]=0x00000004
Feb 10 13:37:13.544: %C7600_ES-DFC9-5-BRIDGE_ASIC_INTR: The Bridge-ASIC-AR[0] interrupt asserted. Addr[0x0200]=0x00000004
Feb 10 13:37:13.552: %C7600_ES-DFC9-5-BRIDGE_ASIC_INTR: The Bridge-ASIC-AR[0] interrupt asserted. Addr[0x0200]=0x00000004
Im talking a gagomoth of informational lines, the alarms were not service affecting, Our syslog server was obviously not amused. Note this only happened on one 7609-s (out of more than 20 ). We ended up swapping the module that cooled things off, figuring out along the way that it was due to a bug -CSCtc16746 (that oly affected that node:-)) - weird.

- one 20 port module somehow lost functionality on half the ports. I assume there's a chip that controls that half that just conked out.
- Another one just 'died' dead dead dead..no light, nothing...it had been working fine, unfortunately that was just before we had installed an external syslog server. So clueless on what happened.

(all modules were replaced by cisco in time and we keep spares).

Im just trying to figure our if Im the only one going through some wacky do's with the ES+20 modules.

Other than that some of the QOS features we have implemented would probably never be possible on other modules....so I still love them):...Moral of the story if any: Pay for support specifically support that replaces modules for you within the shortest time possible.

*PS if you have SAMI blades running STP or GGSN or CSG, let me know how thats working out for you too...

Tuesday, March 29, 2011

Cisco's “unsupported” SFP and making them work - sometimes

We had an interesting incident yesterday. Our network has lots of Huawei and Cisco devices. We use SFP's and XFP's widely on all the platforms. In an ideal world Huawei sfp's should work on Cisco whenever you plug it (there's some HP  switches from which I have borrowed SFP's to cisco and Huawei without any problems) in and this was never an issue on the Huawei end. So it's not a consistent problem for me/us.

Trying it on an ASR 1006 turned out to not be as straight forward. We had to use a 'hidden command'.

*Mar 28 11:07:49.547: %TRANSCEIVER-6-REMOVED: SIP0/3: Transceiver module removed from POS0/3/0

*Mar 28 11:07:59.542: %TRANSCEIVER-6-INSERTED: SIP0/3: transceiver module inserted in POS0/3/0

*Mar 28 11:08:01.160: %ASR1000_RP_ALARM-6-INFO: CLEAR CRITICAL xcvr container 0/3/0 Transceiver Missing - Link Down

*Mar 28 11:08:01.161: %ASR1000_RP_ALARM-6-INFO: ASSERT CRITICAL POS0/3/0 Physical Port Link Down

*Mar 28 11:08:01.124: %TRANSCEIVER-3-NOT_SUPPORTED: SIP0/3: Detected for transceiver module in POS0/3/0, module disabled


Next the command is run:
FW-LB-Rtr(config)#service un?

% Unrecognized command
The above tells you that its hidden and won't come up under context help by typing a question mark.
FW-LB-Rtr(config)#service unsupported-transceiver
You get a warning as shown below so be careful what you do or don't do:
Warning: When Cisco determines that a fault or defect can be traced to

the use of third-party transceivers installed by a customer or reseller,

then, at Cisco's discretion, Cisco may withhold support under warranty or

a Cisco support program. In the course of providing support for a Cisco

networking product Cisco may require that the end user install Cisco

transceivers if Cisco determines that removing third-party parts will

assist Cisco in diagnosing the cause of a support issue.

 

FW-LB-Rtr(config)#


and our interface came up
*Mar 28 12:28:36.354: %LINK-3-UPDOWN: SIP0/3: Interface POS0/3/0, changed state to up

*Mar 28 12:28:37.372: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS0/3/0, changed state to up

Please note the command is supported on some switches.

Thursday, November 25, 2010

CRS1 snmp and other counters

Sooo Im almost done with a very interesting project and wanted to share a little something on what happened when we wanted to monitor the optical interfaces.

First off the core network has a bunch of CRS1's interconnected using 3 x 10G DWDM links to the rest of the network giving us what someone fondly called a 10Gig core, or 30 depending on how/what you calculate.

Now the dwdm link characteristics are very important to ensure the deck of cards above it doesnt come crashing down. Some fiber was not very clean and you'd get parameters like:

RP/0/RP0/CPU0:xxxxxxx1#sh controllers dwdm 0/0/0/2
Thu Nov 25 14:10:16.303 EAT

Port dwdm0/0/0/2

Controller State: up

Loopback: None

G709 Status


OTU
LOS = 0 LOF = 0 LOM = 0
BDI = 0 IAE = 0 BIP = 0
BEI = 0 TIM = 0

ODU
AIS = 0 BDI = 0 OCI = 0
LCK = 0 BIP = 0 BEI = 0
PTIM = 0 TIM = 0

FEC Mode: Enhanced FEC(default)
EC(current second) = 3217480 EC = 602132861570 UC = 43432861570
pre-FEC BER = 2.89E-4 Q = 3.42 Q Margin = 1.74

Remote FEC Mode: Enhanced FEC
FECMISMATCH = 0

Detected Alarms: None
Asserted Alarms: None
Alarm Reporting Enabled for: LOS LOF LOM IAE OTU-BDI OTU-TIM OTU_SF_BER OTU_SD_BER ODU-AIS ODU-BDI OCI LCK PTIM ODU-TIM FECMISMATCH
BER Thresholds: OTU-SF = E-2 OTU-SD = E-5

OTU TTI Sent String ASCII: Tx TTI Not Configured
OTU TTI Received String ASCII: Rx TTI Not Recieved
OTU TTI Expected String ASCII: Exp TTI Not Configured

ODU TTI Sent String ASCII: Tx TTI Not Configured
ODU TTI Received String ASCII: Rx TTI Not Recieved
ODU TTI Expected String ASCII: Exp TTI Not Configured

Optics Status

Optics Type: Cisco 10Gb DWDM
Wavelength Info: C-Band, XXX ITU Channel=27, Frequency=1xx.80THz, Wavelength=1534.976nm
TX Power = 4.45 dBm
RX Power = -10.90 dBm
TDC Info

TDC Not Supported on the Plim



So any way we needed a way to graph and send alerts based on output like that.

IOS/XR's MIB's do not seem to have anything I could work with.

*if you know the MIB/OID to get me TX/RX power,UC,EC and prefec values I'll be very grateful.

*If you also happen to have an explanation on how to interpret the pre fec values eg:

pre-FEC BER = 2.89E-4


I'd also be very happy.

Enter some scripts:
to get this info we used a combination of expect and perl.

Basically we wrote a script that connects to each node and runs runs the command and dumps it on a text file. our perl-fu then comes on along and picks out the bits and pieces we need and dumps it on a database.

Zabbix (our mrtg'like monitoring system) graphs those. A few other scripts send us sms and email alerts.

Sample expect script:

set timeout 3
spawn /bin/bash
match_max 100000
send -- "telnet 192.168.1.1\r"
expect -exact "telnet 192.168.1.1\r
Trying 192.168.1.1...\r
Connected to 192.168.1.1.\r
Escape character is '^\]'.\r
\r
\r
User Access Verification\r
\r
Username: "
send -- "yadada\r"
expect -exact "durangor\r
Password: "
send -- "django\r"
expect -exact "\r
\r
RP/0/RP0/CPU0:xxxxxxxx1#"
send -- "show controllers dwdm 0/0/0/2"
expect -exact "show controllers dwdm 0/0/0/2"
send -- "\r"
send -- " exit\r"
send -- " "
expect eof


its a simple one that one.

other helpful commands for graphing other things:
sh snmp mib object-name
sh snmp interface
sh snmp interface tenGigE 0/0/0/0 ifindex


I am having alot of fun with IOS xr, the crs1 has some interesting features/utilities too....more on this later...

Monday, March 16, 2009

IPv6 From the Ground Up : Part - II

ICMPv6
ICMP for IPv6 is identified by a header value of 58 in the IPv6 next header field. ICMPv6 is used to report errors and perform internet layer functions eg ping for diagnostics. It's the base protocol for IPv6 and has to be fully implemented and understood by aspiring engineers.

Diagram used for this article is:



IPv6 Neighbor discovery and unicast routing.

Unicast routing is off by default, remember to enable it to allow ICMpv6 neighbor discovery that replaces ARP.

Router0(config)#ipv6 unicast-routing
Router0(config)#int f0/0
Router0(config-if)#ipv6 enable
Router0(config-if)#no shutdown

Router0#sh int f0/0
FastEthernet0/0 is up, line protocol is up
Hardware is AmdFE, address is cc00.1368.0000 (bia cc00.1368.0000)

Router0#sh ipv6 interface fastEthernet 0/0
FastEthernet0/0 is up, line protocol is up
IPv6 is enabled, link-local address is FE80::CE00:13FF:FE68:0
No global unicast address is configured
Joined group address(es):
FF02::1
FF02::1:FF68:0
MTU is 1500 bytes
ICMP error messages limited to one every 100 milliseconds
ICMP redirects are enabled
ND DAD is enabled, number of DAD attempts: 1
ND reachable time is 30000 milliseconds

Router0#ping FE80::CE01:13FF:FE68:0
Output Interface: FastEthernet0/0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to FE80::CE01:13FF:FE68:0, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 8/40/164 ms

note the expanded 0's (zeroes below), they mean the same thing.

Router0#ping FE80:0000:0000:0000:CE01:13FF:FE68:0
Output Interface: FastEthernet0/0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to FE80::CE01:13FF:FE68:0, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/32/96 ms

note similar commands have to be run on router 1.
Now lets take some debug to observe this process of enabling th elink local address, but first cover a few basics:
  • IPv6 host adresses are generated from interface mac addresses. from the previouse post (partI), mac addresses are 48 bits and need conversion to 64bit to make a EUI-64 address.
  • ICMPv6 Neighbor discovery is used to resolve layer 3 address to Layer 2 address. in case of ethernet, that would be a mac address to an IP address, or frame relay dlci to an address, or pvc to an ip address etc etc...
  • This is not necessary for point to point links. the router knows that any traffic resolving/recursing to the interface based on the routing table will use whatever layer 2 circuit is assigned to the circuit.
  • no inverse neighbor discovery yet. so all routes should be mapped incase of frame relay (frame relay map ipv6).
ICMPv6 replaces IPV4 ARP for neighbor discovery.
  1. Solicitations - asking other neighbors for info.
  • Neighbor Solicitations - By any general hosts eg desktops and other hosts.
  • Router Solicitations - Devices routing IPV6 eg a default gateway. eg router to router segments.
  • Used to decide what the remote L2 address is of hosts and routers. The two types are there because there is additional info apart from the L2 address. eg routers can tell hosts the network prefix - this way a host just needs to enable IPV6, start sending neighbor solicitations to find out the neighbor, and router solicitation to find out the routers. The router sends back the network bit and the host - stateless autoconfiguration is built into ipv6 protocol stack.
  1. Advertisements - sending informations.

  • Neighbor advertisements
  • Router Advertisemens.
debug ipv6 packets and ipv6nd
Router0(config)#
ICMPv6: Received ICMPv6 packet from ::, type 135
ICMPv6: Received ICMPv6 packet from FE80::CE00:13FF:FE68:0, type 136
ICMPv6-ND: Sending NS for FE80::CE01:16FF:FE0C:0 on FastEthernet0/0
!note the NS (neighbor solicitation) this is basically like asking' can I use this address?"
IPV6: source :: (local)
dest FF02::1:FF0C:0 (FastEthernet0/0)
!solicited node multicast address...used for duplicate address detection (DAD). ie essentially we ask 'is anyone using this address? in the segment.)
traffic class 224, flow 0x0, len 64+16, prot 58, hops 255, originating
IPv6: Sending on FastEthernet0/0
ICMPv6-ND: DAD: FE80::CE01:16FF:FE0C:0 is unique.

!Note chances of having a conflict are rare in this case since the address is derived from your mac address.and ICMPv6 acknowledges that the address is indeed unique.

ICMPv6-ND: Sending NA for FE80::CE01:16FF:FE0C:0 on FastEthernet0/0
!next we are advertising that we're an IPV6 neighbor with the address above.

IPV6: source FE80::CE01:16FF:FE0C:0 (local)
dest FF02::1 (FastEthernet0/0)
traffic class 224, flow 0x0, len 72+8, prot 58, hops 255, originating
IPv6: Sending on FastEthernet0/0
ICMPv6-ND: Address FE80::CE01:16FF:FE0C:0/0 is up on FastEthernet0/0

Router0(config)#
ICMPv6-ND: Sending RA to FF02::1 on FastEthernet0/0
ICMPv6-ND: MTU = 1500
IPV6: source FE80::CE00:16FF:FE0C:0 (local)
dest FF02::1 (FastEthernet0/0)
traffic class 224, flow 0x0, len 72+1428, prot 58, hops 255, originating
IPv6: Sending on FastEthernet0/0

Here - above- R0 sends then sends out an RA - router advertisement

ICMPv6: Received ICMPv6 packet from FE80::CE01:16FF:FE0C:0, type 134
ICMPv6-ND: Received RA from FE80::CE01:16FF:FE0C:0 on FastEthernet0/0
and receives an advertisement from R1. Please note no network addresses are set yet, so what you receive is the routers link local address.
Router0#show ipv6 neighbors
IPv6 Address Age Link-layer Addr State Interface
FE80::CE01:16FF:FE0C:0 0 cc01.160c.0000 REACH Fa0/
note the routers above only have link local processing

other commands that show output for different IP versions:
Router0#sh ipv6 int brief
FastEthernet0/0 [up/up]
FE80::CE00:16FF:FE0C:0

!shows the link local addresses on our interfaces.

Router0#sh ip int br
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 1.1.1.1 YES manual up up

Router0#sh arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 1.1.1.1 - cc00.160c.0000 ARPA FastEthernet0/0

Router0#sh ipv6 neighbors
IPv6 Address Age Link-layer Addr State Interface
FE80::CE01:16FF:FE0C:0 2 cc01.160c.0000 STALE Fa0/0
Router0#sh ipv6 route
IPv6 Routing Table - 2 entries
Codes: C - Connected, L - Local, S - Static, R - RIP, B - BGP
U - Per-user Static route
I1 - ISIS L1, I2 - ISIS L2, IA - ISIS interarea, IS - ISIS summary
O - OSPF intra, OI - OSPF inter, OE1 - OSPF ext 1, OE2 - OSPF ext 2
ON1 - OSPF NSSA ext 1, ON2 - OSPF NSSA ext 2
L FE80::/10 [0/0]
via ::, Null0
L FF00::/8 [0/0]
via ::, Null0
!note the Null0, this is because the traffic is local (remember this are not global addresses yet).
FE80::/10 is the entire range of link local addresses.

Router0#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
ia - IS-IS inter area, * - candidate default, U - per-user static route
o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

1.0.0.0/24 is subnetted, 1 subnets
C 1.1.1.0 is directly connected, FastEthernet0/0

Router0#sh ipv6 int
FastEthernet0/0 is up, line protocol is up
IPv6 is enabled, link-local address is FE80::CE00:16FF:FE0C:0
No global unicast address is configured
Joined group address(es):
FF02::1
!all host multicast, this is where the advertisements are sent to for autoconfiguration.
FF02::2
FF02::1:FF0C:0
MTU is 1500 bytes
ICMP error messages limited to one every 100 milliseconds
ICMP redirects are enabled
ND DAD is enabled, number of DAD attempts: 1
ND reachable time is 30000 milliseconds
ND advertised reachable time is 0 milliseconds
ND advertised retransmit interval is 0 milliseconds
ND router advertisements are sent every 200 seconds
ND router advertisements live for 1800 seconds
Hosts use stateless autoconfig for addresses.

Router0#sh ip int
FastEthernet0/0 is up, line protocol is up
Internet address is 1.1.1.1/24
Broadcast address is 255.255.255.255
Address determined by setup command
MTU is 1500 bytes
Helper address is not set
Directed broadcast forwarding is disabled
Outgoing access list is not set
Inbound access list is not set
all other commands, telnet etc also work but you need to be specific. Is there get a way to make ipv6 default IPversion

another question:
can you disable IPv4 processing and only have IPV6 processing on a router?


Interesting question came up during this writing:
can you disable IPv4 processing and only have IPV6 processing on a router?




Tuesday, March 10, 2009

IPv6 From the Ground Up : Part - I

The Basics

**Please go through rfc2373 in its entirety. Life will be much easier after that.

Our learning topology is a simple: two routers just to show neighbor discovery: This will mainly be used on PartII and any others that follow.


Understanding IPv6 without using it is might not be easy, however playing around with it while planning a CCIE should set you on the right path.

To get a proper grasp of IPv6, you need to understand:
- A link-local address, site-local and global IPv6 address.
- The loopback address (::1) for the loopback interface
- The multicast addresses of joined groups
- Number of bits on an IPv6 address (128 - bits, 16 bytes)

Also very important, what is a modified EUI-address, its purpose and how its generated. I found it also very important to know and understand IEEE 802 addresses.

Basics on MAC addressing:
The IEEE 802 address consist of 24 bit company identifier and a 24 bit extension ID. this is uniquely assigned and gives you a 48-bit address. This 48-bit address is also called the physical, hardware, or media access control (MAC) address.

EUI-64 Addresses
This addressing extends the '24-bit' extension ID on a MAC address to 40 bits. The company/manufacturer ID is still left at 24-bits. This 64 bits are then used to identify the host/node. This is what is called a link local address. Routers do not forward this addresses.

To convert a MAC address to an EUI address, I use the following method. Note this only gives us the link local address, in part 2 or 3 we'll discuss how the rest of the address is completed/generated....lets use an example:

Host X has a MAC address of 12-34-56-78-90-12
on a router, this would be the burnt in address (bia) or the mac address.

First we insert FFFE between the 3rd and 4th bytes ie between the vendor ID and extension ID which results to 12-34-5F-FF-E6-78-90-12/1234-5678-9012. You can easilly do this by slicing the address into two halves.

Next take the first byte (two characters=1 byte) so in our case the first byte is 12 (note this is in hex) and convert it to binary - 0001 0010.. Take the 7th most significant bit and flip it/or invert it, this gives 0001 0000. Convert this back to hex and you get :
10-34-5F-FF-E6-78-90-12
put this in proper notation for IPv6 and get:
1034:56FF:FE78:9012

In case you get hang up on wording

The IEEE now considers the label MAC-48 to be an obsolete term which was previously used to refer to a specific type of EUI-48 identifier used to address hardware interfaces within existing 802-based networking applications and should not be used in the future. Instead, the term EUI-48 should be used for this purpose.

references:
RFC2373 - IP Version 6 Addressing Architecture
My friend tells me most of what he learnt on IPv6 was solidified at an internetwork experts boot camp so go over to their site and grab some work book, have no idea which one in particular.

Part II
IPv6 Neighbor Discovery:

Friday, February 27, 2009

what was cooking today:

after the csg/ggsn exploits, the better part of my week is freed up. I like it when I do something and the impact is felt almost immediately by customers, the feedback is quite refreshing.

An assesor lab some time back showed me weak on IPv6, multicast and i need to polih up on L2 technologies.

Either way I think Im still on track for the CCIE. time to start redirecting my finances towards this goal. This is probably going to be the most painful one since money is hard to come by.

anyway,,...I was on the following links today for other reasons mainly work related, and since its slowly becoming clear that data center networking might be my next bread and butter:-)

**Yes I had nowhere to book mark the pages and needed a quick reference point...:-)

Configuring Enhanced Service-Aware Billing - on the ggsn ( i deal with mobility so ggsn's sgsn,s are sort of my main dish:-)
Technical Resources for the Enterprise - Cisco Design Zone
Data Center Assurance Program (DCAP) 3.0
Protocol Compliance Statements for the CSG 3.1(3)C7(1)
Cisco DCAP Data Mobility Manager (DMM) Design Guide (External)

Tuesday, February 24, 2009

CMX CSG upgrade:

Today had me doing an MOP for some upgrades. This is for a cisco CMX running on a 7613 chassis. It assumes you plug in a CSG module on slot 13 and it doesn't have the right software installed but you have it on disk0 on the supervisor.

Next I will show how to put it together with some ggsn's, configure and test billing for mobile users.


CMX1#dir
Directory of disk0:/
5 -rw- 4736628 Jun 25 2008 02:33:06 +03:00 c6csg-apc.31-3.C7.7.bin

The above shows the image we want is in disk0
Now make the file above accessible via tftp as follows:

CMX1(config)#tftp-server bootflash: c6csg-apc.31-3.C7.7.bin

Now the file above can be reached and picked via tftp by the csg. Another alternative is to use our tftp server.

CMX1#session slot 13 processor 0

The default escape character is Ctrl-^, then x.

You can also type 'exit' at the remote prompt to end the session

Trying 127.0.0.50 ... Open

wwwwwwwwwwwwwwwwwwwwwwww

www.C o n t e n t w

www.S e r v i c e s w

www.G a t e w a y w

wwwwwwwwwwwwwwwwwwwwwwww

CSG> dir

usage

upgrade slot0:|server-ip-addr filename

ping ip-addr

show ...

copy coredump tftp|rcp ip-addr filename [rcp-user]

capture [on|off]

pktlog ...

exit

CSG> upgrade slot0: c6csg-apc.31-3.C7.7.bin

Upgrading System Image 1

CSG ExImage Nov 8 2007

R/W| Reading:lam_ppc.bin..DONE Writing:lam_ppc.bin..DONE

Read 13 files in download image. (13,0,0)

Saving image state for image 1...done.

CSG> exit

Good Bye.

[Connection to 127.0.0.10 closed by foreign host]

You should get output similar to the above.

CMX1# hw-module mod 13 reset

When the module comes up sh mod should give the output almost exactly like below:

CMX1#sh mod

5 0001.c9dd.0f5e to 0001.c9dd.0f65 1.4 3.1(3)C7(7) Ok

That’s it your CSG is upgraded.

Thursday, January 22, 2009

Etherchannel Load Balancing and Forwarding Methods

a couple of WS-C3750E's - had a strange problem (mainly with my assumptions) on how etherchannel load balances. I thought it was automatic:-) it should feel what i need and do it!!!...

Imagine two ports bound to form one port channel giving 200Mbps. Now imagine on one end you have two hosts/servers that generate/carry a lot of traffic (an ftp server for instance) to multiple destinations on the other end - internet, auth servers etc etc...

One of the hosts has more traffic than the other, infact alot more then 98Mb. so when traffic hit 100Mbps, i started noticing random packet drops. why why why...I thought this is a 200mbps interface???

upon further checks we discovered that one of the interfaces within the bundle was dropping packets/frames.

so the checks started:

Gitau-Switch-01-Sw#sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-mac

src-mac was the default.
Now according to cisco :
EtherChannel load balancing can use either source-MAC or destination-MAC address forwarding.

With source-MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on the source-MAC address of the incoming packet. Therefore, to provide load balancing, packets from different hosts use different ports in the channel, but packets from the same host use the same port in the channel (and the MAC address learned by the switch does not change).

With destination-MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on the destination host's MAC address of the incoming packet. Therefore, packets to the same destination are forwarded over the same port, and packets to a different destination are sent on a different port in the channel.

So obviously the default load balancing was not working for me.

Since this was a 3750, I correctly figured that it can also use IP. playing around with the setup on the end that was dropping packets, the following sort of sorted me out:

Gitau-Switch-01-Sw#sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-ip

the command to make this change is:

port-channel load-balance src-ip

you can play around with:


Gitau-Switch-01-Sw#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-ip Src IP Addr
src-mac Src Mac Addr

you can run a test by:

test etherchannel load-balance interface port-channel [#] ip [src] [dst]

references:

http://www.edgenetworks.nl/etherchannel.html
http://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a0080094714.shtml


before:

Gitau-Switch-01-Sw#sh int g1/0/24 | include drop
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 20191

now :

Gitau-Switch-01-Sw#sh int g2/0/24 | include drop
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Gitau-Switch-01-Sw#sh int g1/0/24 | include drop
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0


I figure after some time I'll come across a few drops..because the network hates me!!!

Tuesday, January 20, 2009

Understanding OSPF contd...

After about 20 hours lab and roughly 10 hours going through OSPF, I still can't say I have the confidence to go through OSPF without going down belly fast in the CCIE lab. More practice is definately called for to get at least to 70% confidence levels.

So i decided to just make some notes, go through BGP and start on Multicast as the information digested. Plus Im curious how much of all this material I'll have to cover again.

Apparently OSPF is the most widely used IGP. It brings in the concept of Areas. If you imagine a midsized no actually a midsized company in Europe is probably a LARGE company in Kenya, so if you imagine a LARGE:-) company, an area would probably be a building or a department.

Router ID's, Neighbors, Adjacencies, LSA's *,Hello protocol,Areas, election (DR,BDR) are terms you'll come across and if the books you're reading don't address them:-), google will sort you out.

OSPF sort of goes around the whole network and maps out what you ask it to giving you paths to destinations, its a link state protocol.

OSPF order of operation:
1:A router sends Hello packets, discovers neighbors and elects a Designated Router. Link-state information and a list of neighbors is included in the packet.

*Mar 1 00:15:08.803: OSPF: Send hello to 224.0.0.5 area 1 on Serial1/0 from 1.1.1.1
*Mar 1 00:15:10.903: OSPF: Rcv hello from 2.2.2.1 area 1 from Serial1/0 1.1.1.2
*Mar 1 00:15:10.911: OSPF: End of hello processing
*Mar 1 00:15:11.071: %OSPF-5-ADJCHG: Process 1, Nbr 2.2.2.1 on Serial1/0 from LOADING to FULL, Loading Done
*Mar 1 00:15:18.807: OSPF: Send hello to 224.0.0.5 area 1 on Serial1/0 from 1.1.1.1

*Mar 1 00:15:20.907: OSPF: Rcv hello from 2.2.2.1 area 1 from Serial1/0 1.1.1.2
*Mar 1 00:15:20.911: OSPF: End of hello processing

This is an initial hello.

Debugging ip packet detail shows the messages a bit clearly:

*Mar 1 00:20:48.815: IP: s=1.1.1.1 (local), d=224.0.0.5 (Serial1/0), len 80, sending broad/multicast, proto=89
*Mar 1 00:20:50.899: IP: s=1.1.1.2 (Serial1/0), d=224.0.0.5, len 80, rcvd 0, proto=89


Protocol 89 is OSPF, 224.0.0.5 is the ALLSPFRouters multicast address, from th esource you can also tell whether we originated the hello or not. Since this are serial links, no DR/BDR election takes place.

Let me swap out the links with ethernet to see how that works out....on my next post. Seems the lab is inaccessible....this is a simple lab, I think I'll dynamip on the laptop for a while:...

Friday, January 9, 2009

new day, new solaris + squid :-challenges

Great another morning, my solaris 10, squid installed, but in comes another requirement. Run wccp2 on it. Im happy because I get to mess around with a router, sad because Ive done this countless times on linux and freeebsd but never on solaris. So quickly go to sunfreeware; not a thing when i search for wccp. hmm back to basics, maybe we can work around this by first understanding the requirements ( I always try to know the way a protocol works first before adapting it for whatever system I need it for), this way If i ever need to configure lets say OSPF on juniper, so long as I know how SPF works, i can run it on anything:

I also decided to take some debug output for future reference:

Commands you'll use on the router:

show ip wccp web-cache detail
show ip wccp web-cache view
debug ip wccp events - display WCCP events
debug ip wccp packets - display WCCP packet information

Here's how wccp works with squid:

Once squid is started with the wccp option, the router sees this and sends an I see you message as follows (I had always assumed the here I am message comes first - I was obviously wrong:

Jan 9 08:57:51.584 NAIROBI: WCCP-PKT:S00: Sending I_See_You packet to 196.201.xx.xx w/ rcv_id 00006C9C

and you get this log below on the router - this is not a debug output.

Jan 9 08:58:01.593 NAIROBI: %WCCP-5-SERVICEFOUND: Service web-cache acquired on WCCP client 196.201.xx.xx

Squid then sends back HERE_I_AM packets, and carries on a conversation with the router.

Jan 9 08:58:01.593 NAIROBI: WCCP-PKT:S00: Received valid Here_I_Am packet from 196.201.xx.xx w/rcv_id 00006C9C

If you were also debugging events, you get the following:

Jan 9 08:58:01.593 NAIROBI: WCCP-EVNT:S00: Assignment wait timer started
Jan 9 08:58:01.593 NAIROBI: WCCP-EVNT:S00: Built new router view: 1 routers, 1 usable WCCP clients, change # 0000004D

From then on traffic gets redirected to squid.

and if you disconnect you get the following event/log:

Jan 9 08:57:37.279 NAIROBI: WCCP-EVNT:S00: Assignment wait timer started
Jan 9 08:57:37.279 NAIROBI: WCCP-EVNT:S00: Built new router view: 0 routers, 1 usable WCCP clients, change # 0000004B
Jan 9 08:57:37.279 NAIROBI: WCCP-EVNT:S00: Router 196.201.xxx.xxx removed.
Jan 9 08:57:37.279 NAIROBI: WCCP-EVNT:S00: Assignment wait timer started
Jan 9 08:57:37.279 NAIROBI: WCCP-EVNT:S00: Built new router view: 0 routers, 0 usable WCCP clients, change # 0000004C
Jan 9 08:57:37.279 NAIROBI: %WCCP-1-SERVICELOST: Service web-cache lost on WCCP client 196.201.xx.xx

On the router, the basic config is widely available on the internet....create an access-list for traffic to redirect, enable wccp etc etc...

This conversation (between router and squid) is totally separate from the ip_wccp or ip_gre module --these packets never go through those channels. For so long as the router is receiving WCCP HERE_I_AM packets (with the proper ID), the router will send traffic to the cache IP, encapsulated in a GRE packet, and there in lies the problem or the gotcha as ccie candidates like saying. In a production if you do this without confirming the tunnels, all your redirected traffic will be blackholed by squid.

The GRE decapsulation is a separate process from squid. Squid doesn't talk to the gre/ip_wccp module, The gre decapsulation occurs at the network layer in your kernel, and then the packets enter the normal routing table - on bsd/linux it involves iptables/chains
to hijack any packets destined for port 80 on the internet - In my case thats traffic coming from the router in GRE packets and passes it over to the cache engine's port.

I wonder what happens if I let squid listen on port 80....do you still need something to 'jack' the packets? I'll try that in a while and let you know....

So now with that understanding, you'd think I fix this on solaris pretty fast yes??? nope, nada, my cluelessness gets in the way....AGAIN!!


So again back to google, I can't get an ip_wccp for solaris module but Im sure I will get gre and the ip redirect working...and get more CLUEFUL overall then we can go back to regular work, If you stumble upon this and have a clue...bail me out:-)pleasee....I still have about 5 bgp, 5 ospf LABS to get through before Sunday....

Tuesday, January 6, 2009

Understanding OSPF....

After going through ospf, and to properly understand some concepts, I hit the intanet looking for pre-done dynamips labs:-) to save on time. during my cyber walk, I came across the article
Configuring Basic OSPF (Dynamips). It was excellent and had a very well detailed instruction on things.

However one of the requirements for the lab was:

That the ISP router always be the DR, one of the Remote sites always be a BDR and one remote site never participates in the election process. (Please go to evilrouters.net) for the diagram and full article - but it looks something like below all routers connected through switches.

Remote1===\
======ISP
Remote2===/

I redid the whole lab with emphasis on meeting the designated router (DR)/backup designated router (BDR) requirements
Configurations:

ISP#sh ip int br | exclude un
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 10.10.10.1 YES manual up up
Loopback0 188.46.37.254 YES manual up up

------
Remote1#sh ip int br | ex una
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 10.10.10.3 YES manual up up
Loopback0 192.168.1.1 YES manual up up
Loopback1 192.168.1.161 YES manual up up
-------
Remote2#sh ip int br | ex una
Interface IP-Address OK? Method Status Protocol
FastEthernet0/0 10.10.10.2 YES manual up up
Loopback0 192.168.1.65 YES manual up up
Loopback1 192.168.1.129 YES manual up up

After bringing up the interfaces, I tested connectivity as evilrouter dude had done it - well almost:-):

ISP#tclsh
ISP(tcl)#for
ISP(tcl)#forea
ISP(tcl)#foreach address {
+>10.10.10.2
+>10.10.10.3
+>} {ping $address}

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.2, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 32/58/96 ms
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.10.10.3, timeout is 2 seconds:
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 16/45/88 ms
ISP(tcl)#

Now we get OSPF going, just as he had done it, I'll just show the neighbors here:

ISP#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
192.168.1.129 1 FULL/BDR 00:00:34 10.10.10.2 FastEthernet0/0
192.168.1.161 1 FULL/DROTHER 00:00:38 10.10.10.3 FastEthernet0/0
------
Remote1#sh ip ospf ne

Neighbor ID Pri State Dead Time Address Interface
188.46.37.254 1 FULL/DR 00:00:31 10.10.10.1 FastEthernet0/0
192.168.1.129 1 FULL/BDR 00:00:39 10.10.10.2 FastEthernet0/0
-----
Remote2#sh ip ospf ne

Neighbor ID Pri State Dead Time Address Interface
188.46.37.254 1 FULL/DR 00:00:39 10.10.10.1 FastEthernet0/0
192.168.1.161 1 FULL/DROTHER 00:00:31 10.10.10.3 FastEthernet0/0

-we have proper basic ospf

Since Im interested in the DR/BDR, I'll skip the details here since the original
post
at evilrouters.net still has them.

lets go over the requirements again in proper detail for clarity:
1: ISP - 10.10.10.1 will always be the DR
2: Remote1 - 10.10.10.3 will always be the BDR
3: remote2 - never participates in an election.
Remote2 is easy:

Remote2#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Remote2(config)#interface fastethernet 0/0
Remote2(config-if)#ip ospf priority 0
Remote2(config-if)#


ISP is also easy:

ISP(config)#interface fastethernet 0/0
ISP(config-if)#ip ospf priority ?
<0-255> Priority
ISP(config-if)#ip ospf priority 255

The neighbors currently look like this:

ISP#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
ISP#sh ip ospf ne

Neighbor ID Pri State Dead Time Address Interface
10.10.10.2 0 FULL/DROTHER 00:00:32 10.10.10.2 FastEthernet0/0
10.10.10.3 1 FULL/DR 00:00:35 10.10.10.3 FastEthernet0/0
-------
Remote1#sh ip ospf ne

Neighbor ID Pri State Dead Time Address Interface
10.10.10.1 255 FULL/BDR 00:00:36 10.10.10.1 FastEthernet0/0
10.10.10.2 0 FULL/DROTHER 00:00:38 10.10.10.2 FastEthernet0/0
-------
Remote2#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
10.10.10.1 255 FULL/BDR 00:00:33 10.10.10.1 FastEthernet0/0
10.10.10.3 1 FULL/DR 00:00:38 10.10.10.3 FastEthernet0/0


so according to the above output:

10.10.10.2 - is fine, it won't participate in the election process.

Note:
Immediately something is wrong with the BDR/DR election . Since I reset the Remote2 OSPF process first, it got to be DR first and the ISP router had to be the BDR. That I'm sure starts to make my point but, just to ensure we go through all the evil router guy's/guyette's??:-) steps: I'll go ahead and change the priority of the RemoteRouter to 254 and clear the ISP router's ospf process first - !so it gets to be DR, followed by the others and the output is as follows:

ISP#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
10.10.10.2 0 FULL/DROTHER 00:00:30 10.10.10.2 FastEthernet0/0
10.10.10.3 254 FULL/BDR 00:00:34 10.10.10.3 FastEthernet0/0
-------
Remote1#sh ip ospf ne

Neighbor ID Pri State Dead Time Address Interface
10.10.10.1 255 FULL/DR 00:00:32 10.10.10.1 FastEthernet0/0
10.10.10.2 0 FULL/DROTHER 00:00:34 10.10.10.2 FastEthernet0/0
-------
Remote2#sh ip ospf ne

Neighbor ID Pri State Dead Time Address Interface
10.10.10.1 255 FULL/DR 00:00:30 10.10.10.1 FastEthernet0/0
10.10.10.3 254 FULL/BDR 00:00:36 10.10.10.3 FastEthernet0/0

However, If I now reset the ISP router's ospf interface, or if the link to the ISP were to go off:

Remote1#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
10.10.10.2 0 FULL/DROTHER 00:00:38 10.10.10.2 FastEthernet0/0

Remote2#
*Mar 1 01:06:44.087: %OSPF-5-ADJCHG: Process 1, Nbr 10.10.10.1 on FastEthernet0/0 from FULL to DOWN, Neighbor Down: Dead timer expiredsh ip ospf ne

Neighbor ID Pri State Dead Time Address Interface
10.10.10.3 254 FULL/DR 00:00:39 10.10.10.3 FastEthernet0/0

So now we have a situation where the former BDR was upgraded to DR. The reason I think your requirement for the ISP to always be a BDR cant be met with
this configuration is so long as there's a BDR that gets upgraded to a DR and no pre-emption happens, then your ISP will always be either a DR or BDR even
under normal circumstances. The priority only kicks in if the processes are reset at the same time.

Here's the output after bringing up the ISP router.


ISP(config-if)#
*Mar 1 01:12:17.647: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
*Mar 1 01:12:18.647: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
*Mar 1 01:12:25.631: %OSPF-5-ADJCHG: Process 1, Nbr 10.10.10.2 on FastEthernet0/0 from LOADING to FULL, Loading Done
*Mar 1 01:12:25.719: %OSPF-5-ADJCHG: Process 1, Nbr 10.10.10.3 on FastEthernet0/0 from LOADING to FULL, Loading Done
ISP(config-if)#do sh ip ospf ne

Neighbor ID Pri State Dead Time Address Interface
10.10.10.2 0 FULL/DROTHER 00:00:37 10.10.10.2 FastEthernet0/0
10.10.10.3 254 FULL/DR 00:00:34 10.10.10.3 FastEthernet0/0

ISP#sh ip ospf interface f0/0
FastEthernet0/0 is up, line protocol is up
Internet Address 10.10.10.1/29, Area 0
Process ID 1, Router ID 10.10.10.1, Network Type BROADCAST, Cost: 1
Transmit Delay is 1 sec, State BDR, Priority 255
Designated Router (ID) 10.10.10.3, Interface address 10.10.10.3
Backup Designated router (ID) 10.10.10.1, Interface address 10.10.10.1
Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
oob-resync timeout 40
Hello due in 00:00:05
Supports Link-local Signaling (LLS)
Cisco NSF helper support enabled
IETF NSF helper support enabled
Index 2/2, flood queue length 0
Next 0x0(0)/0x0(0)
Last flood scan length is 0, maximum is 1
Last flood scan time is 0 msec, maximum is 4 msec
Neighbor Count is 2, Adjacent neighbor count is 2
Adjacent with neighbor 10.10.10.2
Adjacent with neighbor 10.10.10.3 (Designated Router) !<<<------ note the DR!
Suppress hello for 0 neighbor(s)

So the question is, how do you ensure that a router is always a BDR? Is that even possible? How to go about it?

Lastly, many thanks to whoever runs http://evilrouters.net for this article

Sunday, January 4, 2009

confused laziness.....and expect:-)

...today (sunday) caught me trying to catch up with OSPF but I just couldn't stand the books, sites - and a friend who'd come for an apparently 'intelligent discussion on nssa's and routers, or sgsn's etc etc', in the end we opened a couple of beers, fed the fish (the only other living thing in my house is 4 fish) and yapped about current affairs, state of african economies and how soon we're going to be travelling all over the place setting up networks sampling local brews:-) - Please note kenya and the general (east) african region has relied on satellite communication for so long, however we have submarine fiber coming in by 2010-2012 so yes I see a very bright future for networkers:-).....so no bootcamps, no authorized training centers for high end certs, no internetwork expert, no narbik...to get certified here you really have to lab up and put in the time...ahh the joys.

Anyway after several hours of that 'drinking', the neurons were perfectly fired up to engage in some creative boredom activity. I thought of one of the most annoying repetitive tasks my team has to undertake daily and decided to get rid of it.

Every once in a while we get customers calling our 1st line support guys just to check if their remote sites have established a pdp session successfully. On a cisco GGSN the command would be:

show gprs gtp pdp-context msisdn 2547221x1x2xx

To get the following:

TID MS Addr Source SGSN Addr MSISDN APN
36xx72xxxxxxxxx 196.xx.xx.01 LOCAL 196.20x.xxx.xx 2547221x1x2xx safaricom
In most cases the next request could be to clear the pdp context so they can re-establish it again

clear gprs gtp pdp-context tid
So in comes EXPECT:
this is an extraordinarilly great tool for writing scripts for the lazy sysadnmin to drive other programs. It recognizes prompts and sends keystrokes in response. It was written by Don Libes of NIST, and you can find papers on it in Usenix LISA (Large Systems Administration) conference proceedings, as well as on the Internet.

My drunken goal was to create a web interface where the users can check this for themselves.

The expect script itself was pretty straight forward:


#!/usr/local/bin/expect -f

set force_conservative 0 ;# set to 1 to force conservative mode even if
;# script wasn't run conservatively originally
if {$force_conservative} {
set send_slow {1 .1}
proc send {ignore arg} {
sleep .1
exp_send -s -- $arg
}
}


set msisdn [lindex $argv 0]
set timeout 3
spawn /usr/bin/bash
match_max 100000
send -- "telnet 196.x.x.x\r" # your GGSN/cisco routers IP address
expect -exact "telnet 196.x.x.x\r # your GGSN/cisco routers IP address
Trying 196.x.x.x...\r
Connected to 196.x.x.x.\r
Escape character is '^\]'.\r
\r
\r
User Access Verification\r
\r
Username: "
send -- "drunkenmaster\r"
expect -exact "drunkenmaster\r
Password: "
send -- "jedimaster\r"
expect -exact "\r
\r
GGSN-xx01>"
send -- "en\r"
expect -exact "en\r
Password: "
send -- "jedimaster\r"
expect -exact "\r
GGSN-xx01#"
send -- "show gprs gtp pdp-context msisdn $msisdn"
expect -exact "show gprs gtp pdp-context msisdn $msisdn"
send -- "\r"
send -- "exit\r"
send -- ""
expect eof


Next do a simple perl,php or whatever makes you happy script, put it on a web server somewhere and guys can access it with a url. The handiwork can be viewed here:
* i just modified an old script we use for users to change unix passwords still using expect and adapted it for my needs.


Please note the above url is only active as a demo - in any case it would only be applicable to our users but im sure it explains the concept

On putting in a valid msisdn the output should show up in full....as follows - or you can use the url above:
Please check from LINE 17 (seventeen) thats where your output will be

GGSN returned following information:
Array
(
[0] => spawn /bin/bash
[1] => telnet 196.x.x.x
[2] => www-data@monitor01:/var/www$ telnet 196.x.x.x
[3] => Trying 196.x.x.x...
[4] => Connected to 196.x.x.x.
[5] => Escape character is '\^]'.
[6] =>
[7] =>
[8] => User Access Verification
[9] => [10] => Username: drunkenmaster
[11] => Password:
[12] =>
[13] => GGSN-xx01>en
[14] => Password:
[15] => GGSN-xx01#show gprs gtp pdp-context msisdn
[16] => TID MS Addr Source SGSN Addr MSISDN APN
[17] => 36xx72xxxxxxxxx 196.xx.xx.01 LOCAL 196.20x.xxx.xx 2547221x1x2xx safaricom
[18] =>
[19] => GGSN-xx01#exit
[20] => Connection closed by foreign host. [
21] => www-data@gitaus-TestServer:/var/www$

please note this is a sample output and no script is actually run to get this output

Please contact the data team! To run another query, : Please click me to run another query! "


Ahh finally success, the php script just inserts the msisdn....if you're interested in it, leave a comment and I'll probably just upload it...

You can run this for virtually anything, show bgp neighbors, for users to change unix passwordsetc etc..just modify the commands as you see fit. You can for instance give your users access a router to check the status of one of their interfaces, or clear statistics or whatever makes you happy, its a bonus if you have the security guys screaming at you:-) no seriously ensure your corporate policy is followed when doing some of this things...

Also if you forget to modify the tty's on a router, users might lock you out of telnet access:-)

please note this could obviously work better, but i wasn't looking for better, i needed to kill some time and be productive at the same time. For instance we could post process the output to only show line 17 ( I also generally prefer if guys especially co-workers to at least have an idea of whats run and what the output would look like from a console)...but time's up, and I think I'll have an easier time tomorrow...maybe some day....

http://oreilly.com/catalog/expect/chapter/ch03.html
http://www.marcelgagne.com/node/582
ftp://ftp.cisco.com/pub/cisco-expect.shar

Saturday, January 3, 2009

Understanding EIGRP from the bottom up Part I I

EIGRP Metrics:
Routers 'discuss' their topology tables. So unlike other protocols, EIGRP has a table with all the routes to a destination.

show ip eigrp topology all-links
!--is the command that shows this table - all links is optional
This topology table is the one holding all the info to make decisions, the outcome is usually a distance and vector to each destination.

Minimum bandwidth and the total delay is the information used to compute the metric. The values are automatically picked from configured values on your interfaces. Generally bandwidth will be more critical for lower bandwidth interfaces while delay is more key where high speed interfaces are used.

Formula to get :
the bandwidth:
bandwidth = 10 000 000 / bandwidth * 256
The delay
delay = delay * 256

The values got for bandwidth and delay are then used in the metric computation. I won't go into detail, please check this cisco link or read a book.

the default behaviour is to just calculate as follows:
metric = bandwidth + delay

(remember to round off your figures after each calculation, I can't remember why but i think floating point math on cisco was the reason)


so if you have the following scenario:


bw 56k
/Delay 2000---Router4---| |
/ | |
Router1== |---------Router2------|
\ |bw10000 |
\__bw 128 |delay100 |Destination Network
Delay 1000-Router3---| |bw 10000
|delay 100


metric = bandwidth + delay
minimum bandwidth = 56k
Total Delay = 100 + 100 + 100 + 2000
= 2200

[(10 000 000/56) + 2200] x 256 = (178571 +2200) x 256
= 180771 x 256
= 46277376

10 000 000/56 is actually = 178571.42857142857142857142857143 but we round it off...
If you calculate the metric through router 3, you find the metric is 20307200

so router one will use the path via Router3 to get to the destination network.
  • The bandwidth is calculated from the configured interface through which the desired network is visible ie where the update is coming in through.
  • The delay is cumulative; ie each router adds a delay and sends it backwards...Im not sure how to explain this if you can't see it....just count the delay from the destination backwards

Feasible Distance (FD)
  • This is the best metric/best path to the destination network - includes the metric to the neighbor advertising the network - from our diagram thats Router2.
reported Distance (RD)
  • This is the total metric as advertised by an upstream router/neighbor. from the ascii diagram above, that will be the distance advertised by Router4 or Router3 (obviously one is going to be used as a FD
feasible Successor (FS)
  • This is the path whose reported distance is less than the feasible distance. This is usually installed in the topology table as a backup.
The reported distance is always calculated from the router advertising the route to the network. so the reported distance from router4 is the metric to get to the destination network from Router4 and the same goes from the reported network from Router3.

If the link between Router1 and Router3 goes down, the convergence is almost instant, users will probably not even notice it since the feasible successor/backup route is immediately picked.

Checking loops:
The FD,FS and RD concepts are used to breack loops using the logic that the reported distance CANNOT be higher than the feasible distance. A route/path with a higher Rd than the FD won't show up on the topology table.

Other loop 'taking care of mechanisms'

Split Horizon - A route WILL NEVER be advertised through the interface it was learnt from.

Poison reverse - after learning of a route through a certain interface, any advertisements back the same interface are sent us unreachable for the said route/network.

Notes:
  • If a FD sends an update, queries regarding the same network are not sent to it.
  • Stuck in active (SIA) occurs if a query takes too lon gto be answered by a neighbor....
Redistribution: Ahh this is when I started having fun....I think I'll deal/write about redistribution after going throroughly through OSPF

now lets see if my trusty old motor bike survived the holidays:-).......happy new year....

Want to read some more.....go here

Friday, January 2, 2009

Understanding EIGRP from the bottom up Part I

When tackling the BSCI, I went through the EIGRP material, just enough to pass the exam. So I concentrated more on the theory end of things. I rarely if ever needed the certifications for my job ie it was'nt a requirement for promotions or anything. (telcos in kenya have very poor skill spotters in my opinion), either way we do run EIGRP so this time I took the time to understand it properly.

Key fields in the EIGRP header are as follows:

* The opcode field specifies the EIGRP packet type (update, query, reply, hello).
* The checksum applies to the entire EIGRP packet, excluding the IP header.
* The rightmost bit in the flags field is the initialization bit and is used in establishing a new neighbor relationship
* The sequence and ack fields are used to send messages reliably
* The AS number identifies the EIGRP process issuing the packet. The EIGRP process receiving the packet will process the packet only if the receiving EIGRP process has the same AS number; otherwise, the packet will be discarded.

EIGRP is very widely documented so googling for EIGRP header will robably give more details.

Other highlights:

  1. Its Hybrid and uses the DUAL (Diffused Update ALgorithm).
  2. Neighbor discovery and maintenance ensures only updates are sent when needed (hellos). The hello interval is 5 and 60 seconds, the default hold time is three times (15 and 180 seconds) the hello timer duration. Hold time is the amount of time a router will consider a neighbor alive without receiving a hello packet. The timers can be adjusted per interface with the ip hello-interval eigrp and ip hold-time eigrp.
Lets take a look:
We'll use the same topology we used for RIP to explore the EIGRP timers.

Loopbacks
R0: : 172.20.1.1/32
R1: : 172.20.2.1/32

Network Addresses:
R0: Serial1/0 : 192.168.10.1/30
R1: Serial1/0 : 192.168.10.2/30

ffr Addresses:
R0: Serial1/1.105 : 192.168.20.1/30
R1: Serial1/1.501 : 192.168.20.2/30

Ethernet interfaces:
R0: Fastethernet0/0: 20.20.20.1/24
r1: FastEthernet0/0: 10.10.10.1/24

Test reachability - before continuing - If all is well, add in EIGRP:

On R0

R0#sh run | section eigrp
router eigrp 1
network 192.168.0.0 0.0.255.255
no auto-summary

on R1

router eigrp 1
network 192.168.0.0 0.0.255.255
no auto-summary

What pops up immediately:


*Mar 1 00:02:32.223: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.10.1 (Serial1/0) is up: new adjacency
*Mar 1 00:02:32.383: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.20.1 (Serial1/1.501) is up: new adjacency
*Mar 1 00:02:35.447: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.20.1 (Serial1/1.501) is resync: peer graceful-restart
*Mar 1 00:02:35.455: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.10.1 (Serial1/0) is resync: peer graceful-restartR0#sh ip eigrp neighbors

IP-EIGRP neighbors for process 1

H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
1 192.168.20.2 Se1/1.105 12 00:10:52 894 5000 0 15
0 192.168.10.2 Se1/0 12 00:10:52 127 762 0 16

The value of the hold column shouldn't be more than the timer unless you're losing packets. Also the Q(queue) count column should always be '0' unless there's a problem.

For instance, lets block EIGRP on R0 and see what shows up on R1:

RO
R0(config)#access-list 100 deny  eigrp any any
R0(config)#access-list 100 permit ip any any
int s1/0
ip access-group 100 in

This pops up on R0
*Mar  1 00:38:34.547: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.10.2 (Serial1/0) is down: holding time expired
and on R1 we now have the following:
R1#
*Mar 1 00:38:34.899: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.10.1 (Serial1/0) is down: Interface Goodbye received
*Mar 1 00:38:39.623: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.10.1 (Serial1/0) is up: new adjacency
*Mar 1 00:39:59.147: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.10.1 (Serial1/0) is down: retry limit exceeded
*Mar 1 00:40:03.619: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.10.1 (Serial1/0) is up: new adjacency
R1#sh ip eigrp neighbors
IP-EIGRP neighbors for process 1
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
0 192.168.10.1 Se1/0 11 00:00:36 1 5000 1 0
1 192.168.20.1 Se1/1.501 11 00:38:08 137 822 0 10
Note the queue count is 1 (one) , this implies an un-acknowledged hello.

On R0 the neighbor is removed:
R0#sh ip eigrp neighbors
IP-EIGRP neighbors for process 1
H Address Interface Hold Uptime SRTT RTO Q Seq
(sec) (ms) Cnt Num
1 192.168.20.2 Se1/1.105 13 00:38:54 894 5000 0 15
Other commands you can run to troubleshoot: I'll just BOLD key info from each command
R1#show ip eigrp interfaces
IP-EIGRP interfaces for process 1

Xmit Queue Mean Pacing Time Multicast Pending
Interface Peers Un/Reliable SRTT Un/Reliable Flow Timer Routes
Se1/0 1 0/0 124 0/15 563 0
Se1/1.501 1 0/0 137 0/15 639 0

R1# show ip eigrp topology
IP-EIGRP Topology Table for AS(1)/ID(172.20.2.1)

Codes: P - Passive, A - Active, U - Update, Q - Query, R - Reply,
r - reply Status, s - sia Status

P 192.168.10.0/30, 1 successors, FD is 2169856
via Connected, Serial1/0
P 192.168.20.0/30, 1 successors, FD is 2169856
via Connected, Serial1/1.501


R1# show ip eigrp traffic
IP-EIGRP Traffic Statistics for AS 1
Hellos sent/received: 1202/1179 <---remember we had blocked some hellos with access list 100
Updates sent/received: 109/9
Queries sent/received: 2/2
Replies sent/received: 4/2
Acks sent/received: 1/12
Input queue high water mark 4, 0 drops
SIA-Queries sent/received: 0/0
SIA-Replies sent/received: 0/0
Hello Process ID: 174
PDM Process ID: 169
R1# show ip eigrp accounting
IP-EIGRP accounting for AS(1)/ID(172.20.2.1) <<--note teh ID is our loopback
Total Prefix Count: 2 States: A-Adjacency, P-Pending, D-Down
State Address/Source Interface Prefix Restart Restart/
Count Count Reset(s)
A 192.168.10.1 Se1/0 1 0 0
A 192.168.20.1 Se1/1.501 1 0 0
R1#show ip eigrp 100 ?
accounting  IP-EIGRP Accounting
interfaces IP-EIGRP interfaces
neighbors IP-EIGRP neighbors
topology IP-EIGRP Topology Table
traffic IP-EIGRP Traffic Statistics
!use this if running more than one AS on the same router


show ip eigrp neighbors
Already covered
!Note since these routes are learnt via connected, our routing table currently wont
show any eigrp routes. To get this we need to advertise our loopbacks via eigrp

R1#sh ip route
!--------edited----------!

192.168.10.0/30 is subnetted, 1 subnets
C 192.168.10.0 is directly connected, Serial1/0
172.20.0.0/32 is subnetted, 1 subnets
C 172.20.2.1 is directly connected, Loopback0
192.168.20.0/30 is subnetted, 1 subnets
C 192.168.20.0 is directly connected, Serial1/1.501
10.0.0.0/24 is subnetted, 1 subnets
C 10.10.10.0 is directly connected, FastEthernet0/0

See!! no EIGRP routes, now lets add the loopback addresses:
R0#sh run | section eigrp
router eigrp 1
network 172.20.1.0 0.0.0.255
network 192.168.0.0 0.0.255.255
no auto-summary

and on R1:
R1#sh run | section eigrp
router eigrp 1
network 172.20.2.0 0.0.0.255
network 192.168.0.0 0.0.255.255
no auto-summary
our output:
R0#sh ip route eigrp
172.20.0.0/32 is subnetted, 2 subnets
D 172.20.2.1 [90/2297856] via 192.168.20.2, 00:01:05, Serial1/1.105
[90/2297856] via 192.168.10.2, 00:01:05, Serial1/0
R1#sh ip route eigrp
172.20.0.0/32 is subnetted, 2 subnets
D 172.20.1.1 [90/2297856] via 192.168.20.1, 00:01:23, Serial1/1.501
[90/2297856] via 192.168.10.1, 00:01:23, Serial1/0
Notes:
*EIGRP doesnt build neighbor relationships over secondary addresses.
*Please note changing the hello interval using ip hello-interval eigrp doesn't adjust the hold. time timer for you.
*Load balancing: routing protocols install routes to your routing table, the switching fabric within the router/switch does the load balancing based on various things. eg per packet load balancing, per destination etc etc...all this depends on your switching (cef, fast switching)
let me make a quick demonstration:

If we try to reach R1's loopback with CEF enabled and the routing table as it is now (by default we use per destination loadbalancing) -
R1#debug ip icmp
ICMP packet debugging is on

R1#
*Mar 1 01:04:19.239: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.20.1
*Mar 1 01:04:19.371: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.20.1
*Mar 1 01:04:19.459: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.20.1
*Mar 1 01:04:19.495: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.20.1
*Mar 1 01:04:19.507: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.20.1
Note the path taken is the same, all packets to the same destination will use
same path

now disable CEF (globally - no ip cef)on R0 and make the same 'ping'
R1#
*Mar 1 01:07:27.411: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.20.1
*Mar 1 01:07:27.495: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.10.1
*Mar 1 01:07:27.539: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.20.1
*Mar 1 01:07:27.583: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.10.1
*Mar 1 01:07:27.591: ICMP: echo reply sent, src 172.20.2.1, dst 192.168.20.1
Note the alternating paths, this is per Packet Load balancing
You can use sh ip interface to figure out what sort of switching is in use. I only did this to show how the routing protocol really only populates the routing database, how traffic is moved across networks is not its business.....I'll follow this up with EIGRP metrics,feasible distance, reported distance and feasible successors.....

References:
Cisco
Oreilly's IP routing by By Ravi Malhotra - I found chapter4 available on oreilly:-) good for you

Tuesday, December 2, 2008

What is this about?

Well I just completed a ccip, I did this mainly to renew my ccnp/na. Having worked on cisco gear for well over 5 years now, I figured its about time I pursued a ccie. - there's the added fact that one of my friends just cleared his ccie a few months back so the motivation could be just trying to match my skill in a lab. BTW Kenya by my last count has just 3 CCIE's, I know one; I'd really like to know the other two and when they certified. I'll need tips. Im taking a few days to collect as much info as I can and make a clear plan and put together some materials....

The one advantage I have apart from the experience is the fact that I have a ready rack and dynamips will more than make up for anything I can't get. No i don't have concrete exam dates, but I should be done with the written by end of January '09, (Im shockingly with some time to spare and dedicate to reading this december and january) - the lab depends, if the company i work for agrees to sponsor me then May/June is my timeline, otherwise slowly saving towards the exam/travel etc etc will probably push me to October....more on this later

No the posts won't be all cisco, or ccie ... I tend to do alot of work on ggsn's, sgsn's, 3G, HSDPA, cycling, biking,living etc etc some system administration and anything I feel might be of help to someone else will definately show up here but hte mainthing will probably be note taking :-)

Comparisons and posts on how things work here vs there will be quite welcome. we for instance still rely on satellite for 'internet', a small network in North America would probably be the biggest here....so for instance dimensioning for a network is a bit different especially if done by consultants remotely....