Thursday, November 25, 2010

CRS1 snmp and other counters

Sooo Im almost done with a very interesting project and wanted to share a little something on what happened when we wanted to monitor the optical interfaces.

First off the core network has a bunch of CRS1's interconnected using 3 x 10G DWDM links to the rest of the network giving us what someone fondly called a 10Gig core, or 30 depending on how/what you calculate.

Now the dwdm link characteristics are very important to ensure the deck of cards above it doesnt come crashing down. Some fiber was not very clean and you'd get parameters like:

RP/0/RP0/CPU0:xxxxxxx1#sh controllers dwdm 0/0/0/2
Thu Nov 25 14:10:16.303 EAT

Port dwdm0/0/0/2

Controller State: up

Loopback: None

G709 Status

LOS = 0 LOF = 0 LOM = 0
BDI = 0 IAE = 0 BIP = 0
BEI = 0 TIM = 0

AIS = 0 BDI = 0 OCI = 0
LCK = 0 BIP = 0 BEI = 0
PTIM = 0 TIM = 0

FEC Mode: Enhanced FEC(default)
EC(current second) = 3217480 EC = 602132861570 UC = 43432861570
pre-FEC BER = 2.89E-4 Q = 3.42 Q Margin = 1.74

Remote FEC Mode: Enhanced FEC

Detected Alarms: None
Asserted Alarms: None
BER Thresholds: OTU-SF = E-2 OTU-SD = E-5

OTU TTI Sent String ASCII: Tx TTI Not Configured
OTU TTI Received String ASCII: Rx TTI Not Recieved
OTU TTI Expected String ASCII: Exp TTI Not Configured

ODU TTI Sent String ASCII: Tx TTI Not Configured
ODU TTI Received String ASCII: Rx TTI Not Recieved
ODU TTI Expected String ASCII: Exp TTI Not Configured

Optics Status

Optics Type: Cisco 10Gb DWDM
Wavelength Info: C-Band, XXX ITU Channel=27, Frequency=1xx.80THz, Wavelength=1534.976nm
TX Power = 4.45 dBm
RX Power = -10.90 dBm
TDC Info

TDC Not Supported on the Plim

So any way we needed a way to graph and send alerts based on output like that.

IOS/XR's MIB's do not seem to have anything I could work with.

*if you know the MIB/OID to get me TX/RX power,UC,EC and prefec values I'll be very grateful.

*If you also happen to have an explanation on how to interpret the pre fec values eg:

pre-FEC BER = 2.89E-4

I'd also be very happy.

Enter some scripts:
to get this info we used a combination of expect and perl.

Basically we wrote a script that connects to each node and runs runs the command and dumps it on a text file. our perl-fu then comes on along and picks out the bits and pieces we need and dumps it on a database.

Zabbix (our mrtg'like monitoring system) graphs those. A few other scripts send us sms and email alerts.

Sample expect script:

set timeout 3
spawn /bin/bash
match_max 100000
send -- "telnet\r"
expect -exact "telnet\r
Connected to\r
Escape character is '^\]'.\r
User Access Verification\r
Username: "
send -- "yadada\r"
expect -exact "durangor\r
Password: "
send -- "django\r"
expect -exact "\r
send -- "show controllers dwdm 0/0/0/2"
expect -exact "show controllers dwdm 0/0/0/2"
send -- "\r"
send -- " exit\r"
send -- " "
expect eof

its a simple one that one.

other helpful commands for graphing other things:
sh snmp mib object-name
sh snmp interface
sh snmp interface tenGigE 0/0/0/0 ifindex

I am having alot of fun with IOS xr, the crs1 has some interesting features/utilities too....more on this later...

No comments:

Post a Comment