Google technician!

Greg had me all giggly this morning with this post and this one :

“Google Technician” — A google tech is one who googles an issue and sys­tem­at­ic­ally tries every sug­ges­tion that is returned without under­stand­ing or regard for the valid­ity of the response.

I find the google techs quite interesting (as per the definition above), I think the same applies to those that pick scripts off the net and try them until something breaks or works (most of the times things work) which sort of fuels the 'google tech' movement.

Cisco's “unsupported” SFP and making them work - sometimes

We had an interesting incident yesterday. Our network has lots of Huawei and Cisco devices. We use SFP's and XFP's widely on all the platforms. In an ideal world Huawei sfp's should work on Cisco whenever you plug it (there's some HP  switches from which I have borrowed SFP's to cisco and Huawei without any problems) in and this was never an issue on the Huawei end. So it's not a consistent problem for me/us.

Trying it on an ASR 1006 turned out to not be as straight forward. We had to use a 'hidden command'.

*Mar 28 11:07:49.547: %TRANSCEIVER-6-REMOVED: SIP0/3: Transceiver module removed from POS0/3/0

*Mar 28 11:07:59.542: %TRANSCEIVER-6-INSERTED: SIP0/3: transceiver module inserted in POS0/3/0

*Mar 28 11:08:01.160: %ASR1000_RP_ALARM-6-INFO: CLEAR CRITICAL xcvr container 0/3/0 Transceiver Missing - Link Down

*Mar 28 11:08:01.161: %ASR1000_RP_ALARM-6-INFO: ASSERT CRITICAL POS0/3/0 Physical Port Link Down

*Mar 28 11:08:01.124: %TRANSCEIVER-3-NOT_SUPPORTED: SIP0/3: Detected for transceiver module in POS0/3/0, module disabled

Next the command is run:

FW-LB-Rtr(config)#service un?

% Unrecognized command

The above tells you that its hidden and won't come up under context help by typing a question mark.

FW-LB-Rtr(config)#service unsupported-transceiver

You get a warning as shown below so be careful what you do or don't do:

Warning: When Cisco determines that a fault or defect can be traced to

the use of third-party transceivers installed by a customer or reseller,

then, at Cisco's discretion, Cisco may withhold support under warranty or

a Cisco support program. In the course of providing support for a Cisco

networking product Cisco may require that the end user install Cisco

transceivers if Cisco determines that removing third-party parts will

assist Cisco in diagnosing the cause of a support issue.

 

FW-LB-Rtr(config)#

and our interface came up

*Mar 28 12:28:36.354: %LINK-3-UPDOWN: SIP0/3: Interface POS0/3/0, changed state to up

*Mar 28 12:28:37.372: %LINEPROTO-5-UPDOWN: Line protocol on Interface POS0/3/0, changed state to up

Please note the command is supported on some switches.

GNS3 Using remote Hypervisors

I have for a while now used Dynamips,Dynagen and Qemu for emulating IOS and Junos. Dynamips as everyone thats tried using it is a huge resource hog and if not properly managed can be quite a pain to work with.

For the troubleshooting exam, you need to load well over 20 routers and a few switches. While this can be done in isolation ie subsections of a lab done separately maybe with 5 routers max, I prefered getting a platform that allowed me to load them all up.

I assume here that you are familiar with GNS3. If not please check out the link below:
http://www.gns3.net/documentation

To get this working for me, I used GNS3 and a remote hypervisor. This sort of outsources the intensive processing to a powerful server and my laptop managed the frontend/topology.

I used just one server but you can use many. On the server (I am using a server running Ubunu 10.10, 128GB memory and several processors), Do the following:

Prepare the serverside environment and download Dynamips (make sure you get the proper version for your platform):

root@scc-sfc-mgt-devsvr:/home/jgitau# mkdir dynamips
root@scc-sfc-mgt-devsvr:/home/jgitau# cd dynamips/
wget http://www.ipflow.utc.fr/dynamips/dynamips-0.2.8-RC2-amd64.bin

Create a folder for your IOS and a temporary folder to use as a working directory

root@scc-sfc-mgt-devsvr:/home/jgitau/dynamips# mkdir ios
root@scc-sfc-mgt-devsvr:/home/jgitau/dynamips# mkdir tmp

Your folder should resemble something like:

root@scc-sfc-mgt-devsvr:/home/jgitau/dynamips# ls
dynamips-0.2.8-RC2-amd64.bin  dynamips_log.txt  ios  tmp

Place the IOS you'll be using in the IOS folder then start dynamips. Also ensure its networked duh! and can 'ping' your laptop/pc/whatever you run the GNS3 on.

Next, start the dynamips engine:

Note on a server with multiple CPU's, it helps if you start several instances on different ports.

root@scc-sfc-mgt-devsvr:/home/jgitau/dynamips# ./dynamips-0.2.8-RC2-amd64.bin -H 7201
Cisco Router Simulation Platform (version 0.2.8-RC2-amd64)
Copyright (c) 2005-2007 Christophe Fillot.
Build date: Oct 14 2007 10:46:52

ILT: loaded table "mips64j" from cache.
ILT: loaded table "mips64e" from cache.
ILT: loaded table "ppc32j" from cache.
ILT: loaded table "ppc32e" from cache.
Hypervisor TCP control server started (port 7201).

Next up, fire up GNS3 on your laptop, we'll create a 20 router topology for demo purposes. I am running GNS3 0.6.

- Click on Edit --> Ios Images and Hypervisors 

Navigate to "External Hypervisors" tab

- Put in the host details , ensure the port matches the port you started on the server (7201).
- Put in the base UDP Port (10004), Base console (2000) and the working directory: /home/jgitau/dynamips/tmp 
(This is the folder you created on the server)
- save.

Next, Navigate to "IOS Images" tab.
Image file is the image location on your server, use whatever platform your IOS matches

Image file: /home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin
platform :7200

On your right,

Unclick the "use Hypervisor manager"
save

Create a topology, I did the one showed on the image below, start up the lab, on the server side you'll see the routers loading up. Sort out the idle PC as usual. Monitor the CPU using top or htop. Sample output follows.

Loading ELF file '/home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin'...
ELF loading skipped, using a ghost RAM file.
ELF entry point: 0x80008000

C7200 'QOA.PE.2': starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.
CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb.
C7200 instance 'QOA.PE.1' (id 19):
  VM Status  : 0
  RAM size   : 256 Mb
  IOMEM size : 0 Mb
  NVRAM size : 128 Kb
  NPE model  : npe-400
  Midplane   : vxr
  IOS image  : /home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin

Loading ELF file '/home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin'...
ELF loading skipped, using a ghost RAM file.
ELF entry point: 0x80008000

C7200 'QOA.PE.1': starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.
CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb.
C7200 instance 'KBW.P' (id 20):
  VM Status  : 0
  RAM size   : 256 Mb
  IOMEM size : 0 Mb
  NVRAM size : 128 Kb
  NPE model  : npe-400
  Midplane   : vxr
  IOS image  : /home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin

Loading ELF file '/home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin'...
ELF loading skipped, using a ghost RAM file.
ELF entry point: 0x80008000

C7200 'KBW.P': starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.
CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb.
C7200 instance 'SCC.RR' (id 21):
  VM Status  : 0
  RAM size   : 256 Mb
  IOMEM size : 0 Mb
  NVRAM size : 128 Kb
  NPE model  : npe-400
  Midplane   : vxr
  IOS image  : /home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin

Loading ELF file '/home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin'...
ELF loading skipped, using a ghost RAM file.
ELF entry point: 0x80008000

C7200 'SCC.RR': starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.
CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb.
C7200 instance 'MSA.P' (id 22):
  VM Status  : 0
  RAM size   : 256 Mb
  IOMEM size : 0 Mb
  NVRAM size : 128 Kb
  NPE model  : npe-400
  Midplane   : vxr
  IOS image  : /home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin

Loading ELF file '/home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin'...
ELF loading skipped, using a ghost RAM file.
ELF entry point: 0x80008000

C7200 'MSA.P': starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled.

The above shows a sample output you get on the server side.

noc@scc-sfc-mgt-devsvr:~$ htop

Sample .net file:

autostart = False
[192.168.13.2:7201]
    workingdir = /home/jgitau/dynamips/tmp
    udp = 10004
    [[7200]]
        image = /home/jgitau/dynamips/ios/c7200-p-mz.124-25d.bin
        ghostios = True
        sparsemem = True
    [[ROUTER MSA.PE.2]]
        console = 2046
        slot1 = PA-GE
        g1/0 = MSA.PE.1 g2/0
        slot2 = PA-GE
        g2/0 = MSA.P g4/0
        x = 874.788888605
        y = 753.254833996
    [[ROUTER SCC.MCE.2]]
        console = 2054
        slot1 = PA-GE
        g1/0 = SCC.MCE.1 g1/0
        slot2 = PA-GE
        g2/0 = SCC.PE.1 g1/0
        slot3 = PA-GE
        g3/0 = SCC.PE.2 g2/0
        x = -1235.54415588
        y = 811.297690046
    [[ROUTER EDT.PE.2]]
        console = 2065
        slot1 = PA-GE
        g1/0 = EDT.PE.1 g2/0
        slot2 = PA-GE
        g2/0 = SCC.P g6/0
        slot3 = PA-GE
        slot4 = PA-GE
        slot5 = PA-GE
        slot6 = PA-4E
        x = -1679.1445866
        y = -389.695526217
    [[ROUTER EDT.PE.1]]
        console = 2066
        slot1 = PA-8E
        e1/0 = NKU.P e6/0
        slot2 = PA-GE
        g2/0 = EDT.PE.2 g1/0
        slot3 = PA-GE
        slot4 = PA-GE
        slot5 = PA-GE
        slot6 = PA-4E
        x = -1506.61053199
        y = -389.695526217
    [[ROUTER SCC.MCE.1]]
        console = 2055
        slot1 = PA-GE
        g1/0 = SCC.MCE.2 g1/0
        slot2 = PA-GE
        g2/0 = SCC.PE.2 g1/0
        slot3 = PA-GE
        g3/0 = SCC.PE.1 g2/0
        x = -1499.27416998
        y = 810.611398545
    [[ROUTER SCC.P]]
        console = 2021
        f0/0 = QOA.P f0/0
        slot1 = PA-GE
        g1/0 = MSA.P g2/0
        slot2 = PA-GE
        g2/0 = NKU.P g1/0
        slot3 = PA-GE
        g3/0 = SCC.PE.2 g3/0
        slot4 = PA-GE
        g4/0 = QOA.PE.1 g2/0
        slot5 = PA-GE
        g5/0 = SCC.RR g1/0
        slot6 = PA-GE
        g6/0 = EDT.PE.2 g2/0
        x = -1260.55277785
        y = 323.885822331

- Note:
You can use as many servers as you want. technically the server side stuff can be a friends laptop when he's not using it. Since there's not much in the name of installation, a friendly systems admin can also 'allow' you to run this on a test server, it can also be done remotely. There's a clearer video I found showing the same thing here: http://www.brainbump.net/tutorials/gns3/multiplepcgns3topology.htm

- The other upside is anyone I share this with on our network won't suffer CPU issues and can just focus on the simulations. It comes quite handy as a training tool as well since multiple hypervisors can be used. This particular server has at one point run 80 routers....(4 guys doing 20'something routers each) thats some serious steroids right there.

Work Study habits

Studying and time management: It's now a skill I should probably list on my CV alongside everything else I have in there.

I wake up early. Sleep early. By early I mean depending on what's on my plate, don't be shocked to find me at my desk either in the office or the study at home at 3-4am. By 9 am on these days Im pretty much done with any pressing creative (hard focus work) and can 'trade' email or chat by the water cooler all day.

Fact: the only thing that matters is the result. In the case of a ccie, a pass, in a competitive environment, no one wants to be number 2....I want to be at the top...ALWAYS...I want the gold, keep the silver....

Life has never been fair, the path to success is not fair, its not linear, it's negotiable and it most definately changes. I know this, I have been lucky in some ways...very lucky actually .....

I don't do facebook. I totally weaned off facebook during the last 3 months of my CCIE preparation. I might use it to market something in the future but it has no relevance whatsoever in my personal life. All my friends know how to reach me. I am checking out twitter. This doesn't mean you have to, I personally just couldnt handle the distraction.

I like knowing well before hand whats expected of me for me to appropriately schedule it.I was brought up in a household that avoided ambiguity, yes no answers were common, you either did something or didn't.

This is a big pain at my current workplace since a restructuring is taking place. Ad hoc requests and implied responsibilities tend to come up alot. I hate it. It frustrates me and my efforts towards effective time and other resource management.

I don't measure my productivity with Hours worked. I prefer working with results. Your results are a direct reflection of the strategy you employ to get a certain grade or outcome.

Avoid meaningless meetings, infact don't go for any meeting without an agenda. Wait for the minutes. If none come then it can't have been important. On the same note avoid people that bring negative energy to your life. If you can't eg they are your boss...learn to 'zone out' (a skill i don't have but wish I did - just in case:-))

Almost all core concepts on the CCIE blueprint require 20 - 60 minutes of some serious focus including time spent lab'bing up simple scenarios before you 'get it' well enough to answer related questions or teach it. Which is why trying to rush through the material doesn't work very well. If you don't have a study plan that respects this need for the tasks, you are most likely destined for failure.

Time management is a technical skill. There is a huge difference between technique and effort. Learn to schedule work then do it, dont make excuses, do not procrastinate, do the mental labour, don't focus on time, just get it done to get the most satisfactory result possible.

IMHO hours spent on study and work are a terrible metric, to this end the 9-5 sort of job alot of people are into offers no appeal to me.  I think compared to most people I do have a very flexible work schedule. Use the time saved to do things you love.

Avoid transcribing, aim to understand. If listening to an on demand video on OSPF/bgp/etc, pause it occassionally to test the scenarios, don't try to take 'blind' notes. Even typing the commands and expected output on notepad if you have no access to routers/gns3/dynamips/dynagen helps more than taking notes as you listen and move along. so learn it like you are going to teach it, synthesize the material. break it down to its littlest pieces, then put it together again.

In the end the point Im trying to make here is : counting hours is meaningless (It was for me) when it comes to studying. (On my first ccie attempt I kept a very detailed log of how long I spent studying for the exam - While the data is good to a statician, I didn't find it to be of any use). So the last time round I focused on learning the material and the concepts, got the knowledge ,acquired the skills until I felt ready. Do this and you might avoid landing yourself in a failure situation.

If you are working on something else that requires 'some serious focus' alongside something like preparing for the CCIE, stop being silly, drop one. There's a high chance you'll miss out on both.

Do not give up, especially if you have already spent well over $10K in personal dollars, Money spent on a CCIE can be a big incentive to working harder, to passing:-)....can also be quite frustrating......

Lastly just have fun, relax enjoy the journey.....

Why It's not about the "G' 's! and the 'b' 's

Every once in a while it happens that my TV is on while Im messing around on the laptop. One of the 'news' items on the last week was that one of the Telco's was launching a broadband service that would guarantee well over 8MegaBytes of data. huh! This was followed by the usual market speak....it will be super fast..yada yada yada!!!

On the same note the papers have all sort's of G's being advertised. 2G,2.5G,3G and next up for your consumption is 4G.....

Sometimes the market speak can be quite hilarious, other times- not so much. On the Kictanet mailing list, a search for complaints, promises, bandwidth, unlimited etc gives an indicator of the kind of confusion marketing a service purely on offered capacity and 'take home bits' without making an attempt at educating the consumer can create. The case linked above helped get out some very interesting issues.

One I thing we (the industry) need to take a keen on is the fact that the more info consumers have, the easier it gets. Information on coverage,regional QOS (site to site latency/jitter) etc shoul d probably be availed. Customers should be allowed a forum to vent and networks use that as a honey pot of sorts. Do we support IPv6? provide the info before its asked for. Network engineers need to collaborate more.

Managing customer expectations/interactions and the role of CCK    as a consumer advocate is also not very clear.

Considering ISP's have been there longer than traditional telco's in the data arena, its funny they still do not have a QOS policy for data. (I take that back, even if they had one I'm not too sure they can enforce it - they can try but ehhh)....It might turn out to be the old Jambonet 'block' all VOIP musical chairs with ISP's...somehow Jambonet used to be left standing....now those were good days!!!

If I am in a coverage hole for 3G, am I right in accusing the vendor of poor service? what if my phone 'hops' from 3G to 2G, If on an unlimited data plan what gives, what is this downgrade my speed (bps) maneno? did I pay for a specific QOS? If I did how do I measure it? can the guy selling it to me even measure it? what of Wimax, LTE when it gets sold...?....

My CCIE lab

My practice lab for the CCIE was made up of 3 x 3750, 1 3550 switches, 2 x 2821 routers , 2 x 2811 (one acted as a console - from the image thats the one with the green cables, 1 x 2851 router and three lower end routers to inject backbone routes. Just in case shit hit the fan, I had (still have) more than 1000 rack hours over at graded labs.

For the troubleshooting scenarios, I got extremely lucky. Happened to have a Sun Fire X4470 Server, 128GB of memory and some extremely fast processors that allowed me to load well over 30 routers. Virtualizing would have got me more capacity. Dynamips and Dynagen took care of the rest. I didn't even have to meddle with the idlepc. So obviously I had more than enough equipment to get this done.

*Being ready and having access to the right equipment and people clearly came out as one of the key factors if you expect to pass the ccie. Out of the 15 or so guys that sat the mobile lab, I think only my colleague and I passed which to me is very telling.

*I have never paid Narbik a single cent but for some reason he answered all my questions promptly - ON CHAT!....including after the lab when I had trouble accessing information on how to become a ccsi. So thanks for that...

I had (still have about 1500hrs rented rack over at INE) and the lab above was set up for the full INE R&S topology. I also have to add that INE/graded lab racks are probably more advanced than any other commercial rack I ever tried (and I tried quite a number). My time zone (+3 GMT) allowed me to get excellent service/rack hours.

Considering all the above, the only thing stopping me from taking the CCIE SP is the change in blueprint.

To the physical devices above will be added some IOS-XR devices and I expect that to form the basis for future blogs if I'm not too lazy to post that here. I am yet to decide on a vendor vs self study for workbooks and how much 'noteput' to produce (I could do my own workbooks). IOS-XR is not very well covered, there's not many examples for various scenarios, neither is the CRS-1/ASR's and other high end devices that I might have access to....I would expect that to add value here in between my rambling...we'll see
!!

CCIE Laziness

I think I have what I can only call "Post CCIE Syndrome": Symptoms include :

- Sleeping early, getting up late, having breakfast without panicking because you seem to have forgotten under what circumstances a default route cannot be injected to different OSPF areas.
- Occassionally slapping your face to confirm 'its not a dream, the 8 hrs happened and you passed'.
- Somehow having time to go to the gym, say a smiling hello to the neighbors (who hated you for being a snob).
- General cheer and high energy levels - if you passed, low energy levels if you failed (Imagining that you'll go through it all again sucks).
- Having time for girls/boys/dogs/cycling/hiking....
- Wondering where all the money people say you get after a CCIE is...(My CCIE debts are astronomical).
- but the one that really gets me is how lazy I have become. Yesterday I watched day time TV (I swear I could feel some IQ points flying away as a consequence but who needs those for another week right:-))...

**

Done

ccie#28352