Imagine two ports bound to form one port channel giving 200Mbps. Now imagine on one end you have two hosts/servers that generate/carry a lot of traffic (an ftp server for instance) to multiple destinations on the other end - internet, auth servers etc etc...
One of the hosts has more traffic than the other, infact alot more then 98Mb. so when traffic hit 100Mbps, i started noticing random packet drops. why why why...I thought this is a 200mbps interface???
upon further checks we discovered that one of the interfaces within the bundle was dropping packets/frames.
so the checks started:
Gitau-Switch-01-Sw#sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-mac
src-mac was the default.
Now according to cisco :
EtherChannel load balancing can use either source-MAC or destination-MAC address forwarding.
With source-MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on the source-MAC address of the incoming packet. Therefore, to provide load balancing, packets from different hosts use different ports in the channel, but packets from the same host use the same port in the channel (and the MAC address learned by the switch does not change).
With destination-MAC address forwarding, when packets are forwarded to an EtherChannel, they are distributed across the ports in the channel based on the destination host's MAC address of the incoming packet. Therefore, packets to the same destination are forwarded over the same port, and packets to a different destination are sent on a different port in the channel.
So obviously the default load balancing was not working for me.
Since this was a 3750, I correctly figured that it can also use IP. playing around with the setup on the end that was dropping packets, the following sort of sorted me out:
Gitau-Switch-01-Sw#sh etherchannel load-balance
EtherChannel Load-Balancing Configuration:
src-ip
the command to make this change is:
port-channel load-balance src-ip
you can play around with:
Gitau-Switch-01-Sw#port-channel load-balance ?
dst-ip Dst IP Addr
dst-mac Dst Mac Addr
src-dst-ip Src XOR Dst IP Addr
src-dst-mac Src XOR Dst Mac Addr
src-ip Src IP Addr
src-mac Src Mac Addr
you can run a test by:
test etherchannel load-balance interface port-channel [#] ip [src] [dst]
references:
http://www.edgenetworks.nl/etherchannel.html
http://www.cisco.com/en/US/tech/tk389/tk213/technologies_tech_note09186a0080094714.shtml
before:
Gitau-Switch-01-Sw#sh int g1/0/24 | include drop
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 20191
now :
Gitau-Switch-01-Sw#sh int g2/0/24 | include drop
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
Gitau-Switch-01-Sw#sh int g1/0/24 | include drop
Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
I figure after some time I'll come across a few drops..because the network hates me!!!
Posts
I'm a little green to etherchannel setups. I'm trying to pass a LOT of data (>8gb files via ftp) to 15+ simultaneas hosts. Etherchannel on my 2960G's and adapters teamed. As you, the default wasn't cutting it. What sort of load balancing do you suggest.. I only have 1 server hosting ftp. cheers !!!
ReplyDeletesorry for the late reply.
ReplyDeleteIf the switch is purely doing Layer2, then you only have the Non-IP: Source MAC address options to choose from.
on the switch with 15+ simultaneous connections, use src-mac and for the sqitch connecting the server do dst-mac.
This ensures that the many mac addresses are distributed between 'channels'.
the problem will show up if on both ends a sh arp shows only one mac address on either side. in this case traffic only uses one link in any one direction.
I suggest just going layer 3 if this is the case.
Hello,
ReplyDeleteAbout the source/destination mac address of the packet, I have read some references that say that those are updated on each hop of the network:
http://networkers-online.com/blog/2008/07/etherchannel-load-balancing-case-study/#comment-1347
http://www.eventhelix.com/realtimemantra/networking/ip_routing.htm
If that is true, I believe that on a multi-hop situation, on a certain node, all the packet that came from the same neighbor would follow the same phisical path (on a configuration that selects the path by the MAC, source or destination or both). So it could happen that all the hosts from a certain subnet would follow the same physical path on that node.
What do you think about that?
I would be really apreciated if you could abou what I wrote and post your option, if that is true or not.
Thank you!
in my opinion:
ReplyDeletesrc-dest-IP
would be the option thast gives the best distrubtion of the traffic in a IP network - all links will be used more or less in a random pattern
Src-dest-MAC is needed for non IP networks
for IP networks this will cause that the default gateway is using all traffic to a specific System over the same link because Src and Dest mac are the same
hi thank you very much it is very useful for me
ReplyDeletei was struggling for the past 2 days why only one ingerface is getting chocked after this port-channel load-balance src-ip command both the interface share the traffice
Thank you
Karthik
This comment has been removed by the author.
ReplyDelete