Cisco Catalyst 3750 SVI MAC Addresses

The following has been tested on a Cisco Catalyst 3750. I suspect the behaviour is the same on a 3560 but I would like someone to test that out and confirm. Also, if you find that this is documented better elsewhere please comment and let me know and I'll link to it.

I have a particularly unusual customer setup that I won't go into the details of here, but the long and short of it is that we need to know the MAC address assigned to a particular SVI as the other end can't dynamically learn addresses, but there's a potential gotcha around expecting the MAC to be static.

Let's take a look at the Catalyst 3750 I was testing with. The base MAC (from show version) is as follows:

Base ethernet MAC Address       : 08:17:35:E6:46:80

On the 3750 the first 64 MAC addresses beginning at the base MAC are reserved for layer 2 switchports. The 65th MAC address is assigned to the first layer 3 interface and this is where things get interesting. So, the last part of our base MAC address is 0x80 (hex). Let's convert to decimal to make things a bit easier:

0x80 = 128
128 + 65 = 193
193 = 0xC1

So we would expect our first layer 3 interface to have a MAC address of 08:17:35:E6:46:C1.

If we configure an interface Vlan 40 and do a show interface we get the following output:

Vlan40 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c1 (bia 0817.35e6.46c1)

which is what we expected. And if we configure another SVI, say Vlan 50, we would expect the MAC address to be increased by 1:

Vlan50 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c2 (bia 0817.35e6.46c2)

and it is. So far, so good.

So, we know we can predict the MAC address of an SVI given the base MAC address of the switch, but does it survive a switch reload? Let's save the config and reload to find out. The following is a show interface for vlans 40 and 50 after the reload:

Vlan40 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c1 (bia 0817.35e6.46c1)

Vlan50 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c2 (bia 0817.35e6.46c2)

They're still the same, vlan 40 ending C1 and vlan 50 ending C2. Cool.

What happens if we add vlan 60 now after the reload?

Vlan60 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c3 (bia 0817.35e6.46c3)

As you might have predicted, vlan 60 gets assigned a MAC address ending in C3.

This all makes sense so far, our vlans are contiguous and predictable, but what would happen if we added vlan 30 now? Let's add it and do a show interface:

Vlan30 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c4 (bia 0817.35e6.46c4)

Vlan 30 gets assigned the next available MAC address, ending C4. Okay, so that's fairly predictable; we created a new vlan and it got assigned the next available MAC address. What do you think will happen if the switch is reloaded? Let's save the config and reload to find out:

Vlan30 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c1 (bia 0817.35e6.46c1)

Vlan40 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c2 (bia 0817.35e6.46c2)

Vlan50 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c3 (bia 0817.35e6.46c3)

Vlan60 is up, line protocol is down
  Hardware is EtherSVI, address is 0817.35e6.46c4 (bia 0817.35e6.46c4)

Now we see a change in behaviour. The switch has assigned out the first available layer 3 MAC address to the first assigned vlan, which, when reading the config, is no longer vlan 40, but vlan 30 which appears first in the config.

Normally this isn't going to be a problem, but in my particular scenario where I need to know the MAC addresses for static assignment it's a bit of a big deal.

In summary, MAC addresses are assigned to SVI interfaces in the order the SVI interface comes up, which is numerically in the IOS configuration file.

Cisco ASR 1001 Queuing on PPPoE Interfaces

The ASR 1001 has a few interesting limitations, mostly around some features being missing from the IOS XE codebase. The latest one I've hit is the inability to apply a shaping policy to a PPPoE virtual-access interface.

My setup looks like this: two GigE interfaces in a port-channel with a number of VLANs configured as sub-interfaces to the port-channel and PPPoE sessions terminated on those VLANs. Simple enough.

If I apply a simple policer to the virtual-template or configure it via RADIUS for a router dialling in with a PPPoE client the policy is applied, no questions asked. But a simple policer isn't really much good to me because I'm going to want a child-policy in order to define an LLQ for, say, VoIP traffic.

So, I need to apply a shaper parent policy effectively defining the bandwidth of the interface and a child policy containing my LLQ for my VoIP class and my other traffic classes specific bandwidth requirement.

All straightforward enough stuff, bread and butter for SPs and enterprises. Until you bring up your PPPoE client, do a show policy-map interface on the virtual-access subinterface and... nothing. A quick look at the log shows an error like this:

Sep  7 14:30:02 BST: %QOS-6-POLICY_INST_FAILED:
 Service policy installation failed

After digging around a bit and trying a few things I was confident enough that I'd hit at best some sort of bug, at worst a "feature limitation" so I got in touch with Cisco TAC. After a bit of troubleshooting the TAC engineer homed in on my port-channel configuration and suggested that I upgrade to IOX XE 3.7 as there is limited support for queuing on port-channel interfaces. Fine, I went ahead with that and tried again. Show policy-map interface and... nothing.

Another look at the logs showed the familiar policy installation error from before but this was now accompanied with this additional error:

Sep 11 08:19:32.251: Port-channel1 has more than one active member link

Yes, this port-channel has more than one active member link because, you know, it's a port-channel!

At this point I could see where this was going and spoke to the TAC engineer again. Apparently this is what he meant by "limited support" for PPPoEoVLAN on port-channels. So, in order to prove that the rest of my configuration worked I went ahead and re-configured the port-channel for fast-switchover (active / standby - see here (cisco.com)). Now the policy applied without a problem.

This leaves me in a bit of a predicament though. On one hand I now have policy-maps applying successfully PPPoE virtual-access interfaces. On the other hand I now have an ASR 1001 that can do 2.5Gbps throughput by default (and is upgradeable by license to 5Gbps throughput) that I've had to "waste" an interface on to get uplink resilience. And, on a box with only four SFP ports that's expensive wastage. I could use the other two SFP ports and artificially load-balance traffic by targeting different PPPoE VLANs on each port-channel, but I honestly would much rather see Cisco fix queuing on load-balanced port-channels. Hopefully this will happen sometime soon, before this box starts getting pushed beyond 1Gbps!