Over the last two years or so, I have been on adventure with Data Centre Infrastructure renewal. As past posts may allude to, ACI was a big part of what we did, but before anyone gets all dogmatic about it, know that we didn’t go “All in” with that one product, since I personally don’t subscribe to the “DC Fabrics cure all ills” mantra.

CLOS fabrics and the various approaches to overlays within them are great at providing stable platforms with predictable properties for speed, latency and scale. Unsurprisingly, they go on to do a great job in server farms that can make the best use of that flexibility. During recent conversations on DC refresh, our Arista friends have been extremely keen to try and get us to run our Internet BGP border on the fabric as well. The 7280SR2K can handle 2M routes in FIB they say, just lob stuff into a VRF, bit of policy and voila. Yeah.

So I bought my ACI bundles so long ago that they’re still running 1.0(3f). Right now mainline is 1.2(1k), so i’m a bit behind.

Using the official Cisco doc I did the first staged upgrade from 1.0 to 1.1 using the Web GUI. I wanted to see what happened in a visual sense.

Basically you setup a connection between the APIC and a host that has staged the firmware files, then you setup a policy defining what versions the fabric should be on, and when that should be made active. For me it was 1.1(4f) and now basically.

ACI brings with it many different constructs for operating networks, some of which have analogous equivalence with classical networking, some of which are literally bat-poop crazy.

As per usual, you can find lots of resources on how to structure ACI fabrics elsewhere, i’m not going to waste time on what you can do and focus on what I am going to do (roughly).

The below Image was unceremoniously stolen from Cisco themselves, in the critical read ACI Fundamentals

Before I get too wound up I should probably say that all of this was directed to my friends there first, and whilst I won’t say much about their thoughts, I don’t think this is particularly new to them, or out of place.

I have a fondness for ACI. I think its innovative, and once you break through the naming conventions and the terminology, it’s exactly what I think Enterprise should be doing in terms of Next Generation Networking. That said, INSBU are not helping themselves penetrate the market, and as such, are putting themselves at risk of falling behind to Openstack.

Plumbing ACI is something that YouTube has you covered on. I wont reinvent that wheel. For the initial standup, I am doing the bare minimum connectivity; each leaf has one 40G uplink to each spine, meaning, 80G of North/South Bandwidth. This will double up when we are preparing for Production service, matching my UCS/FI Bandwidth between each Chassis (4x10G links to each side of my 2208XPs). My 3 APICs are configured as follows: