As I’m working on a larger “cloud bootstrapping” white paper (look for a pending Dell release), I stumbled on an apparent unifying principle for hyperscale cloud design. I’m interested in feedback about this concept to see if it fairly encapsulates a common target for cloud hardware, networking and software design.
“Flatness at the Edges” is one of the guiding principles of hyperscale cloud designs.
Flatness means that cloud infrastructure avoids creating tiers where possible. For example, having a blade in a frame aggregating networking that is connected to a SAN via a VLAN is a tiered design in which the components are vertically coupled. A single node with local disk connected directly to the switch has all the same components but in a single “flat” layer.
Edges are the bottom tier (or “leaves” to us CS geeks) of the cloud. Being flat creates a lot of edges because most of the components are self contained. To scale and reduce complexity, clouds must rely on the edges to make independent decisions such as how to route network traffic, where to replicate data, or when to throttle VMs. The anti-example of edge design is using VLANs to segment tenants because VLANs (a limited resource) require configuration at the switching tier to manage traffic generated by an edge component. We are effectively distributing an intelligence overhead tax on each component of the cloud rather than relying on a “centralized overcloud” to rule them all.
Combining flatness and edges evolves the sympathetic concepts into full-fledged cloud design principle.
Interested in discussing this face to face? I’ll presenting this and other cloud setup concepts that the SJC OpenStack meetup on 2/3.
I’d sure love to hit that event up.
I might just have to coordinate one up here in Seattle. 🙂
Pingback: Tweets that mention “Flatness at the Edges” guides hyperscale cloud design « Rob Hirschfeld's Blog -- Topsy.com
Interesting post and look forward to the whitepaper.
Funnily I read this right after discussing with a colleague cloud design without VLAN trunks where any layer 2 segments are virtualised over layer 3 p2p, thus avoiding VLAN scale limitations, and also potentially further enabling cloud bursting.
We were also discussing BGP peering for control nodes so network addresses were advertised as available resources when added to the network and therefore dynamically assigned.
Current VLAN and routing configurations are largely static due to the tiers you speak of. Even when automated it is essentially a number of scripted static assignments, which still create scale issues. I agree that intelligence and resource decisions at the edge will enable far more scalable cloud pods.
OpenFlow may be of interest to you. It’s not quite prime time (yet) but will enable more edge networking by effectively creating tunnels like you are describing between edges.
Pingback: Here I go again on my own…. « Land of the Long White Cloud
Pingback: Bootstrapping Hyperscale OpenStack Clouds – slides from 2/3 OpenStack SJC Meetup « Rob Hirschfeld's Blog
Pingback: Dell to spin bare iron into #OpenStack gold « Rob Hirschfeld's Blog
Pingback: How #OpenStack installer (#crowbar + #chefops) works (demo 3/14 #SXSW) « Rob Hirschfeld's Blog