Mayflies and Dinosaurs (extending Puppies and Cattle)

Dont Be FragileJosh McKenty and I were discussing the common misconception of the “Puppies and Cattle” analogy. His position is not anti-puppy! He believes puppies are sometimes unavoidable and should be isolated into portable containers (VMs) so they can be shuffled around seamlessly. His more provocative point is that we want our underlying infrastructure to be cattle so it remains highly elastic and flexible. More cattle means a more resilient system. To me, this is a fundamental CloudOps design objective.

We realized that the perfect cloud infrastructure would structurally discourage the creation of puppies.

Imagine a cloud in which servers were automatically decommissioned after a week of use. In a sort of anti-SLA, any VM running for more than 168 hours would be (gracefully) terminated. This would force a constant churn of resources within the infrastructure that enables true cattle-like management. This cloud would be able to very gracefully rebalance load and handle disruptive management operations because the workloads are designed for the churn.

We called these servers mayflies due to their limited life span.

While this approach requires a high degree of automation, the most successful cloud operators I have met are effectively building workloads with this requirement. If we require application workloads to be elastic and fault-resilient then we have a much higher degree of flexibility with the underlying infrastructure. I’ve seen this in practice with several OpenStack clouds: operators with helped applications deploy using automation were able to decommission “old” clouds much more gracefully. They effectively turned their entire cloud into a cow. Sadly, the ones without that investment puppified™ the ops infrastructure and created a much more brittle environment.

The opposite of a mayfly is the dinosaur: a server that is so brittle and locked that the slightest disturbance wipes out everything it touches.

Dinosaurs are puppies grown into a T-Rex with rows of massive razor sharp teeth and tiny manicured hands. These are systems that are so unique and historical that there’s no way to recreate them if there’s a failure. The original maintainers exit happy hour was celebrated by people who were laid-off two CEOs ago. The impact of dinosaurs goes beyond their operational risk; they are typically impossible to extend or maintain and, consequently, ossify other server around them. This type of server drains elasticity from your ops team.

Puppies do not grow up to become dogs, they become dinosaurs.

It’s a classic lean adage to do hard things more frequently. Perhaps it’s time to start creating mayflies in your ops infrastructure.

9 thoughts on “Mayflies and Dinosaurs (extending Puppies and Cattle)

  1. If you’re trying to build elastic scalability into a system to exceed the demands of predators, err i mean customers then this metaphor may work, otherwise, you’re just spending an awful lot of energy on an evolutionary dead end
    I think the stronger theme one could take from such mayfly experimentation isn’t in the rather dogmatic “pets vs cattle” concept but more along the lines of mimicking the biology of mayflies – how they evolve, how natural selection chooses the winners and how selective evolution could also be in place to help build resilient “species” of systems. As long as your life cycle isn’t just the expenditure of energy but the measurement of that expenditure against your survivability and whether or not the short life improves your ability survive then you’re good to go. Otherwise, its a very expensive system that i believe others will out evolve.

    Like

  2. Pingback: Mayfiles and Dinosaurs – Metamorphosis and Epigenetics in Devops | New Goliath

  3. Pingback: DevOps for Non-Profits?! The Miracle Foundation does IRL Puppies v. Cattle | Rob Hirschfeld

  4. Pingback: Hyperscale Invades the Enterprise Data Center [Infographic] « Wikibon Blog

  5. Pingback: Networking in Cloud Environments, SDN, NFV, and why it matters [part 1 of 2] | Rob Hirschfeld

  6. Pingback: Hyperscale Invades the Enterprise Data Center [Infographic] | OSINT ZONE

  7. Pingback: Research showing that Short Lived Servers (“mayflies”) create efficiency at scale [DATA REQUESTED] | Rob Hirschfeld

  8. Pingback: Short lived VM (Mayflies) research yields surprising scheduling benefit | Rob Hirschfeld

Leave a comment