A post by Randy Bias OpenStack Track Chair.
There’s no shortage of stories written about the success of OpenStack. In just three short years, OpenStack has catapulted from a nifty idea advanced by Rackspace and NASA/Anso Labs to a juggernaut of open source software.
Consider this: The more than 10,000 OpenStack Foundation members comprise over 1,000 developers, about 300+ of whom are contributing code on average any given month. Those developers work for 200 companies in more than 100 countries globally, and they contributed more than 3,200 code commits in the latest release, Grizzly. In total, for Grizzly, 517 devs contributed 7,600 patches and 230 new features.
That’s impressive, no doubt. But what’s more impressive is project’s software development life cycle (SLDC). It has matured into an impressive machine that has made possible the project’s semi-annual release cycle with few or no code regressions that complicate compatibility for application developers and cloud architects.
OpenStack benefits greatly from the leadership of folks like Russell Bryant and Mark McLoughlin, both with Red Hat, Monty Taylor of HP and Thierry Carrez, who came to the OpenStack Foundation from Canonical. These leaders have brought decades of open source experience to OpenStack, contributing new ideas and energy that have directly led to the processes and systems that make our semi-annual release cycle possible.
The Continuous Integration (CI) team in OpenStack has built testing systems that have made it possible to scale from a few dozen contributors for the Bexar release to more than 700 developers now pushing patches *daily* to the project.
The CI system runs as a single app across two OpenStack-powered public clouds, with resources donated by HP, Rackspace and eNovance. The system merges about 150 patches each day into the code base. There are more than 500 commits that don’t make it — every day.
Automation is a big part of the CI system. Google’s Gerrit code review system feeds into a system called Zuul for gating (Ghostbuster fans will recall that Zuul was the Gatekeeper). All this is connected to a Jenkins server with Gearman worker support for scaling. Tests are tun in parallel with optimistic pipelining to save time.
This may seem like a lot of moving parts, and it is. Without the CI system, there would be no way that 500+ devs could contribute hundreds of code commits each day and the project still ship a new version every six months.
The OpenStack infrastructure team uses the CI system to deploy and test OpenStack more than 700 times every day day using Tempest integration tests. To date, the system has performed more than 15,000 unit tests, an incredible feat for an automation system that has only recently come online.
This is all the more impressive when you consider that the CI and other QA projects in OpenStack have been designed, implemented and iterated in an environment where the number of projects has grown from just two (Nova and Swift) to nine (adding Glance, Cinder, Keystone, Heat, Horizon, Neutron (nee Quantum), and Ceilometer).
The success of OpenStack is substantially the responsibility of the developers and leaders who have given the community a top-notch CI system. We’ll take a closer look at OpenStack’s SLDC during the OpenStack Boot Camp at Cloud Connect Chicago.
Randy Bias is CEO and Co-founder of Cloudscaling, and a board member of the OpenStack Foundation. He is also listed as a Top 10 Cloud Computing Pioneer by Informationweek.