Presentation Material
Abstract
Using an open-sourced stack from bottom to top, this presentation will discuss the practical challenges alongside the security opportunities of standing up a private cloud infrastructure. Looking beyond the obvious upside of programmatically definable systems, private clouds present a new platform for security controls that are generally unattainable in traditional infrastructure models. Building on this concept, this talk will present a few interesting security ideas that are currently under development and some that could be the subject of future research. Read more about Building Securable Infrastructure: Open-Source Private Clouds
AI Generated Summary
Bloomberg developed the Bloomberg Clustered Private Cloud (BCPC) to address the specific needs of its financial data and analytics operations, which are poorly served by public cloud models due to strict data locality, sensitivity, and low-latency requirements. The project focused on leveraging a private cloud to enhance security assurance and operational efficiency.
The BCPC architecture is built on a stack of open-source components designed for resilience and automation. It uses Ceph as a distributed storage layer with configurable replication policies aware of physical failure domains (e.g., racks, rows). For state management, it employs MySQL with the Galera plugin for multi-master synchronous replication and RabbitMQ in a clustered mode for reliable messaging. OpenStack provides the core infrastructure APIs for compute, network, and volume management. A custom orchestration layer sits atop OpenStack to monitor service health, automatically replace failed capacity, and manage load balancing, ensuring recovery at the service layer rather than the server level.
Key design principles include treating servers as disposable capacity, eliminating single points of failure, and ensuring clusters are standalone with localized DNS, authentication, and access control to avoid cascading failures. The system scales from as few as three nodes to thousands and can be deployed identically across Bloomberg’s global network of points-of-presence.
Practical benefits include significant developer productivity through self-service, ephemeral sandbox environments and immutable infrastructure where all configurations are codified in repositories (using Chef). Production efficiency is gained via elastic scaling to handle extreme trading peaks without permanent over-provisioning. Security is improved through automated, declarative hardening and the ability to grant developers temporary root access on disposable instances for debugging, eliminating persistent production access. Bloomberg open-sourced all Chef recipes for BCPC deployment, providing a reproducible reference implementation for organizations with similar requirements for controlled, automated, and fault-tolerant private cloud infrastructure