Presented: Len Wyatt
There are commercial capacity management tools out there, so why would anyone build their own? For Blackbaud, there were both technical and organizational reasons. There have been some surprising benefits. This talk surveys the motivations, the architecture, the benefits and the tradeoffs of creating your own Capacity Management Data Warehouse (CMDW).
The primary uses for CMDW include monitoring current systems for capacity concerns, trying to anticipate upcoming needs, forecasting resource needs as we move from physical servers to a virtualized data center and to a cloud-based infrastructure, and troubleshooting issues based on data that was not visible before. On the monitoring front, as the number of systems monitored has grown, we have moved toward a process of looking for statistical anomalies in the data and having people investigate data that was first uncovered by the statistics. Forecasting is still a spreadsheet-driven process, but the CMDW provides data that lets us project from current environments to future configurations.
This talk will outline the variety of data sources that feed into CMDW, dive into how a traditional data warehouse architecture using a relational database has worked as the core mechanism, and where we extended the concepts of a relational warehouse for synchronizing disparate data sources and for doing statistical analysis. A look at the varied uses of CMDW comes next: both the things we planned to do and the things that popped up once people saw the ability to collect and analyze data in new ways. At the end, we’ll “fess up” to some of the issues that came up as well.
About the Presenter
Len Wyatt is a Laureate Software Engineer at Blackbaud, where he is involved in system performance and capacity management for the systems that power thousands of non‐profit and social‐good organizations. Before that, Len was a Program Manager at Microsoft, often engaged in crazy projects like building the largest On‐Line Analytical Processing (OLAP) database ever, breaking the world record for ETL processing, or creating an industry standard ETL benchmark through the Transaction Processing Council. In these roles he was a speaker at many industry events. Len was also an Engineer in testing and in performance at Sequent Computer Systems, which created the first commercial multiprocessing systems. He started out at Bell Labs working on packet switching systems and doing performance simulations of not‐yet‐implemented systems.