ML-Based Power Models for CPUs. Innovation in Verification

An effective learned power model for a CPU could be a significant advantage in early estimation of power performance against realistic workloads in complex architectures. Paul Cunningham (GM, Verification at Cadence), Raúl Camposano (Silicon Catalyst, entrepreneur, former Synopsys CTO and lecturer at Stanford, EE292A) and I continue our series on research ideas. As always, feedback welcome.

The Innovation

This month’s pick is Machine Learning-Based Microarchitecture-Level Power Modeling of CPUs. The authors are from UT Austin, also Hyperstone and U Stuttgart, both in Germany. The paper (2023) has been published in the IEEE Transactions on Computers and has 18 citations.

We covered another paper on this topic back in 2021: Learning-Based Power Modeling. Innovation in Verification (senior author in common with this paper). The primary advance from that earlier paper is extending the method from fixed function accelerators to programmable in-order and out-of-order CPUs. The learning method here is based on DNNs; it would be interesting see how transformer-based methods could advance these ideas.

Paul’s view

Fun paper this month on estimating power from a cycle-accurate micro-architecture level simulation of a CPU – something at a resolution similar to what a high-level synthesis tool would be able to compile into RTL.

The authors’ approach is to train decision-tree models to predict power for a single clock cycle on low level CPU building blocks such as the fetch, decode, and load-store. The decision-trees are trained using golden power data using a post-P&R gate-level netlist and simulation traces. The models are then assembled hierarchically to construct a cycle-accurate power profile from a cycle-accurate micro-architectural simulation trace of the complete design.

The key challenge to make the approach work picking the right features to use for branches in the decision-trees. Here, the authors have several neat ideas. All primary inputs to a block are used as features – control signals have a feature for each bit, datapath signals have a feature for each byte, where the value of that feature is the number bits in the byte that change value in the cycle. Where FIFOs and other pipelined structures are detected, their internal stages get their own features in the decision-tree. Glue logic at each intermediate level in the hierarchy is treated as a virtual sub-block and given its own dedicated decision-tree.

The authors present results on two open source designs, a simple in-order CPU, and a complex out-of-order CPU, BOOM. They are implemented on an open source 45nm process and library. With only 20k cycles of training data using 8 very simple test programs (quick sort, Towers of Hanoi, Dhrystone, …), power estimation for BOOM on the CoreMark benchmark has an average error in any clock cycle of less than 4%. At the resolution of individual building blocks, 80% of the blocks are within a 15% error in any clock cycle. The runtime overhead of processing the decision trees is negligible, so the entire estimation process basically takes as long as the architecture simulation takes. Nice.

Raúl’s view

The motivation for this work is the long runtime of power estimation. Signoff power analysis requires gate-level activity from a synthesized and physically implemented design, making it too expensive for architectural exploration. The authors report an “accurate gate-level reference power estimation a total of 40-65 hours running at 2-3 cycles/s” of the BOOM processor (approx. 460K standard cells, https://slam.ece.utexas.edu/pubs/tc22.LACPo.pdf), compared with 7–9 hours using their learned models and Verilator-based simulation, a speedup of roughly 5–9x. The catch is that these savings come only after the models have been trained using expensive gate-level reference simulations, and it’s not clear these are accurate when exploring different implementations. The methodology is based on decision trees which are calibrated against gate-level power estimates from a particular synthesized and placed-and-routed design. Changes in synthesis, cell libraries, clock trees, physical implementation, etc. affect power consumption even when the microarchitectural functionality remains identical. The authors suggest that characterized blocks could be reused in different configurations but provide no evidence that this preserves accuracy. What they convincingly demonstrate is generalization across workloads running on the same implementation. Applicability to design-space exploration, where the hardware implementation itself is changing, is unclear.

The reported accuracy also deserves a closer look. The Mean Absolute Error (MAE) is normalized to the average power of the entire CPU, rather than averaging the percentage error of each individual cycle. Thus, the 3.6% MAE on CoreMark does not mean that power is predicted within 3.6% in every cycle. Also, accuracy varies strongly across individual blocks: the ALU has a 15% MAE, the issue unit 18%, and the memory issue unit 23%. Errors in individual blocks can partially cancel when aggregated, even though absolute errors cannot cancel across cycles in the MAE calculation. This raises questions about the accuracy of localized power estimates for identifying power hotspots and comparing alternative implementations. Nevertheless, the reported results are impressive for whole-core power prediction. It gives an idea of how AI-based models may transform power estimation, shifting much of the analysis from expensive gate-level simulations to fast, learned predictions.