Dynamic power management based on continuous-time Markov decision processes

By Qinru Qiu, Massoud Pedram. 1999

This paper introduces a continuous-time, controllable Markov process model of a power-managed system. The system model is composed of the corresponding stochastic models of the service queue and the service provider. The system environment is modeled by a stochastic service request process. The problem of dynamic power management in such a system is formulated as a policy optimization problem and solved using an efficient "policy iteration" algorithm. Compared to previous work on dynamic power management, our formulation allows better modeling of the various system components, the power-managed system as a whole, and its environment. In addition it captures dependencies between the service queue and service provider status. Finally, the resulting power management policy is asynchronous, hence it is more power-efficient and more useful in practice. Experimental results demonstrate the effectiveness of our policy optimization algorithm compared to a number of heuristic (time-out and N-policy) algorithms.

Subfields: ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In ML Systems & Hardware

1. Giovanni De Micheli
2. Luca Benini

Citations in the corpus

• Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classification (2011, background): “Thus DPM policies that are statically optimized (and are considered to be globally optimal for the modeled system) may in reality not achieve optimal performance in the presence of such uncertainties and variations.”
• A survey of design techniques for system-level dynamic power management (2000, background): “Continuous-time Markov models have been studied as well [37], [38], [40].”
• Energy Harvesting for Structural Health Monitoring Sensor Networks (2008, methodology): “An alternative to the DTMDP model is a continuous-time Markov decision process (CTMDP) model [57].”
• System-level power optimization: techniques and tools (2000, background): “Related work relates to industrial standards [77, 78], general approaches to power management [67, 68], predictive [83, 84, 85] and stochastic [87, 86, 87, 88] control policies.”
• Software Architecture Optimization Methods: A Systematic Literature Review (2013, methodology): “A considerable number of papers (20 percent of overall papers) use exact methods, most of which are standard optimization techniques such as Linear Programming [174], [193], [208], [211], [212], while some propose problemspecific exact methods based on knowledge or assumptions on the problem [8], [2”