soroush rastegarpour - Academia.edu (original) (raw)

Papers by soroush rastegarpour

IEEE Access, Dec 31, 2022

This paper presents a model predictive control (MPC)-based reinforcement learning (RL) approach f... more This paper presents a model predictive control (MPC)-based reinforcement learning (RL) approach for a home energy management system (HEMS). The house consists of an air-to-water heat pump connected to a hot water tank that supplies thermal energy to a water-based floor heating system. Additionally, it includes a photovoltaic (PV) array and a battery storage system. The HEMS is supposed to exploit the house thermal inertia and battery storage to shift demand from peak hours to off-peak periods and earn benefits by selling excess energy to the utility grid during periods of high electricity prices. However, designing such a HEMS is challenging because the discrepancies due to model mismatch make erroneous predictions of the system dynamics, leading to a non-optimal decision making. Besides, uncertainties in the house thermodynamics, misprediction in the forecasting of PV generation, outdoor temperature, and user load demand make the problem more challenging. We solve this issue by approximating the optimal policy by a parameterized MPC scheme and updating the parameters via a compatible delayed deterministic actor-critic (with gradient Q-learning critic, i.e., CDDAC-GQ) algorithm. Simulation results show that the proposed MPC-based RL HEMS can effectively deliver a policy that satisfies both indoor thermal comfort and economic costs even in the case of inaccurate model and system uncertainties. Furthermore, we conduct a thorough comparison between the CDDAC-GQ algorithm and the conventional twin delayed deep deterministic policy gradient (TD3) algorithm, the results of which affirm the efficacy of our proposed method in addressing complex HEMS problems. INDEX TERMS Model predictive control (MPC), reinforcement learning (RL), home energy management system (HEMS), inaccurate model, system uncertainties.

Social Science Research Network, 2022

Frontiers in Energy Research, Sep 2, 2022

This paper presents a comparative analysis of different modeling and control techniques that can ... more This paper presents a comparative analysis of different modeling and control techniques that can be used to tackle the energy efficiency and management problems in buildings. Multiple resources are considered, from generation to storage, distribution and delivery. In particular, it is shown what are the real needs and advantages of adopting different techniques, based on different applications, type of buildings, boundary conditions. This contribution is based widely on the experience performed by the authors in the recent years in dealing with existing residential, commercial and tertiary filed buildings, with application ranging from local temperature control up to smart grids where buildings are seen as an active node of the grid thanks to their ability to shape the thermal and electrical profile in real time. As for control models, a wide range of modeling techniques are here investigated and compared, from linear time-invariant models, to time-varying, to nonlinear ones. Similarly, control techniques include adaptive ones and real-time predictive ones.

Due to the growing energy demand in residential building, the need to reduce carbon footprint, an... more Due to the growing energy demand in residential building, the need to reduce carbon footprint, and the smart grid paradigm, thermal energy control and overall power consumption reduction have become a hot research topic. The development of an energy management system able to modify consumer's energy consumption patterns while preserving comfort is a substantial solution. Hence, for load shaping in demand side management, particularly useful is the usage of a thermal energy storage (TES). It gives the possibility to shape the demand profile in an economic way based on dynamic electricity tariffs, by storing energy in thermal terms during off-peak hours. This paper focuses on the development of a novel control model for the integration of TES, HVAC system, building and local renewable energy sources to be used with optimization techniques. The presented control framework is based on Model Predictive Control (MPC) to better anticipate the effects of disturbances (e.g. weather conditions and user requirements on the load side, electricity price, etc.). A distributed structure has also been considered, to follow the modular structure of the system under control with the aim of optimizing the energy consumption costs and improving the indoor comfort level. Furthermore, the novel configuration of TES coupled with a heat pump and a radiant floor building giving rise to a more complex model with respect to the literature ones.

Inverted pendulum system is one of popular and important laboratory models for teaching control s... more Inverted pendulum system is one of popular and important laboratory models for teaching control system engineering. This paper presents a multiple layers sliding mode controller and pole assignment controller and LQR for inverted pendulum system. In multiple layers sliding mode control, firstly, the given system is divided into several subsystems. Then, one subsystem is selected to construct the first layer sliding mode surface and it is used to construct the second layer sliding mode surface with the sliding mode surface of another subsystem. This process continues till all states of all subsystems are included. The controller is designed according to this multiple layers structure. For optimization of sliding surfaces constants are used genetic algorithm.

2022 IEEE 18th International Conference on Automation Science and Engineering (CASE), Aug 20, 2022

Thermal science and engineering progress, Dec 1, 2022

IFAC-PapersOnLine, 2020

This paper studies the impact of different types of energy storage integrated with a heat pump to... more This paper studies the impact of different types of energy storage integrated with a heat pump to improve energy efficiency in multiple radiant-floor buildings. In particular, the buildings and the heating generation system are decoupled through a 3-element mixing valve, which enforces a fixed flow rate but a variable temperature in the inlet water entering the building pipelines. The paper presents an optimal control formulation based on an Economic Nonlinear MPC scheme, in order to find the best compromise among different goals: make the heat pump work when it is more efficient, store electrical energy when it is cheap, store thermal energy in the tank when the heat pump is more effective, modulate the inlet water temperature to satisfy the user's comfort constraints, exploit the buildings thermal inertia. The nonlinearity of the system stems from the variable flow rate into the hot water tank due to the variable action of the mixing valve. The model is also time-varying due to the fact that the heat pump efficiency depends on external conditions. The simulation results show that the proposed optimal control algorithm is able to economically distribute energy among all storages in order to insure cost benefits (almost 20% electricity cost saving) and comfort satisfaction with the feasible computational effort.

Journal of Dynamic Systems Measurement and Control-transactions of The Asme, Jun 27, 2019

This paper studies the impact of using different types of energy storages integrated with a heat ... more This paper studies the impact of using different types of energy storages integrated with a heat pump for energy efficiency in radiant-floor buildings. In particular, the performance of the building energy resources management system is improved through the application of distributed model predictive control (DMPC) to better anticipate the effects of disturbances and real-time pricing together with following the modular structure of the system under control. To this end, the load side and heating system are decoupled through a three-element mixing valve, which enforces a fixed water flow rate in the building pipelines. Hence, the building temperature control is executed by a linear model predictive control, which in turn is able to exchange the building information with the heating system controller. On the contrary, there is a variable action of the mixing valve, which enforces a variable circulated water flow rate within the tank. In this case, the optimization problem is more complex than in literature due to the variable circulation water flow rate within the tank layers, which gives rise to a nonlinear model. Therefore, an adaptive linear model predictive control is designed for the heating system to deal with the system nonlinearity trough a successive linearization method around the current operating point. A battery is also installed as a further storage, in addition to the thermal energy storage, in order to have the option between the charging and discharging of both storages based on the electricity price tariff and the building and thermal energy storage inertia. A qualitative comparative analysis has been also carried out with a rule-based heuristic logic and a centralized model predictive control (CMPC) algorithm. Finally, the proposed control algorithm has been experimentally validated in a well-equipped smart grid research laboratory belonging to the ERIGrid Research Infrastructure, funded by European Union's Horizon 2020 Research and Innovation Programme.

The dynamic model of thermal zones has nonlinear characteristics, large inertia and time variabil... more The dynamic model of thermal zones has nonlinear characteristics, large inertia and time variability. Dominating these difficulties in order to increase overall energy efficiency in buildings and improve the comfort level of building occupants is a controversial issue nowadays. Although there are several traditional, simple and effective control strategies such as thermostatic valves on radiators (TVR) or PID controllers to cope with these difficulties, some alternative solutions may be more efficient. In this study, the dynamic model of a single zone including hydronic system will be formulated according to the energy balance equation of the wall, pavement, zone and pipelines. This model can be easily extended to large thermal zones for instance for commercial buildings. Then, the suboptimal second order Sliding Mode Control (SMC) will be implemented on the simplified dynamic model. Besides, first order SMC will be analyzed for comparison and cross-validation of the suboptimal method. At the end the application of sliding mode approach will be analyzed for a radiant floor model including mixing valve and return water as an uncertainty resource. The main purpose of this study is to formulate a reliable dynamic model for thermal zones and make a cross-validation on the implemented first order SMC and suboptimal second order SMC.

This paper aims at modeling and controlling an air-to-water heat pump for building energy efficie... more This paper aims at modeling and controlling an air-to-water heat pump for building energy efficiency applications. First, a detailed model of a generic heat pump, here called reference model, is developed and experimentally validated. Then, the reference model is used to formulate several control-oriented models of the heat pump, namely the ones defining the Coefficient of Performance (COP) based on increasing level of complexity. Finally, the paper explores the impact of the simplification level of the heat pump model on the overall quality of temperature control in a building, and on the electrical energy consumption. In particular, the pilot case here considered consists of a heat pump supplying water to a load through a hot water tank, and for control structure a linear time varying Model Predictive Control is designed. The impact of the power peaks is also investigated, which shows a significant improvement in the COP prediction based on the level of approximation. This study presents also how the load flexibility can take advantage from a correct COP prediction. The results can be seamlessly extended to the application of the real-time pricing, where the prediction of the COP trajectory can be used for the economic load shifting, thanks to the inertia of the hot water tank and the load.

The present paper focuses on the development of a control system strategy on medium size industri... more The present paper focuses on the development of a control system strategy on medium size industrial boilers (up to 1 MW) with the aim of having safe and efficient operation for the boiler itself. The class of the considered boiler is D-type water tube boiler. The basic plant model is based on Åström and Bell nonlinear dynamic model with simple adaptation due to specific geometries and physical constraints. The control system is mainly a combination of a pressure control loop and a three-element level controller. The pressure control loop here proposed consists of a gain scheduling PID control strategy to operate on heat power in order to keep the pressure at its desired value. The three-element level controller is a two-loop cascade control with feed forward water aimed at correcting the mismatch between the demand (steam flow) and feed water flow: level variation must be considered during this process because of the non-minimum phase behaviour of the level. Due to switching behaviour of gain scheduling approach, an adaptive control rule is also investigated in order to simplify the overall control structure and alleviate the adverse effects of the switching among many controllers in industrial applications.

This paper studies the problem of performance improvement and energy consumption reduction of the... more This paper studies the problem of performance improvement and energy consumption reduction of the heating, ventilation and air conditioning system of a large-scale university building through the application of nonlinear predictive control strategies concerning also practical and implementation issues. The system consists of two heat pumps, a water-to-water and an air-to-water type, and two different air handling units, which regulate and circulate air in all thermal zones. In such applications, prediction of the future dynamical behavior of the heat pumps is extremely important to enforce efficiency, but it is also very challenging due to the load dependency and nonlinearity of the coefficient of performances of those heat pumps. On the other hand, another source of potential model mismatch is the nonlinear characterization of the heat transfer coefficients of the AHU induced by variable air and water velocity, which gives rise to a non-trivial nonlinear system. To do so, two nonlinear model predictive control strategies are investigated to deal with many physical constraints and nonlinear problems. Finally, a sensitivity and robustness analysis are performed to highlight the merits, defects and impacts of those control algorithms on the energy performance of the building.

IEEE robotics and automation letters, Oct 1, 2020

Air-to-water heat pumps are quite often integrated with a hot-water tank, to better decouple the ... more Air-to-water heat pumps are quite often integrated with a hot-water tank, to better decouple the generation from the delivery of heat in buildings and to improve the overall performance of the system. The estimation and prediction of the coefficient of performance of the heat pump is extremely important to enforce efficiency, but it is also a very challenging task, due to the strong dependency of the performance on disturbances and operating conditions. Another source of potential model mismatch is the variable water flow rate in the condenser side induced by the heat pump low-level control logic, which gives rise to a non-trivial nonlinear system. In this letter, we tackle the problem to develop and properly tune an equivalent control-oriented model for the system, i.e. heat pump and tank, under variable flow rate conditions on both the condenser and load side, while still preserving good prediction capabilities of the model, with no tank temperature nor mass flow rate sensors. In particular, we focus on a real case study consisting in an air-to-water heat pump system and a $ 150{m^2}$ building located in SYSLAB, Department for Electrical Engineering, Risoe Campus, Denmark Technical University. The quality of the developed models is then evaluated through a nonlinear model predictive controller suitably designed and checked against detailed reference models previously developed. Finally, different sensitivity analyses are performed, which witness the robustness of the proposed algorithm.

Journal of Process Control, Mar 1, 2021

Abstract Air-to-water heat pumps are one of the most common and energy efficient heating systems ... more Abstract Air-to-water heat pumps are one of the most common and energy efficient heating systems for buildings, particularly floor-heating plants. One way to further improve their effectiveness is to control the heat pump exploiting the dependence of its coefficient of performance (COP) on the external temperature and temperature of the return water from the load. In particular, it is possible to exploit the heat pump when its efficiency is higher, so optimizing its performance in a predictive manner, anticipating the impact of external conditions. For the case of an air-to-water heat pump, the optimization problem is nonlinear due to the load dependence of the heat pump COP and variable supply water flow rate. This may pose implementation problems. If we address a standard control hardware, simplified optimal control formulations are more effective. In this paper, we specifically address this issue, and a reduced-order, linear, but adaptive time-varying predictive model of the heat pump COP is designed. Our solution takes into account the variation of the heat pump efficiency based on the external temperature and the load profile, which are changing within the control horizon. The proposed COP model is then used within a linear time-varying model predictive controller formulation which provides a prediction of the heat pump dynamical behavior based on the load dependence of the heat pump COP, while tackling the nonlinearities of the system imposed by the variable water flow rate in the hot water tank and also by the load dependence of the heat pump COP. The proposed approach has been implemented and in detail tested on a reference model based on a real case study from the Denmark Technical University, Riso Campus, SYSLAB. An intensive simulation analysis and complements the testing, showing the accuracy and the potential of the method, also in the perspective of practical implementation.

ABSTRACT The Purpose of this paper is based on Inverse Optimal Control method. There are two meth... more ABSTRACT The Purpose of this paper is based on Inverse Optimal Control method. There are two methods for designing optimal control stabilizer In order To minimize the presented cost function for any nonlinear systems, (Direct method and Inverse method). In direct method, in order to solve the optimization problem, HJB equation must be solved, in which there is no exist exact feasible mathematical techniques for solving Hamilton-Jacobi-Bellman equation, while in inverse method, by considering Control Lyapunov Function and an obtained feedback control law, a cost function will be designed so that the presented control law will be optimal for designed cost function. In this paper a new approach is presented so that there will be no need to solve the Hamilton-Jacobi-Bellman equation and the suboptimal controller will be designed without numerical method. In this approach, by using Inverse Optimal Control method and determining cost function for the system, Control Lyapunov Function and suboptimal control law are designed simultaneously in which the Control Lyapunov Function will be designed by intelligent algorithm such as PSO algorithm and GA separately. To analyze the optimal level of designed controller and Control Lyapunov Function, different performance criteria will be used. In this method any structure of Control Lyapunov Function could be considered.

Control Engineering Practice, Feb 1, 2020

A modulating heat pump and water tank result in a nonlinear model due to the load dependency of t... more A modulating heat pump and water tank result in a nonlinear model due to the load dependency of the heat pump performance, and variable water flows. Nonlinear model predictive control is an effective way to deal with many physical constraints and nonlinear formulations. Alternatively, linear time-varying MPC can be used, based on successive linearizations around a reference trajectory. The goal of the paper is to analyze the advantages and disadvantages of those MPC techniques for temperature control in radiantfloor buildings. The results show that nonlinear on-line optimization is real-time feasible for the application considered here, as the slow dynamics allows for a fairly long sampling time. Alternatively, the linear timevarying MPC approach shows a significantly better performance compared to the Standard MPC scheme if a feasible reference trajectory is provided. Nonlinear MPC can save up to 6% energy and improve the comfort by 4% with respect to Standard MPC for the given application, while its difference with LTV-MPC is negligible. Moreover, a robustness analysis has been conducted, showing the impact of the heat pump efficiency on the control performance.

Frontiers in Energy Research

2022 IEEE 18th International Conference on Automation Science and Engineering (CASE)