Data-Driven Energy Management for Renewable Energy Communities: Leveraging Smart Monitoring and Machine Learning for Grid Flexibility

The paradigm shift necessitated by the global imperative for decarbonization and for energy security is fundamentally reshaping the architecture of electric power systems, transitioning from a centralized, dispatchable model to a decentralized ecosystem largely influenced by stochastic Renewable Energy Sources (RES). This structural transformation has caused a critical "flexibility gap," characterized by the increasing divergence between variable generation profiles and inelastic demand, which threatens grid stability at both transmission and distribution levels. This thesis addresses this challenge by proposing an integrated, data-driven framework designed to unlock Demand-Side Flexibility (DSF) within the emerging context of Renewable Energy Communities (RECs), carrying out evaluations based on the acquisition and processing of real data from different types of case studies. Against the backdrop of the evolving European and Italian regulatory landscapes, specifically the establishment of Configurations for Self-Consumption and Sharing of Renewable Energy and of Local Flexibility Markets (MLF), the technological enablement of Distributed Energy Resources (DERs) constitutes a fundamental necessity for effective market participation. While the regulatory architecture establishes the principle of technological neutrality, the operational reality requires scalable platforms capable of bridging the gap between physical assets and market mechanisms through robust observability and control. To overcome these barriers, the thesis presents a multi-layered methodology focused on the enabling technologies for aggregation and flexibility. First, it investigates diverse advanced energy monitoring architectures suitable for heterogeneous environments, ranging from complex infrastructures to residential units. Within this monitoring framework, specific attention is dedicated to Non-Intrusive Load Monitoring (NILM) as a promising technique to enhance data granularity where sub-metering is economically inconvenient, particularly in the residential sector. This data acquisition layer serves as the foundation for a data-driven Energy Management System (EMS), which integrates Machine Learning-based forecasting models with optimization algorithms. The research culminates in the design and development of an intelligent platform for the optimized management of energy resources, capable of real-time control over the dispatch of flexible assets. By maximizing shared energy incentives while simultaneously offering ancillary services to the grid, this work provides a technical pathway for making RECs active nodes within the smart grid ecosystem.