We demonstrate TimeTravel, an efficient DBMS system for seamless integrated querying of past and (forecasted) future values of time series, allowing the user to view past and future values as one joint time series. This functionality is important for advanced application domain like energy. The main idea is to compactly represent time series as models. By using models, the TimeTravel system answers queries approximately on past and future data with error guarantees (absolute error and confidence) one order of magnitude faster than when accessing the time series directly. In addition, it efficiently supports exact historical queries by only accessing relevant portions of the time series. This is unlike existing approaches, which access the entire time series to exactly answer the query.
To realize this system, we propose a novel hierarchical model index structure. As real-world time series usually exhibits seasonal behavior, models in this index incorporate seasonality. To construct a hierarchical model index, the user specifies seasonality period, error guarantees levels, and a statistical forecast method. As time proceeds, the system incrementally updates the index and utilizes it to answer approximate and exact queries. TimeTravel is implemented into PostgreSQL, thus achieving complete user transparency at the query level. In the demo, we show the easy building of a hierarchical model index for a real-world time series and the effect of varying the error guarantees on the speed up of approximate and exact queries.

Integrating sophisticated statistical methods into database management systems is gaining more and more attention in research and industry in order to be able to cope with increasing data volume and increasing complexity of the analytical algorithms. One important statistical method is time series forecasting, which is crucial for decision making processes in many domains. The deep integration of time series forecasting offers additional advanced functionalities within a DBMS. More importantly, however, it allows for optimizations that improve the efficiency, consistency, and transparency of the overall forecasting process. To enable efficient integrated forecasting, we propose to enhance the traditional 3-layer ANSI/SPARC architecture of a DBMS with forecasting functionalities. This article gives a general overview of our proposed enhancements and presents how forecast queries can be processed using an example from the energy data management domain. We conclude with open research topics and challenges that arise in this area.

Nowadays, Renewable Energy Sources (RES) are attracting more and more interest. Thus, many countries aim to increase the share of green energy and have to face with several challenges (e.g., balancing, storage, pricing). In this paper, we address the balancing challenge and present the MIRABEL project which aims to prototype an Energy Data Management System (EDMS) which takes benefit of flexibilities to efficiently balance energy demand and supply. The EDMS consists of millions of heterogeneous nodes that each incorporates advanced components (e.g., aggregation, forecasting, scheduling, negotiation). We describe each of these components and their interaction. Preliminary experimental results confirm the feasibility of our EDMS.

The contribution describes the demand response solution, which was developed within the international project Mirabel and cofounded by the European 7th Framework Program. The main goal of the project is to provide a solution, which controls the peak consumption and provide a more efficient integration of the renewable sources into the electricity grid. The uniqueness of the solution is the economic concept, where the consumer makes a closed energy supply contract for that part of its consumption, which is put under the control of the external party - Balance responsible party. That approach is in relation of the energy transfer on the organized market, which is close to the BRP's energy handling practice. The algorithms of the data aggregation, consumption and production forecasting and energy flow scheduling developed within the project enable the integration of the large amount of consumers into the Mirabel solution. The preliminary results presented are generated during the testing phase of the project, which is being executed for the time of the generating this paper. The testing is being executed in the simulation environment and with the real data provided by the utility project partner.

The trial cases used for the testing of the Mirabel product follows the strategic impact defined in the Description of Work. It is expected that within the mass of the electricity consumers the Mirabel solution will enable increase the integrated share of RES in EES by 5% and reduce the peak demand for at least 5%, with a targeted approximate 8-9% for the total grid.
The delivery describes and resolves the issues necessary to execute the testing and provide the laboratory evaluation of the Mirabel solution. That includes description of the scenarios for the trial cases, definition of the testing environment and the definition of the successfulness criteria.

Use of renewable energy sources is enforced by national and international regulations.
Drivers for such policies include mitigation of climate change due to emission of
greenhouse gasses and reducing dependency on fossil fuel reserves. Due to the
intermittent character of renewable energy sources such as photovoltaic or wind power,
integration of such sources creates a challenge in maintaining balance between demand
and supply. Indications of such challenges in countries with e.g. a high penetration of
wind power are already showing in prices on power exchanges reaching zero or negative
energy prices. In general, without mitigation measures, an increase in the use of
intermittent renewable energy sources leads to a diminished ability to guarantee security
of supply.

The aim of the MIRABEL project is to exploit potential flexibility in both demand for and
generation of electricity to compensate for the intermittent nature of Renewable Energy
Sources (RES). For instance by postponing certain electricity demand until energy from a
RES (e.g. solar panel) becomes available.

In order to leverage flexibility in demand and generation means for expressing and
communicating this flexibility are needed. For this purpose, this MIRABEL deliverable
provides a data model. This document contains the final version of the data model, a
previous version of the data model was presented in deliverable D2.2 which was
published in December 2010.

Forecasting is an important analysis technique to support decisions and functionalities in many application domains. While the employed statistical models often provide a sufficient accuracy, recent developments pose new challenges to the forecasting process. Typically the available time for estimating the forecast models and providing accurate predictions is significantly decreasing. This is especially an issue in the energy domain, where forecast models often consider external influences to provide a high accuracy. As a result, these models exhibit a higher number of parameters, resulting in increased estimation efforts. Also, in the energy domain new measurements are constantly appended to the time series, requiring a continuous adaptation of the models to new developments. This typically involves a parameter re-estimation, which is often almost as expensive as the initial estimation, conflicting with the requirement for fast forecast computation. To address these challenges, we present a framework that allows a more efficient integration of external information. First, external information are handled in a separate model, because their linear and non-linear relationships are more stable and thus, they can be excluded from most forecast model adaptations. Second, we directly optimize the separate model using feature selection and dimension reduction techniques. Our evaluation shows that our approach allows an efficient integration of external information and thus, an increased forecasting accuracy, while reducing the re-estimation efforts.

The so-called smart grid is emerging in the energy domain as a solution to provide a stable, efficient and sustainable energy supply accommodating ever growing amounts of renewable energy like wind and solar in the energy production. Smart grid systems are highly distributed, manage large amounts of energy related data, and must be able to react rapidly (but intelligently) when conditions change, leading to substantial real-time business intelligence challenges. This paper discusses these challenges and presents data management solutions in the European smart grid project \Mirabel. These solutions include real-time time series forecasting, real-time aggregation of the flexibilities in energy supply and demand, managing subscriptions for forecasted and flexibility data, efficient storage of time series and flexibilities, and real-time analytical query processing spanning past and future (forecasted) data. Experimental studies show that the proposed solutions support important real-time business intelligence tasks in a smart grid system.

Time series forecasting is challenging as sophisticated forecast models are computationally expensive to build. Recent research has addressed the integration of forecasting inside a DBMS. One main benefit is that models can be created once and then repeatedly used to answer forecast queries. Often forecast queries are submitted on higher aggregation levels, e.g., forecasts of sales over all locations. To answer such a forecast query, we have two possibilities. First, we can aggregate all base time series (sales in Austria, sales in Belgium ...) and create only one model for the aggregate time series. Second, we can create models for all base time series and aggregate the base forecast values. The second possibility might lead to a higher accuracy but it is usually too expensive due to a high number of base time series. However, we actually do not need all base models to achieve a high accuracy, a sample of base models is enough. With this approach, we still achieve a better accuracy than an aggregate model, very similar to using all models, but we need less models to create and maintain in the database. We further improve this approach if new actual values of the base time series arrive at different points in time. With each new actual value we can refine the aggregate forecast and eventually converge towards the real actual value. Our experimental evaluation using several real-world data sets, shows a high accuracy of our approaches and a fast convergence towards the optimal value with increasing sample sizes and increasing number of actual values respectively.

Flexibility objects, objects with flexibilities in time and amount dimensions (e.g., energy or product amount), occur in many scientic and commercial domains. Managing such objects with existing DBMSs is infeasible due to the complexity, data volume, and complex functionality needed, so a new kind of flexibility database is needed. This paper is the first to consider flexibility databases. It formally defines the concept of flexibility objects (flex-objects), and provide a novel and efficient solution for aggregating and disaggregating flex-objects. This is important for a range of applications, including smart grid energy management. The paper considers the grouping of flex-objects, alternatives for computing aggregates, the disaggregation process, their associated requirements, as well as efficient incremental computation. Extensive experiments based on data from a real-world energy domain project show that the proposed solution provides good performance while still satisfying the strict requirements.

Forecasting is an important analysis technique used in many application domains such as electricity management, sales and retail and, traffic predictions. The employed statistical models already provide very accurate predictions, but recent developments in these domains pose new requirements on the calculation speed of the forecast models. Especially, the often used multi-equation models tend to be very complex and their estimation is very time consuming. To still allow the use of these highly accurate forecast models, it is necessary to improve the data processing capabilities of the involved data management systems. For this purpose, we introduce a partitioning approach for multi-equation forecast models that considers the specific data access pattern of these models to optimize the data storage and memory access. With the help of our approach we avoid the redundant reading of unnecessary values and improve the utilization of the CPU cache. Furthermore, we utilize the capabilities of modern multi-core hardware and parallelize the model estimation. Our experimental results on real-world data show speedups of up to 73x for the initial model estimation. Thus, our partitioning and parallelization approach significantly increases the efficiency of multi-equation models.

Integrating sophisticated statistical methods into database management systems is gaining more and more attention in research and industry. One important statistical method is time series forecasting, which is crucial for decision management in many domains. In this context, previous work addressed the processing of ad-hoc and recurring forecast queries. In contrast, we focus on subscription-based forecast queries that arise when an application (subscriber) continuously requires forecast values for further processing. Forecast queries exhibit the unique characteristic that the underlying forecast model is updated with each new actual value and better forecast values might be available. However, (re-)sending new forecast values to the subscriber for every new value is infeasible because this can cause signicant overhead at the subscriber side. The subscriber therefore wishes to be notied only when forecast values have changed relevant to the application. In this paper, we reduce the costs of the subscriber by optimizing the notications sent to the subscriber, i.e., by balancing the number of notifications and the notification length. We introduce a generic cost model to capture arbitrary subscriber cost functions and discuss dierent optimization approaches that reduce the subscriber costs while ensuring constrained forecast values deviations. Our experimental evaluation on real datasets shows the validity of our approach with low computational costs.

In the MIRABEL project, a data management system for a smart grid is developed to enable smarter scheduling of energy consumption such that, e.g., charging of car batteries is done during night when there is an overcapacity of green energy from windmills etc. Energy can then be requested by means of flexoffers which define flexibility with respect to time, amount, and/or price. In this paper, we describe MIRABEL DW, a data warehouse (DW) for the management of the large amounts of complex energy data in MIRABEL. We present a unified schema that can manage data both at the level of the entire electricity network and at the level of individual nodes, such as a single consumer node. The schema has a number of complexities compared to typical DW schemas. These include facts about facts and composed non-atomic facts and unified handling of different kinds of flex-offers and time series. We also discuss alternative data modeling strategies and present typical queries from the energy domain and a performance study.

Jack Verhoosel, Diederik Rothengatter, Frens-Jan Rumph and Mente Konsman. In: 3rd Workshop on eeBuildings Data Models (Energy Efficiency Vocabularies), July 2012.

To address the needs of rapidly changing energy markets, an energy data management system capable of supporting higher utilization of renewable energy sources is being developed. The system receives flexible offers from producers and consumers of energy, aggregates them on a regional level and schedules the aggregated flexible offers to balance forecast energy supply and demand. This paper focuses on formulating and solving the optimization problem of scheduling aggregated flexible offers within such a system. Three metaheuristic scheduling algorithms (a randomized greedy search, an evolutionary algorithm and a hybrid between the two) tailored to this problem are introduced and their performance is assessed on a benchmark test problem and two realistic problems. The best results are achieved by the evolutionary algorithms, which can efficiently handle thousands of aggregated flex-offers.

Changing electricity markets call for new ways of handling supply and demand. The desired goal is to increase the utilization of renewable energy while ensuring reliable supply and minimizing the costs. We present an approach aiming at this goal that handles a large number of flexible energy offers from producers and consumers by aggregating them and scheduling these aggregates to minimize a cost function. We explore the influence of aggregation on the performance of scheduling, establishing that a trade-off between keeping the flexibilities of flexible offers and reducing their number is what yields the best results.

Modern demand-side management techniques are an integral part of the envisioned smart grid paradigm. They require an active involvement of the consumer for an optimization of the grid's efficiency and a better utilization of renewable energy sources. This applies especially in so called demand dispatch systems, where consumers are required to proactively communicate their flexibilities. However, a monetary compensation may not sufficiently motivate the individual consumer for a sustainable participation in such a program. The proposed approach uses a motivational framework leveraging the novel area of gamification, which applies well-known game mechanics, such as points and leaderboards, to engage customers in the system. This is accomplished by embedding a special scoring system and social competition aspects into a stimulating user interface for the definition and management of flexible energy demand. In a first user study, the system showed a high user acceptance and the potential to engage consumers in participation.

Rapidly changing electricity markets call for innovative solutions to support balancing of energy production and consumption, and utilize the increasing amount of energy from renewable sources. MIRABEL is a future energy data management system based on flexible offers (flex-offers) for energy production and consumption. One of its core functionalities is scheduling of aggregated flex-offers to minimize the costs of the balance responsible party. This paper presents a formulation of this scheduling problem in terms of decision variables, constraints and the objective function, and discusses the problem characteristics.

The use of renewable energy sources is increasing due to national and international regulations. Such energy sources are less predictable than most of the classical energy production systems, like coal and nuclear power plants. This causes a challenge for balancing the electricity system. A possibility to meet this challenge is to use the flexibility in electricity demand for balancing with unpredictable electricity supply. In this paper, we introduce a model for flexibility in electricity demand and supply and give some examples of flexibility models for existing devices. In addition, we describe the business advantages for using flexibility and some pricing mechanisms that provide financial incentives for using flexibility by the consumer and the balance responsible party in the grid.

The two smart grid development projects - Kibernet and Mirabel - are used to expose the problems and unresolved issues of the demand side management (DSM) usage from several aspects. The article describes two different approaches of automatic controlling of the demand and consumer communication. Beside the technical solution the article exposes the problems at defining the user of the DSM system in the EE scheme and its economic justification. Separate and significant problems are also the social problems of accepting the system by the end consumers.

Forecasting of time series data is crucial for decision-making processes in many domains as it allows the prediction of future behavior. In this context, a model is fit to the observed data points of the time series by estimating the model parameters. The computed parameters are then utilized to forecast future points in time. Existing approaches integrate forecasting into traditional relational query processing, where a forecast query requests the creation of a forecast model. Models of continued interest should be deployed only once and used many times afterwards. This however leads to additional maintenance costs as models need to be kept up-to-date. Costs can be reduced by choosing a well-defined subset of models and answering queries using derivation schemes. In contrast to materialized view selection, model selection opens a whole new problem area as results are approximate. A derivation schema might increase or decrease the accuracy of a forecast query. Thus, a two-dimensional optimization problem of minimizing the model cost and model usage error is introduced in this paper. Our solution consists of a greedy enumeration approach that empirically evaluates different configurations of forecast models. In our experimental evaluation, with data sets from different domains, we show the superiority of our approach over traditional approaches from forecasting literature.

Continuous balancing of energy demand and supply is a fundamental prerequisite for the stability and efficiency of energy grids. This balancing task requires accurate forecasts of future electricity consumption and production at any point in time. For this purpose, database systems need to be able to rapidly process forecasting queries and to provide accurate results in short time frames. However, time series from the electricity domain pose the challenge that measurements are constantly appended to the time series. Using a naive maintenance approach for such evolving time series would mean a re-estimation of the employed mathematical forecast model from scratch for each new measurement, which is very time consuming. We speed-up the forecast model maintenance by exploiting the particularities of electricity time series to reuse previously employed forecast models and their parameter combinations. These parameter combinations and information about the context in which they were valid are stored in a repository. We compare the current context with contexts from the repository to retrieve parameter combinations that were valid in similar contexts as starting points for further optimization. An evaluation shows that our approach improves the maintenance process especially for complex models by providing more accurate forecasts in less time than comparable estimation methods.

The energy sector is in transition. Firstly, the liberalisation process forces companies to restructure their value chain in order to increase their market efficiency. Secondly, in order to reduce carbon emissions, the use of renewable energy sources is enforced by national and international regulations. Thirdly, smart metering is being widely adopted and with it consumers will be involved actively. The main goal of the project is to develop an ICT system that fits the future liberalised energy sector and enables the integration of a higher rate of distributed and renewable energy sources into the electricity grid. We will explore an approach for demand (and supply) side management in which electricity consumers and producers issue flex-offers indicating flexibilities in time and amount of the electricity. These flex-offers will be processed by our system in order to balance electricity supply and demand in near real-time and thus allow to use not-schedulable renewable energy sources much better.

This document describes the strategy and instruments to be used for disseminating and exploiting results obtained in the MIRACLE project.

Deregulation of energy markets and environmental sustainability pose major challenges for modern energy systems. As increased efficiency and flexibility are sought at all levels, new services are needed to ensure reliable supply, utilize the renewable energy sources (RES), and balance the costs and benefits of the involved parties. Information and communication technology (ICT) plays a key role in pursuing these goals. In this context, the FP7 project MIRACLE (Micro-Request-Based Aggregation, Forecasting and Scheduling of Energy Demand, Supply and Distribution) is developing a conceptual and infrastructural approach allowing electricity distributors to balance supply and demand and increase the amounts of energy from RES.

Most renewable energy sources (RES; e.g., windmills or solar panels) pose the challenge that the production depends on external factors such as wind speed and the amount of sunlight. Hence, available power from RES cannot be planned as traditional energy sources. As a result, there is the need for balancing energy demand and supply in order to integrate more renewable energy sources. Accurate and efficient forecasts for short-term and mid-term horizons of energy consumption and production are a fundamental precondition for this dynamic and fine-grained scheduling of energy demand and supply. The state-of-the-art of forecasting energy demand and supply mainly focus on high accuracy forecasting of energy demand, while only few techniques exist for energy supply. In addition, there are further challenges. First, with regard to balancing energy demand and supply, forecasting takes place in a distributed system architecture that is inherently given by the hierarchy of involved organizations. Second, the large scale of the distributed system, in combination with a continuous stream of updates, leads to the requirement of efficient forecasting and forecast model maintenance for evolving time series. Beside these general challenges, we observe the need for (1) awareness of specified flexibilities and (2) integration of external information sources in order to achieve high accuracy forecast results.

The new EU directive sets ambitious targets for all EU countries, such as to reach 20% share of energy from renewable sources by 2020. Many countries which started implementing this directive are facing the same problem - the production from renewable energy sources (RES, e.g., windmills, solar panels) cannot be planned, but can only be predicted. Thus, electrical power produced by RES usually does not match the energy consumption, and it must be discarded or given away for free.

The aim of the MIRACLE project is to exploit potential flexibility in both demand for and generation of electricity to compensate for the intermittent nature of Renewable Energy Sources (RES). For instance by postponing certain electricity demand until energy from a RES (e.g. solar panel) becomes available.

In order to leverage flexibility in demand and generation means for expressing and communicating this flexibility are needed. For this purpose, this MIRACLE deliverable provides a data model. This document contains the first version of the data model; a subsequent deliverable D2.3 will follow in M18 of the MIRACLE project and will provide further enhancements and elaborations.

The energy sector is in transition. Firstly, the deregulation process forces companies to restructure their value chain in order to increase their market efficiency. Secondly, in order to reduce carbon emissions, the use of renewable energy sources is enforced by national and international regulations. Thirdly, smart metering is being widely adopted. The main goal of the project is to develop an Information Communication Technology (ICT) system that fits the future deregulated energy sector and enables the integration of a higher rate of distributed and renewable energy sources into the electricity grid. We will explore an approach for demand (and supply) side management in which electricity consumers and producers issue flex-offers indicating flexibilities in time and amount of the electricity. These flex-offers will be processed by our system in order to balance electricity supply and demand in near real-time.

Most renewable energy sources (RES; e.g., windmills or solar panels) pose the challenge that the production depends on external factors such as wind speed and the amount of sunlight. Hence, available power from RES cannot be planned as traditional energy sources. As a result, there is the need for balancing energy demand and supply in order to integrate more renewable energy sources. Accurate and efficient forecasts for short-term and mid-term horizons of energy consumption and production are a fundamental precondition for this dynamic and fine-grained scheduling of energy demand and supply. The state-of-the-art of forecasting energy demand and supply mainly focus on high accuracy forecasting of energy demand, while only few techniques exist for energy supply. In addition, there are further challenges. First, with regard to balancing energy demand and supply, forecasting takes place in a distributed system architecture that is inherently given by the hierarchy of involved organizations. Second, the large scale of the distributed system, in combination with a continuous stream of updates, leads to the requirement of efficient forecasting and forecast model adaptation.

The energy sector is in transition. First, the deregulation forces companies to restructure their value chain in order to increase their market efficiency. Second, to reduce carbon emissions, the use of renewable energy sources is enforced by national and international regulations. Third, smart metering is being widely adopted. The main goal of the MIRACLE project is to develop an ICT system that will enable the integration of a higher rate of distributed and renewable energy sources into the electricity grid by using flexibilities in energy demand and supply. The system will provide the means to issue so-called micro-requests indicating these power profile flexibilities (e.g., shifting in time or changing the energy amount) and to process the micro-request data in a hierarchical fashion. Consumers and producers own appliances and devices such as electric heat pumps, electric vehicles, washing machines, dishwashers, photovoltaic cells, urban wind turbines and micro combined heat and power units. The developed system enables customers and energy companies to balance energy demand and supply in near real-time and thus, allows the integration of more renewable energy sources whose availability cannot be influenced.

This  document  presents  a  roadmap  towards  the  standardisation  of  the  MIRACLE specifications  for  the  exchange  of  information  around  management  of  energy  demand and supply. A long list of 15 potential standardisation organisations has been set-up and briefly  described  with  respect  to  three  criteria:  type  of  standardisation  organisation,  its geographic  scope  and  focus  on  energy  management.  From  this  list  a  short  list  of  4 different  standardisation  organisations  has  been  selected.  These  are  CEN/CENELEC, IEC,  IEEE  and  the  combination  of  ebIX  and  ENTSO-E.  For  these  organisations,  an additional four criteria have been described, namely openness of organisation, complexity of  procedures,  impact on  the  energy  sector  and  potential  success  of  standardisation  of the MIRACLE specifications.
 
Based  on  this  set  of  seven  criteria  in  total,  CEN/CENELEC  has  been  selected  as  the target  standardisation  organisation.  The  MIRACLE  project  will  strive  to  get  a  so-called CEN  Workshop  Agreement  (CWA)  as  the  standardisation  product  at  the  end  of  the project. The roadmap towards this CWA is worked-out in more detail and the coordination with  CEN/CENELEC  during  the  project  is  described.  Thereby,  the  CWA  process  is aligned  with  the  original  planning  of  the  WP7  deliverables  of  MIRACLE.  The  resulting CWA  can  be  used  after  the  lifetime  of  the  MIRACLE  project  as  a  basis  for  further standardisation  towards  a  European  Norm  (EN),  which  is  the  highest  level  of  formal standardisation to be achieved within Europe.
 
The  main  conclusion  and  advice  is  that  the  MIRACLE  project  should  adhere  to  the
roadmap sketched in this document and try to align the WP7 activities during M18 and
M36 with a CWA process at CEN/CENELEC. [read more]

Authors:
Jack P.C. Verhoosel, Roel E. Stap, TNO

Integration, testing and validation of the developed Miracle methodologies and algorithms will be performed in two testing environments. In one, an important part will be simulation based data analyses of the MeRegio project; and in the other, the running of the test scenarios on in the CRES test bed.

The MeRegio project plans to set up the pilot region with 1,000 consumers, with the goal to minimize the CO2 emissions at energy supply. A side result of the project will be also substantial amount of the measured energy consumption data, which shall be used as an input for the testing and validation of the Miracle project.

The CRES test bed provides the necessary facilities and resources like RES, measurement equipment and corresponding infrastructure to implement and test the Miracle features in a controlled environment.

Both in the development as well as in the testing phase of the project, the approaches, solutions and experiences of other similar projects can be of benefit. In its second part, the report briefly describes the projects in the same general field as the Miracle project, and which are now in the developing or concluding phases and which - to the extent that the concepts and solutions are already formulated and available for interested parties outside the project partners - could be used as references for state-of-the-art in the neighboring and cross-section fields with Miracle. The projects identified, shortly presented and their intersection with Miracle discussed, are: FENIX, EU DEEP, AEOLUS, MORE MICROGRIDS, ADDRESS, EDISON, DLC-VIT4IP and Smart House/Smart Grid. [read more]

Authors:
Hellmuth Frey, EnBW; Stathis Tselepis, Evangelos Rikos, CRES; Matjaž Bobnar, INEA; Torben Bach Pedersen, AAU; Matthias Boehm, TUD; Henrike Berthold, SAP; Gregor Černe, Zoran Marinšek, INEA

Energy market deregulation and environmental sustainability increase the need for efficiency and flexibility of energy systems. New services are sougt to ensure reliable supply, utilize the renewable energy sources (RES), and balance the costs and benefits of the involved parties. In this context, the European FP7 project MIRACLE (Micro-Request-Based Aggregation, Forecasting and Scheduling of Energy Demand, Supply and Distribution) proposes a conceptual and infrastructural approach allowing electricity distributors to manage higher amounts of renewable energy and balance supply and demand. For this purpose, MIRACLE introduces the concept of micro-requests that allow consumers and producers to specify flexibilities of their energy profiles in terms of the energy amounts and their time shifts. Such micro-requests from numerous consumers and producers will enable fine-grained scheduling of consumption and production of electricity, and maintaining a system-wide balance between demand and supply.

Work Package 5 (WP5) of the MIRACLE project deals with scheduling and negotiation in the proposed approach. Based on the forecast of energy supply and demand, negotiation will take place to determine how and when consumption and production can be matched, and a schedule for production and consumption will be determined. The goals of WP5 are to specify a framework to schedule production and consumption for the forthcoming period, specify a negotiation framework, implement and integrate the two frameworks, and validate them on real data from the project trial cases.

This deliverable is a result of the WP5 preparatory phase and reports on the state-of-the-art in scheduling and negotiation approaches. Regarding scheduling, it first presents a common type of scheduling problems together with their properties, and introduces some characteristic aspects of the scheduling domain. It then focuses on scheduling in energy sector where it identifies particular problems: generation scheduling, unit commitment and economic dispatch. Finally, it reviews methods applied in solving scheduling problems in energy sector, including deterministic and meta-heuristic techniques, and with a special attention to the approaches for deregulated markets. The state-of-the art survey on negotiation approaches starts with an introduction to negotiations and two negotiation types: bilateral contracts and auctions. Energy exchange auctions are then described with the focus on hourly bids, block bids, pricing and trading phases. Examples of multi-agent negotiation systems are then presented, taken from related projects and the literature. The report concludes with comments related to further work on both scheduling and negotiation in MIRACLE. [read more]

Authors:
Bogdan Filipič, Erik Dovgan, JSI; Alexandr Savinov, SAP

In the energy domain, one of the most important goals is the integration of more renewable energy sources (RES, e.g., windmills, solar panels). Unfortunately, most RES depend on external factors such as wind speed and the amount of sunlight. Hence, available power from RES cannot be planned as traditional energy sources and thus, there is the need of balancing energy demand and supply. We address this requirement within the MIRACLE (Micro-Request-Based Aggregation, Forecasting and Scheduling of Energy Demand, Supply and Distribution) project with a micro-request-based approach, where acceptable flexibilities (e.g., timeshifts or variable amount) can be specified by consumers and producers along with concrete energy demand and supply, respectively. These flexibilities allow for fine-grained scheduling and thus, balancing of demand and supply. Accurate and efficient forecasts for short-term and long-term horizons of energy consumption and production as well as for requests with timeshifts are a fundamental precondition for dynamic and fine-grained scheduling of energy demand and supply.

In this survey, we give a detailed overview of forecast models for energy demand and supply. First, we reveal typical data characteristics of energy demand and supply. Second, we describe the mathematical background of time series forecasting in general. Furthermore, we review existing domain-specific techniques of forecasting energy demand, supply and prices as well as how these techniques can be integrated into a system architecture of a data management system. Third, we select representative forecast models from the main categories of existing techniques and evaluate their accuracy with regard to different time horizons. Fourth, we identify major challenges and open problems that should be addressed in order to enable accurate and efficient forecasting as a fundamental prerequisite for scheduling energy demand and supply. Finally, the scheduling of energy demand and supply will allow (1) to smoothen cost-extensive peaks, (2) to integrate more renewable energy sources, and (3) to balance energy demand and supply. [read more]

Authors:
Lars Dannecker, SAP; Matthias Boehm, Ulrike Fischer, Frank Rosenthal, TUD; Gregor Hackenbroich, SAP; Wolfgang Lehner, TUD

Today, many countries aim to increase the share of energy consumed that comes from renewable sources. Unfortunately, the electrical power produced from weather-dependent renewable energy sources (RESs; e.g., wind turbines, solar panels) is produced in varying quantities that do not match the varying energy needs. As more and more such renewable energy becomes available, it becomes an increasingly difficult challenge to maintain an energy system that enables the effective use of all available renewable energy. Consequently, tackling this problem is one of the top goals in the energy domain.

The Miracle (Micro-Request-Based Aggregation, Forecasting and Scheduling of Energy Demand, Supply and Distribution) project aims to invent and prototype key elements of an energy system that is better able to accommodate large volume of electricity from renewable sources. The approach, taken is based on micro-requests that allow an individual consumer/producer to specify acceptable flexibilities in the amounts of energy consumed and the times when this is done. The introduction of such micro-requests from millions of consumers/producers enables fine-grained scheduling of consumption and production of electricity while maintaining a system-wide balance between demand and supply. In order to appropriately manage very large volumes of micro-requests, a reliable, distributed, and highly scalable computer  system infrastructure is needed.

This deliverable concerns Work Package 3, "State-of-the-art report on data collection and analysis" in the Miracle project. We first introduce Miracle's application scenario along with the consequent requirements to the data management infrastructure. Then we survey the state-of-the-art of relevant, existing work on data collection, data integration, query processing, and query optimization from the perspective of the project's requirements. Specifically, the survey covers the following key topics: (1) virtually and materialized integrated systems, including column stores; (2) data exchange solutions, including ETL tools, EAI servers, and data stream management systems; (3) web-scale data management; (4) management of uncertainty in the context of probabilistic databases, OLAP, and data streams; (5) management of  multi-version data; (6) efficient tracking of continuous processes; and (7) query optimization based on early aggregation and materialized views. Moreover, relevant existing computer systems in the energy domain are covered. For all technologies surveyed, the relation to Miracle is discussed. [read more]

Authors:
Laurynas Šikšnys, AAU; Matthias Boehm, TUD; Torben B. Pedersen, Christian S. Jensen, Dalia Martišiuté, AAU

This document provides an overview of the state of the art that is relevant for WP2 of the Miracle project; Data Specification. There are two subjects that form the main focus of this state of the art report; modeling approaches and existing models. Three modeling approaches are described; Unified Modeling Language (UML), UN/CEFACT's Modeling Methodology (UMM) and Object Role Modeling (ORM). For the existing models, two international standard organizations are relevant; ebiX and IEC. ebiX has developed models for Customer Switching Process and for the Exchange of Metered Data. These models describe the business processes and the corresponding message definitions and do so by using the aforementioned UN/CEFACT's Modeling Methodology. The Common Information Model (CIM) is a data model by the IEC that aims to describe all major objects that an electric utility enterprise is typically involved with.

The following conclusions are drawn:

• UMM is the methodology of choice for the development of the WP2 deliverables. It is an international standard that describes different viewpoints that help guide the process of modeling. The artifacts that are part of these viewpoints are UML based. UMM has also been adopted by ebiX which serves as a good example of the application of UMM for the energy area.

• The main subject of the Miracle project; shiftable consumption and/or production is not being covered by the existing models in the energy area. Therefore specific models will have to be developed in Miracle that are able to cope with these concepts.

• Part of the Common Information Model by the IEC provides a solid basis for the Miracle data model. Basic energy concepts that already have been modeled can be reused.

[read more]

Authors:
M.J. Konsman, F.J. Rumph, TNO

The energy sector is in transition. Firstly, the deregulation process forces companies to restructure their value chain in order to increase their market efficiency. Secondly, in order to reduce carbon emissions, the use of renewable energy sources is enforced by national and international regulations. Thirdly, smart metering is being widely adopted. The main goal of the MIRACLE project is to develop an ICT system that fits the future deregulated energy sector and enables the integration of a higher rate of distributed and renewable energy sources into the electricity grid. We will explore a micro-request-based approach for demand side management in which electricity producers and consumers issue micro-requests indicating flexibilities in time and amount of the electricity profiles. These requests will be processed by our system in order to balance electricity supply and demand in near real-time.

In this deliverable, we describe the conceptual architecture of the energy data management system (EDMS) developed in the MIRACLE project. The architecture reflects the hierarchical organization of the energy domain in balance groups and market balance areas. The requirements for the system are derived from the project goals and an estimation of the volume and number of messages exchanged within the EDMS and the volume of the data to be stored persistently in the EDMS.

A major prerequisite to design the EDMS in a way that it will be applicable to the future deregulated energy sector is to understand the current situation of the energy sector in different European countries and foresee its future structure. We therefore describe the current national electricity markets for some European countries in detail and compare the national roles to the roles defined in the ETSO harmonized model. We then describe characteristics of the MIRACLE system and based on that we specify three use cases that represent these characteristics. The processes associated with the use cases, the ETSO roles involved in them and the base processes identified are described and listed.  The description of the MIRACLE role model reflects the status of the current discussion within the MIRACLE team. The final specification of the MIRACLE roles and processes is planned for the next deliverable (D1.2 Final role model and process specification). [read more]

Authors:
Henrike Berthold, Alexandr Savinov SAP; Laurynas Šikšnys, Torben Bach Pedersen, Christian S. Jensen, AAU; Hellmuth Frey, EnBW; Christos Nychtis, CRES; Mente Konsman, Frens Jan Rumph, TNO; Matjaž Bobnar, Zoran Marinšek, Gregor Černe, INEA; Bogdan Filipič, JSI; Matthias Boehm, TUD