An easy step by step tutorial about how to extract Market Data on Artesian Portal.
Artesian gives you straightforward access to the data history to perform an analysis and produce a plan in the most compatible way.
Let’s see step-by-step how to proceed.
The goal
Extract data from the Artesian Portal.
The reference data is fictitious, created exclusively for this case. With Artesian, it is possible to write any data attributable to a Time Series, making it suitable for saving your data production.
Let’s see how to proceed step by step.
Time Series Search
On the main screen, at the top page next to the user, you can decide whether to extract a TimeSeries or GME Public offers.
Based on your choice, our screen will change:
For the Time Series, we will have an extended menu (R1) che ci aiuta a selezionare i market data di nostro interesse tramite selezione di categorie o ricerca testuale libera.
Filtrando i dati usando la tabella delle categorie avremo a disposizione un set di default come: il Tipo di market data (che potrà essere Actual, Versioned, Market Assessment, Bid Ask o Auction), i Provider elencati per nome, il tipo di Granularità in cui sono scritti i market data e la loro TimeZone; oppure usando categorie custom create dall’utente per classificare i dati e ritrovarli in maniera più agevole.
Alcuni esempi visibili nelle schermate sotto sono: l’Operator Country Label che è stato assegnato ai dati del porvider ENTSO-G per facilitare il filtro del paese, così come l’Operator Label che lo denomina, l’Operator Type Label, l’Interruption Type (Planned, Actual o Unplanned), etc..
Usando invece la ricerca testuale libera (R2) è possibile costruire una vera e propria query di ricerca usando i comandi visibili andando col mouse sulla i del box di ricerca
The selection of Market Data to extract.
If we were to extract “Actual” market data from the “PythonSDK” provider, we would notice a drastic reduction of menu choices available on the initial screen (M1); this happens due to the type of selected Market Data and the Provider. Therefore, a filter is applied to the data to exclude all records unrelated to the selection and simplify further filtering.
Since this specific market data was written in a Granularity of “Day”, with a TimeZone “CET”, we can see these details on the menu, under the provider.
The number next to each sub-category indicates the number of curves available if you select that specific filter.
By selecting our market data, we can see how on the left of the menu (M2) that the number of TimeSeries (TS: 1) and Market Data (#MarketData: 1) selected so far is again indicated.
Market Data Extraction
Once we have decided on the market data of interest, we can proceed to the “Extract” section. In this screen, we can define the parameters for the extraction. Additionally, we will find a brief summary on the left regarding the ID of the selected market data, the provider’s name, the type of market data and the Aggregation rule.
– “Date Options” allows you to set the extraction Time Range.
Artesian supports 4 options:
- “AbsoluteDateRange” : an absolute fixed period of time (e.g.: from “2022-06-22” to “2022-06-23” will allow you to extract the data of “2022-06-22”).
- “RelativePeriod” : represents a relative period of time, before or after today (e.g.: Considering that today is “2022-06-22”, requesting the period “P-5D” will mean extracting data from “2022-06- 17 “to” 2022-06-21 “. Requesting the period” P5D “will mean extracting the data from” 2022-06-22 “to” 2022-06-26 “). For the syntax, refer to the ISO8601 standard.
- “RelativePeriodRange”: (e.g.: from “P-5D” to “P5D” will extract data from “2022-06-17” to “2022-06-26”).
- “RelativeInterval” is a fixed dimension “rolling” time-span. The possible options are: “RollingWeek”, “RollingMonth”, “RollingQuarter”, or “RollingYear” or the last 7, 30, 90, 365 days of data (with the current day included); “WeekToDate”, “MonthToDate”, “QuarterToDate” or “YearToDate” or considering from the current day to the beginning of the week, month or year.
– “Granularity” allows you to establish the extraction Granularity; this can coincide with that of the original data or be different, as long as an “Aggregation Rule” has been configured on the curve of interest.
– “Aggregation Rule” allows you to extract data even at different granularities from the original one. The aggregation/disaggregation operation applied to the data is defined through the enhancement of this property. The possible options are “Undefined”, “SumAndDivide” or “AverageAndReplicate”. In the case of “Undefined”, extracting data with different Granularity from the original will obviously not be possible.
– “Timezone” allows you to establish the time zone for extracting the Market Data; if different from the original, it will be the system’s responsibility to carry out the necessary conversion to return the data in the correct form.
– “Time Transform” allows you to apply time shifts to the data to bring them back to different dimensions from the standard, e.g., Gas Day or Thermal Year.
Once we decide on this last step, we can download the data in an Excel file format or use the API link built by the UI.
GME Public Offer Extraction.
If you want to extract GME Public Offer data, you will need to select this option from the main page on Artesian.
There is no longer a menu choice of possible extractions due to the selected single data type.
Let’s see the methods and limits for this type of extraction. Among the various filters available, there are some mandatory ones, such as the date (you can only extract the data of a specific day, at a time), the Purpose of the Public Offers (if Bid or Offer) and the Status of the Public Offers (“ACC “,” REJ “,” INC “,” REP “,” REV “,” SUB “).
Once these mandatory parameters are defined, we can insert optional filters for the extraction:
We can decide to extract a specific Market (“MGP”, “MI2”, “MSD”, “MB”, …), a specific Scope (“NULL”, “ACC”, “AS”, “CA”, “GR1” , …), a BAType (“NULL”, “NETT”, “NREV”, “REV”), a specific Zone (“AUST”, “BRNN”, “CALA”, “GREC”, …), a specific Unit Type (“UP”, “UPV”, “UC”, “UCV”) and a specific Generation Type (“AUTOGENERATION”, “BIOMASS”, “COAL”, “WIND”, “GAS”, …), the latter option is usable only if the customer has shared with us the mapping between Unit and Generation Type.
Once these extraction parameters are defined, we can obtain a “Preview” of the data or download them directly in Excel format.
It is sufficient to employ just once and then have it entirely reproducible and automated in our workflow.
Not only does it save you time, but it allows you to minimize human errors caused by repeated operations on substantial amounts of data or different Excel files.
An undeniable advantage that allows us to focus on data analysis instead of its management and optimization.
