Differences

This shows you the differences between two versions of the page.

--- the_capri_data_base [2022/01/10 11:38] – [Annex: Testing procedure and checking intermediate steps in COCO (biofuels)] matsz
+++ the_capri_data_base [2024/05/31 09:58] – add annex: fertiliser data sources massfeller
@@ Line 174: / Line 174: @@
 Example from file //‘Eurostat _agriculture_mapping.gms’//. The results of the program run are gdx-files loaded by files (e.g. coco/coco1_eurostat.gms) which are in turn loaded by coco1.gms or coco2.gms.
+<code fortran>
-<html>
+SET EcoActMAP(ASS_COLS,ASS_ROWS,eco_act_ori_eurostat) "mapping" /
-<head>
+EAAP.CERE. aact_eaa01_01000_PROD_PP_MIO_EUR
-<META http-equiv=Content-Type content="text/html; charset=UTF-8">
+EAAP.SWHE. aact_eaa01_01110_PROD_PP_MIO_EUR
-<title>Exported from Notepad++</title>
+EAAP.DWHE. aact_eaa01_01120_PROD_PP_MIO_EUR /;
-<style type="text/css">
-span {
-	font-family: 'Consolas';
-	font-size: 9pt;
-	color: #000000;
-}
-.sc0 {
-}
-.sc9 {
-	font-weight: bold;
-	color: #0000FF;
-}
-.sc12 {
-	font-weight: bold;
-	color: #6600CC;
-}
-.sc13 {
-	font-weight: bold;
-	color: #6600CC;
-}
-.sc16 {
-	color: #880000;
-}
-.sc24 {
-}
-</style>
-</head>
-<body>
-<div style="float: left; white-space: pre; line-height: 1; background: #FFFFFF; "><span class="sc9">SET</span><span class="sc24"> </span><span class="sc0">EcoActMAP</span><span class="sc13">(</span><span class="sc0">ASS_COLS</span><span class="sc12">,</span><span class="sc0">ASS_ROWS</span><span class="sc12">,</span><span class="sc0">eco_act_ori_eurostat</span><span class="sc13">)</span><span class="sc24"> </span><span class="sc16">"mapping"</span><span class="sc24"> </span><span class="sc12">/</span><span class="sc24">
- </span><span class="sc0">EAAP.CERE.</span><span class="sc24"> </span><span class="sc0">aact_eaa01_01000_PROD_PP_MIO_EUR</span><span class="sc24">
- </span><span class="sc0">EAAP.SWHE.</span><span class="sc24"> </span><span class="sc0">aact_eaa01_01110_PROD_PP_MIO_EUR</span><span class="sc24">
- </span><span class="sc0">EAAP.DWHE.</span><span class="sc24"> </span><span class="sc0">aact_eaa01_01120_PROD_PP_MIO_EUR</span><span class="sc24"> </span><span class="sc12">/;</span><span class="sc24">
-</span><span class="sc9">SET</span><span class="sc24"> </span><span class="sc0">AgriProdMAP</span><span class="sc13">(</span><span class="sc0">ASS_COLS</span><span class="sc12">,</span><span class="sc0">ASS_ROWS</span><span class="sc12">,</span><span class="sc0">agri_prod_ori_eurostat</span><span class="sc13">)</span><span class="sc24"> </span><span class="sc16">"mapping"</span><span class="sc24"> </span><span class="sc12">/</span><span class="sc24">
-</span><span class="sc0">CERE.LEVL.</span><span class="sc13">(</span><span class="sc24"> </span><span class="sc0">apro_cpnh1_C1000_AR</span><span class="sc12">,</span><span class="sc0">apro_cpnh1_h_C1000_AR</span><span class="sc13">)</span><span class="sc24">
-</span><span class="sc0">SWHE.LEVL.</span><span class="sc13">(</span><span class="sc24"> </span><span class="sc0">apro_cpnh1_C1110_AR</span><span class="sc12">,</span><span class="sc0">apro_cpnh1_h_C1110_AR</span><span class="sc13">)</span><span class="sc24">
-</span><span class="sc0">SWH1.LEVL.</span><span class="sc13">(</span><span class="sc24"> </span><span class="sc0">apro_cpnh1_C1111_AR</span><span class="sc12">,</span><span class="sc0">apro_cpnh1_h_C1111_AR</span><span class="sc13">)</span><span class="sc24"> </span><span class="sc12">/;</span><span class="sc24">
-</span></div></body>
-</html>
+SET AgriProdMAP(ASS_COLS,ASS_ROWS,agri_prod_ori_eurostat) "mapping" /
+CERE.LEVL.( apro_cpnh1_C1000_AR,apro_cpnh1_h_C1000_AR)
+SWHE.LEVL.( apro_cpnh1_C1110_AR,apro_cpnh1_h_C1110_AR)
+SWH1.LEVL.( apro_cpnh1_C1111_AR,apro_cpnh1_h_C1111_AR) /;
+</code>
 === Western Balkan Countries and Turkey ===
@@ Line 246: / Line 207: @@
 === Other additional input data ===
-COCO1: Biofuels
+COCO1: Biofuels FIXME (most links are not working anymore, remove or re-link)
   * Production, market balance and feedstock quantities for biodiesel and bioethanol are collected from a multitude of sources:
@@ Line 1413: / Line 1374: @@
 The COCO module produces various reporting files on the intermediate data processing steps. These files can be used to trace back potential errors in the COCO database to their origin. These debugging files also contain meta-information on the input data and settings used for producing the COCO database.
-The following example is a walk-through on the typical data processing steps, covring biofuels data preparation in France.
+The following example is a walk-through on the typical data processing steps, covering biofuels data preparation in France.
 The reporting file 'output/results/coco/biof_data_with_prep/chk_biof_data_with_prep_FR000000.gdx' reports on the data preparation for biofuels for France (FR000) in COCO1. The file includes the set ‘meta_prepare_biofuel_data’, with meta-information on the recent coco1 run (e.g. creation date of file, GAMS version used).
@@ Line 1436: / Line 1397: @@
 | MAPRagr | Ethanol production from agricultural sources |
 | EloBio | Biofuel production and demand data from DG Energy project EloBio |
-| DG\_Agri | Ethanol data from DGAgri website and supplementary files |
+| DG_Agri | Ethanol data from DGAgri website and supplementary files |
 | ProdCom | Eurostat: PRODCOM ANNUAL SOLD (NACE Rev. 2.) [DS-066341] |
 | EIA | Independent Statistics &amp; Analysis, US Energy Information Administration |
 | comext | Eurostat: Comext |
-| Energy\_bal | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg\_107a] |
+| Energy_bal | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg_107a] |
-| Energy\_dem | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg\_102a, nrg\_1073a] |
+| Energy_dem | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg_102a, nrg_1073a] |
 | final | results of the calculations |
 | ODOM | other domestic use (activity from biostock calculations |
-| APRagr | Ethanol production from agricultural sources |
 | INDt | Sum of model results for BIOF and INDM |
 | BIOi, INDi, DOMi | intermediate activities to save data from model initialisation for later documentation. |
@@ Line 1482: / Line 1442: @@
 {{coco_biof_6.png?nolink|}}
-The parameter //p_biofDataMS// reports on production (MAPR), trade (import:IMPT, export:EXPT), production from non-agricultural sources (NAGR), prices (UVAD, UVAP) and consumer taxes (CTAX). The distribiutio of total biodiesel processing to the feedstock is also reported, for rapeseed oil (RAPO), sunflower oil (SUNO), soya oil (SOYO) and palm oil (PLMO).
+The parameter //p_biofDataMS// reports on production (MAPR), trade (import:IMPT, export:EXPT), production from non-agricultural sources (NAGR), prices (UVAD, UVAP) and consumer taxes (CTAX). The distribiution of total biodiesel processing to the feedstock is also reported, for rapeseed oil (RAPO), sunflower oil (SUNO), soya oil (SOYO) and palm oil (PLMO).
@@ Line 1534: / Line 1494: @@
 Under activity dairy cows (DCOW) the following items are reported: yield, total production (GROF), feed use (FEDM) and losses on market (LOSM). Eurostat’s //National Accounts of Agriculture (EAA)// only supply data for the aggregate milk (MILK). The equation //e_EAAMLK// in the consolidation model //AnimNSSQ// ensures the consistency of EAA values for MILK, as they are split up between cow milk (COMI) and sheep and goat milk (SGMI).
-<code gams>
+<code fortran>
 e_EAAMLK("%MS%000",T)
      $ (p_NobsP("%MS%000","EAAP","MILK") AND ESTR("MILK") and
@@ Line 1701: / Line 1661: @@
 To assess the reliability of the CAPRI database in terms of GHG results against official UNFCCC notifications, results from the first step (time series) were insufficient, as the GHG accounting also requires information on the feed allocation. This problem was addressed within the scope of the IDEAg (Improving the quantification of GHG emissions and flows of reactive nitrogen) project((The IDEAg project was commissioned by the JRC-IES in Ispra in 2015 and was carried out by the Thünen Institute in cooperation with the JRC-IES (August 2015 – August 2016). A more detailed explanation of the CAPRI task “Build GHG inventories” and its use has been prepared by the Thünen contributors at the time, Sandra Marquardt and Alexander Gocht, see capri/doc/GHG_inventory_module.docx. )), where an option has been introduced to allow for a consistent accounting of GHG emissions over time. This is able to combine input information from CAPREG time series runs as well as (short run, nowcasting-style) CAPMOD simulation results. Furthermore, an R-based tool was introduced to the CAPRI GUI that maps GHG emissions data from CAPRI to the GHG emission balances contained in the National Inventory Reports (NIRs) that are submitted annually by countries in compliance with UNFCCC GHG reporting obligations.
+==== Annex: Fertilizer data used in Capri ====
+CAPRI combines time series data on **total fertilizer** quantities at member state level from various sources:
+  - total fertilizer from Eurostat: consumption of inorganic fertilizers [Eurostat table: aei_fm_usefert] (2000-2020) (“TOTdom”)
+  - total fertilizer consumption (IFA) (1980-2020) (“TOTifa”)
+  - total fertilizer from Eurostat: sales of manufactured fertilizers (source: EFMA) [Eurostat table: aei_fm_manfert] (1985-2019) (“TOTdem”)
+  - total nitrogen consumption from UNFCCC (CRF) (“TOTcrf”)
+  - total fertilizer from Eurostat: gross nutrient balance [Eurostat table: aei_pr_gnb] (1985-2019) (“TOTgnb”)
+From these sources the total fertilizer consumption for use in CAPRI (TOTsel) is selected or computed. The application of fertilizer per crop is scaled to TOTsel, so that the sum over all crops (“TOTapp”) is equal TOTsel, the finally consolidated time series.
+In the following the selections process of fertilizer sources are described in more detail:
+The preferred and most reliable source of total fertilizer use in CAPRI (TOTsel) for the time period beginning in 2000 is assumed to be Eurostat’s total consumption of inorganic fertilizers (TOTdom).
+This source is completed and replaced by total fertilizer consumption of IFA if Eurostat data is not available or does not appear to be sufficiently reliable. Potassium (K20) fertilizer consumption is missing from Eurostat’s TOTdom series for all European countries. Also, total fertilizers for Western Balkan countries apart from Albania are missing and taken over from IFA (TOTifa).
+Some exceptions to these general rules:
+  * In Albania we have the particularity that the potassium series from IFA appeared to be unrealistic. These originally very volatile data were therefore smoothed and interpolated.
+  * Furthermore, the fertilizer data for Portugal is based on IFA data, as the Eurostat time series TOTdom appeared implausible and was discarded therefore.
+  * Italy’s total fertilizer use is relied on EFMA data (TOTdem), which provides long and reliable time series for all three kinds of fertilizer.
+  * Potassium for Cyprus from 2011 onwards is taken from TOTdem as TOTifa data shows a high volatility.
+Our preferred Eurostat data (TOTdom) start only in 2000 such that a backward extension is necessary for CAPRI. Here the data sources are used hierarchically in the following order for completion: (1) Eurostat-EFMA data (TOTdem), (2) IFA fertilizer data which starts 1980 for several countries (TOTifa), (3) UNFCCC total nitrogen consumption (TOTcrf), and (4) Eurostat gross nutrient balance (TOTgnb).
+If a different data source is used for the period before 2000, the conversion to the level of the later data source is carried out based on the relation between this source and TOTsel from the oldest common year. All backward extensions involving this level adjustment are marked “bas” in the table below.
+Slovenian's nitrogen consumption before 2000 is based on TOTcrf and its total phosphorus consumption is based on TOTgnb because these data series apparently permit a reasonable backward extension.
+Final fertilizer sources (TOTsel) applied in CAPRI:
+{{table_fertisource.png?627x280}}
+Finally, the fertilizer use per crop is scaled to the total fertilizer use in CAPRI (TOTsel), so that the sum over all crops (TOTapp) is equal to our preferred series TOTsel.
+In addition to the total fertilizer time series, CAPRI also uses data on **fertilizer utilization by crop (FUBC).** The following list gives (in a less detailed manner) an overview on the data sources collected and prepared for CAPRI:
+  - Nonpublic FUBC data from the EFMA forecasting exercise in 2008 (fubc0608 (EFMA)). Data exist for the years 2005, 2006, 2007, 2008, 2012, 2017. The years 2004, 2005 are forecasted by also using EFMA data from 2006 (“FUBCefm”).
+  - IFA fertilizer data for the 90s (“FUBCifa”) exist for certain European member states, but is missing the Western Balkan Countries apart from Albania as well as Malta, Cyprus, Slovenia, Bulgaria, and Romania.
+  - Data from a publication in Nature (“FUBCnature”) can be activated for all CAPRI “RMS” regions by setting the global %fert_from_Nature%==ON. The source can be found here: https://datadryad.org/stash/dataset/doi:10.5061/dryad.2rbnzs7qh
+FUBCefm data is backward extended by using FUBCifa. The use of FUBCnature can optionally be used. In the default setting this data source is not considered. If FUBCnature is activated, the source has priority to the combined source of FUBCefma and FUBCifa.
 ===== Input Allocation =====