User Tools

Site Tools


the_capri_data_base

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
the_capri_data_base [2022/01/11 16:40] – [Annex: Testing procedure and checking intermediate steps in COCO (dairy)] gochtthe_capri_data_base [2026/03/13 13:20] (current) – add herd dynamics (Anna) massfeller
Line 174: Line 174:
 Example from file //‘Eurostat _agriculture_mapping.gms’//. The results of the program run are gdx-files loaded by files (e.g. coco/coco1_eurostat.gms) which are in turn loaded by coco1.gms or coco2.gms. Example from file //‘Eurostat _agriculture_mapping.gms’//. The results of the program run are gdx-files loaded by files (e.g. coco/coco1_eurostat.gms) which are in turn loaded by coco1.gms or coco2.gms.
  
- +<code fortran
-<html+SET EcoActMAP(ASS_COLS,ASS_ROWS,eco_act_ori_eurostat) "mapping"
-<head> +EAAP.CERE. aact_eaa01_01000_PROD_PP_MIO_EUR 
-<META http-equiv=Content-Type content="text/html; charset=UTF-8"> +EAAP.SWHE. aact_eaa01_01110_PROD_PP_MIO_EUR 
-<title>Exported from Notepad++</title> +EAAP.DWHE. aact_eaa01_01120_PROD_PP_MIO_EUR /;
-<style type="text/css"> +
-span { +
- font-family: 'Consolas'; +
- font-size: 9pt; +
- color: #000000; +
-+
-.sc0 { +
-+
-.sc9 { +
- font-weight: bold; +
- color: #0000FF; +
-+
-.sc12 { +
- font-weight: bold; +
- color: #6600CC; +
-+
-.sc13 { +
- font-weight: bold; +
- color: #6600CC; +
-+
-.sc16 { +
- color: #880000; +
-+
-.sc24 { +
-+
-</style> +
-</head> +
-<body> +
-<div style="float: left; white-space: pre; line-height: 1; background: #FFFFFF; "><span class="sc9">SET</span><span class="sc24"> </span><span class="sc0">EcoActMAP</span><span class="sc13">(</span><span class="sc0">ASS_COLS</span><span class="sc12">,</span><span class="sc0">ASS_ROWS</span><span class="sc12">,</span><span class="sc0">eco_act_ori_eurostat</span><span class="sc13">)</span><span class="sc24"> </span><span class="sc16">"mapping"</span><span class="sc24"> </span><span class="sc12">/</span><span class="sc24"> +
- </span><span class="sc0">EAAP.CERE.</span><span class="sc24"> </span><span class="sc0">aact_eaa01_01000_PROD_PP_MIO_EUR</span><span class="sc24"> +
- </span><span class="sc0">EAAP.SWHE.</span><span class="sc24"> </span><span class="sc0">aact_eaa01_01110_PROD_PP_MIO_EUR</span><span class="sc24"> +
- </span><span class="sc0">EAAP.DWHE.</span><span class="sc24"> </span><span class="sc0">aact_eaa01_01120_PROD_PP_MIO_EUR</span><span class="sc24"> </span><span class="sc12">/;</span><span class="sc24"> +
-  +
-  +
-  +
-</span><span class="sc9">SET</span><span class="sc24"> </span><span class="sc0">AgriProdMAP</span><span class="sc13">(</span><span class="sc0">ASS_COLS</span><span class="sc12">,</span><span class="sc0">ASS_ROWS</span><span class="sc12">,</span><span class="sc0">agri_prod_ori_eurostat</span><span class="sc13">)</span><span class="sc24"> </span><span class="sc16">"mapping"</span><span class="sc24"> </span><span class="sc12">/</span><span class="sc24"> +
-</span><span class="sc0">CERE.LEVL.</span><span class="sc13">(</span><span class="sc24"> </span><span class="sc0">apro_cpnh1_C1000_AR</span><span class="sc12">,</span><span class="sc0">apro_cpnh1_h_C1000_AR</span><span class="sc13">)</span><span class="sc24">  +
-</span><span class="sc0">SWHE.LEVL.</span><span class="sc13">(</span><span class="sc24"> </span><span class="sc0">apro_cpnh1_C1110_AR</span><span class="sc12">,</span><span class="sc0">apro_cpnh1_h_C1110_AR</span><span class="sc13">)</span><span class="sc24"> +
-</span><span class="sc0">SWH1.LEVL.</span><span class="sc13">(</span><span class="sc24"> </span><span class="sc0">apro_cpnh1_C1111_AR</span><span class="sc12">,</span><span class="sc0">apro_cpnh1_h_C1111_AR</span><span class="sc13">)</span><span class="sc24"> </span><span class="sc12">/;</span><span class="sc24"> +
- +
-</span></div></body> +
-</html> +
- +
  
  
  
 +SET AgriProdMAP(ASS_COLS,ASS_ROWS,agri_prod_ori_eurostat) "mapping" /
 +CERE.LEVL.( apro_cpnh1_C1000_AR,apro_cpnh1_h_C1000_AR)
 +SWHE.LEVL.( apro_cpnh1_C1110_AR,apro_cpnh1_h_C1110_AR)
 +SWH1.LEVL.( apro_cpnh1_C1111_AR,apro_cpnh1_h_C1111_AR) /;
 +</code>
  
 === Western Balkan Countries and Turkey === === Western Balkan Countries and Turkey ===
Line 246: Line 207:
 === Other additional input data === === Other additional input data ===
  
-COCO1: Biofuels+COCO1: Biofuels FIXME (most links are not working anymore, remove or re-link)
  
   * Production, market balance and feedstock quantities for biodiesel and bioethanol are collected from a multitude of sources:   * Production, market balance and feedstock quantities for biodiesel and bioethanol are collected from a multitude of sources:
Line 1413: Line 1374:
 The COCO module produces various reporting files on the intermediate data processing steps. These files can be used to trace back potential errors in the COCO database to their origin. These debugging files also contain meta-information on the input data and settings used for producing the COCO database.  The COCO module produces various reporting files on the intermediate data processing steps. These files can be used to trace back potential errors in the COCO database to their origin. These debugging files also contain meta-information on the input data and settings used for producing the COCO database. 
  
-The following example is a walk-through on the typical data processing steps, covring biofuels data preparation in France.+The following example is a walk-through on the typical data processing steps, covering biofuels data preparation in France.
  
 The reporting file 'output/results/coco/biof_data_with_prep/chk_biof_data_with_prep_FR000000.gdx' reports on the data preparation for biofuels for France (FR000) in COCO1. The file includes the set ‘meta_prepare_biofuel_data’, with meta-information on the recent coco1 run (e.g. creation date of file, GAMS version used). The reporting file 'output/results/coco/biof_data_with_prep/chk_biof_data_with_prep_FR000000.gdx' reports on the data preparation for biofuels for France (FR000) in COCO1. The file includes the set ‘meta_prepare_biofuel_data’, with meta-information on the recent coco1 run (e.g. creation date of file, GAMS version used).
Line 1436: Line 1397:
 | MAPRagr | Ethanol production from agricultural sources | | MAPRagr | Ethanol production from agricultural sources |
 | EloBio | Biofuel production and demand data from DG Energy project EloBio | | EloBio | Biofuel production and demand data from DG Energy project EloBio |
-DG\_Agri | Ethanol data from DGAgri website and supplementary files |+DG_Agri | Ethanol data from DGAgri website and supplementary files |
 | ProdCom | Eurostat: PRODCOM ANNUAL SOLD (NACE Rev. 2.) [DS-066341] | | ProdCom | Eurostat: PRODCOM ANNUAL SOLD (NACE Rev. 2.) [DS-066341] |
 | EIA | Independent Statistics &amp; Analysis, US Energy Information Administration | | EIA | Independent Statistics &amp; Analysis, US Energy Information Administration |
 | comext | Eurostat: Comext | | comext | Eurostat: Comext |
-Energy\_bal | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg\_107a] | +Energy_bal | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg_107a] | 
-Energy\_dem | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg\_102anrg\_1073a] |+Energy_dem | Eurostat: Supply, transformation, consumption - renewable energies - annual data [nrg_102anrg_1073a] |
 | final | results of the calculations | | final | results of the calculations |
 | ODOM | other domestic use (activity from biostock calculations | | ODOM | other domestic use (activity from biostock calculations |
-| APRagr | Ethanol production from agricultural sources | 
 | INDt | Sum of model results for BIOF and INDM | | INDt | Sum of model results for BIOF and INDM |
 | BIOi, INDi, DOMi | intermediate activities to save data from model initialisation for later documentation. | | BIOi, INDi, DOMi | intermediate activities to save data from model initialisation for later documentation. |
Line 1482: Line 1442:
 {{coco_biof_6.png?nolink|}} {{coco_biof_6.png?nolink|}}
  
-The parameter //p_biofDataMS// reports on production (MAPR), trade (import:IMPT, export:EXPT), production from non-agricultural sources (NAGR), prices (UVAD, UVAP) and consumer taxes (CTAX). The distribiutio of total biodiesel processing to the feedstock is also reported, for rapeseed oil (RAPO), sunflower oil (SUNO), soya oil (SOYO) and palm oil (PLMO).+The parameter //p_biofDataMS// reports on production (MAPR), trade (import:IMPT, export:EXPT), production from non-agricultural sources (NAGR), prices (UVAD, UVAP) and consumer taxes (CTAX). The distribiution of total biodiesel processing to the feedstock is also reported, for rapeseed oil (RAPO), sunflower oil (SUNO), soya oil (SOYO) and palm oil (PLMO).
  
  
Line 1701: Line 1661:
  
 To assess the reliability of the CAPRI database in terms of GHG results against official UNFCCC notifications, results from the first step (time series) were insufficient, as the GHG accounting also requires information on the feed allocation. This problem was addressed within the scope of the IDEAg (Improving the quantification of GHG emissions and flows of reactive nitrogen) project((The IDEAg project was commissioned by the JRC-IES in Ispra in 2015 and was carried out by the Thünen Institute in cooperation with the JRC-IES (August 2015 – August 2016). A more detailed explanation of the CAPRI task “Build GHG inventories” and its use has been prepared by the Thünen contributors at the time, Sandra Marquardt and Alexander Gocht, see capri/doc/GHG_inventory_module.docx. )), where an option has been introduced to allow for a consistent accounting of GHG emissions over time. This is able to combine input information from CAPREG time series runs as well as (short run, nowcasting-style) CAPMOD simulation results. Furthermore, an R-based tool was introduced to the CAPRI GUI that maps GHG emissions data from CAPRI to the GHG emission balances contained in the National Inventory Reports (NIRs) that are submitted annually by countries in compliance with UNFCCC GHG reporting obligations. To assess the reliability of the CAPRI database in terms of GHG results against official UNFCCC notifications, results from the first step (time series) were insufficient, as the GHG accounting also requires information on the feed allocation. This problem was addressed within the scope of the IDEAg (Improving the quantification of GHG emissions and flows of reactive nitrogen) project((The IDEAg project was commissioned by the JRC-IES in Ispra in 2015 and was carried out by the Thünen Institute in cooperation with the JRC-IES (August 2015 – August 2016). A more detailed explanation of the CAPRI task “Build GHG inventories” and its use has been prepared by the Thünen contributors at the time, Sandra Marquardt and Alexander Gocht, see capri/doc/GHG_inventory_module.docx. )), where an option has been introduced to allow for a consistent accounting of GHG emissions over time. This is able to combine input information from CAPREG time series runs as well as (short run, nowcasting-style) CAPMOD simulation results. Furthermore, an R-based tool was introduced to the CAPRI GUI that maps GHG emissions data from CAPRI to the GHG emission balances contained in the National Inventory Reports (NIRs) that are submitted annually by countries in compliance with UNFCCC GHG reporting obligations.
 +
 +==== Annex: Fertilizer Data used in CAPRI ====
 +
 +CAPRI combines time series data on **total fertilizer** quantities at member state level from various sources:
 +
 +  - total fertilizer from Eurostat: consumption of inorganic fertilizers [Eurostat table: aei_fm_usefert] (2000-2020) (“TOTdom”)
 +  - total fertilizer consumption (IFA) (1980-2020) (“TOTifa”)
 +  - total fertilizer from Eurostat: sales of manufactured fertilizers (source: EFMA) [Eurostat table: aei_fm_manfert] (1985-2019) (“TOTdem”)
 +  - total nitrogen consumption from UNFCCC (CRF) (“TOTcrf”)
 +  - total fertilizer from Eurostat: gross nutrient balance [Eurostat table: aei_pr_gnb] (1985-2019) (“TOTgnb”)
 +
 +From these sources the total fertilizer consumption for use in CAPRI (TOTsel) is selected or computed. The application of fertilizer per crop is scaled to TOTsel, so that the sum over all crops (“TOTapp”) is equal TOTsel, the finally consolidated time series.
 +
 +In the following the selections process of fertilizer sources are described in more detail:
 +
 +The preferred and most reliable source of total fertilizer use in CAPRI (TOTsel) for the time period beginning in 2000 is assumed to be Eurostat’s total consumption of inorganic fertilizers (TOTdom).
 +
 +This source is completed and replaced by total fertilizer consumption of IFA if Eurostat data is not available or does not appear to be sufficiently reliable. Potassium (K20) fertilizer consumption is missing from Eurostat’s TOTdom series for all European countries. Also, total fertilizers for Western Balkan countries apart from Albania are missing and taken over from IFA (TOTifa).
 +
 +Some exceptions to these general rules:
 +
 +  * In Albania we have the particularity that the potassium series from IFA appeared to be unrealistic. These originally very volatile data were therefore smoothed and interpolated.
 +  * Furthermore, the fertilizer data for Portugal is based on IFA data, as the Eurostat time series TOTdom appeared implausible and was discarded therefore.
 +  * Italy’s total fertilizer use is relied on EFMA data (TOTdem), which provides long and reliable time series for all three kinds of fertilizer.
 +  * Potassium for Cyprus from 2011 onwards is taken from TOTdem as TOTifa data shows a high volatility.
 +
 +Our preferred Eurostat data (TOTdom) start only in 2000 such that a backward extension is necessary for CAPRI. Here the data sources are used hierarchically in the following order for completion: (1) Eurostat-EFMA data (TOTdem), (2) IFA fertilizer data which starts 1980 for several countries (TOTifa), (3) UNFCCC total nitrogen consumption (TOTcrf), and (4) Eurostat gross nutrient balance (TOTgnb).
 +
 +If a different data source is used for the period before 2000, the conversion to the level of the later data source is carried out based on the relation between this source and TOTsel from the oldest common year. All backward extensions involving this level adjustment are marked “bas” in the table below.
 +
 +Slovenian's nitrogen consumption before 2000 is based on TOTcrf and its total phosphorus consumption is based on TOTgnb because these data series apparently permit a reasonable backward extension.
 +
 +Final fertilizer sources (TOTsel) applied in CAPRI:
 +
 +{{table_fertisource.png?627x280}}
 +
 +Finally, the fertilizer use per crop is scaled to the total fertilizer use in CAPRI (TOTsel), so that the sum over all crops (TOTapp) is equal to our preferred series TOTsel.
 +
 +In addition to the total fertilizer time series, CAPRI also uses data on **fertilizer utilization by crop (FUBC).** The following list gives (in a less detailed manner) an overview on the data sources collected and prepared for CAPRI:
 +
 +  - Nonpublic FUBC data from the EFMA forecasting exercise in 2008 (fubc0608 (EFMA)). Data exist for the years 2005, 2006, 2007, 2008, 2012, 2017. The years 2004, 2005 are forecasted by also using EFMA data from 2006 (“FUBCefm”).
 +  - IFA fertilizer data for the 90s (“FUBCifa”) exist for certain European member states, but is missing the Western Balkan Countries apart from Albania as well as Malta, Cyprus, Slovenia, Bulgaria, and Romania.
 +  - Data from a publication in Nature (“FUBCnature”) can be activated for all CAPRI “RMS” regions by setting the global %fert_from_Nature%==ON. The source can be found here: https://datadryad.org/stash/dataset/doi:10.5061/dryad.2rbnzs7qh
 +
 +FUBCefm data is backward extended by using FUBCifa. The use of FUBCnature can optionally be used. In the default setting this data source is not considered. If FUBCnature is activated, the source has priority to the combined source of FUBCefma and FUBCifa.
 +
 +
 +
  
 ===== Input Allocation ===== ===== Input Allocation =====
Line 1818: Line 1826:
 |Bull fattening (BULF) |BULL: 20% lower meat output, variable inputs besides feed an young animals at 80% of average |BULH: 20% higher meat output, variable inputs besides feed an young animals at 120% of average| |Bull fattening (BULF) |BULL: 20% lower meat output, variable inputs besides feed an young animals at 80% of average |BULH: 20% higher meat output, variable inputs besides feed an young animals at 120% of average|
 |Heifers fattening (HEIF)| HEIL: 20% lower meat output, variable inputs besides feed an young animals at 80% of average |HEIH: 20% higher meat output, variable inputs besides feed an young animals at 120% of average| \\ Source: CAPRI Modelling System |Heifers fattening (HEIF)| HEIL: 20% lower meat output, variable inputs besides feed an young animals at 80% of average |HEIH: 20% higher meat output, variable inputs besides feed an young animals at 120% of average| \\ Source: CAPRI Modelling System
 +
 +=== Implementation of herd dynamics in coding ===
 +
 +The dynamic relationships shown in Fig 3 are implemented in the animal sector specific equations in file coco1_estimA.gms, more specifically equations “e_yanib” (young animal balance) and “e_stocksA” (animal stock change equivalence):
 +
 +{{ :e_yanib.jpg?nolink&800 |}}
 +
 +Equation e_yaniB defines the stock change of all young animal types as the difference between production of young  animals and their use to source the flow of slaughtered animals. At the example of piglets ready for fattening or becoming young sows:
 +
 +Equation 1
 +\begin{align}
 +\text{STCM}_{YPIG}(t) = \text{GROF}_{YPIG}(t) - \text{GROF}_{IPIG}(t)
 +\end{align}
 +
 +{{ :e_stocksa.jpg?nolink&800 |}}
 +
 +Equation e_stocksA is more complex, because it handles basically 3 different cases. The first is the situation in the non-cattle sectors (pigs, poultry, sheep) where production of young animals by breeding activites (sows, hens, ewes) and their raising is assumed to take place within the current year. Taking into account the mappings in the code, then the equation e_stocksA reads for the case of piglets:
 +
 +Equation 2
 +
 +\begin{align}
 +&\text{SOWS}_{LEVL}(t+1) - \text{SOWS}_{LEVL}(t) = \text{STCM}_{YPIG}(t) \\
 +&= \text{GROF}_{YPIG}(t) - \text{GROF}_{IPIG}(t)
 +\end{align}
 +
 +As the input use of piglets only covers their use for slaughtered animals (either fatteners or replacement of old sows), the excess of piglet production and this “replacement” use matches with the change in the size of the breeding stock (sows) in the next year. Conversely, a piglet production falling short of the replacement need (STCMYPIG(t) < 0) means that the sow herd will be lower in the next year. 
 +In the cattle sector raising from calves to the adult animals may extend over several years and is therefore represented with specific raising activities that produce one animal category for the next stage, for example raising of male calves to become young bulls for the final fattening process. In this example (case #2), equation e_stocksA reads for the case of young bulls:
 +
 +Equation 1
 +
 +\begin{align}
 +&\text{CAMR}_{LEVL}(t) - \text{CAMR}_{LEVL}(t-1) = \text{STCM}_{YBUL}(t) \\
 +&= \text{GROF}_{YBUL}(t) - \text{GROF}_{IBUL}(t)
 +\end{align}
 +
 +This equation is more intuitive considering that the activity level CAMRLEVL(t) is defined to be equal to the output = production of young bulls GROFYBUL(t) in the current year such that this appear on both sides of the equation, which may be simplified to read: 
 +
 +Equation 1
 +
 +\begin{align}
 +&\text{GROF}_{YBUL}(t) - \text{CAMR}_{LEVL}(t-1) =\\
 +&\text{GROF}_{YBUL}(t) - \text{GROF}_{IBUL}(t) \\
 +&\iff \text{CAMR}_{LEVL}(t-1) = \text{GROF}_{IBUL}(t)
 +\end{align}
 +
 +In this form the equation just states that the current input use of young bulls (into the fattening activities BULL and BULH) must have been raised from young calves in the previous year such that e_stocksA just ensures this aspect of the herd category linkages in the cattle sector. 
 +The third case handeled by equation e_stocksA is the dynamics for young cows where the equation reads as follows:
 +
 +Equation 1
 +
 +\begin{align}
 +&\text{HEIR}_{LEVL}(t) - \text{HEIR}_{LEVL}(t-1) \\
 +&\quad + \text{COWS}_{LEVL}(t+1) - \text{COWS}_{LEVL}(t) \\
 +&= \text{STCM}_{YCOW}(t) \\
 +&= \text{GROF}_{YCOW}(t) - \text{GROF}_{ICOW}(t)
 +\end{align}
 +
 +Considering again that it holds for the activity level of the raising process that HEIRLEVL(t) = GROFYCOW(t), the equation boils down to the requirement
 +
 +Equation 1
 +
 +\begin{align}
 +&\text{COWS}_{LEVL}(t+1) - \text{COWS}_{LEVL}(t) \\
 +&= \text{GROF}_{YCOW}(t-1) - \text{GROF}_{ICOW}(t)
 +\end{align}
 +
 +The change in the total cow herd (DCOW+SCOW) in the next year must be equal to the excess of last year’s production of young cows (thus ready to enter the current cow herd) over the replacement need derived from slaughterings of old cows in the current year. 
 +
  
 ====Input allocation for feed==== ====Input allocation for feed====
Line 2060: Line 2136:
   * Output of manure at tail –depending on animal type, regional animal population and animal yields, as final weights or milk yields (see section on Output at tail).   * Output of manure at tail –depending on animal type, regional animal population and animal yields, as final weights or milk yields (see section on Output at tail).
   * Manure imports and exports (to the region)   * Manure imports and exports (to the region)
-  * Input of mineral fertiliser –as given from national statistics at sectoral level.+  * Input of mineral fertiliser –as given from national statistics at sectoral level. For details see[[https://www.capri-model.org/dokuwiki_help/doku.php?id=the_capri_data_base#annexfertilizer_data_used_in_capri|"Annex: Fertilizer Data used in CAPRI" in the CAPREG description]]
   * Input of crop residues, biological fixation, atmospheric deposition   * Input of crop residues, biological fixation, atmospheric deposition
   * Emissions (NH3, NOx, N2, N2O, CO2, CH4, NO3, C from soil erosion) only for nitrogen and carbon, and removals (carbon sequestration) only for carbon   * Emissions (NH3, NOx, N2, N2O, CO2, CH4, NO3, C from soil erosion) only for nitrogen and carbon, and removals (carbon sequestration) only for carbon
the_capri_data_base.1641919236.txt.gz · Last modified: 2022/11/07 10:23 (external edit)

Except where otherwise noted, content on this wiki is licensed under the following license: CC0 1.0 Universal
CC0 1.0 Universal Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki