the_capri_data_base
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
the_capri_data_base [2021/12/14 11:22] – [Annex: Testing procedure and checking intermediate steps in COCO (biofuels)] himics | the_capri_data_base [2021/12/16 14:33] – [Data Import] himics | ||
---|---|---|---|
Line 172: | Line 172: | ||
* // | * // | ||
- | Example from file // | + | Example from file // |
+ | |||
+ | |||
+ | < | ||
+ | < | ||
+ | <META http-equiv=Content-Type content=" | ||
+ | < | ||
+ | <style type=" | ||
+ | span { | ||
+ | font-family: | ||
+ | font-size: 9pt; | ||
+ | color: #000000; | ||
+ | } | ||
+ | .sc0 { | ||
+ | } | ||
+ | .sc9 { | ||
+ | font-weight: | ||
+ | color: #0000FF; | ||
+ | } | ||
+ | .sc12 { | ||
+ | font-weight: | ||
+ | color: #6600CC; | ||
+ | } | ||
+ | .sc13 { | ||
+ | font-weight: | ||
+ | color: #6600CC; | ||
+ | } | ||
+ | .sc16 { | ||
+ | color: #880000; | ||
+ | } | ||
+ | .sc24 { | ||
+ | } | ||
+ | </ | ||
+ | </ | ||
+ | < | ||
+ | <div style=" | ||
+ | </ | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | |||
+ | |||
+ | </ | ||
+ | </ | ||
+ | </ | ||
+ | </ | ||
+ | |||
+ | </ | ||
+ | </ | ||
- | <file gams Eurostat _agriculture_mapping.gms> | ||
- | SET EcoActMAP(ASS_COLS, | ||
- | | ||
- | | ||
- | | ||
- | ... | ||
- | SET AgriProdMAP(ASS_COLS, | ||
- | CERE.LEVL.( apro_cpnh1_C1000_AR, | ||
- | SWHE.LEVL.( apro_cpnh1_C1110_AR, | ||
- | SWH1.LEVL.( apro_cpnh1_C1111_AR, | ||
- | </ | ||
- | The results of the program run are gdx-files loaded by files (e.g. coco/ | ||
=== Western Balkan Countries and Turkey === | === Western Balkan Countries and Turkey === | ||
Line 1146: | Line 1183: | ||
| GFCO | Other gross fixed capital formation | | | GFCO | Other gross fixed capital formation | | ||
+ | **Table: Codes used for storing the original REGIO tables in the data base and their description (columns)** | ||
+ | |||
+ | ^ Codes used in CAPRI REGIO tables ^ Original REGIO description ^ | ||
+ | | LEVL | Herd size / Area / # of persons | | ||
+ | | LSUN | Live stock units | | ||
+ | | PROP | Physical production | | ||
+ | | YILD | Yield | | ||
+ | | VALE | EAA position in ECU | | ||
+ | | VALN | EAA position in NC | | ||
+ | |||
+ | **Table: Connection between CAPRI and REGIO crop areas, crop production and herd sizes** | ||
+ | |||
+ | ^ SPEL-code ^ REGIO-code ^ REGIO-code ^ REGIO-code ^ REGIO-code ^ Description of SPEL activity ^ | ||
+ | | SWHE | WHEA | CERE | ARAB | | Soft wheat | | ||
+ | | DWHE | WHEA | CERE | ARAB | | Durum wheat | | ||
+ | | RYE | | CERE | ARAB | | Rye | | ||
+ | | BARL | BARL | CERE | ARAB | | Barley | | ||
+ | | OATS | | CERE | ARAB | | Oats | | ||
+ | | MAIZ | MAIZ | CERE | ARAB | | Maize | | ||
+ | | OCER | | CERE | ARAB | | Other cereals (excl. rice) | | ||
+ | | PARI | RICE | CERE | ARAB | | Paddy rice | | ||
+ | | PULS | | | ARAB | | Pulses | | ||
+ | | POTA | POTA | | ARAB | | Potatoes | | ||
+ | | SUGB | SUGA | | ARAB | | Sugar beet | | ||
+ | | RAPE | RAPE | OILS | ARAB | | Rape and turnip rape | | ||
+ | | SUNF | SUNF | OILS | ARAB | | Sunflower seed | | ||
+ | | SOYA | | OILS | ARAB | | Soya beans | | ||
+ | | OLIV | | OLIV | PERM | | Olives for oil | | ||
+ | | OOIL | | OILS | ARAB | | Other oil seeds and oleaginous fruits | | ||
+ | | FLAX | | | ARAB | | Flax and hemp \*\*\* (faser) \*\*\* | | ||
+ | | TOBA | TOBA | | ARAB | | Tobacco, unmanufactured, | ||
+ | | OIND | | | ARAB | | Other industrial crops | | ||
+ | | CAUL | | | ARAB | | Cauliflowers | | ||
+ | | TOMA | | | ARAB | | Tomatoes | | ||
+ | | OVEG | | | ARAB | | Other vegetables | | ||
+ | | APPL | | | PERM | | Apples, pears and peaches | | ||
+ | | OFRU | | | PERM | | Other fresh fruits | | ||
+ | | CITR | | | PERM | | Citrus fruits | | ||
+ | | TAGR | | VINE | PERM | | Table grapes | | ||
+ | | TABO | | OLIV | PERM | | Table olives | | ||
+ | | TWIN | | VINE | PERM | | Table wine | | ||
+ | | OWIN | | VINE | PERM | | Other wine | | ||
+ | | NURS | | | PERM | | Nursery plants | | ||
+ | | FLOW | | | ARAB | | Flowers, | ||
+ | | OCRO | | | ARAB | | Other final crop products | | ||
+ | | MILK | DCOW | | | | Dairy cows | | ||
+ | | BEEF | BUL2 | BUL3 | | | Bulls fattening | | ||
+ | | CALF | CALF | | | | Calves fattening (old VEAL) | | ||
+ | | PORK | PIG3 | PIG2 | PIG1 | | Pig fattening | | ||
+ | | MUTM | GOAT | SHEP | | | Ewes and goats | | ||
+ | | MUTT | GOAT | SHEP | | | Sheep and goat fattening | | ||
+ | | EGGS | POUL | | | | Laying hens | | ||
+ | | POUL | POUL | | | | Poultry fattening | | ||
+ | | OANI | | | | | Other animals | | ||
+ | | OROO | | | ARAB | | Other root crops | | ||
+ | | GRAS | GRAS | | | | Green fodder | | ||
+ | | SILA | GREF | | ARAB | | Silage | | ||
+ | | CALV | CALV | | | | Suckler cows | | ||
+ | | RCAL | CABM | CABF | | | Calves, raising | | ||
+ | | HEIF | H2SL | H2BR | H3SL | H3BR | Heifers | | ||
+ | | PIGL | SOW2 | | | | Pig breeding | | ||
+ | | FALL | | | FALL | | Fallow land | | ||
+ | |||
+ | **Tables: Codes of the input allocation estimation** | ||
+ | |||
+ | ^FADN inputs (FI) ^Label | ||
+ | | TOIN | total inputs | | ||
+ | | COSA | animal specific inputs | | ||
+ | | FEDG | self grown feedings | | ||
+ | | ANIO | other animal inputs | | ||
+ | | FEDP | purchased feedings | | ||
+ | | COSC | crop specific inputs | | ||
+ | | SEED | seeds | | ||
+ | | PLAP | plant protection | | ||
+ | | FERT | fertilisers | | ||
+ | | TOIX | other inputs (overheads) | | ||
+ | |||
+ | |||
+ | ^CAPRI inputs (CI) used in the reconciliation ^label ^ | ||
+ | | TOIN | total inputs | | ||
+ | | FEED | feedings | | ||
+ | | IPHA | other animal inputs | | ||
+ | | COSC | crop specific inputs | | ||
+ | | SEED | seeds | | ||
+ | | PLAP | plant protection | | ||
+ | | FERT | fertilisers | | ||
+ | | REPA | repairs | | ||
+ | | ENER | energy | | ||
+ | | SERI | agricultural services input | | ||
+ | | INPO | other inputs | | ||
+ | |||
+ | 1. The set of //Other// activities that had been omitted from the econometric estimation: | ||
+ | |||
+ | * OTHER={OCER, | ||
+ | |||
+ | 2. The set of activity groups, and their elements, used in the replacement or missing/ | ||
+ | |||
+ | * GROUPS = {YOUNG, VEGE, SETT, PULS, PIG, OILS, MILK, MEAT, INDS, HORSE, GOAT, FRU, FOD, FLOWER, DENNY, COW, CHICK1, CHICK2, CHICK3, CERE, ARAB} | ||
+ | * YOUNG={YBUL, | ||
+ | * VEGE={TOMA}, | ||
+ | * SETT={SETA, NONF, FALL, GRAS}, | ||
+ | * PULS=PULS | ||
+ | * PIG={PIGF, SOWS}, | ||
+ | * OILS={RAPE, SOYA, SUNF, PARI, OLIV}, | ||
+ | * INDS={TOBA, TEXT, TABO}, | ||
+ | * GOAT={SHGM, SHGF}, | ||
+ | * FRU={APPL, CITR, TAGR, TWIN}, | ||
+ | * FOD={ROOF, MAIF}, | ||
+ | * FLOWER={FLOW, | ||
+ | * DENNY={PORK, | ||
+ | * COW={DCOW, SCOW, HEIF, HEIR, CAMF, CAFF, BULF, CAMR, CAFR}, | ||
+ | * CHICK1={HENS, | ||
+ | * CERE={SWHE, DWHE, BARL, OATS, RYEM, MAIZ}, | ||
+ | * ARAB={POTA, SUGB} | ||
+ | |||
+ | 3. The sets of Northern European, Southern European countries: | ||
+ | |||
+ | * NEUR={NL000, | ||
+ | * SEUR={El000, | ||
+ | |||
+ | |||
+ | ** Table: Codes of land use classes (Set LandUse)** | ||
+ | |||
+ | ^Code ^Label | ||
+ | | OART | artificial | | ||
+ | | ARAO | (other) arable crops - all arable crops excluding rice and fallow (see also definition of ARAC below) | | ||
+ | | PARI | paddy rice (already defined) | | ||
+ | | GRAT | temporary grassland (alternative code used for CORINE data, definition identical to TGRA | | ||
+ | | FRCT | fruit and citrus | | ||
+ | | OLIVGR | Olive Groves | | ||
+ | | VINY | vineyard (already defined) | | ||
+ | | NUPC | nursery and permanent crops (Note: the aggregate PERM also includes flowers and other vegetables | | ||
+ | | BLWO | board leaved wood | | ||
+ | | COWO | coniferous wood | | ||
+ | | MIWO | mixed wood | | ||
+ | | POEU | plantations (wood) and eucalyptus | | ||
+ | | SHRUNTC | shrub land - no tree cover | | ||
+ | | SHRUTC | shrub land - tree cover | | ||
+ | | GRANTC | Grassland - no tree cover | | ||
+ | | GRATC | Grassland - tree cover | | ||
+ | | FALL | fallow land (already defined) | | ||
+ | | OSPA | other sparsely vegetated or bare | | ||
+ | | INLW | inland waters | | ||
+ | | MARW | marine waters | | ||
+ | | KITC | kitchen garden | | ||
+ | |||
+ | |||
+ | ** Table: Codes of land use aggregates (Set LandUseAgg)** | ||
+ | |||
+ | ^Code ^Label | ||
+ | | OLND | other land - shrub, sparsely vegetated or bare | | ||
+ | | ARAC | arable crops | | ||
+ | | FRUN | fruits, nursery and (other) permanent crops | | ||
+ | | WATER | inland or marine waters | | ||
+ | | ARTIF | artificial - buildings or roads | | ||
+ | | OWL | other wooded land - shrub or grassland with tree cover (definition to be discussed) | | ||
+ | | TWL | total wooded land - forest + other wooded land | | ||
+ | | SHRU | shrub land | | ||
+ | | FORE | forest (already defined) | | ||
+ | | GRAS | grassland (already defined) | | ||
+ | | ARAB | arable (already defined) | | ||
+ | | PERM | permanent crops (already defined) | | ||
+ | | UAAR | utilizable agricultural area (already defined) | | ||
+ | | ARTO | total area - total land and inland waters | | ||
+ | | ARTM | total area including marine waters | | ||
+ | | CROP | crop area - arable and permanent | | ||
+ | |||
+ | **Table: Codes of mutually exclusive subset adding up to total area - ARTO (Set LandUseARTO)** | ||
+ | |||
+ | ^Code ^Label | ||
+ | | OLND | other land - shrub, sparsely vegetated or bare | | ||
+ | | ARTIF | artificial - buildings or roads | | ||
+ | | FORE | forest | | ||
+ | | UAAR | utilizable agricultural area | | ||
+ | | INLW | Inland waters | | ||
====Annex: Detailed description of Eurostat data processing in COCO (coco1_eurostat.gms)==== | ====Annex: Detailed description of Eurostat data processing in COCO (coco1_eurostat.gms)==== | ||
Line 1201: | Line 1413: | ||
The COCO module produces various reporting files on the intermediate data processing steps. These files can be used to trace back potential errors in the COCO database to their origin. These debugging files also contain meta-information on the input data and settings used for producing the COCO database. | The COCO module produces various reporting files on the intermediate data processing steps. These files can be used to trace back potential errors in the COCO database to their origin. These debugging files also contain meta-information on the input data and settings used for producing the COCO database. | ||
+ | The following example is a walk-through on the typical data processing steps, covring biofuels data preparation in France. | ||
- | In addition | + | The reporting file ' |
+ | |||
+ | {{coco_biof_1.png? | ||
+ | |||
+ | The set // | ||
+ | |||
+ | {{coco_biof_2.png? | ||
+ | |||
+ | The complete list of the biofuel items in // | ||
| bioECere | Ethanol processed from cereals | | | bioECere | Ethanol processed from cereals | | ||
Line 1226: | Line 1447: | ||
| INDt | Sum of model results for BIOF and INDM | | | INDt | Sum of model results for BIOF and INDM | | ||
| BIOi, INDi, DOMi | intermediate activities to save data from model initialisation for later documentation. | | | BIOi, INDi, DOMi | intermediate activities to save data from model initialisation for later documentation. | | ||
+ | |||
+ | |||
+ | Biofuels production (levels) are calculated for biodiesel (BIOD) and bioethanol (BIOE). Input data and final initialization values before the consistency models are run are documented on the parameter // | ||
+ | |||
+ | |||
+ | |||
+ | **Example 1: Bioethanol** | ||
+ | |||
+ | The screenshot demonstrates the input data and final initialization values collected on parameter // | ||
+ | |||
+ | {{coco_biof_3.png? | ||
+ | |||
+ | The results of the model m_bioFitE (BIOE) are documented on the parameter p_biofDatatMS. | ||
+ | |||
+ | {{coco_biof_4.png? | ||
+ | |||
+ | We take soft wheat (SWHE) as an example for biofuel feedstock, and walk through the initialization and consistency model results. From data input (Eurostat and FAO) we received in 2002 an industrial use of 894 1000t, saved on INDi. For production of bio-ethanol 631 1000t were initialized, | ||
+ | |||
+ | |||
+ | |||
+ | **Example 2: Biodiesel** | ||
+ | |||
+ | |||
+ | The first dimension of the reporting parameter // | ||
+ | The second dimension of the parameter shows the activity. | ||
+ | |||
+ | {{coco_biof_5.png? | ||
+ | |||
+ | For Bio-diesels, | ||
+ | |||
+ | ^Data source code ^Data source description ^ | ||
+ | |Primes |PRIMES MODEL, EC3MLAB of ICCS, National University of Athens| | ||
+ | |||
+ | {{coco_biof_6.png? | ||
+ | |||
+ | The parameter // | ||
Line 1231: | Line 1488: | ||
====Annex: Testing procedure and checking intermediate steps in COCO (dairy)==== | ====Annex: Testing procedure and checking intermediate steps in COCO (dairy)==== | ||
- | The following three examples show how to use the intermediate reporting files to trace the data preparation steps. Screenshots demonstrate the arrangement of the reporting parameters by using the CAPRI Graphical User Interface. | + | The following three examples show how to use the intermediate reporting files to trace the data preparation steps. Screenshots demonstrate the arrangement of the reporting parameters by using the CAPRI Graphical User Interface. |
+ | |||
+ | |||
+ | **Example 1: Production of cow (COMI) and sheep (SGMI) milk** | ||
+ | |||
+ | In order to document the procedure of data consolidation and rebooking, we look at the reporting file for France “chk_estima_FR000.gdx”. | ||
+ | |||
+ | {{: | ||
+ | |||
+ | The codes in the rows show the activity code, the product code and its status. For activity codes see Annex 1: Code list. | ||
+ | |||
+ | Status codes: | ||
+ | |||
+ | * INI: | ||
+ | * COCO1: | ||
+ | |||
+ | The initialization of the production of COMI and SGMI is done in the module // | ||
+ | |||
+ | * COMI: Milk from cows (CMLK) and buffaloes (BMLK) is added up. | ||
+ | * SGMI: Milk from ewes (EMLK) and goats (GMLK) is added up. | ||
+ | * If data on cow or sheep and goat milk is not available separately, but total milk production (RMLK) is available, then production of COMI is set equal to total milk production. | ||
+ | * Only COMI and SGMI are included in the estimation in // | ||
+ | * The production of RMLK and its components CMLK, BMLK, EMLK and GMLK are only copied from raw data tables into this check parameter for documentation purposes. | ||
+ | |||
+ | |||
+ | ** Example 2: data consolidation for cow milk** | ||
+ | |||
+ | The procedure of data consolidation and rebooking of all activities for the CAPRI product “COMI” (cow milk) is shown in the following screenshot (only part of the reporting parameter p_estimAnimMS is shown, but the full scope of the table is visible in the GUI). | ||
+ | |||
+ | {{: | ||
+ | |||
+ | |||
+ | The codes in the rows show the activity code and its status. For activity codes see Annex 1: Code list. | ||
+ | Additional codes for status include the following. | ||
+ | |||
+ | ^Status code | ||
+ | |StdeData |Final (small) Stde (standard deviation) attached to priors from raw data | | ||
+ | |StdeScale |Final (large) Stde attached to priors from trends but not from raw data | | ||
+ | |Upplim |Soft upper limits triggering extra penalties if violated | ||
+ | |Lowlim |Soft lower limits triggering extra penalties if violated | ||
+ | |Supps |Prior value = support: comes from raw data or trends plus HP filter | ||
+ | |Err2rev |Original error term from preest: to steer speed of bound opening | ||
+ | |||
+ | |||
+ | Under activity dairy cows (DCOW) the following items are reported: yield, total production (GROF), feed use (FEDM) and losses on market (LOSM). Eurostat’s //National Accounts of Agriculture (EAA)// only supply data for the aggregate milk (MILK). The equation // | ||
+ | |||
+ | <code gams> | ||
+ | e_EAAMLK(" | ||
+ | $ (p_NobsP(" | ||
+ | (p_NobsP(" | ||
+ | * | ||
+ | v_EstimY(" | ||
+ | | ||
+ | + v_EstimY(" | ||
+ | |||
+ | </ | ||
+ | |||
+ | Finally. the producer prices (UVAP) are calculated directly from the monetary EEA values and production. | ||
+ | The following picture shows the data processing steps (states) for the EAA values for milk. | ||
+ | |||
+ | {{: | ||
+ | |||
+ | From the example for COMI above you can also understand the influence of the standard deviation from raw data (e.g. FEDM.StdeData), | ||
+ | |||
+ | The first initialisation of // | ||
+ | |||
+ | |||
+ | |||
+ | **Example 3: data consolidation for cow dairy cow activity (DCOW)** | ||
+ | |||
+ | The procedure of data consolidation and booking intermediate data processing results for the dairy cow activity (DCOW) is demonstrated in the following screenshot. | ||
+ | |||
+ | {{: | ||
+ | |||
+ | The rows of the table show the product item code for the production activity DCOW, and the data processing steps (status). The first two lines show the coco1 results for slaughtering. The items starting with Y and I stand for the output and input of calves. The initialization, | ||
+ | |||
===== The Regionalised Data Base (CAPREG) ===== | ===== The Regionalised Data Base (CAPREG) ===== | ||
==== Data requirements and sources at the regional level ==== | ==== Data requirements and sources at the regional level ==== |
the_capri_data_base.txt · Last modified: 2024/07/05 06:24 by massfeller