CAPRI Online Manual (update)

After the Task on Trends generation have been successfully completed, meaning that the projections for the defined (in GUI or a batch file) future years (currently, 2015, 2020, 2025 and 2030 are available) have been produced, the next step in the Baseline generation process (“Generate baseline” workstep in CAPRI GUI) is to calibrate the CAPRI global trade model. In the CAPRI GUI this refers to the task “Baseline calibration of market model”.

The calibration of the market model is steered by the C:\…\CAPRI\gams\capmod.gms file. The relevant parts of the code are activated by setting the setglobal 'BASELINE' to ON.

The CAPRI database is composed of many different data sources, and requires data processing before the market model equations can be calibrated against the data set. Sources of potential problems include missing data and price-quantity framework that is inconsistent with the behavioural assumptions (e.g. profit maximizing producers, utility maximizing consumers).

Stage I of the market model calibration makes the CAPRI database consistent, and creates a dataset for the global agri-food markets against which the market model can be calibrated. As CAPRI is a comparative static model, the market model is calibrated only against the simulation year. But technically the CAPRI dataset is first made consistent to the model structure in the base year, and then shifted to the simulation year. More specifically the main steps in this stage include:

1. Prepare the necessary data by
   1. loading them from various intermediate data files;
   2. mapping them to correct code lists;
   3. adjusting if necessary, often by applying security bounds;
2. Ensure the consistency of the dataset to the market model structure for the base year (BAS)
3. Shift the consistent dataset from the base year to the simulation year
4. Ensure the consistency of the dataset to the market model structure for the simulation year (SIMY)

Before actually performing the calibration of the market model parameters, CAPRI first loads the necessary sets, parameters and data. These refer to periods (years), regions, activities, commodities, agricultural policies (e.g., premiums, quotas, rural development payments, set-aside requirements), environmental indicators, feed and fertilizer requirements, nutrient content of the commodities, global warming potentials, and other necessary input. The data loaded also includes two very important for this calibration step files: C:\…\CAPRI\output\results\capreg\res_BBCC.gdx and C:\…\CAPRI\output\results\baseline\trends_BBYY.gdx. The first file, res_BBCC.gdx, includes the results of generation of data for the base year (BB, currently 2012) for European countries and Turkey (CC) at NUTS0, NUTS1 and NUTS2 aggregation levels (GUI workstep “Build database”, task “Build regional database”). The second file, trends_BBYY.gdx, includes the results of trends generation task (see sections above) for all of the European countries and Turkey at NUTS0, NUTS1 and NUTS2 aggregation levels for the target simulation year (currently, 2030).

Constraints, requirements, policies and other data loaded including base year and trends data (i.e., of res_BBCC.gdx and trends_BBYY.gdx files) are subject to certain (mainly non-major) adjustments, additional calculations and assumptions that serve the purposes of data balancing, checking and provision of necessary for the calibration information. These include, for example, deleting positions not needed during the calibration run, (re-)assigning parameter names, deleting tiny quantities, checks for production without activity levels, possible empty projections and negative inland waters, setting the output coefficients for young animals equal to the ones at country (EU MSs, as young animals are not presented in the non-EU countries) level if missing at regional level, correcting fat and protein content of raw milk, assumption that second generation biofuels are produced 50/50 by agricultural residuals and new energy crops, etc.

Next, FAO data on the non-European countries as well as the trade flows among all of the countries (country trade blocks) accounted for in CAPRI are loaded. These FAO data together with the European data, which has already been subjected to certain adjustments as described in the previous paragraph, undergo the, so-called, data preparation step. This process is controlled by C:\…\CAPRI\gams\arm\market1.gms file which calls the C:\…\CAPRI\gams\arm\data_prep.gms file - specifically for this step. The data preparation step mostly refers to the base year and includes: among else, modification of GDP to fit the sum of final household expenditure, final government expenditure, gross capital formation and current account balance; import and export flows to be in line with net trade from production minus demand; scaling of demand side to fit production plus net trade; estimation of consumer prices for some countries, if missing; calculation of nutrient consumption per head and day as net of losses in distribution and households; scaling of outliers in prices etc. This step as well provides with estimation of yearly change factors beyond the base year: for prices, GDP, population, quantities and areas. Additionally, i) substitution elasticities (i.e., p_rhoX, where X indicates continuation of the parameter name) for bio-fuel feedstocks, feed, dairy products, sugar, table grapes, tobacco, cheese, fresh milk products, fruits, vegetables, distilled dried grains and rice for the CAPRI demand system¹⁾, and ii) transformation elasticity for oil seed processing and land supply elasticities are assigned.

Together with the data, equations of the CAPRI market module are loaded. They are described in detail in section Market module for agricultural outputs. These equations include behavioural functions for market demand including expenditure function, feed demand, blocks for dairy products, oilseeds processing and biofuels, netput functions, trade equations and balances, equations for prices and price transmission, functions for trade policies and for intervention stocks. There are additionally two crucial for data calibration functions: minimization of deviation of estimated values from the observed data. These two functions are described in detail later in this section.

After data preparation, data calibration for the base (currently, 2012) and simulation years (currently, 2030) take place. The main file steering the data balancing process is C:\…\CAPRI\gams\arm\data_cal.gms, which in turned is included in arm\market1.gms.

Data balancing for the base year

Data calibration for the base year aims at modifying the base year data to fit the system of equations of the market module. Some of the parameters defined in Stage I (e.g., p_rhoX) as well as parameter values and bounds defined at this stage are used. For example, starting points and corridors for quantity variables are set (e.g., calculating of world production to define correction corridor for calibration of production/demand/trade flows globally), global TRQ data are converted into ad valorem tariffs and checked for consistency and completeness, policy variables for the EU market model such as e.g., intervention stocks, are loaded. Also, starting values for prices of dairy products are estimated. In particular, a non-linear programming model is used, where the objective function is formulated as a Highest Posterior Density function. The value of this objective function equals sum of squared deviations of fat and protein prices, fat and protein content of milk products and processing margins of milk products from the respective means, weighted with the a priori variances. The means are defined as parameters based on the prices and fat and protein content of milk in the base year. The objective function is restricted by the balance: fat and protein of raw milk delivered to dairies shall equal fat and protein content of dairy products. The model is solved by minimizing the value of highest posterior density, hence minimizing the differences between the variables and their means. Prices of milk products are then defined as: product of fat and protein content and of far and protein prices plus processing margin. Furthermore, administrative prices for cereals and dairy products, and minimal import prices for cereals are constructed.

With the file C:\…\CAPRI\gams\arm\cal_models.gms, the so-called, models, used in calibration of data base are defined and solved. These models represent collection of equations, solutions of which provide with parameter values used for data calibration. The first model (MODEL m_trimSubsExports) calibrates the parameters of the function which defines the values of subsidized exports with and without the increase of market price above the administrative price. The second model (MODEL m_trimInterv) defines parameters of equations for intervention stock changes. It includes an objective function defined as a sum of: squared scaled difference of estimated and observed intervention stock changes and squared scaled parameters for behavioural function of intervention stock changes. This objective function is minimized subject to constraints represented by equations for intervention sales, probability for an undercut of administrative price, release from intervention stock, intervention stock changes and value of the intervention stock. The constraints are equations of the market model (see section Market module for agricultural outputs).

The model that calibrates base year data (MODEL m_calMarketBas) is defined in cal_models.gms file as well and includes almost all equations of the market model. In particular: equations for processing margin for dairy products (ProcMargM_), fat and protein balance between raw milk and dairy products (FatsProtBal_), processing margin for oilseeds ProcMargO_, processing yields of oilseeds (procYield_), 1st generation output of biofuels (prodBiof_) and total output of biofuels (MaprBiof_); balancing and adding up equations: equations which add production, processing demand, human consumption, feed demand quantities and quantities for processing from single countries (or block of countries) to trade blocks (ProdA_, ProcA_, HconA_, FeedUseA_, Proca_), adding up inside of the Armginton aggregate (total domestic consumption) (ArmBal1_), supply balance (SupBalM_) and imports and exports added up to bilateral trade flows (excluding diagonal element) (impQuant_); price equations: 1st stage Armington quantity aggregate (ArmFit1_), 2nd stage Armington quantity aggregate (ArmFit2_), import price relation to producer price (impPrice_), consumer price as average of domestic and import prices (arm1Price_), average price as average of different import prices (arm2Price_), average import price (arm2Val_), consumer price (Cpri_), producer price (PPri_), market price (PMrk_), average market price (MarketPriceAgg_); trade and tariff equations: aggregated trade flows (TradeFlowsAgg_), average transportation costs (TransportCostsAgg_), sum of imports under a non-allocated TRQ (TRQImports_), share of the tariff applied for the EU entry price system (EntryPriceDriver_), tariff specific entry price (tarSpecIfEntryPrice_), Cif price (cifPrice_), equation for defining levy (replaces tariff) in case of minimal border prices (FlexLevyNotCut_), cuting flexible levy by specific tariff if it exceeds the bound rate (FlexLevy_), tariffs under bi-lateral TRQs (trqSigmoidFunc_), specific tariffs as function of import quantities, if TRQ is present (tarSpec_, prefTriggerPrice_), tariffs under globally open (not bilaterally allocated) TRQs (tarSpecW_), ad valorem tariffs, if TRQ is present (tarAdval_), ad valorem tariff under not bilaterally allocated TRQs (tarAdValW_), export quantities from bi-lateral trade flows (expQuant_), exports included in the calculation of the export unit values excluding flows under double-zero agreements (nonDoubleZeroExports_), unit value exports (unitValueExports_, valSubsExports_), subsidised export values (EXPs_); equations for intervention stocks: probability weight for an undercut of administrative price (probMarketPriceUnderSafetyNet_), intervention sales (buyingToIntervStock_), intervention stock end size (intervStockLevel_), intervention stock changes (intervStockChange_), release from intervention stocks (releaseFromIntervStock_), aggregators for intervention purchases; equation for world market price (wldPrice_), and equation for minimization of deviation from given base year data and estimated data (NSSQ_). The model is solved by minimizing the SSQ value of NSSQ equation which is constrained by all of the rest of the equations included in the model.

The NSSQ equation is crucial to the data calibration as it, in its essence, minimizes the difference between the estimated and the observed (already adjusted at the previous stage) data of the base year. Its logic is analogues to the one of equation below:

im dokument nicht nummeriert \begin{equation} SSQ\cdot \sum_{RMS} \sum_{XXX} p\_weight_{RMS}^i=\sum_{RMS} \sum_{XXX} \left( \frac{v_{RMS,XXX}^i-DATA_{RSM,XXX,BAS}^i}{max(DATA_{RSM,XXX,BAS}^i,0.1) \cdot p\_weight_{RMS}^i} \right)^2 \end{equation}

where SSQ is an artificial variable to be minimized, indices RMS, XXX, BAS and i indicate, respectively, regions, commodities, base year and activities (e.g., production, processing, imports etc.), and p_weight is a parameter of weights between 1 and 100 assigned to regions and activities. These weights are necessary to achieve plausible calibrated values and their specification is the outcome of a trial and error process, inspecting results from data calibration and retrying. They depend on the results of global database and trends generation. On the right hand-side of the equation v stands for a variable to be estimated and DATA – for base year data already adjusted at the data preparation and balancing stage. Hence with this equation squared sum over regions and commodities of differences between estimated and observed values (and or quantities), these differences being scaled by the observed data times the weight parameters, is minimized. Respectively, calibrated base year data fits the system of the market equations, given certain parameter values, and resembles the observed data as closely as possible. The activities implied under the I index include quantities of production, human consumption, feed, processing, processed to biofuels, import and export, producer, consumer and market prices, difference between market prices and import prices to reduce differences between physical and Armington aggregation, consolidated gap between producer and market prices, processing margin, trade flows and transport costs.

The process of model solving is navigated with C:\…\CAPRI\gams\arm\data_fit.gms file. Its main function is to assure model solving by keeping the market balances closed and price system consistent. Because of the very large number of equations with the exact similar number of variables (36 thsds) that makes the system of equations square, as well as non-linear formulation of some of the equations, it is very likely that infeasibilities will occur during the model solving. To ensure the feasibility as far as possible, code elements such as widening of variable bounds, once they become binding, reducing non-smoothness of the functional forms and introduction of slack variables are introduced. More detailed information on this process can be found in a technical document by Wolfgang Britz and Heinz-Peter Witzke Infeasibilities in the market model of CAPRI – how they are dealt with at https://www.capri-model.org/docs/infes.pdf.

After solving the MODEL m_calMarketBas, the calibrated data are stored, new producer prices for agricultural outputs are set, sugar beet prices as a function of – sugar market price – sugar export price (pre-reform) or ethanol market price (post-reform) – processing yield (specific to CUR to calibrate to any set of projected beet prices) – levying model for A- and B- sugar (pre-reform) are calculated, share and shift parameters of CES-functions used in the Armington approach to determine import shares as a function of import prices are defined (file C:\…\CAPRI\gams\arm\cal_armington.gms). Furthermore, energy conversion factors for animal products are defined with MODEL m_fitFeedConv (in file C:\…\CAPRI\ gams\arm\feed_conv_decl.gms).

Data balancing for the simulation year
Aim of data calibration for the simulation year aims at generating such quantity, price and other market values (see list below) for the simulation year that they fit the system of equations of the market module and variable and parameter lower and upper bounds, as well as remain as close as possible to the values to which they are calibrated (e.g., trends, estimated with growth rates from the base year, Aglink-COSIMO values, GLOBIOM values etc.). Thus process, basically, follows similar approach as for the base year. There are, however, a few differences. The main is that the model used for calibration is MODEL m_calMarketFin. As the model for base year calibration (MODEL m_calMarketBas), it is defined in cal_models.gms file and includes similar equations of the market model with the exception of NSSQ_ equation. The latter equation is replaced by NSSQ1_. Its major difference from NSSQ_ is that DATA parameter includes not values of the base year, but values projected in trend generation step for some of the factors and values shifted to the simulation year based on assumptions or growth rates for the other factors. Thus, it is used for minimizing the differences between estimated and projected (with trend generation step or growth rates) values of the variables in question. Another difference of NSSQ1_ with NSSQ_ is that it includes the differences in intervention stock changes and excludes the differences in consumer prices and gaps between producer and market prices.

Before MODEL m_calMarketFin is solved, values of DATA parameter for the simulation year are defined. For example, administrative prices for dairy products and cereals and minimum import prices for cereals (in C:\…\CAPRI\gams\arm\prep_pol.gms) and policy data are defined, market prices, quantity variables are shifted with growth rates (C:\…\CAPRI\gams\arm\shift_quantities.gms) and tariffs are defined. Bounds for tariff variables, market prices, milk fat and protein as well as upper and lower limits on quantity variables are assigned as well. At this point, models to calibrate TRQs and entry price equations (MODEL m_fitTrq) and parameters of equations for the intervention stock changes (MODEL m_trimInterv) are solved as well (now for the simulation year, as before it was solved for base year values).

As m_calMarketBas model, m_calMarketFin model is solved by minimizing SSQ value by applying the approach of assuring feasibility via data_fit.gms file. After the solution is found and energy conversion factors for animal products are defined with MODEL m_fitFeedConv, the results are stored in C:\…\CAPRI\ output\results\baseline\data_market_1230.gdx.

Elasticity trimming in CAPRI aims at adjusting prior estimations of elasticities so that

• the behavioural functions can be parameterized/calibrated to the given prices/quantity framework with the elasticities;
• the calibrated elasticities satisfy regulatory conditions (homogeneity, additivity) and correct curvature in line with microeconomic theory;
• the calibrated elasticities are as close as possible to prior elasticities (minimize deviation).

At first, parameters for land use market are calculated based on data from FAO world food market model. Among them are land use classes, crop yields, land demand of non-crop activities, areas used for fodder and average land price, total energy use for feeding and producer price of feed. Next, starting elasticity values, as well as their lower and upper bounds are loaded (e.g., demand elasticities used in SPEL/MFSS). Finally, elasticities are trimmed.

Elasticities trimming is controlled by C:\…\CAPRI\gams\arm\trim_par.gms file. The elasticity groups are: for calibration of demand and supply systems, feed demand system, oilseeds crush, oil processing and dairy industry. Elasticities of supply system, oilseeds crushing, oil processing and dairy industries, as well as for feed demand, are estimated with MODEL m_trimElas. It is solved by minimising absolute squares between given and calibrated elasticities including land elasticities (FitElas_) subject to the following constraints: marginal effects from price and quantity for current elasticity estimate (Hess_), homogeneity of degree zero for elasticities in prices (HomogN_), Cholesky decomposition of marginal effects to ensure correct curvature (Chol_), Ensure that own price elasticity exceeds yield elasticity * 1.5 (YieldElas_) and elasticities for total energy and protein intake from feeding (ReqsElas_).

Human consumption elasticities are estimated with MODEL m_trimDem by minimizing absolute squares between given and calibrated elasticities (FitElas_). Apart from the objective function the model includes several equations related to the definition of the demand system as Generalized Leontief, homogeniety of degree zero for elasticities in prices, additivity of income elasticities weighted with budget shares and elasticities for total calorie intake.

In this stage, the feed system is calibrated against the primary product prices of the market model (both marketable and non-marketable feed). The nutrient requirements of the crops are calculated together with the nutrient and energy requirement of the animal production activities.

The fertilizer flows are also calibrated here. The prior parameters for the fertilizer flows are defined based on the posterior mode of the base year, by modifying them with land use changes: the fertilization per ha is computed in the base year situation and then multiplied with the areas in the calibration point. The fertilizer flows are calibrated with the same calibration model as used for the base year in the database tasks.

The file C:\…\CAPRI\gams\capmod\def_fert_and_requirements.gms defines animal nutrient requirements and the nutrient requirements of the crops given trend forecasted yields. In particular, feed input coefficients are defined and calibrated, days in production process of fattening are defined, and manure output is taken into consideration as an input for fertilizer calibration. Fertilizer calibration is basically a merge of trend based forecasts from the ex-post CAPREG results. The fertilizer need is calculated as a function of yield, and adjusted according to the exogenous assumptions. Furthermore, crop nutrient need factors from trends are scaled and logistic function is used to calculate average growth rate of fertilizer use. The calculations must as well comply with the fertilizer equations of the supply model.

After the behavioural blocks of the market model are calibrated (one-by-one), the whole model should be also tested for being correctly calibrated. In essence, the test initializes the model with the data against the model was calibrated, and then executes/solves the market model. In theory, a perfectly calibrated model can be solved in one single iteration, without adjustments in the values of the model variables. That is why the iteration limit is technically set to zero (i.e. not allowing for adjustment in the model variables) for the test solve. In practice, a number of infeasibilities might exist due to the accuracy of the numerical solution. But infeasibilities stemming from rounding errors must be small, so the sum of all infeasibilities gives a good indication on the quality of the model calibration.

At the final stage, some of the starting values and bounds for the market model are set, and agricultural policy data are loaded, adjusted and extended to the simulation year. The policy data include single area payment scheme, set-aside regulations, differentiation between old and new MSs payments, special national envelopes, Nordic schemes, changes in administrative prices, rural development policy and other major CAP post-2014 instruments. Policy files used for the baseline are located in C:\…\CAPRI\ gams\scen\base_scenarios folder. Their loading into the baseline process is controlled by CAP_2014_2020.gms file. With the data mentioned, the outcome of calibration of the CAPRI market module can be tested. In particular, the market model is solved at “trend values” and, thus, the calibration outcome is checked for fitting to the square system of market model equations. This is controlled by C:\…\CAPRI\.gams\arm\prep_market.gms file.

Note that the task “Baseline calibration of market model” deletes the sim_ini.gdx file, but does not create a new one at the end of the calibration process. The new sim_ini.gdx file will be only created at the first simulation run after the calibration. That is also the reason why a specific GUI option 'Kill simini file' is provided for the simulation tasks. The simini file can be deleted upon request at the beginning of any scenario run, forcing CAPRI to re-create it before the scenario shock is introduced.

Technically, the calibration of the biofuel demand system and the Armington bilateral trade system is not directly linked to the BASELINE mode, but also executed every time when the simini file is missing (by create_sim_ini_gdx module).