Differences

This shows you the differences between two versions of the page.

--- the_complete_and_consistent_data_base_coco_for_the_national_scale [2020/02/13 08:04] – [COCO2: Data Preparation] matsz
+++ the_complete_and_consistent_data_base_coco_for_the_national_scale [2020/02/13 09:39] – [COCO2: Estimation procedure] matsz
@@ Line 552: / Line 552: @@
 ====COCO2: Estimation procedure====
+**Include file //‘coco2_def.gms’//**
+The approach to determine consumer prices is to distribute food expenditure on groups with consumption quantities given from COCO1 results such that endogenous consumer prices link endogenous expenditure with exogenous quantities. Deviations of estimated expenditure and consumer prices from their supports is penalised in an entropy framework. Estimation is done year by year, starting with the most recent year where hard data are usually available to a greater extent than for the oldest years in the database. Including consumer price changes (always relative to the previously solved year) serves to stabilise the results to some extent such that the objective does not only have supports for the consumer prices, but also for their changes. The entropy problem is solved by maximizing:
+\begin{align}
+\begin{split}
+max_t &- \sum_{m,j,k} CPS_{m,j,2}*HCOM_{m,j,k}/1000/TOFO_{m,t}*\\
+&PE_{m,j,k}*LOG(PE_{m,j,k}/PQ_k)\\
+&-\sum_{m,j,k} CPS_{m,j,2}*HCOM_{m,j,k}/1000/TOFO_{m,t}*\\
+&PED_{m,j,k}*LOG(PED_{m,j,k}/PQ_k)\\
+&-\sum_{m,FOPOS,k} EXS_{m,FOPOS,2}/TOFO_{m,t}*\\
+&PEX_{m,FOPOS,k}*LOG(PEX_{m,FOPOS,k}/PQ_k)\\
+&-\sum_{m,j,k} PFAC_{m,k}*LOG(PFAC_{m,,k}/PQ_k)*1000\\
+\end{split}
+\end{align}
+where //m// represents the region, //j// the food item with consumer price, FOPOS the food group, //t// stands for the current estimation year, t_1 for the year estimated before and k for the number of support points (=3).
+Parameters are
+| \(HCOM_{m,j,t}\) |Human consumption, result from COCO1|
+| \(UVAD_{m,j,t\_1}\)	|Consumer price from last simulation of year t+1|
+|\(CPS_{m,j,k}\)	|Support points for consumer prices |
+|\(DCPS_{m,j,k}\)	|Support points for consumer price changes|
+|\(EXS_{m,FOPOS,k}\)	|Support points for group expenditures|
+|\(TOFACS_{m,k}\)	|Support points for total food expenditure slack|
+|\(PQ_k\)		|A priori probabilities for support points|
+|\(TOFO_{m,t}\)	|Total food expenditure |
+|and entropy variables||
+|\(PE_{m,j,t}\)	|Probability of support points for consumer prices|
+|\(PED_{m,j,t}\)	|Probability of support points for consumer price changes|
+|\(CP_{m,j}\)	|Consumer prices|
+|\(DCP_{m,j}\)	|Consumer price changes|
+|\(PEX_{m,FOPOS,t}\)	|Probability of support points for group expenditure|
+|\(PFAC_{m,k}\)	|Probability of support points for food expenditure slack|
+|\(EX_{mFOPOS}\)	|Group expenditures|
+|\(TOFAC_m\)	|Food expenditure slack|
+Constraints are as follows:
+Summing up probabilities for support points
+\begin{equation}
+\sum_{k\forall_{m,j}(CP.L_{m,j}\ge 0\wedge HCOM_{m,j,i}\ge 0)} PE_{m,j,k}=1
+\end{equation}
+\begin{equation}
+\sum_{k\forall_{m,j}(DCPS_{m,j}\ge 0\wedge HCOM_{m,j,i}\ge 0)} PE_{m,j,k}=1
+\end{equation}
+\begin{equation}
+\sum_{k\forall_{m,j}(EX.L_{m,FOPOS}\ge 0)} PE_{m,FOPOS,k}=1
+\end{equation}
+\begin{equation}
+\sum_{k\forall_{m}(TOFAC.LO_m\ge TOFAC.UP_m)} PFAC_{m,k}=1
+\end{equation}
+Define consumer price changes from support points
+\begin{equation}
+DCP_{m,j} = \sum_{k\forall_{m,j}(CP.L_{m,j}\ge 0\wedge HCOM_{m,j,i}\ge 0 \wedge DCPS_{m,j,2}\ge 0)} PED_{m,j,k}*DCPS_{m,j,k}
+\end{equation}
+Of course consumer prices changes are also related to the last simulation result (which is for T+1 due to backward looping)
+\begin{equation}
+DCP_{m,j} =UVAD_{m,j,t\_1}-CP_{m,j}
+\end{equation}
+Define consumer prices from support points and probabilities
+\begin{equation}
+CP_{m,j} = \sum_{k\forall_{m,j}(CP.L_{m,j}\ge 0\wedge HCOM_{m,j,i}\ge 0)} PE_{m,j,k}*CPS_{m,j,k}
+\end{equation}
+Define group expenditure from support points and probabilities
+\begin{equation}
+EX_{m,FOPOS} = \sum_{k\forall_{m,j}(EX_{m,FOPOS}\ge 0)} PEX_{m,FOPOS,k}*EXS_{m,FOPOS,k}
+\end{equation}
+Define total expenditure slack from support points and probabilities
+\begin{equation}
+TOFAC_m=\sum_{k\forall_{m}(TOFAC.LO_m\ge TOFAC.UP_m)} PFAC_{m,k}*TOFACS_m
+\end{equation}
+Exhaustion of food expenditure may be relaxed with a slack factor different from one. However, this “last resort” to achieve feasibility in the expenditure allocation problem is limited to years and countries with precarious data and subject to strong penalties.
+\begin{equation}
+\sum_{FOPOS} EX_{m,FOPOS}=TOFO_{m,t}*TOFAC_{m,k}
+\end{equation}
+Consistency of group expenditure
+\begin{equation}
+EX_{m,FOPOS}=\sum_{j\forall_{m,FOPOS}(j\in FOPOS\wedge HCOM_{m,j} \ge 0)}CP_{m,j}*HCOM_{m,j}/1000
+\end{equation}
+For most countries the exhaustion of total expenditure is the only evident hard constraint (and even this is relaxed in problem cases). However, as the penalties for group expenditure are set high, and furthermore as the range of expenditure supports defines additional implicit hard constraints, the problem may turn out infeasible (typically solved by additional leeway). To meet the expenditure constraints the solver would tend to concentrate deviations from supports on the most important expenditure items while setting the less important items close to their supports. A more balanced distribution of deviations from supports was achieved in practice by weighting all contributons to the overall objective (except the last one for the total expenditure slack) with expected expenditure shares. The weights may be interpreted as expected expenditure shares because supports are specified in a symmetric way such that the central, second (of three) supports, which is used in the objective function, is equal to the expectation.
+**Include file //‘coco2_solve.gms’//**
+The initialisation, solving, reporting and storage is organised in the next include files with a few elements worth mentioning
+  * The initialisation tries to ensure positive consumer margins by the assignments of expected values and by specifying bounds on estimated consumer prices. The reference point for these margins is an average of EU and national prices that reflects the importance of domestic sales vs. imports.
+  * Bounds and spread of supports around expected consumer prices are set high for items without ILO style prices (say “table olives” TABO) or where the fit of available price information is questionable (e.g. cabbage prices for “OVEG”).
+  * A checking parameter (“p_checks”) permits to check the iniitalisation in case of infeasibilites. The most frequent case observed in the last years is that lower bounds on oils expenditure become binding, suggesting the need for some systematic mismatch of price and expenditure information for this group.
+====COCO2: Final completions====
+At this point it may be motivated why there is at all a need for a COCO2 module instead of handling all further topics in COCO1, that is MS by MS. There are basially two motives:
+  * In some cases it is convenient to have the completed COCO1 results of all countries at hand for comparison purposes and in order to achieve a balanced picture across MS. This is the main motive for the assignments of consumer loss rates (Section 3.2.7.1).
+  * Whenever averages of consolidated data (from COCO1) across several or all MS are involved, a solution in a loop requires certain sequence (such as first solving for non-candidate countries to form the averages that are input to candidate countries) or is better solved in a new module like COCO2. This applies to the expenditure allocation problem (Section 3.2.5), to completions for certain feedstuffs (Section 3.2.7.2, EU averages used due to the scarcity of data), and to corrections of LULUCF coefficients (Section 3.2.7.3).
+===Assignment of consumer loss rates and nutrient intake per head ===
+Since a number of years diet shift scenarios have increase in importance and therefore the plausibility of per capita consumption projectios and hence their starting values, per capita consumption in the data base. A common yardstick to assess plausibility is nutrient (e.g. calorie) consumption per head where the nutrition literature offers guidance in terms of recommendable as well as “observed” consumption. For nutrition issues it is intake, so consumption after losses, which matters, such that the assignment of these loss rates becomes a critical element of the database. The starting values are due to an FAO study and stored in the \dat folder
+{{::code_p53.png?600|}}
+The aggregate food share (= 1-loss shares) links intake (INHA(i)) to total consumption (sum(i, HCOM(i)*foodSh(i)) / INHA(levl) and is therefore stored in the database as well.
+{{:code_p53_2.png?600|}}
+In spite of the FAO study the real loss rates are highly uncertain. Therefore they are reduced if the estimate of calorie intake based on the FAO loss rates strongly falls short of recommendations (most strongly in a set of “low calory regions”). Conversely loss rates are increased, if the estimate of calorie intake based on the FAO loss rates strongly exceeds recommendations (e.g. in Turkey).
+{{:code_p53_3.png?600|}}
+=== Completion of feed related data in coco2_feed ===
+The first sections of coco2_feed handle completions for certain by-products and other product so far ignored in coco1. These are by-products of the milling and the brewing industry and for corn gluten feed, sugarbeet pulp, manioc and fish meal where the database is completed for market balance positions production, imports, exports and feed. This relies on discontinued Eurostat tables (collected on p_feedAgri) which are extended using national data and external trade data from Comext. After completion the detailed by-products are aggregated to the CAPRI rows FENI (Rich energy fodder imported or industrial) and FPRI (Rich protein fodder imported or industrial). Based on completed data for all feedingstuffs nutrient contents for the CAPRI feed “bulks” (cereal feed FCER, protein feed FPRO etc) are assigned as an aggregate of their components.
+These completions are useful as such but they also permit a balancing of (preliminary) total nutrient supply and demand in the animal sector that ultimately serves to adjust loss rates for fodder with the help of a number of include files:
+**Include files //‘feed_decl.gms’// and //‘req_or_man_fcn.gms’//**
+These files are not only active in COCO2, but also in CAPREG, and in the baseline calibration of CAPMOD. This “reuse” of the same files in different modules is efficient and ensures consistency, but usually also requires some adaptations of set definitions:
+{{::code_p54.png?600|}}
+The previous snippet from coco2_feed gives an example that some sets (RS, R_RAGG) are assigned specifically to ensure functionality in different modules (here COCO2).
+As the name should signal file //‘feed_decl.gms’// mainly collects a number of declarations but it also specifies some bounds for process length DAYS and daily growth DAILY that are imposed throughout of CAPRI (example: maximum daily growth for male cattle = 1.5kg/day). The second include file (//‘req_or_man_fnc.gms’//) specifies the requirement functions (with the argument “req” passed on) for animal activities of CAPRI.
+Requirement functions are specified that determine: