Methodology

Codification of policy proposals

In the first stage, the policy proposals contained in Development Plan for the Greek Economy they are gathered and coded according to the policy area they belong to. It should be clarified that in this project only policy proposals are codified and not principles, values and general findings. The reason for choosing the policy proposals is their testability, i.e. the fact that the phrase that has been formulated can be tested in a concrete way, without having abstract questions of interpretation, at least not to a large extent. An example of this important distinction is how the phrase:

"a series of direct more specific interventions aimed at a fairer, more productive and more developmentally friendly distribution of tax burdens on households and businesses"

concerns principles of the tax system, as the word "fairer" is neither defined nor verifiable. What is fair is subject to subjective interpretation. However, what can be tested is a policy proposal that specifies the above principle as the phrase:

"similation and uniform tax treatment of income from different sources"

where here either the incomes from different sources are taxed uniformly, or they are not taxed uniformly, so there is no element of subjectivity.

It should be emphasized that in some cases, if there is a question of interpretation of certain terms, this is explained in the evaluation of the policy proposal (see below).

Also, it is useful to note that because many times certain policy proposals may be repeated in other areas in a similar way, we apply the following criteria:

• Policy proposals are coded once in the policy base field.
• Policy proposals are coded based on the clearest wording that appears.

Finally, some policy proposals refer to objectives, that is, to measurable data for which it is checked whether they have been achieved or not. One such example is the target to increase Greek exports to 50.5% of gross domestic product by 2030.

Screening and evaluation of policy proposals

The control and evaluation of the implementation status of the policy proposals that have been codified is carried out in 2+1 stages:

• From the research team
• From expert experts
• External agencies

A. The review and evaluation by the research team

In the first stage, the 3 independent researchers check the content of each policy proposal and evaluate it as:

• Applied

A policy proposal is considered as applied if all or the greater part of it has been enacted and is or can be enforced. An example of an implemented policy is the abolition of additional tax for private property owners, which was legislated and is valid from March 2022 with Law 4916 and specifically Articles 40-49.

• Partially implemented

A policy proposal is considered as partially applied if much of it is legislated and implemented. An example of a partially implemented policy is the expansion of the tax base, which was carried out in the first two years of the ND administration, however, according to the recommendations of the Governor of the Bank of Greece and an OECD report, it has not reached the desired (as an economic result) level. It is useful to note that this category contains the greatest degree of variation in the evaluation, that is, proposals with different degrees of application may be in this category. For example, if one proposal has 2 of the 5 proposed measures implemented while another has 3 of the 5 proposed measures implemented, even though the second proposal has been implemented to a greater extent and the 2 proposals are included in the same category of partially implemented proposal.

• Under implementation

A policy proposal is considered to be in place under implementation if there is a national action plan, acceptance of a recommendation, a bill in consultation and an official announcement of implementation, but it has not yet been legislated and implemented. An example of a policy in action is the proposal to transfer ENFIA revenues to local governments and to adjust transfers from central government to local governments accordingly, which has been announced as a measure to be implemented in January 2023.

• Not applied

A policy proposal is considered as not implemented if much of it is not legislated and not implemented. An example of an unimplemented policy is the proposal that environmental taxes transparently replace existing taxes or be returned to taxpayers as an apparent reduction in income tax.

After the control and evaluation by the 3 independent researchers is completed, the reliability of their evaluation is checked with the coefficient Krippendorff alpha. The agreement of evaluators as to whether a policy proposal has been implemented or not exceeds 0.73 in the control carried out, while the minimum acceptable limit according to Krippendorff is 0.67.

Then, the research team briefly explains the evaluation reasons (rationale) of each proposal according to the above categorization and refers to the corresponding sources. In all proposals, their implementation from 2020 onwards, the year the report was written, is evaluated, and not the implementation of earlier regulations.

B. The control and evaluation by special experts

In the second stage, the evaluation and explanation of each proposal is sent to a panel of experts who in turn approve the evaluation or reject it, explaining the reasons and referring to relevant data. The final evaluation is the result of this review process and is judged by the research team.

C. The continuous control and suggestions from external bodies

From the publication of the results onwards, the research team acts in the same way to check each policy proposal and then updates the evaluation status of each policy proposal accordingly.

The research group encourages citizens, political society actors, research and academic institutions, parties and organizations and other actors to send their comments on the screening and evaluation of policy proposals. All comments are studied and in cases where new information or information that had escaped control is found, the evaluation will be adjusted accordingly. It is important to note that the project is ongoing and the research team is making every effort to continuously review the vast amount of legislation in order to gather new audit evidence for each proposal.

Sources of control

• Key sources of control:
  • Regulations: laws, amendments, presidential decrees, ministerial decisions, circulars.
• • Accompanying reports: regulatory impact analysis, special report, explanatory report, consultation report. Official websites / archives of the state: information on ministries, reports, data, announcements of Ministries and government agencies, national strategies.
  • Announcements from state agencies: Local Authorities, Independent Authorities, ELSTAT.
• Additional control sources:
  • Articles in print and online.
  • Reports and data of domestic and international organizations: OECD, Eurostat, European Union, European Commission.

 

The methodological framework

The methodological framework for coding and evaluating the implementation of policy proposals (or commitments to policy implementation) is modeled on the analysis standards of the most successful PolitiFact from the Poynter Institute for Media Studies, which since 2007 has been checking the central policies of US presidents (fact checking journalism).

The main methodological tools of the analysis come from studies concerning the measurement of the implementation of political commitments by parties in modern democracies, see for example:

Louwerse, T. (2011). The spatial approach to the party mandate. Parliamentary Affairs, 64(3), 425-47.

Naurin, E. (2014) Is a Promise a Promise? Election Pledge Fulfillment in Comparative Perspective Using Sweden as an Example, West European Politics, 37:5, 1046-1064, DOI: 10.1080/01402382.2013.863518.

Pétry, F., & Collette, B. (2009). Measuring How Political Parties Keep Their Promises: A Positive Perspective from Political Science. In LM Imbeau (Ed.), Do They Walk Like They Talk? Speech and Action in Policy Processes (pp. 65-80). New York: Springer Dordrecht Heidelberg London New York.

Thomson, R., Royed, T., & Naurin, E. (2010). The program-to-policy linkage: a comparative study of election pledges and government policies in the United States, the United Kingdom, the Netherlands and Ireland. Paper presented at the annual meeting of the APSA, Washington, DC, Sept. 2-5.