How can amendments be judged in a neutral and objective way?
When we started our analysis of more than 3.100 amendments the question occured how to measure countless political decisions in one statistic. There is not a single proper method – so we had to develop one that is as transparent and clear as possible.
We agreed to evaluate every single amendment by it’s own: Does the change lead to stronger or weaker protection of privacy? Most of the changes can be categorized quite easily. If the consequences of an amendment are unclear or if the amendment does not change the level of protection we have marked it as “neutral”. The calculation of the overall impact of an MEP follows a simple formula: If a MEP suggests 45 changes which strengthen privacy and 30 changes that weaken the protection his final balance score will be 15.
Of course this approach also has some weaknesses (see below).
Our rating criteria:
To be absolutely transparent we want to be open about our rules and explain them. Everybody should be able to rate the collected amendments by him- or herself.
- Basis of our collection are 3.113 amendments of the LIBE committee on the EU Data Protection Regulation. Not included (for now) are changes to chapter 5. to 7. concerning international data traffic and administration - which are less relevant than the other chapters.
- We reviewed every single amendment, regardless of which its length and influence. It would not be possible to objectively measure the amount political influence of every amendment.
- The author of every amendment is taken from the official documents. If amendments are co-signed by multiple MEP it will be attributed to every MEP.
- Our judgement is based on an general assessment if the amendment will cause the privacy rights of an average user to be strengthened or weakened.
- Scale is the legal situation of an average citizen compared to the draft by the European commission from january 2012.
- The following changes were therefore graded as privacy-weakening amendments:
- Limitations of rules for companies.
- Exceptions from rule for companies.
- New allowances for additional data processing.
- Higher thresholds for duties to apply.
- Ambiguous wording or alternative concepts which are inherently unclear or vague.
- Exemptions from duties for companies, if they have consequences for the rights of data subjects.
- Easier data exchange with non-EU countries.
- The following changes were therefore graded as privacy-enhancing amendments:
- Extension of the rules to cover a greater number of people.
- Deletion of exceptions for companies.
- Deletion of legitimate reasons for data processing.
- Lower threshold for duties of companies.
- Clear wording and alternative concepts with more rights for users.
- Introduction of new rules, duties or procedures for companies, if they have consequences for the rights of data subjects.
- Limitation of data exchange with non-EU countries.
- Amendments that could not be clearly identified as having impact on privacy were marked as “neutral”.
- The person evaluating amendments was unable to see the MEP that has proposed the amendments.
- Only if an amendment could not be evaluated on its own, it was allowed to look at the justification and other amendments by the same MEP to enable the reviewer to grade the amendment.
- Amendments relating to delegated acts by the commission were generally regarded as “neutral”, because they do not weaken or strengthen privacy rights per se.
- Evaluations by different reviewers were compared, if contradictory evaluations were found, a second round was taken in order to reach a consensus.
Comparison with documents from Privacy NGOs:
The results from LobbyPlag were again matched with documents from different privacy NGOs. Different results were reexamined another time and checked, but always graded in line with our own set of rules as mentioned above.
Weaknesses of our method:
Of course there are some weaknesses in our method that we want to be frank about:
- There is no unit in which one can measure more of less privacy. We have therefore counted the number of amendments. This means that a small change is counted the same way as a major amendment. The only alternative would have been an rating of how much every amendment changes the right to privacy, which cannot be done objectively. But especially when MEPs have submitted many amendments this problem should be evened out.
- Not every limitation of the right to privacy is bad, not every additional right is good. As an example: If the trade unions get the right to agree on processing of data with employers, we think that this is a good idea. But this also means that there is another ground on which data can be processed. Therefore these amendments were marked as weakening the right to privacy according to the principles explained above.
- With more than 3.100 amendments there can always be some mistakes. One can click the wrong button or there might be some misunderstanding. After reviewing our results we are sure that there are no more than 2-3% of all amendments that might be marked wrong. If someone finds a mistake he can always report it by clicking on the „report“ button on every amendments. In addition everyone can discuss every amendment on our page. We hope that this allows us to steadily improve our data and get results for even more amendments.