Page 69 - Reforming Benefits Decision-Making -(updated - August 2021)

Two systems in the Netherlands have demonstrated the dangers of automated
fraud detection for individuals’ rights and the need for transparency.

SyRI

In February 2020 a system named SyRI that was used by the Dutch
government to identify citizens who were considered more likely to commit
fraud, including in welfare benefits and tax, was found to be unlawful by a
Dutch court. The court held that SyRI breached the right to private life
190
under Article 8 of the European Convention on Human Rights as it lacked
191
transparency and verifiability. The risk models it used, including the
algorithm, risk indicators, data inputs and output, were not disclosed to the
192
court. Citizens also did not know how their data was being processed and,
absent more information on how it worked, the court warned that the system
might discriminate on the basis of socio-economic or migrant status.
193

Childcare benefits

Over more than a decade the Dutch tax authority used an automated system to
identify parents suspected of committing fraud in relation to state subsidies
for childcare fees. As a result of the system more than 20,000 parents were
falsely accused of having committed fraud, had their benefits stopped and
were told to repay back what they had received. 194 Many of those affected
were from an immigrant background and the errors forced hundreds into
severe financial difficulty.
195

190 The Hague District Court, NJCM cs/ De Staat der Nederlanden (February 2020); Koen Vervloesem,
‘How Dutch activists got an invasive fraud detection algorithm banned’ (Algorithm Watch, 2020).
191 De Rechtspraak, ‘SyRI legislation in breach of European Convention on Human Rights’ (February
2020).
192 NJCM cs/ De Staat der Nederlanden (see n.190 above), para. 6.49.
193 NJCM cs/ De Staat der Nederlanden, (see n.190 above), paras. 6.91 – 6.93.
194 The Economist, ‘A benefits scandal sinks the Dutch Government’ (January 2021).
195 BBC, ‘Dutch Rutte government resigns over child welfare fraud scandal’ (January 2021).

60