Page 68 - Reforming Benefits Decision-Making -(updated - August 2021)

replace rules-based processing steps. However, this could cover a whole range
188
of processes, many of which may be important to decision making.
2.89 One of the issues in the Johnson and Pantellerisco cases was a lack of
transparency about what the actual constraints of the UC system and other
computer systems it interfaces with are, and the ability to adapt the UC
system to deal with unforeseen issues caused by the automation.

2.90 There is legitimate concern amongst claimants and civil society organisations
regarding the use of automation and artificial intelligence in fraud detection
and prevention. Privacy International sent a series of FOIA requests to the
DWP asking for further information about the use of artificial intelligence for
fraud prevention referenced in the DWP’s annual report. The DWP refused to
provide information on the criteria or indicators used by the system to flag
someone as likely to be committing fraud, or the code of the algorithm being
used. They relied on FOIA exemption that release would prejudice the
prevention and detection of fraud and crime.
189

188 The only processes that we are aware that have been automated are pensions claims. See Convedo,
‘How Intelligent Automation is improving public services’.
189 Privacy International, ‘Shedding light on the DWP Part 2 – A Long Day’s Journey Towards
Transparency’ (February 2021).

59