Voice Stress Analysis - Junk Science used by Government to frighten and intimidate benefit claimants
Harrow Council Communications Management Strategy 2003 - 2006 - Successful communications are at the heart of any successful enterprise...... Harrow Council tries to be open and honest in its dealings.
Independent observers therefore might find it difficult to reconcile this worthy objective with the details of the London Borough of Harrow - The introduction of Voice Risk Analysis to Housing Benfits and Council Tax assessments - which is a nominee for the e-Government National Awards 2007.
Public technology.net provides an effusive and excited review of this exciting initiative ;
"Harrow's Housing Benefit Service is already a national benchmark of excellence, with a four star CPA rating and fastest assessment times in the country. The recent implementation of Voice Risk Assessment (VRA) has further improved performance, secured a DWP easement on checking claimants' details, and saved money. VRA does not deter benefits claimants - Harrow's caseload is the highest it has ever been but helps to drive out fraud or error and to target resources faster. The use of this technology in the public sector is a national 'first' for Harrow. Following its success both Lambeth and Birmingham councils have adopted the package."Apparently Mike Carney (Head of Assessments) consulted with Capita in January 2007 and decided to use VRA within its contact centre , " to innovate the review process "(?) Capita are the Council's strategic partner. Harrow worked with them to apply their general VRA technology to a Local Government assessments environment. They have provided technical expertise and supported Harrow's project manager in implementing the system.
As a national 'first' implementing VRA required courage (?). But the DWP's confidence in the project was such that it funded the pilot. By February operators were being trained and the scripts developed by the Council.
1.VRA went live on May 21 for Housing and Council Tax Benefits interventions.
2. The system was extended in June for review of single person discounts
3. It was further extended in August for new claims for Housing and Council Tax Benefits.
4. We have worked with Job Centre Plus who plan to pilot VRA from September.
5.Plans are also in place to extend the service to cover a range of other assessment based Council services, such as Blue Badges and Freedom Passes."
6. We are on track to achieve the Government's PI (?) within 2007.
John Hutton, who was then the Secretary for Work and Pensions (eGov monitor 10 April 2007) said , "Harrow Council is already recognised as providing one of the best Housing Benefit services in the UK and I welcome its willingness to pilot this innovative technique to make further improvements." What he failed to mention was that no surrogate marker of deception has yet proved reliable enough to be admitted as evidence in a court of law in the UK.
Benefits ? - well they claim ..." So far we have reviewed 588 cases and fast-tracked processing for 402 of them. Of these 135 declared a change, and 73 are being followed up further. This has resulted in a saving in this area alone of £98,192, and £140,000 in total. The one year trial costs only £63,000 and has saved £140,000 in the first four months. "
Recent figures from the DWP are said to show that Benefit fraud in the UK is estimated to run to a total of £800m in 2006/7, and that there were more than 52,000 prosecutions in 2005/6. The Fraud Act 2001 gave fraud investigators more powers to catch and punish offenders, including greater powers to check records with other organisations.
Curiously in the Guradian December 2nd they say ... "In the first three months Harrow council saved £300,000, suggesting levels of benefit fraud may be higher than government estimates." ...the Monbiot Calculus at work again, think of a number ...er.. then make it bigger. In the Economist on May 8th 2008 they report ..."This week the council said that in the ten months from May 2007, when the system was installed, it had saved £420,000 ($828,000).
A press report in the Birmingham Post May 8th 2008 , "Phone lie detector led to 160 Birmingham benefit cheat investigations" , reports that of 2,656 calls subject to checking, 6%, around 160, were classified as "high risk", leading to further inquiries. It refrains from saying how many of those investigated then proved to be cheating the system or lying. See also BBC video report posted at the same time...."Fifteen councils across Britain are to be offered lie detectors to help catch out benefit cheats who cost taxpayers up to £400m a year. "
As recently as November 20th the BBC reported that the technology will be used for the first time on telephone calls of unemployment benefit claimants in Lincoln and Nottingham.
Harrow, partnered with Capita, are one of the 68 Finalists and are included in Section 9) Local e-Government excellence: Team - they will hear their fate at an exclusive black-tie e-Government National Awards dinner which takes place 6.45pm - 11.15pm on 20th January 2009 at the prestigious Guildhall in the City of London.(details)
What exactly is Harrow's / Capita's magical Voice Risk Assessment (VRA) system
Here we have to thank the Yorkshire Ranter - Dr. Benway strikes again, with Venture Capital who with sound knowledge and experience and with a wry literary touch provides a detailed and lethal de-bunking and also The Ministry of Truth for the Government Bullshit Risk Detector .
Briefly ..
Voice stress analysis (VSA), is not new and , works it is claimed, by detecting involuntary psychophysiological modulations in a subsonic and inaudible component of the voice called the ‘Lippold Tremor’ at a frequency of between 8Hz and 12Hz. As far as can be ascertained the original research was undertaken on live voice recording and computer analysis - not on digitally transmitted voice data over telephone networks.
Within the world of Detection Of Deception (DOD) this issue is hotly debated with polygraphists (Lie Detectors) and few independent research studies have been undertaken. A study by VThe State of Virginia in 2003, (35 page pdf) at which time the technique was in widespread use, concluded that
The technique used by Harrow is a proprietary, patented (only in Canada it appears - registrant is Amir Liberman, an Israeli - 16th Dec. 1998) system utilising a subset of VSA called Layered Voise Analysis which is peddled by a company called Nemesysco Ltd. a privately held security-oriented company founded in 2000, and headquartered in Netanya, Israel and sold through Digilog UK."Because there have been no independent scientific studies conducted on the reliability of the computer voice analyzer to detect deception, the Board recommends to the Director of the Department of Professional and Occupational Regulation that computer voice analyzer equipment should not be approved in Virginia at this time."
The suppliers do not claim that their devices detect lies; rather, they claim that VSA detects microtremors, which are caused by the stress of trying to conceal or deceive. hence it's usefulness in their personal use product ... the PC based Love Detector - The Love Detector technology is capable of detecting love-like feelings and other emotions via the telephone or a microphone.
There is little experimental or independent evidence of the utility of this technique but the Natonal Institute of Justice - a section of the US Department of Justice published a report by Kelly R. Damphousse, Ph.D. in their Journal dated March 17, 2008 - entitled , "Voice Stress Analysis: Only 15 Percent of Lies About Drug Use Detected in Field Test" in which they address the question ...Can VSA Accurately Detect Deception? and they conclude ;
Our findings suggest that these VSA software programs were no better in
determining deception about recent drug use among arrestees than flipping a
coin.
The Yorkshire Ranter however points to a fatal weakness in that the original work on VSA relies upon
... a change in a signal in your voice between 8 and 12Hz, but even the highest-quality voice codecs used for public telephony filter out everything below 50Hz, so a VSA system based on - well - science couldn't possibly work.All this is of course to some extent an academic and amusing diversion - but it isn't.
In the Queen's Speech debate the Guradian reports ;
The government also highlighted a pilot covering 25 local councils administering housing benefit to 500,000 claimants, saying "voice risk analysis technology" will be used to test whether a claimant is providing false information. The government first introduced the technology in Harrow in north London last year, but now says it plans to make the technology available nationwide.The Government have of course anticipated resistance to the system and response is outlined here by Department for Work & Pensions Anti-Fraud Minister, James Plaskitt yesterday during their media blitz on May 8th 2008.
His ad hominem is remarkable (or not in the circumstances) in that it justifies the use of such systems by their results, not by any attempt to validate the techniques.
This is because, by simply telling the caller such a system is in use acts as a deterrant to continue with the application. This may be because they are attempting to commit fraud, but it may not be.
The use of this technology is merely to cow and frighten, the guilty and the inoccent will fall foul of this technique unsupported by any rigorous scientific studies. Furthermore no surrogate marker of deception has yet proved reliable enough to be admitted as evidence in a court of law in the UK.
So the state is using a technique which is fraudulent to possibly identify the fraudsters.... but... but ... Harrow Council tries to be open and honest in its dealings.
Recent figures from the DWP are said to show that Benefit fraud in the UK is estimated to run to a total of £800m in 2006/7 - equivalent to the loss to the HMRC of 2 months carousel fraud.
1 comment:
The most recent scientific study dealing with the layered voice analysis (LVA) system marketed by Nemesysco is Eriksson, A. and Larceda, F., "Charlatanry in forensic speech science: A problem to be taken seriously", IJSLL, Vol. 14, 2007.
Eriksson and Larceda review the previous scientific literature dealing with various vendors' LVA systems and particularly examine the program code published in Nemesysco's patent specification for its LVA system.
The authors conclude:
"To sum up by saying that there is absolutely no scientific basis for the claims made by the LVA proponents is an understatement. The ideas on which the products are based are simply complete nonsense."
Academic research published in peer reviewed journals invariably concludes that the LVA method completely lacks theoretical basis or methodological validity. Because of this, Eriksson and Larceda emphasise that it makes no sense to consider whether or not the LVA method is reliable. They do, however, note that existing research into the reliability of the LVA method has invariably concluded that LVA is no more (or less) reliable than tossing a coin. They point out that the reason for this is that LVA is so completely methodologically invalid as to be simply arbitrary.
The following extract from Eriksson and Larceda's paper provides some background in relation to the man behind Nemesysco.
"We might as well have asked: Who is Nemesysco, the company behind the LVA products, because Mr Liberman and Nemesysco seem to be one and the same. Damphousse et al. (2008:14) report as follows: 'The LVA was developed in Israel by Amir Lieberman [sic] who applied mathematical algorithm science to voice frequencies', giving the impression that the program is based on some advanced mathematical theory. As we have pointed out, this is far from the truth. When we first became aware of the LVA, in connection with an attempt in 2004 to introduce the LVA on the Scandinavian market, we too were given the impression that Mr Liberman was indeed a high ranking Israeli mathematician. We do not know the origin of these rumours. It has been said that the information once appeared on the Nemesysco home pages but we have not been able to confirm this. Screening the Nemesysco home pages we became highly suspicious of these claims, however. To acquire more information about the person behind the products we consulted an Israeli colleague who is an active speech science researcher and asked him if he knew a mathematician by that name. He did not. A controversy arose between us and the Scandinavian representatives of the LVA whom we, after a careful study of the LVA claims, accused of trying to peddle a bogus product. This controversy, partly fought in a newspaper, caught the interest of a journalist, Arne Lapidus, who was working in Israel for the Swedish daily Expressen. After some research he managed to locate Mr Liberman, a 32 year old (in 2004) businessman in a small office in the town of Natania. The business appeared to be a one-man operation. Mr Lapidus interviewed Mr Liberman about his academic background and was told he basically had none. He had no degree (never had time to get one, he explains) but has taken some courses in marketing at an Israeli open university. As we have explained above, the LVA is a simple program written in rather amateurishly used Visual Basic. Given what we now know about Mr Liberman, that is about what one would expect rather than '8,000 mathematical algorithms applied to 129 voice frequencies' (Damphousse et al. 2007:15).
In view of the evidence that Amir Liberman is simply a fraud selling a bogus product, Eriksson and Larceda go on to ask why otherwise sophisticated companies and government agencies are willing to pay Nemesysco's business partners around USD 25,000 per licence/training package:
"What still remains for us to understand is how insurance companies, security agencies, police departments can be willing to invest hundreds of thousands of dollars, pounds and euros in equipment without ever asking who are behind the products, what are their qualifications, what are the scientific principles upon which the products are based. The program code is part of the patent documents and may be downloaded from patents on-line. Any qualified speech scientist with some computer background can see at a glance, by consulting the documents, that the methods on which the program is based have no scientific validity. Why did those who so willingly invested huge amounts of money not even bother to look? For us this is the real puzzle."
The answer is straightforward. By using LVA, insurance companies achieve "success" in terms of reduced claims paid and police departments achieve "success" in terms of increased convictions.
Eriksson and Larceda explain that this is so despite LVA's complete lack of validity and reliability because of two factors well known to researchers: first, 'the Bogus Pipeline Effect' ('no one wants to be second-guessed by a machine'); and secondly, because these users aren't concerned about false positives (in the logic of their organizations, they actually benefit from false positives).
Ironically, insurers and police could achieve identical "benefits" without investing in LVA. Simply advising claimants or detainees that their answers are being automatically analysed for veracity would be no more misleading and would deliver identical "improved success" without any up-front costs.
Either way, there are obvious legal and moral problems with these situations. The problems are further compounded when security agencies begin to rely on LVA to the exclusion of basically valid and at least somewhat reliable traditional methods. False LVA positives would distract security staff from their real work and false LVA negatives would carry catastrophic risks.
Mr Liberman's reaction to Eriksson and Larceda's paper tends to confirm Eriksson and Larceda's description of him. He didn't adopt the accepted course of submitting a research paper to the journal rebutting Eriksson and Larceda's findings. How could he? He has no academic credentials. Instead, he instructed lawyers to threaten the publisher, Equinox Publishing, with legal action for defamation if it did not withdraw the article and publish an apology.
To the discredit of Equinox Publishing, it allowed itself to be intimidated. It is also declining to answer enquiries from the press. As it is not clear whether Nemesysco's threats against Equinox are limited to legal action, it is perhaps premature to judge Equinox's apparent lack of fortitude too harshly.
See: http://www.equinoxjournals.com/ojs/index.php/IJSLL/article/view/3775
Eriksson and Larceda's paper as originally published may nevertheless be consulted in the printed journal at a university library or online at:
http://www.scribd.com/doc/9673590/Eriksson-Lacerda-2007
Francisco Lacerda's blog at Stockholm University appears at:
http://blogs.su.se/frasse/frasse-mrW2v4nd
(in Swedish). According to his blog, he intends to continue publishing on the issue.
CDS
Post a Comment