{"id":115,"date":"2015-01-27T01:41:07","date_gmt":"2015-01-27T06:41:07","guid":{"rendered":"http:\/\/www.datagoneodd.com\/blog\/?p=115"},"modified":"2015-05-20T23:05:31","modified_gmt":"2015-05-21T03:05:31","slug":"how-to-lie-and-mislead-with-rape-statistics-part-2","status":"publish","type":"post","link":"http:\/\/www.datagoneodd.com\/blog\/2015\/01\/27\/how-to-lie-and-mislead-with-rape-statistics-part-2\/","title":{"rendered":"How To Lie And Mislead With Rape Statistics: Part 2"},"content":{"rendered":"

In Part 1<\/a> we talked about the reasons the authors felt some of the existing research was not credible. \u00a0They wrapped up their critique of the Kanin study<\/a> with the following quote (and if you read Part 1, you can probably guess who they are going to cite):<\/p>\n

As a result of these and other serious problems\u00a0with the \u201cresearch,\u201d Kanin\u2019s (1994)\u00a0article can be considered \u201ca provocative\u00a0opinion piece, but it is not a scientific study\u00a0of the issue of false reporting of rape. It certainly\u00a0should never be used to assert a scientific\u00a0foundation for the frequency of false\u00a0allegations\u201d (Lisak, 2007, p. 1).<\/em><\/p>\n

In contrast, when more methodologically\u00a0rigorous research has been conducted,\u00a0estimates for the percentage of\u00a0false reports begin to converge\u00a0around 2-8%.<\/em><\/p><\/blockquote>\n

Before I get into the primary study that they reference, let me give you an idea of just what kind of games they have in store for us. \u00a0Here is what they say about the second study used to back up their 2-8% range:<\/p>\n

For example, Clark and Lewis (1977) examined\u00a0case files for all 116 rapes investigated\u00a0by the Toronto Metropolitan Police\u00a0Department in 1970. As a result, they concluded\u00a0that seven cases (6%) involved false\u00a0reports made by victims.<\/em><\/p><\/blockquote>\n

Ok, so we are back to using data from a single police department, in a single year, and in this case from about 40 years prior to when the article was written, but at least it is in the range they are claiming. They then go on to say\u00a0this<\/em><\/p>\n

There were also\u00a0five other reports made by someone other\u00a0than the victim that were deemed by these researchers to be false (e.g., a relative or\u00a0boyfriend).<\/em><\/p><\/blockquote>\n

So, to clarify, the study found that 7+5 = 12 out of 116 reported rapes were false. \u00a0For most people that would be a false reporting rate of 10.3%, which is outside the 2-8% range 1<\/a><\/sup>, but apparently not for the authors. \u00a0It would seem that rape reports made by someone other than the victim do not count as false despite being, well, false2<\/a><\/sup> 3<\/a><\/sup>. At this point all I can say is that at least they are quoting the research of others – if this is the kind of logic they are using, I can only\u00a0imagine what the results would be if they got to design the study themselves.<\/p>\n

Back to their other sources – what is the primary study that they cite?<\/p>\n

For example, in a multi-site study of eight\u00a0<\/em>U.S. communities involved in the \u201cMaking a\u00a0<\/em>Difference\u201d (or \u201cMAD\u201d) Project, data were\u00a0<\/em>collected by law enforcement agencies for\u00a0<\/em>all sexual assault reports received in an 18-\u00a0<\/em>24 month period. Of the 2,059 cases that\u00a0<\/em>were included in the study, 140 (7%) were\u00a0<\/em>classified as false.<\/em><\/p><\/blockquote>\n

A multi-site, multi-year study? \u00a0Seems like we actually have some good data to work with here. \u00a0They then give a website where you can find out more information about the MAD Project, http:\/\/www.evawintl.org<\/a>. \u00a0Clicking through you’ll find out that their primary source for a “methodologically rigorous” study on the number of false rape reports comes from the\u00a0organization End Violence Against Women International. If someone tried to present their view that the number of people killed by cigarettes each year\u00a0is actually pretty low and their primary source was a study done by Phillip Morris, how much credence would you lend that research? \u00a0What about a study on gun violence done by the NRA? And yet, for some unknown reason, the authors don’t seem to think that End Violence Against Women International might in some way be biased as to frequency of\u00a0false rape reports.<\/p>\n

As it turns out, there is actually a very good reason why the authors don’t see a problem with that, which they are pretty upfront about – and by “are pretty upfront about” I mean “bury it in the footnotes.”4<\/a><\/sup><\/p>\n

1 Dr. Lonsway is the Research Direcotr <\/em>[sic] of EVAW International;\u00a0<\/em>Sgt. Archambault is the Executive Director of EVAW Internatiaonal;…<\/em>\u00a0This article is an adapted excerpt from\u00a0<\/em>the training module of the same name in the On-Line Training institute\u00a0<\/em>hosted by End Violence Against Women (EVAW) International,\u00a0<\/em>at http:\/\/www.evawintl.org\/evaw_courseware.<\/em><\/p><\/blockquote>\n

So not only do they try to disguise the fact that the main source for their hypothesis is a study that two of the authors conducted themselves for an organization with a clear stake in the matter, but the entire article is essentially an advertisement for their training sessions. Think I’m exaggerating? Check out the last page of article:<\/p>\n

\"Ad<\/a><\/p>\n

(click to enlarge)<\/span><\/em><\/p>\n

Perhaps I have not been keeping up with the times, but I hadn’t realized that scholarly articles came with coupons now. And we are supposed to believe that Kanin’s study is the “provocative opinion piece”?<\/p>\n

So how did they get to the 7% number? The first problem we are going to run into has to do with the research itself. Take a look at EVAW’s page on the MAD Project<\/a>. Despite the data collection period being from 2005-2006, there is very little in the way of actual published research. \u00a0They have a link to research materials<\/a>\u00a0where you can see some of the data collection forms they used, a pdf on the research methods<\/a>, and a collection of slides listing some of the basic data results<\/a>, but that is pretty much it. \u00a0That basic data results document is over 5 years old at this point, it has been over 8 years since the data collection periods, and there is still no comprehensive study document available, which strikes me as a little strange.<\/p>\n

One of the key questions in evaluating their claim is going to be “How, exactly, is a false report defined?”\u00a0As it turns out, that part isn’t entirely\u00a0clear cut. \u00a0In fact, it isn’t until paragraph 6 of the section entitled “What is the Actual Definition of a False Report?” that we get to this:<\/p>\n

Rather, investigators and prosecutors must\u00a0base all final judgments of a sexual assault\u00a0report on the findings from a thorough, evidence-based investigation. The determination\u00a0that a report is false can then only be\u00a0made when there is sufficient evidence to\u00a0establish that the sexual assault did not happen\u00a0(was not completed or attempted.)<\/em><\/p><\/blockquote>\n

OK, so not a terribly unreasonable definition, but to understand how this has the potential to skew the data, we have to go back to the research methods I mentioned before. \u00a0If you read through it, you’ll find that the 8 sites were not chosen randomly. \u00a0Communities had to apply to be in the study and there were 88 US applications in total. \u00a0These applications were then evaluated by a selection committee to select the final 8 cities5<\/a><\/sup> 6<\/a><\/sup>. After selection, the 8 communities came together for a 3 days training conference, and that is where I think things get particularly interesting. \u00a0While they don’t specify the exact training that occured, we know that this article is adapted from some of their training materials. \u00a0So what kind of things does their training have to say about false rape reports?<\/p>\n

To start off with, here are some\u00a0things that someone filing a rape report might do, which the authors say happens for perfectly justifiable reasons<\/p>\n