Wiley also stopped me doing my research

In November, I wrote about how Elsevier wanted me to stop downloading scientific articles for my research. Today, Wiley also ordered me to stop downloading.

As a quick recapitulation: I am a statistician doing research into detecting potentially problematic research such as data fabrication and estimating how often it occurs. For this, I need to download many scientific articles, because my research applies content mining methods that extract facts from them (e.g., test statistics). These facts serve as my data to answer my research questions. If I cannot download these research articles, I cannot collect the data I need to do my research.

I was downloading psychology research articles from the Wiley library, with a maximum of 5 per minute. I did this using the tool quickscrape, developed by the ContentMine organization. With this, I have downloaded approximately 18,680 research articles from the Wiley library, which I was downloading solely for research purposes.

Wiley noticed my downloading and notified my university library that they detected a compromised proxy, which they had immediately restricted. They called it “illegally downloading copyrighted content licensed by your institution”. However, at no point was there any investigation into whether my user credentials were actually compromised (they were not). Whether I had legitimate reasons to download these articles was never discussed. The original email from Wiley is available here.

As a result of Wiley denying me to download these research articles, I cannot collect data from another one of the big publishers, alongside Elsevier. Wiley is more strict than Elsevier by immediately condemning the downloading as illegal, whereas Elsevier offers an (inadequate) API with additional terms of use (while legitimate access has already been obtained). I am really confused about what the publisher’s stance on content mining is, because Sage and Springer seemingly allow it; I have downloaded 150,210 research articles from Springer and 12,971 from Sage and they never complained about it.

10 thoughts on “Wiley also stopped me doing my research

  1. Graham Steel (@McDawg)

    So if your proxy was not compromised, and you’ve been able to use the same system to download >160,000 articles from other publishers, it seems to me that is it Wiley that have acted inappropriately.

  2. Ross Mounce

    In a time when Sci-Hub is utterly mainstream, why are Wiley & Elsevier blocking individual researchers legitimately downloading paid-for research content from their own websites? The content is already available for free online at Sci-Hub, the ‘cat is out of the bag’ so to speak. Blocking the one legal route of access a researcher has is just plain stupid. What are they thinking? This is mad.

    1. Mr. Gunn

      They’re either crazy, or they don’t yet recognize Sci-hub as mainstream and thus are conducting business as usual.

      Sorry you’re having to deal with this, Chris. Can I ask what it is about the Science Direct text-mining API that you found insufficient? (I vaguely recall discussing this once already… did we?)

    1. Mr. Gunn

      I was responding to Ross’s comment, “In a time when Sci-Hub is utterly mainstream,…”

      Maybe the threading got messed up for you? It looks right for me.

    1. Roger Mutt

      SciHub.io uses Captchas on many records, so one couldn’t simply crawl that content programmatically.

  3. scotiabankpr.com

    These data can also be used as baseline data to identify extreme statistical results in the literature by determining their percentile score, or to replicate other meta-research.


Leave a Reply

Your email address will not be published. Required fields are marked *

e.g. 0000-0002-7299-680X

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>