All posts by Danny Kingsley

Who is requesting what through Cambridge’s Request a Copy service?

In October last year we reported on the first four months of our Request a Copy service. Now, 15 months in, we have had over 3000 requests and this provides us with a rich source of information to mine about the users of our repository.  The dataset underpinning the findings described here is available in the repository.

What are people requesting?

We have had 3240 requests through the system since its inception in June 2016. Of those the vast majority have been for articles 1878 (58%) and theses 1276 (39%). The remaining requests are for book chapters, conference objects, datasets, images and manuscripts. It should be noted that most datasets are available open access which means there is little need for them to be requested.

Of the 23 requests for book chapters, it is perhaps not surprising that the greatest number  – 9 (39%) came for chapters held in the collections from the School of Humanities and Social Sciences. It is however possibly interesting that the second highest number – 7 (30%) came for chapters held in the School of Technology.

The School of Technology is home to the Department of Engineering which is the University’s largest department. To that end it is perhaps not surprising that the greatest number of articles requested were from Engineering with 311 of the 1878 requests (17%) from here. The areas with next most requested number of articles were, in order, the Department for Public Health and Primary Care, the Department of Psychiatry, the Faculty of Law and the Judge Business School.

What’s hot?

Over this period we have seen a proportional increase in the number of requests for theses compared to articles. When the service started the requests for articles were 71% versus 29% for theses. However more recently, theses have overtaken request for articles to a ratio of 54% to 46%.

The most requested thesis, by a considerable amount, over this period was for Professor Stephen Hawking’s thesis with double the number of requests of the following ten most requested theses. The remaining top 10 requested theses are heavily engineering focused, with a nod to history and social research. These theses were:

The top 10 requested articles have a distinctly health and behavioural focus, with the exception of one legal paper authored by Cambridge University’s Pro Vice Chancellor for Education, Professor Graham Virgo.

When are people requesting?

Looking at the day of the week people are requesting items, there is a distinct preference for early in the week. This reflects the observations we have made about the use of our helpdesk and deposits to our service – both of which are heaviest on Tuesdays.

When in the publication cycle are the requests happening?

In our October 2016 blog we noted that of the articles requested in the four months from when the service started in June 2016 to the end of September 2016, 45% were yet to be published, and 55% were published but not yet available to those without a subscription to the journal.  The method we used for working this out involved identifying those articles which had been requested and determining if the publication date was after the request.

Now, 15 months after the service began it is slightly more difficult to establish this number. We can identify items that were deposited on acceptance because we place these items on a very long embargo (until 2100) until we can establish the publication date and set the embargo period. So in theory we could compare the number of articles with this embargo period against those that have a different date.

However articles that would provide a false positive (that appear to have been requested before publication) would be ones which had been published but we had not yet identified this – to give an indication of how big an issue this is for us, as of the end of last week there were 1768 articles in our ‘to be checked’ pile. We would also have articles that would provide a false negative (that appear to have been requested after publication) because they had been published between the request and the time of the report and the embargo had been changed as a result. That said, after some analysis of the requests for articles and conference proceedings, 19% are before publication. This is a slightly fuzzy number but does give an indication. 

How many requests are fulfilled?

The vast majority of the decisions recorded (35% of the total requests for articles, but 92% of the instances where we had a decision) indicate that the requestor shared their article with the requestor. The small number (3%) of  ‘no’ recordings we have indicate the request was actively rejected.

We do not have a decision recorded from the author in 62% of the requests. We suspect that in the majority of these the request simply expires from the author not doing anything. In some cases the author may have been in direct correspondence with the requestor. We note that the email that is sent to authors does look like spam. In our review of this service we need to address this issue.

Next steps

As we explained in October, the process for managing the requests is still manual. As the volume of requests is increasing the time taken is becoming problematic. We estimate it is the equivalent of 1 person day per week. We are scoping the technical requirements for automating these processes. A new requirement at Cambridge for the deposit of digital theses means there will be three different processes because requests for these theses will be sent to the author for their decision. These authors will, in most cases, no longer be affiliated with Cambridge. Requests for digitised theses where we do not have the author’s permission are processed within the Library and requests for articles are sent to the Cambridge authors.

Given the challenges with identifying when in the publication process the request has been made, we need to look at automating the system in a manner that allows us to clearly extract this information. The percentage of requests that occur before publication is a telling number because it indicates the value or otherwise of having a policy of collecting articles at the acceptance point rather than at publication.

Published 12 September 2017
Written by Dr Danny Kingsley
Creative Commons License

Sustaining long-term access to open research resources – a university library perspective

In the third in a series of three blog posts, Dave Gerrard, a Technical Specialist Fellow from the Polonsky-Foundation-funded Digital Preservation at Oxford and Cambridge project, describes how he thinks university libraries might contribute to ensuring access to Open Research for the longer-term.  The series began with Open Resources, who should pay, and continued with Sustaining open research resources – a funder perspective.

Blog post in a nutshell

This blog post works from the position that the user-bases for Open Research repositories in specific scientific domains are often very different to those of institutional repositories managed by university libraries.

It discusses how in the digital era we could deal with the differences between those user-bases more effectively. The upshot might be an approach to the management of Open Research that requires both types of repository to work alongside each other, with differing responsibilities, at least while the Open Research in question is still active.

And, while this proposed method of working together wouldn’t clarify ‘who is going to pay’ entirely, it at least clarifies who might be responsible for finding funding for each aspect of the task of maintaining access in the long-term.

Designating a repository’s user community for the long-term

Let’s start with some definitions. One of the core models in Digital Preservation, the International Standard Open Archival Information System Reference Model (or OAIS) defines ‘the long term’ as: 

“A period of time long enough for there to be concern about the impacts of changing technologies, including support for new media and data formats, and of a changing Designated Community, on the information being held in an OAIS. This period extends into the indefinite future.”

This leads us to two further important concepts defined by the OAIS:

Designated Communities” are an identified group of potential Consumers who should be able to understand a particular set of information”, i.e. the set of information collected by the ‘archival information system’. 

A “Representation Information Network” is the tool that allows the communities to explore the metadata which describes the core information collected. This metadata will consist of:

  • descriptions of the data contained in the repository
  • metadata about the software used to work with that data,
  • the formats in which the data are stored and related to each other, and so forth.  

In the example of the Virtual Fly Brain Platform repository discussed in the first post in this series, the Designated Community appears to be: “… neurobiologists [who want] to explore the detailed neuroanatomy, neuron connectivity and gene expression of Drosophila melanogaster.” And one of the key pieces of Representation Information, namely “how everything in the repository relates to everything else”, is based upon a complex ontology of fly anatomy.

It is easy to conclude, therefore, that you really do need to be a neurobiologist to use the repository: it is fundamentally, deeply and unashamedly confusing to anyone else that might try to use it.

Tending towards a general audience

The concept of Designated Communities is one that, in my opinion, the OAIS Reference Model never adequately gets to grips with. For instance, the OAIS Model suggests including explanatory information in specialist repositories to make the content understandable to the general community.

Long term access within this definition thus implies designing repositories for Designated Communities consisting of what my co-Polonsky-Fellow Lee Pretlove describes as: “all of humanity, plus robots”. The deluge of additional information that would need to be added to support this totally general resource would render it unusable; to aim at everybody is effectively aiming at nobody. And, crucially, “nobody” is precisely who is most likely to fund a “specialist repository for everyone”, too.

History provides a solution

One way out of this impasse is to think about currently existing repositories of scientific information from more than 100 years ago. We maintain a fine example at Cambridge: The Darwin Correspondence Project, though it can’t be compared directly to Virtual Fly Brain. The former doesn’t contain specialist scientific information like that held by the latter – it holds letters, notebooks, diary entries etc – ‘personal papers’ in other words. These types of materials are what university archives tend to collect.

Repositories like Darwin Correspondence don’t have “all of humanity, plus robots” Designated Communities, either. They’re aimed at historians of science, and those researching the time period when the science was conducted. Such communities tend more towards the general than ‘neurobiologists’, but are still specialised enough to enable production and management of workable, usable, logical archives.

We don’t have to wait for the professor to die any more

So we have two quite different types of repository. There’s the ‘ultra-specialised’ Open Research repository for the Designated Community of researchers in the related domain, and then there’s the more general institutional ‘special collection’ repository containing materials that provide context to the science, such as correspondence between scientists, notebooks (which are becoming fully electronic), and rough ‘back of the envelope’ ideas. Sitting somewhere between the two are publications – the specialist repository might host early drafts and work in progress, while the institutional repository contains finished, publish work. And the institutional repository might also collect enough data to support these publications, too, like our own Apollo Repository does.

The way digital disrupts this relationship is quite simple: a scientist needs access to her ‘personal papers’ while she’s still working, so, in the old days (i.e. more than 25 years ago) the archive couldn’t take these while she was still active, and would often have to wait for the professor to retire, or even die, before such items could be donated. However, now everything is digital, the prof can both keep her “papers” locally and deposit them at the same time. The library special collection doesn’t need to wait for the professor to die to get their hands on the context of her work. Or indeed, wait for her to become a professor.

Key issues this disruption raises

If we accept that specialist Open Research repositories are where researchers carry out their work, that the institutional repository role is to collect contextual material to help us understand that work further down the line, then what questions does this raise about how those managing these repositories might work together?

How will the relationship between archivists and researchers change?

The move to digital methods of working will change the relationships between scientists and archivists.  Institutional repository staff will become increasingly obliged to forge relationships with scientists earlier in their careers. Of course, the archivists will need to work out which current research activity is likely to resonate most in future. Collection policies might have to be more closely in step with funding trends, for instance? Perhaps the university archivist of the digital future might spend a little more time hanging round the research office?

How will scientists’ behaviour have to change?

A further outcome of being able to donate digitally is that scientists become more responsible for managing their personal digital materials well, so that it’s easier to donate them as they go along. This has been well highlighted by another of the Polonsky Fellows, Sarah Mason at the Bodleian Libraries, who has delivered personal digital archiving training to staff at Oxford, in part based on advice from the Digital Preservation Coalition. The good news here is that such behaviour actually helps people keep their ongoing work neat and tidy, too.

How can we tell when the switch between Designated Communities occurs?

Is it the case that there is a ‘switch-over’ between the two types of Designated Community described above? Does the ‘research lifecycle’ actually include a phase where the active science in a particular domain starts to die down, but the historical interest in that domain starts to increase? I expect that this might be the case, even though it’s not in any of the lifecycle models I’ve seen, which mostly seem to model research as either continuing on a level perpetually, or stopping instantly. But such a phase is likely to vary greatly even between quite closely-related scientific domains. Variables such as the methods and technologies used to conduct the science, what impact the particular scientific domain has upon the public, to what degree theories within the domain conflict, indeed a plethora of factors, are likely to influence the answer.

How might two archives working side-by-side help manage digital obsolescence?

Not having access to the kit needed to work with scientific data in future is one of the biggest threats to genuine ‘long-term’ access to Open Research, but one that I think it really does fall to the university to mitigate. Active scientists using a dedicated, domain specific repository are by default going to be able to deal with the material in that repository: if one team deposits some material that others don’t have the technology to use, then they will as a matter of course sort that out amongst themselves at the time, and they shouldn’t have to concern themselves with what people will do 100 years later.

However, university repositories do have more of a responsibility to history, and a daunting responsibility it is. There is some good news here, though… For a start, universities have a good deal of purchasing power they can bring to bear upon equipment vendors, in order to insist, for example, that they produce hardware and software that creates data in formats that can be preserved easily, and to grant software licenses in perpetuity for preservation purposes.

What’s more fundamental, though, is that the very contextual materials I’ve argued that university special collections should be collecting from scientists ‘as they go along’ are the precise materials science historians of the future will use to work out how to use such “ancient” technology.

Who pays?

The final, but perhaps most pressing question, is ‘who pays for all this’? Well – I believe that managing long-term access to Open Research in two active repositories working together, with two distinct Designated Communities, at least might makes things a little clearer. Funding specialist Open Research repositories should be the responsibility of funders in that domain, but they shouldn’t have to worry about long-term access to those resources. As long as the science is active enough that it’s getting funded, then a proportion of that funding should go to the repositories that science needs to support it. The exact proportion should depend upon the value the repository brings – might be calculated using factors such as how much the repository is used, how much time using it saves, what researchers’ time is worth, how many Research Excellence Framework brownie points (or similar) come about as a result of collaborations enabled by that repository, etc etc.

On the other hand, I believe that university / institutional repositories need to find quite separate funding for their archivists to start building relationships with those same scientists, and working with them to both collect the context surrounding their science as they go along, and prepare for the time when the specialist repository needs to be mothballed. With such contextual materials in place, there don’t seem to be too many insurmountable technical reasons why, when it’s acknowledged that the “switch from one Designated Community to another” has reached the requisite tipping point, the university / institutional repository couldn’t archive the whole of the specialist research repository, describe it sensibly using the contextual material they have collected from the relevant scientists as they’ve gone along, and then store it cheaply on a low-energy medium (i.e. tape, currently). It would then be “available” to those science historians that really wanted to have a go at understanding it in future, based on what they could piece together about it from all the contextual information held by the university in a more immediately accessible state.

Hence the earlier the institutional repository can start forging relationships with researchers, the better. But it’s something for the institutional archive to worry about, and get the funding for, not the researcher.

Published 11 September 2017
Written by Dave Gerrard

Creative Commons License

Continuing the conversation: a CRUK workshop on RDM

In May 2017 the Office of Scholarly Communication organised a workshop with Paola Quattroni from Cancer Research UK (CRUK) focusing on data sharing policy and practices. It was a great opportunity for the funder to outline its policies and current initiatives on data sharing and for the Cambridge researchers to discuss the issues, suggest further solutions and give feedback to the funder about the changes they would like to see implemented. This blog highlights the main points of the workshop.

This session was continuing  the conversation from February last year when the CRUK and Wellcome Trust came to Cambridge to speak to our research community.

CRUK’s grand ambition

In her presentation “Data sharing in policy and practice with Cancer Research UK“, Paola Quattroni began with CRUK’s grand ambition: “To bring forward the day all cancers are cured” and “see three quarters of people surviving cancer within the next 20 years.

One of the key elements to materialise this and maximise public benefit is data sharing. CRUK firmly believes that transparency, research integrity and swift dissemination and reproducibility of research results are key ingredients to the success.

“Our goal is to improve how research is carried out,” explained Paola, who is the Research Funding Manager – Data at CRUK. “We fund the best science and expect researchers to follow best practices… Improving patient benefit and health is our ambition.”

She emphasised the need to have ongoing discussions with the research community and work together on how to overcome barriers to data sharing. Appropriate sharing and dissemination of research data are particularly important for CRUK, and good data management is the first step to get most from the data and facilitate sharing and re-use. In this context, CRUK is actively working to increase and improve data sharing by being instructive but not necessarily demanding in its requirements.

The audience

The majority of the attendees came from the fields of Biological Sciences and Clinical Medicine. When asked why they came to the workshop the consensus was to be informed regarding the CRUK policy and what actions they needed to take. Examples of individual responses included:

  • To learn how to fulfil funders’ requirements.
  • To learn more about processing data.
  • To know the policy on sharing code and data.
  • To learn the difference between data sharing and open data.
  • To discuss about the costs of storing data and how to be able to forecast costs for periods of more than 10 years.
  • To learn more about contractual agreements.
  • To learn what the funder expects regarding data sharing.
  • To learn and inform other colleagues about it.

The structure

The workshop started with an icebreaker. The audience was asked to pinpoint why they came to the workshop and what they hoped to gain from it. Following that, Paola Quattroni presented CRUK’s policy on the management and sharing of data, explained why data sharing is important, what are the barriers and outlined current initiatives to improve data sharing among researchers.

Paola highlighted some of the work CRUK is doing to increase data sharing such as the recent signing of the San Francisco Declaration of Research Assessment (DORA) and the fact that CRUK is continuing to work with others to put it into practice. Other future activities include:

  • Encouraging grant applicants to explain the significance and impact of their discoveries, publications and a broad range of other outputs (e.g. policy influence).
  • Being more explicit about evaluating grant applicants’ publications according to their scientific content, rather than simply consider where they are published.
  • Working with reviewers and committee members to evaluate the impact of all research outputs.
  • Measuring the re-use of research.
  • Encouraging replication studies.
  • Recognising and rewarding researchers who share their data.

After the presentation, everybody split into groups and identified various challenges of data sharing which were then analysed by the teams and the trainers. The last part of the workshop concentrated on group feedback and suggestions from the audience on what funders could do to further enhance collaboration with the research community.

Challenges

The workshop continued by splitting into groups. Each group identified challenges and problems of data sharing with regard to Publishing, Skills and Training, Rewards and Data Infrastructure:

Publishing

A recurring item among all groups was the fear of being scooped and the loss of publication opportunities. Also, that the impact factor is still be-all, end-all. Other challenges included:

  • Accepting citations of preprints as a metric of achievement – can be dangerous as groups can release data non-peer reviewed online to discourage innovation of competitors.
  • Range of requirements across different journals/publishers.
  • Need to take care not to kill analytical innovation.
  • The larger the collaboration the higher the importance of a standardised data format and analysis.

Skills & Training

The Skills and Training section concentrated on how to write data management plans and standardise laboratory notes as well as the necessary training to catch up with technology. Other points included:

  • Lack of computer skills/knowledge to physically upload data.
  • Formatting data.
  • Version Control.

Rewards

It was apparent in most of the groups that time, cost and re-usability problems were significant inhibitors regarding rewards and incentives:

  • There is a need to overcome the ‘time burden’ aspect of sharing.
  • Cost and Time – solution: Electronic Laboratory Notebooks (ELN) – one or many? Public or private?
  • New PI (Persistent Identifier) for metrics.
  • Re-usability – how do you measure it?
  • DMPs are required at the time of grant submission. However, the researcher needs to report after one year because various parameters can change and might need to be re-adjusted.

Data Infrastructure

The need for standardisation in data acquisition, storage and analysis methods and how ‘big data’ is handled by the funders were common themes in this category. In addition, it was pinpointed that individual Institutes should have the infrastructure to support data sharing and DMP writing.

Other data infrastructure challenges included:

  • Data formats – for example there are so many different scanners for imaging, which all have different formats.
  • EU project testing imaging modality across 20 sites where integrating the data is a challenge. The analogy is a clinical trial where protocols and practices have to produce comparable data.
  • Cost of the software: there are open source imaging software available. However, you may need different imaging analysis tools.

Solutions

Although there was not enough time to concentrate on all challenges, the ongoing discussions turned into ideas that provided the seeds for possible solutions or change of strategies regarding how data is being valued and shared.

For example, what if you are just scooped? Would citations help? One solution is that if you have a DOI stamp this can be evidence that you were first.

Currently, publications are considered to be the sole reward so there is a wide fear of loss of publication opportunities. However, if your data is more valuable than the paper, then the dataset becomes the incentive and is highly valued. How can this be achieved? Micropublishing? If you can build a career on data publishing instead of papers, it would change the incentive strategy. Instead of relying on the old system where there is a big story, what about writing a small story or event data papers? Data in conjunction with data notes is a type of article. These kind of outputs are valuable and publishers should consider this.

Despite the fact that staff working for funders have often been researchers themselves, they could visit researchers from different disciplines to get an idea of what is needed, especially with discipline specific DMPs. Some participants suggested that DMPs should be discipline-specific and standardised. As an example, if preclinical and clinical data had the same format, such data could easily be compared.

Another solution proposed by the participants to the financial challenges associated with data sharing could be an open access fund for data, similar to COAF that supports the cost of infrastructure and rewards openness.

Conclusions

As already mentioned, the discussions evolved to the point that there was no time left to analyse all challenges and talk about practical issues.

For example, there was a clear need from the participants’ point of view for practical guidance on data plans and distinct approaches per field (STEM/HASS). Questions arose about the use and cost of ELNs and any implications in the future.  Similarly, about what happens if data needs to be deposited somewhere else or in the middle of the plan. What would the rules be for additional funding midway in such instances? Lastly, preservation and infrastructure costs that associate projects in the long term was another big topic as well future funders’ strategies regarding ‘big data’. (See this blog for a discussion on the cost issue).

This workshop brought together researchers from different disciplines interested in learning more about data management and sharing at CRUK. From the funder’s perspective, it was a great opportunity to discuss policies and initiatives in data sharing and to hear directly from researchers about the main barriers to data sharing. CRUK strives to help researchers overcome these barriers and is actively working to facilitate the way research is carried out and ultimately shared.

It was agreed that this workshop was only the beginning and highlighted that collaboration is key to overcome some of these challenges.

The main outcomes, however, were clear from the onset:

  • There is a recognised need for ongoing collaboration between funders, researchers and institutions.
  • A global view is required – all funders should have the same vision and aims regarding data sharing.
  • Reporting and disseminating all data is key.
  • Data needs to be available and reusable.
  • We need to overcome the technical and infrastructure challenges of how to measure the “journey” of the data and its re-usability.

Published 07 September 2017
Written by Maria Angelaki

Creative Commons License