Depositing and evaluating datasets: een sketch of four disciplines in four countries

50 %
50 %
Information about Depositing and evaluating datasets: een sketch of four disciplines in...
Education

Published on March 4, 2014

Author: FrankvanderMost1

Source: slideshare.net

Description

This presentation gives an overview of the answers that my interviewees during the ACUMEN project gave on questions about their data deposition practices. It was my 'farewell' presentation at DANS.

Depositing and evaluating datasets: een sketch of four disciplines in four countries Frank van der Most eHumanities and DANS ! DANS lunch-presentation, 4 March 2014 1

ACUMEN Academic Careers Understood through
 Measurement and Norms www.research-acumen.eu ! “Currently, there is a discrepancy between the criteria used in performance assessment and the broader social and economic function of scientific and scholarly research.” (proposal, p. 3) !2

ACUMEN Additional problems ✦ “lack of resources for qualitative evaluation due to increased scale of research” ✦ “quantitative measures are often not applicable at the individual level” ✦ “lack of recognition for new types of work”
 (ACUMEN description of work, p. 3) !3

ACUMEN ! ✦ so, ACUMEN will develop •a portfolio for researchers, and •criteria and guidelines for Good Evaluation Practices ✦ Done … see the website !4

My part in ACUMEN !5

Research Question ✦ How do evaluations influence the development of individuals’ careers in academia?
 (What is the impact of evaluations?)
 •Wide notion of evaluations •Careers + ‘work’ !6

Method ✦ Quite structured, in depth interviews ✦ 4 Countries : UK, Germany, the Netherlands, Poland ✦ 4 Disciplines : Astronomy/astrophysics, Environmental engineering, Philosophy (+), Public Health ✦ Individual academics •3 levels of seniority: PhD + 5 to 10 years, + 11 to 25 years, + 26 years or more (or equivalents) ✦ Deans, HoDs, or HR manager !7

Method Number of interviews done AA EE P+ PH Total UK 4 (3+1) 4 (3+1) 3 (2+1) 4 (3+1) 15 (11+4) Germany 2 (2+0) 3 (3+0) 3 (2+1) 3 (3+0) 11 (10+1) The Netherlands 3 (3+0) 4 (3+1) 3 (2+1) 4 (3+1) 14 (11+3) Poland 2 (1+1) 1 (1+0) 4 (3+1) 1 (1+0) 8 (6+2) Total 48 11 (9+2) 12 (10+2) 13 (9+4) 12 (10+2) (38+10) Numbers in brackets: individuals + deans/HR managers !8

Questions relevant for DANS" Academic individuals ✦ Have you ever deposited your research data, or have you ever reviewed a data set? •Could you describe the occasions in a few words? ✦ In which evaluations is data deposition, creation, maintenance, or review addressed? !9

AI: Have you ever deposited your research data? Answer (coded) Discipline AA EE P+ PH Total No 1 2 1 4 8 No, because we do not produce data 1 1 5 1 8 Yes, it is done automatically by the facilities 5 !10 5

European Space Agency Example of European Southern Observatory archives ESO / B. Tafreshi (twanight.org) http://www.eso.org/public/archives/images/screen/potw1239a.jpg
 Accessed 4 March 2014 !11

http://archive.eso.org/eso/eso_archive_main.html Accessed 4 March 2014 !12

http://archive.eso.org/eso/eso_archive_main.html Accessed 4 March 2014 !13

http://archive.eso.org/eso/eso_archive_main.html Accessed 4 March 2014 !14

AI: Have you ever deposited your research data? Answer (coded) Discipline AA EE P+ PH Total No 1 2 1 4 8 No, because we do not produce data 1 1 5 1 8 Yes, it is done automatically by the facilities 5 5 Question not asked 1 No, but I store privately 1 No, but I know that we should 2 No in general, but Yes, one time 1 2 2 3 3 1 3 Yes, we produce catalogues 3 2 No, but financing parties claim the data 2 2 1 1 Yes, some data is deposited at a central place in the group 1 1 Yes, in the library of the university 1 1 10 38 Total 9 !15 10 9

AI: have you ever reviewed a dataset? ✦ No, except … ✦ In AA: implicit reviewing when re-using a dataset •‘Reduction’ of data / processing of data •Need to know about the instrument and trust the data ✦ Exception of one specific project (EE) !16

AI: In which evaluations is data deposition, creation, maintenance, or review addressed? Answer (coded) Discipline Do not know of any AA 3 EE P+ PH Total 8 8 24 2 Question not asked 5 1 2 5 The process is evaluated 2 2 Incidentally, when data is re-processed 2 2 Through peer review of publications 1 1 2 No, it should be done but probably is not 1 1 Through peer review of claims to discoveries 1 1 Yes, through pipeline processing 1 Total 9 1 10 9 10 38

AI: In which evaluations is data deposition, creation, maintenance, or review addressed? Answer (coded) Discipline Do not know of any AA 3 EE P+ PH Total 8 8 24 2 Question not asked 5 1 2 5 The process is evaluated 2 2 Incidentally, when data is re-processed 2 2 Through peer review of publications 1 1 2 No, it should be done but probably is not 1 1 Through peer review of claims to discoveries 1 1 Yes, through pipeline processing 1 Total 9 1 10 9 10 38

AI: In which evaluations is data deposition, creation, maintenance, or review addressed? Answer (coded) Discipline Do not know of any AA 3 EE P+ PH Total 8 8 24 2 Question not asked 5 1 2 5 The process is evaluated 2 2 Incidentally, when data is re-processed 2 2 Through peer review of publications 1 1 2 No, it should be done but probably is not 1 1 Through peer review of claims to discoveries 1 1 Yes, through pipeline processing 1 Total 9 1 10 9 10 38

AI: In which evaluations is data deposition, creation, maintenance, or review addressed? Answer (coded) Discipline Do not know of any AA 3 EE P+ PH Total 8 8 24 2 Question not asked 5 1 2 5 The process is evaluated 2 2 Incidentally, when data is re-processed 2 2 Through peer review of publications 1 1 2 No, it should be done but probably is not 1 1 Through peer review of claims to discoveries 1 1 Yes, through pipeline processing 1 Total 9 1 10 9 10 38

Questions relevant for DANS" Deans, HR managers and Heads of Departments ✦ Is your institution promoting data deposition and data review?
 Since when?
 How? ✦ In which evaluations is data deposition, creation, maintenance, or review addressed? !21

DH: Is your institution promoting data deposition and data review? Answer (coded) Discipline AA EE P+ PH Total Question not asked 1 2 1 4 Yes, university promotes 1 1 1 3 No promotion, but it may happen 1 No promotion 1 Total 2 !22 2 1 2 1 4 2 10

Dean in EE ✦ Motivation for researchers? •Feels like extra bureaucracy •Sit on their data •What’s the point once you finish the project •Checking for fraud ✦ Problems of depositing and re-use •Meta-data > experimental setting !23

Dean in PH ✦ Yes, the university is promoting ✦ Many questions to answer •Should we store everything? Capacity •Consent from patients, anonymity ✦ External institute or at the university? •Trust in institution •Technical reliability (15 years), back-ups etc. !24

DH: In which evaluations is data deposition, creation, maintenance, or review addressed? Answer (coded) Discipline AA EE P+ PH Do not know of any 1 2 1 2 Question not asked 1 Total 6 2 1 No evaluations, but interesting question 2 Total !25 2 3 1 4 2 10

Conclusions ✦ Differences between disciplines, but also within them ✦ DANS may get more busy than expected •Outside pressure, from research councils and through incidents such as Stapel case •Or less busy … ? ✦ How to ‘sell’ the effort needed? •How to turn data deposition into academic credit? !26

!27

Thank you for your attention Questions, comments? !28

Add a comment

Related presentations

Related pages

Ecosystems - Scribd

Evaluating progress ... In many countries, ... The current debate now seems to be focusing on four ...
Read more

Auto Insurance Quotes – Get Free Quotes on a New Policy ...

Eliana and Brody ... The company website explains mandatory coverages in the car collection this business was removed from the services collection 7 From ...
Read more

www.ipres2016.ch

2 3 Proceedings of the 13th International Conference on Digital Preservation iPRES 2016 conference proceedings will be made available under a Creative ...
Read more

web.mit.edu

... /s by four order s of ... By testing our approach on datasets related to the ... During triaxial compaction the exponent n only varies betw een 2 ...
Read more

Forensic Human Identification - Scribd

FORENSIC HUMAN IDENTIFICATION An Introduction FORENSIC HUMAN IDENTIFICATION An ... member countries that they should ... of the four different types of ...
Read more

standard boundary element: Topics by Science.gov

... out of the four ... by moving the field point inside the fluid medium and evaluating some surface and ... large numbers of datasets.
Read more

Combining Genetic Diversity and Spatio-Temporal Data to ...

1 COMBINING GENETIC DI VERSITY AND SPATIO TEMPORAL DATA TO CHARACTERIZE THE SPA TIAL ECOLOGY OF ... the four soil variables ... from three countries, ...
Read more

META-INF/MANIFEST.MFname/audet/samuel/shorttyping

cou,countries clu,column cntn,continue cge,charge crer,career cra,create clf,california ... coms,comments cntc,contact cion,collection calg,calling chan,chain
Read more