Skip to content

Commit a0b31bd

Browse files
committed
Merge branch 'pr/PetrosStav/94' into pr/PetrosStav/86
2 parents 8d81e4d + 1985b39 commit a0b31bd

18 files changed

+789
-133
lines changed

.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,5 @@
66
/.luarc.json
77
**/_targets
88
**/_freeze/
9+
10+
*.docx

_quarto.yml

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -101,6 +101,7 @@ website:
101101
- sections/3_societal_impact/uptake_in_education.qmd
102102
- sections/3_societal_impact/uptake_in_medical_practice.qmd
103103
- sections/3_societal_impact/uptake_in_the_legal_sector.qmd
104+
- sections/3_societal_impact/OS_access_advantage.qmd
104105

105106
- title: Economic Impact
106107
contents:
@@ -128,10 +129,6 @@ website:
128129
- sections/5_reproducibility/reuse_of_code_in_research.qmd
129130
- sections/5_reproducibility/reuse_of_data_in_research.qmd
130131

131-
comments:
132-
hypothesis:
133-
theme: clean
134-
135132
format:
136133
html:
137134
toc: true
@@ -148,8 +145,6 @@ format:
148145
zotero: PathOS
149146
bibliography: references.bib
150147

151-
editor: visual
152-
153148
license:
154149
text: >
155150
Open Science Impact Indicator Handbook © 2024 by PathOS is licensed under CC BY 4.0

references.bib

Lines changed: 264 additions & 26 deletions
Large diffs are not rendered by default.

sections/0_causality/open_data_citation_advantage.qmd

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,6 @@ affiliations:
1515
title: The effect of Open Data on Citations
1616
---
1717

18-
1918
::: {.callout collapse="true"}
2019
# History
2120

@@ -25,7 +24,7 @@ title: The effect of Open Data on Citations
2524
| 1.0 | 2024-11-13 | First draft | V.A. Traag |
2625
:::
2726

28-
We here provide some idea of what it would take to try to infer the causal effect of one specific Open Science practice on citation impact. In particular, we consider the effect of [Open Data](../1_open_science/prevalence_open_fair_data_practices.qmd) on [citation impact](../2_academic_impact/citation_impact.qmd). That is, papers that share their data might be more likely to be cited. This is something that has been called the "Open Data Citation Advantage", and in the PathOS scoping review of the academic impact of Open Science [@klebel_academic_2024], evidence was found for a small positive effect of sharing data.
27+
We here provide some idea of what it would take to try to infer the causal effect of one specific Open Science practice on citation impact. In particular, we consider the effect of [Open Data](../1_open_science/prevalence_open_fair_data_practices.qmd) on [citation impact](../2_academic_impact/citation_impact.qmd). That is, papers that share their data might be more likely to be cited. This is something that has been called the "Open Data Citation Advantage", and in the PathOS scoping review of the academic impact of Open Science [@klebel_academic_2025], evidence was found for a small positive effect of sharing data.
2928

3029
Inferring the causal effect of open data on citation impact is not straightforward and cannot easily be done in an experimental setting. Although an experiment study could in principle be done, it would require researchers to participate and follow the experimental, randomised "treatment" of sharing data or not, which will be challenging, especially where more and more data policies mandate that data should be shared. This means that, barring such experiments, we have to rely on observational studies of citations to publications that have (not) shared their data. Note that we here only focus on whether data was shared or not, not whether the data is FAIR or not, or the extent to which it is FAIR, although that might be a relevant confounder to consider.
3130

sections/2_academic_impact/introduction_academic_impact.qmd

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,6 @@ repo-actions: true
2727

2828
Of the various areas of impact (academic, societal and economic) of Open Science, academic impact is probably the most well-established and researched impact area. In particular, indicators of [citation impact](citation_impact.qmd) and [collaborations](collaboration_intensity.qmd) have a long history and a long development and are well-established. At the same time, there are also more recently developed indicators that are becoming relevant in this area, such as the [reuse of data](reuse_of_data_in_research.qmd) and [code](reuse_of_code_in_research.qmd) in research, and are still actively being developed. The rise of reference managers enabled tracking [readership](academic_readership.qmd) instead of only citations, making it possible to distinguish between papers that are read widely and papers that are cited widely. At the same time, aspects of academic impact such as “quality”, [“novelty” or “interdisciplinarity”](interdisciplinarity.qmd) remain complicated to develop robust indicators for based on existing data sources, thus requiring more in-depth manual assessments.
2929

30-
In addition, academic impact is most closely associated with Open Science practices, meaning that we can track potential effects of Open Science more easily than in societal and academic impact. This is also visible in the available literature on specific impacts. For instance, the PathOS’ scoping review of academic impact [@klebel_academic_2024] lists 489 studies compared to 196 reviewed studies in societal impact [@cole_societal_2024], and even less for economic impact. There is also a streetlight effect, with studies of the impact of Open Access on Citation Advantage constituting a large part of the literature on academic impact.
30+
In addition, academic impact is most closely associated with Open Science practices, meaning that we can track potential effects of Open Science more easily than in societal and academic impact. This is also visible in the available literature on specific impacts. For instance, the PathOS’ scoping review of academic impact [@klebel_academic_2025] lists 485 studies compared to 196 reviewed studies in societal impact [@cole_societal_2024], and even less for economic impact. There is also a streetlight effect, with studies of the impact of Open Access on Citation Advantage constituting a large part of the literature on academic impact.
3131

3232
Still, although establishing a connection between Open Science and academic impact is more straightforward, inferring the causal effect remains a common challenge across impact areas.

sections/2_academic_impact/use_of_code_in_research.qmd

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,11 +39,11 @@ This indicator can be useful to provide a more comprehensive view of the impact
3939

4040
### Connections to Reproducibility Indicators
4141

42-
This indicator focuses on identifying and measuring the presence and contribution of code or software within research activities, providing insight into how these tools support the research process itself. In contrast, reproducibility-focused indicators such as [Reuse of Code in Research](https://handbook.pathos-project.eu/indicator_templates/quarto/5_reproducibility/reuse_of_code_in_research.html) examine the extent to which code or software is adopted and utilized in subsequent studies, reflecting its broader applicability, reusability and role in reproducibility. Additionally, the [Impact of Open Code in Research](https://handbook.pathos-project.eu/indicator_templates/quarto/5_reproducibility/impact_of_open_code_in_research.html) highlights the value of openly shared code or software in fostering transparency, collaboration, and validation across the scientific community.
42+
This indicator focuses on identifying and measuring the presence and contribution of code or software within research activities, providing insight into how these tools support the research process itself. In contrast, reproducibility-focused indicators such as [Reuse of Code in Research](../5_reproducibility/reuse_of_code_in_research.html) examine the extent to which code or software is adopted and utilized in subsequent studies, reflecting its broader applicability, reusability and role in reproducibility. Additionally, the [Impact of Open Code in Research](../5_reproducibility/impact_of_open_code_in_research.html) highlights the value of openly shared code or software in fostering transparency, collaboration, and validation across the scientific community.
4343

4444
### Connections to Reproducibility Indicators
4545

46-
This indicator focuses on identifying and measuring the presence and contribution of code or software within research activities, providing insight into how these tools support the research process itself. In contrast, reproducibility-focused indicators such as [Reuse of Code in Research](https://handbook.pathos-project.eu/indicator_templates/quarto/5_reproducibility/reuse_of_code_in_research.html) examine the extent to which code or software is adopted and utilized in subsequent studies, reflecting its broader applicability, reusability and role in reproducibility. Additionally, the [Impact of Open Code in Research](https://handbook.pathos-project.eu/indicator_templates/quarto/5_reproducibility/impact_of_open_code_in_research.html) highlights the value of openly shared code or software in fostering transparency, collaboration, and validation across the scientific community.
46+
This indicator focuses on identifying and measuring the presence and contribution of code or software within research activities, providing insight into how these tools support the research process itself. In contrast, reproducibility-focused indicators such as [Reuse of Code in Research](../5_reproducibility/reuse_of_code_in_research.html) examine the extent to which code or software is adopted and utilized in subsequent studies, reflecting its broader applicability, reusability and role in reproducibility. Additionally, the [Impact of Open Code in Research](../5_reproducibility/impact_of_open_code_in_research.html) highlights the value of openly shared code or software in fostering transparency, collaboration, and validation across the scientific community.
4747

4848
# Metrics
4949

sections/2_academic_impact/use_of_data_in_research.qmd

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -39,11 +39,11 @@ Nevertheless, this document attempts to summarize what indicators can be used to
3939

4040
### Connections to Reproducibility Indicators
4141

42-
This indicator focuses on identifying and measuring how data is utilized in research activities, providing insight into its contribution to academic outputs and innovation. In contrast, the [Reuse of Data in Research](https://handbook.pathos-project.eu/indicator_templates/quarto/5_reproducibility/reuse_of_data_in_research.html) examines the extent to which existing datasets are adopted for subsequent studies, emphasizing reusability and reproducibility. Additionally, the [Impact of Open Data in Research](https://handbook.pathos-project.eu/indicator_templates/quarto/5_reproducibility/impact_of_open_data_in_research.html) highlights the broader effects of openly sharing data, fostering transparency, and driving advancements across scientific communities.
42+
This indicator focuses on identifying and measuring how data is utilized in research activities, providing insight into its contribution to academic outputs and innovation. In contrast, the [Reuse of Data in Research](../5_reproducibility/reuse_of_data_in_research.html) examines the extent to which existing datasets are adopted for subsequent studies, emphasizing reusability and reproducibility. Additionally, the [Impact of Open Data in Research](../5_reproducibility/impact_of_open_data_in_research.html) highlights the broader effects of openly sharing data, fostering transparency, and driving advancements across scientific communities.
4343

4444
### Connections to Reproducibility Indicators
4545

46-
This indicator focuses on identifying and measuring how data is utilized in research activities, providing insight into its contribution to academic outputs and innovation. In contrast, the [Reuse of Data in Research](https://handbook.pathos-project.eu/indicator_templates/quarto/5_reproducibility/reuse_of_data_in_research.html) examines the extent to which existing datasets are adopted for subsequent studies, emphasizing reusability and reproducibility. Additionally, the [Impact of Open Data in Research](https://handbook.pathos-project.eu/indicator_templates/quarto/5_reproducibility/impact_of_open_data_in_research.html) highlights the broader effects of openly sharing data, fostering transparency, and driving advancements across scientific communities.
46+
This indicator focuses on identifying and measuring how data is utilized in research activities, providing insight into its contribution to academic outputs and innovation. In contrast, the [Reuse of Data in Research](../5_reproducibility/reuse_of_data_in_research.html) examines the extent to which existing datasets are adopted for subsequent studies, emphasizing reusability and reproducibility. Additionally, the [Impact of Open Data in Research](../5_reproducibility/impact_of_open_data_in_research.html) highlights the broader effects of openly sharing data, fostering transparency, and driving advancements across scientific communities.
4747

4848
# Metrics
4949

0 commit comments

Comments
 (0)