add operationalization details for PathOS case studies across multiple indicators

PetrosStav · PetrosStav · commit b6eb84c09c8d · 2025-09-16T15:41:41.000+03:00
diff --git a/sections/1_open_science/availability_data_repositories.qmd b/sections/1_open_science/availability_data_repositories.qmd
@@ -80,4 +80,11 @@ It can be difficult to obtain data on all existing data repositories, given the
 
 ##### Core Trust Seal
 
-Core Trust Seal is a non-profit organisation that labels data sources with their seal if data sources adhere to the FAIR principles. On the website a [list](https://amt.coretrustseal.org/certificates/) is maintained with all the data sources that the seal has been assigned to. Data stored in these sources can thus be considered to be produced in accordance with the FAIR principles. When performing research related to the availability of data repositories, one can consider repositories that have received the CoreTrustSeal, the Nestor Seal DIN31644, the ISO16363 certification, or similar, to be automatically trusted [@jahn2023].
+Core Trust Seal is a non-profit organisation that labels data sources with their seal if data sources adhere to the FAIR principles. On the website a [list](https://amt.coretrustseal.org/certificates/) is maintained with all the data sources that the seal has been assigned to. Data stored in these sources can thus be considered to be produced in accordance with the FAIR principles. When performing research related to the availability of data repositories, one can consider repositories that have received the CoreTrustSeal, the Nestor Seal DIN31644, the ISO16363 certification, or similar, to be automatically trusted [@jahn2023].
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- Effects of Data Repositories on Data Usage
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/1_open_science/availability_preprint_repositories.qmd b/sections/1_open_science/availability_preprint_repositories.qmd
@@ -143,6 +143,13 @@ OpenAIRE Graph and Directory of Open Access Repositories (DOAR) (integrated in t
 
 All repositories in both data sources are Open Access.
 
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
+
 # Known correlates
 
 The availability of preprint repositories **may** correlate with specific fields of study, given that certain scientific disciplines are more inclined to use preprints than others. Furthermore, regional or national open science policies, funding opportunities, and research culture can also influence the presence or absence of preprint repositories in a particular country or region.
diff --git a/sections/1_open_science/availability_publication_repositories.qmd b/sections/1_open_science/availability_publication_repositories.qmd
@@ -68,6 +68,13 @@ OpenAIRE Graph and Directory of Open Access Repositories (DOAR) (integrated in t
 
 All repositories in both data sources are Open Access.
 
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
+
 # Known correlates
 
 Various factors **may** influence the availability of publication repositories in specific fields, countries, or organizations such as: research output, research funding, technological infrastructure, Open Access mandates, research collaboration, overall research infrastructure.
diff --git a/sections/1_open_science/prevalence_open_access_publishing.qmd b/sections/1_open_science/prevalence_open_access_publishing.qmd
@@ -127,6 +127,13 @@ To facilitate interaction with the Unpaywall API there are also supporting packa
 
 Once we know the Open Access status of each publication, we can easily calculate the percentage, simply as the number of publications out of the total that are Open Access. Additionally, we can do this for each separate Open Access type. We could for example refer to the % Green OA or % Gold OA. Note that percentages may not add up to 100%, but the total may also even exceed 100%, because publications can have both a Green Open Access status and another Open Access status. For that reason, you should not report percentages of Open Access statuses in a cumulative fashion (e.g. not in a stacked bar chart), in particular not when reporting on Green Open Access.
 
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
+
 # Known correlates
 
 There is a large ongoing debate whether Open Access publishing increases the citation impact of publications, known as the so-called Open Access citation advantage. A recent systematic review on the topic suggests that the evidence is inconclusive [@langham-putrow2021].
diff --git a/sections/2_academic_impact/citation_impact.qmd b/sections/2_academic_impact/citation_impact.qmd
@@ -259,6 +259,16 @@ The database is closed access, and we therefore do not provide more details abou
 
 The database is closed access, and we therefore do not provide more details about API usage.
 
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- Impact of Artefact Reuse in COVID-19 Publications
+- Impact of Open Access Routes on Topic Persistence
+- Portuguese Repository Infrastructure RCAAP
+- ELIXIR´s Bioinformatics Resources
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
+
 # Known correlates
 
 As already clarified, citations are affected in general by field and publication year, and these are quite clearly causal effects. There are many other factors that correlate with citations [@onodera2015], for which most it is unclear whether the effect is causal. One factor that is consistently associated with more citations is collaboration [@larivière2015], which is potentially driven by network effects [@schulz]. In addition, there is evidence for a clear causal effect of the journal where something is published on citations [@traag2021].
diff --git a/sections/2_academic_impact/extra-academic_collaboration.qmd b/sections/2_academic_impact/extra-academic_collaboration.qmd
@@ -150,3 +150,10 @@ The four types of non-academic organisations and the six types of possible colla
 The data sources described in the previous section can be used to populate this double entry table for a given piece of research, a set of researches or (even more interestingly) several sets of researches whose different collaboration-profile can be assessed comparing their value for each of the cell, row or column as well as for the entire table.
 
 In particular, all the metrics matrix and the data source described above can be employed to assess collaboration in OS but also in non-OS projects, allowing to investigate whether open science is less or more prone to facilitate specific types of collaboration. Similarly, the comparison can be carried out by disciplinary field, type of research, project size or duration, type of funding, etc.
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- ELIXIR´s Bioinformatics Resources
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/2_academic_impact/thematic_persistence.qmd b/sections/2_academic_impact/thematic_persistence.qmd
@@ -137,3 +137,10 @@ The continuity framework [@Yan2014ResearchDM] quantifies persistence by categori
 
 Continuity is evaluated by the **strength of inter-year linkages** among topic clusters.\
 This allows distinguishing different **modes of persistence** and topic evolution.
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- Impact of Open Access Routes on Topic Persistence
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/2_academic_impact/use_of_data_in_research.qmd b/sections/2_academic_impact/use_of_data_in_research.qmd
@@ -113,4 +113,11 @@ Based on the data retrieval information from the data repositories one can compi
 
 ##### Standardization protocols
 
-To combat the issue of standardization for the tracking of clicks/views/downloads of data use in research [The COUNTER Code of Practice for Research Data](https://www.countermetrics.org/code-of-practice/) gives guidance on how to standardize count systems for data use. The guidelines provided by The COUNTER Code of Practice for Research Data can be followed to make inquiries in data retrieval information more comparable among researchers.
+To combat the issue of standardization for the tracking of clicks/views/downloads of data use in research [The COUNTER Code of Practice for Research Data](https://www.countermetrics.org/code-of-practice/) gives guidance on how to standardize count systems for data use. The guidelines provided by The COUNTER Code of Practice for Research Data can be followed to make inquiries in data retrieval information more comparable among researchers.
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- Effects of Data Repositories on Data Usage
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/3_societal_impact/OS_access_advantage.qmd b/sections/3_societal_impact/OS_access_advantage.qmd
@@ -91,3 +91,10 @@ The indicator described in these pages is meant to be computed based on access d
 As an example, one can (1) focus on the scientific domains associated with the accessed resources and (2) aggregate the IP-related information in countries and industrial sectors, for example follwiong the NACE typology ([ec.europa.eu/eurostat/web/nace](https://ec.europa.eu/eurostat/web/nace)).
 
 This data structure allows to observe not only the general access advantage (or disadvantage) of different type of scientific ressource, but to break the calculation down to different combination of scientific domains, countries and industrial sectors, in order to facilitate comparison and detect potential confounding factors.
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/3_societal_impact/scientific_literacy.qmd b/sections/3_societal_impact/scientific_literacy.qmd
@@ -62,3 +62,10 @@ While the PISA instrument is not publicly available, the framework includes deta
 Besides the PISA instrument, there is a great variety of other instruments used to assess scientific literacy, commonly aligned with vision II and the PISA conceptualization. In general, the focus seems to have shifted from viewing scientific literacy as one single ability to defining it as a multidimensional construct. Most of the instruments' contents focus on a specific domain or context (e.g., biology), employing various formats to test skills. The majority of instruments have been developed for use in (secondary school) students and have mainly been employed in that context [@coppi2023; @opitz2017].
 
 A relatively recent and broadly used instrument is the Test of Scientific Literacy Skills (TOSLS), developed to be a comprehensive and psychometrically sound measure that can be employed on a large scale. The TOSLS includes 28 multiple-choice questions presented through realistic scenarios and requiring respondents to use skills that relate to understanding scientific methods and interpreting data [@gormally2012]. The instrument is available as the supplement to @gormally2012.
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/3_societal_impact/uptake_and_impact_on_societal_issues.qmd b/sections/3_societal_impact/uptake_and_impact_on_societal_issues.qmd
@@ -117,4 +117,11 @@ Given the complexity of the assessment described in these pages, the metrics sug
 ## Metrics for task 4 Open Science uptake
 
 -   Number of references to OS resources in the statements and documents of each actor or actor type active on each societal issue.
--   Ratio of OS over non-OS references to OS resources in the statements and documents of each actor or actor type active on each societal issue.
+-   Ratio of OS over non-OS references to OS resources in the statements and documents of each actor or actor type active on each societal issue.
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/3_societal_impact/uptake_by_media.qmd b/sections/3_societal_impact/uptake_by_media.qmd
@@ -123,6 +123,13 @@ In addition to citation, views and download information, PlumX collects data fro
 -   Wikipedia
 -   Facebook
 
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
+
 # Known correlates
 
 Correlations among various measure of uptake by media have been studies [@fang2020].
diff --git a/sections/3_societal_impact/uptake_by_patient_groups.qmd b/sections/3_societal_impact/uptake_by_patient_groups.qmd
@@ -100,4 +100,11 @@ This is an automated tool [@stavropoulos2023], leveraging Deep Learning and Natu
 
 To measure the proposed metric, the tool can be used to identify the reused and created OS inputs in the texts of the patient group websites / reports.
 
-One limitation of this methodology is that it may not capture all instances of research artifacts if they are not explicitly mentioned in the scientific text. Additionally, the machine learning algorithms used by the tool may not always accurately classify whether a research artifact has been reused or created, and may require manual validation.
+One limitation of this methodology is that it may not capture all instances of research artifacts if they are not explicitly mentioned in the scientific text. Additionally, the machine learning algorithms used by the tool may not always accurately classify whether a research artifact has been reused or created, and may require manual validation.
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/3_societal_impact/uptake_by_policy_makers.qmd b/sections/3_societal_impact/uptake_by_policy_makers.qmd
@@ -67,6 +67,13 @@ Overton was founded in 2019 also by Euan Adie. It is a novel database with the g
 
 Beyond the methodologies of the different data providers mentioned above (Overton and Altmetric), and other past applications related to the extractions of mentions to scientific concepts, there are no other large methodologies focused on extracting mentions to scientific publications or scholarly entities from policy sources.
 
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
+
 # Known correlates
 
 Previous studies have reported mostly positive weak correlations between policy citations and other metrics, including scholarly citations [@fang2024; @bornmann2016a; @szomszor2022].
diff --git a/sections/3_societal_impact/uptake_in_education.qmd b/sections/3_societal_impact/uptake_in_education.qmd
@@ -96,4 +96,11 @@ This is an automated tool [@stavropoulos2023], leveraging Deep Learning and Natu
 
 To measure the proposed metric, the tool can be used to identify the reused and created OS inputs in the texts of the educational material.
 
-One limitation of this methodology is that it may not capture all instances of research artifacts if they are not explicitly mentioned in the scientific text. Additionally, the machine learning algorithms used by the tool may not always accurately classify whether a research artifact has been reused or created, and may require manual validation.
+One limitation of this methodology is that it may not capture all instances of research artifacts if they are not explicitly mentioned in the scientific text. Additionally, the machine learning algorithms used by the tool may not always accurately classify whether a research artifact has been reused or created, and may require manual validation.
+
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
diff --git a/sections/3_societal_impact/uptake_in_medical_practice.qmd b/sections/3_societal_impact/uptake_in_medical_practice.qmd
@@ -108,6 +108,15 @@ To measure the proposed metric, the tool can be used to identify the reused and
 
 One limitation of this methodology is that it may not capture all instances of research artifacts if they are not explicitly mentioned in the scientific text. Additionally, the machine learning algorithms used by the tool may not always accurately classify whether a research artifact has been reused or created, and may require manual validation.
 
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- Impact of Artefact Reuse in COVID-19 Publications
+- French Open Access Infrastructure
+- ELIXIR´s Bioinformatics Resources
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
+
 # References
 
 Stavropoulos, P., Lyris, I., Manola, N., Grypari, I., & Papageorgiou, H. (2023). Empowering Knowledge Discovery from Scientific Literature: A novel approach to Research Artifact Analysis. In L. Tan, D. Milajevs, G. Chauhan, J. Gwinnup, & E. Rippeth (Eds.), Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023) (pp. 37–53). Association for Computational Linguistics. https://doi.org/10.18653/v1/2023.nlposs-1.5
diff --git a/sections/3_societal_impact/uptake_in_the_legal_sector.qmd b/sections/3_societal_impact/uptake_in_the_legal_sector.qmd
@@ -159,6 +159,13 @@ Finally, it is crucial to include in any qualitative study or questionnaire a sa
 
 Once these limitations and pointers to qualitative assessment have been acknowledged, it is nevertheless possible to perform the same automatic queries than for other indicators, for instance the patient groups community or the policy-makers, in order to provide a possibly limited, top of the iceberg understanding of the reuse of OS by the legal sector, without evidencing all the real impact.
 
+# Operationalization in PathOS Case Studies
+
+This indicator has been operationalized in the following case studies:
+- French Open Access Infrastructure
+
+Further details are available in Deliverable D3.3 of PathOS, accessible through the [PathOS Zenodo community](https://zenodo.org/communities/pathos/records?q=&l=list&p=1&s=10&sort=newest).
+
 # Known correlates
 
 In the absence of citation, it is difficult to assert that a legal decision has been taken following a research result published in an Open Access journal possibly relying on an Open Data set. Correlation (or coincidence, as explained in the Societal Issues indicator) cannot be interpreted as causation, or direct and only impact.
diff --git a/sections/4_economic_impact/cost_savings.qmd b/sections/4_economic_impact/cost_savings.qmd
diff --git a/sections/4_economic_impact/innovation_output.qmd b/sections/4_economic_impact/innovation_output.qmd
diff --git a/sections/4_economic_impact/science_industry_collaboration.qmd b/sections/4_economic_impact/science_industry_collaboration.qmd
diff --git a/sections/5_reproducibility/reuse_of_code_in_research.qmd b/sections/5_reproducibility/reuse_of_code_in_research.qmd
diff --git a/sections/5_reproducibility/reuse_of_data_in_research.qmd b/sections/5_reproducibility/reuse_of_data_in_research.qmd
