Course materials for the co-design of (open) science research projects - Version 2

At the Center for Research and Interdisciplinarity (CRI) in Paris the Master students of the Master students of the digital, learning and life sciences take a joint course on open science in their first year. After a two-day kick-off workshop, the course 2020-2021 was designed around project-based learning, in which interdisciplinary teams of 4-6 students run their own small research project from start to finish over the course of 12 weeks. To facilitate their work they are accompanied by fortnightly group sessions and the course materials we are sharing here.

The overall topic or “challenge” for the course research projects, in this case, was about learning processes at CRI, but these research design materials can be adapted for other topics and areas.

• Authors: Enric Senabre Hidalgo, Bastian Greshake Tzovaras, Ignacio Atal & Ariel Lindner
• Date: March 2021
• Source: Materials based on previous work from authors, Profs Chercheurs project and Research co-design toolkit. For complementary sources see the legal notes.

This version of open course materials allow to copy and paste it completely or any of its components (sections, subsections, tables, paragraphs, etc), in order to adapt them to the specific needs for other research courses of activities. It’s a first tested version (and a work in progress through future similar courses) which can of course be improved so feedback welcome!

1. Format: This is a template with a suggested sequence of specific steps that can guide you as a team to collaboratively design a research project. It is based on previous materials from researchers at CRI, and requires moving things around and completing different sections.
2. Participation: Each team has a template like this one for the same general challenge (but deal autonomously and independently with their own research project on learning). At some parts you will be invited to contribute individually here with your own ideas, comments and suggestions, followed by sections and ways to decide as a group what to select and prioritize. Please try to consider and respect all contributions equally, and move with collective intelligence and agreements at each stage (and also having fun, when possible).
3. Discussion: One tip is to also generate notes and discussions anywhere during the process, to expand the discussion and other relevant considerations as you move (in parallel to the course meetings). This way, you will also document your learning and research process.

Sequence of the course and assignments

Guided by the calendar of the course and the indications from facilitators, students will follow the sequence summarised in this info box to:

Get to know and depart from the research topics / concerns regarding learning [Assignment #1] Develop the project’s research question(s) [Assignment #1] Produce a first diagram and protocol of the research: concepts, methods, participants, logistics [Assignment #2] Perform data collection and analysis [Assignment #3] Work on synthesis and results obtained: Via presentation from all teams [Assignment #4] Via manuscript draft [Assignment #5] Sharing final manuscript / report + Feedback on the research process (by mentors and peers): During synchronous video-call sessions On documents derived from this template Adding comments in other platforms or tools used by students (especially regarding final manuscript document)

The following discrete list of topics and potential areas of inquiry represent a first, incomplete but series of motivating issues prepared from the course facilitators at CRI in a short brainstorming session preceding the open science course (in this case, with a focus on learning-related experiences and challenges). Please consider them as departing points or inspiration, so following the next sections you can try to “adopt” or “adapt” them, but also think about different ones which motivate you as a team…

1. Time spent on learning...
   1. Virtual vs physical
   2. Time of the day
   3. Time between classes
2. Sleep patterns during the course...
3. Resources used for learning...
4. Collaborative networks...
5. Stress / mood analysis when studying...
6. Language uses...
7. Learning styles...
8. Phone / screen time (distractions)...
9. ...what else?

Sandbox for new related questions or reformulating the above ones. Reflect here your brainstorming:

A research question is what any research project wants to answer. Figuring out and choosing a research question is an essential part of any type of scientific process (open or not). Afterwards the investigation will require data collection and analysis, for which the choice of a methodology is also critical (but we will get to this later).

Good research questions seek to improve knowledge on an important challenge or topic, and should usually be as narrow and specific as possible.

1. Think about the brainstorming round based on previous examples of learning challenges and topics, and which specific questions could be derived from them;
2. Write at least one question individually (we recommend anonymously), following the structure suggested below;
3. Become a “peer-reviewer” of the rest of questions following the template, just adding your name to the “Selection criteria” columns to show support (and comments if needed);
   1. Originality: Regarding the impression you have that the question has been addressed before, or not, or not properly (good moment for a quick literature review/check on Dimensions, Semantic Scholar, Google Scholar or similar). Remember that all this is about radical openness, so the way you can make things open science also counts here.
   2. Feasibility: Remember that you will work together to answer the selected question! Consider if it maybe leads to a very complex process, or some variables will be too difficult, or finding useful data is going to be a hard mission, etc. Here is also a good moment to think about openness, because doing things openly does not usually mean doing things easily.
   3. Impact: Here it can be tricky because impact can mean many things. We would like you to think basically about (1) “academic” impact (that is, in relation to originality and what you assume is the state of the art) and (2) “social” impact (which relates to how the project can help to improve or change things for good in a real-life given context).
4. As a general rule, add maximum 10 times your name on RQ table below (so each participants has in total a maximum of “10 votes”) and do not vote as reviewer for your own question(s), in order to allow the most coherent selection process possible.

Structure of possible research questions

Descriptive questions	Relational questions
How...? What...? How often...? What percentage...? What proportion...? How far...? What value...?	What is the relationship between...and...? What is the effect of...on...?

Reflect here the different research questions (RQ) generated during your discussion:
	List of research questions	Selection criteria
		Originality	Feasibility	Impact
RQ #1		Student name 1, etc
RQ #2
RQ #3
[etc]

Here in this second step, after checking results from the previous section and another round discussing them (if needed), please use the template to indicate the first or main research question your team would like to work on, considering that:

1. You can still “merge” some of the previous questions or modify and improve a given one together, as result of your deliberation.
2. You can also pick up one or two additional questions from the previous stage in case you think they are interrelated, or there’s important subquestions (or you think you may need a “B plan” at some point).
3. Derived from that, we also invite you to describe a hypothesis in relation to the question(s). It is not always necessary (and your project may not need it) but in case you think about one remember that...

A hypothesis is a tentative explanation for a phenomenon, but for a hypothesis to be scientific it requires to be tested. It is usually based on previous observations that cannot satisfactorily be explained with available scientific studies, theories or literature.

...which means that, during this stage, you may need to go back to “reading mode” and invest some time in checking more literature and studies :)

In this section you will also find additional areas with estimation of effort or time for answering the RQ, or where to reflect previous literature and findings, etc. Please use them under your best criteria, and only if you consider them useful!

Reflect here the selected RQ and rest of needed details:
Selected research question(s):
Keywords:
Type of people / participants involved (students, teachers, researchers, laypeople, institutions...):
Generic description of the context: What is the problematic situation? What could be the underlying causes? What is the goal to achieve? What would need to happen in order to consider that the question has been addressed satisfactorily?

Additional questions / subquestion #1:
Additional questions / subquestion #2:

Literature research - What scientific articles address similar questions and/or hypotheses?

Summary of previous research - What are scientific studies and previous results saying about it?

This phase will help you to reflect how the protocol of the data collection process will take place, as well as other needed logistics for your research.

First of all, think about the key elements to include (from the main icons in the subsections below): research question(s) and hypothesis, concepts or units of analysis, people and groups involved, methods to apply, etc. Select everything that your team thinks would be necessary, discuss it and then organize it as a sequence following a temporary order (from left to right).

In order to reflect the overall approach of your research, you have to work on a shared presentation where to reflect the different areas to cover according to your selected research questions and / or hypothesis in the previous stage. Consider this canvas as a basic “collage” which reflects (1) the overall concepts to be addressed, as well as the sampling population of participants to involve; (2) the “flow” of data collection and data analysis methods; and finally (3) some logistics or needed tasks to take into account as well. These important ingredients of your research project range from more conceptual to more practical, and have an implicit sequence or progression you have to consider as well.

Larger description / outline of the research project:
Please add here a couple of paragraphs explaining the diagram with more details, as well as the overall research goals and process.

In order to fill this canvas by “copy and paste” from the options below, your team has a dedicated template like this one, where you can add more information or details regarding each selected icon, in order to make more clear for you and rest of the course participants what the research plan is about, in general terms. This won't cover more precise considerations or even changes of plans as you move through the data collection and analysis phases in the next stages (and sections of this document), but consider it as a sort of visual point of departure.

The following lists contain the main elements or “ingredients” for your research diagram on the canvas, as icons you can directly copy and paste and put in a sequential order on your slides document. It is important, as mentioned, that you add titles or short descriptions to them once selected and reflected on the diagram, for a better general understanding of the research process. Although the list is long, you only need to select a few of them!

High-level concepts related to the research challenge

This icon is for questions regarding learning (in relation to this specific topic of the course), as already discussed in your team brainstorming in connection with the selected RQ or hypothesis. It is simply a reminder of key concepts derived from it (it can be many things, according to what you plan to do: accessibility, equality, formal education, stress, learning materials, concentration, motivation or any other of the many derived issues you are already wondering about).

Sampling population / participants

Use this icon (more than one time, if needed) to reflect the type of participants you need to observe, ask or obtain data from / about during the research. Think about your team as “self-researchers” but additionally the broad learning and teaching community: students of different types, teachers or mentors, family, education institutions, etc. Depending on the ambition and needs of the research you are planning, now is a good moment to be as specific about them as possible! + Info on sampling

Methods for data collection

We show below a short list of possible data collection methods and techniques among the big diversity of methods usually used for research. Consider it also as a set of recommendations, based on this type of research challenge (about learning experiences) and the specific research questions you plan to answer empirically. You will probably just need one or two of them, since the more you implement the more complex your research will probably be.

	Online survey: This is a data generation and collection method in which you present a list of questions to a selected group of participants. It can have multiple choice questions, or only one choice, or instead open ended questions (or a combination of them). The way you formulate the questionnaire influences whether or not you can use quantitative or qualitative methods of its analysis. You can use several online tools for this, like LimeSurvey, Google Forms or Typeform, among many other possibilities. + Info on surveys
	Content analysis: Although it can also be seen as an analytical tool, we suggest this possibility as a specific data collection tool in terms of accessing existing content or knowledge reflected in open or accessible online formats. In this case this can be course materials, curriculum descriptions, results from evaluations, books or manuals, for example. Once you collect them, you can classify them into specific categories, compare them to other types of analysis regarding your research, or as the base for additional methods.
	Open datasets: This is another possible collection method, for already existing data from previous studies or sources, where you access and work with available data from repositories and process it with a new purpose or research objective. It can be the main source of your research (combining different open datasets) or complementary to it, if you have also generated new data from other methods. One popular resource is the Harvard Dataverse repository (where you can find several datasets related to education and learning using the search features).
	Web scraping: This technique will require some specific skills for accessing data which is not open or treated for research purposes, but instead contained on PDFs, online spreadsheets or other online sources to explore. Once you identify those sources (in this case regarding learning and education, but it can also be sociodemographic information) you need to “copy & paste” or use more sophisticated coding processes to recover interesting data. + Info on web scraping
	Wearables: This is a recently popular and relatively accessible way of gathering personal data about oneself, which depending on the aim of your research could be useful as well. However, it could be the case that not all participants have a wearable at their disposal or can get one for the sole purpose of the study. Popular wearables today like Fitbit or similar allow to track data like steps walked, heart rate, sleep patterns and more, which can offer interesting insights even with small samples or participants (and also taking into account its reliability as well as possible problems accessing raw data, depending on the tool).

If you prefer to use other data collection methods or tools, you can check for more for example at the Guide to research and research methods for Master’s (M.A.) students at the Faculty of Humanities of University of Jyväskylä (Finland), here. Also, for finding the corresponding icons to reflect them, we recommend you to use TheNounProject platform, with all types of icons like these ones, available under Creative Commons licenses.

Methods for data analysis

Below there’s another list of methods to start to consider at this stage of your research project design, in close relation to the ones suggested above. That is, several possible ways to analyze the data you obtain, regarding the scope and objectives of your research question. Again, it is just a series of suggestions for you to consider, which can also influence the previous choices regarding methods for data gathering. For this reason, on your research design diagram they should be placed in close connection to the data gathering methods. Although there could be different things to know in advance and take into account when analyzing your data, we will cover that part of the project in other sections of this document.

	Time series analysis: Time-series analysis tries to measure the existence of a phenomenon in relation to time or periods, stages, etc. For this you need to make observations or measurements of the phenomena during a certain sequence of time. Then you usually categorize and describe the observations or measurements with statistical methods, as the base for obtaining and interpreting results. + Info on time series analysis
	Correlation analysis: Correlation methods of analysis aim to describe the correlation between two variables. Correlation analysis attests the relation between two or more variables, but does not usually measure the causal relation between them. This type of analysis may also indicate the intensity of the relationship between variables. + Info on correlation analysis
	Causal analysis: Causal analysis aims to explain the causal relations between variables. If you want to indicate explicit causality, your study must include some sort of experiment or experimental approach. This way you can compare control and treatment groups, for example, with some sort of variance analysis. You may also use regression analysis, which measures causality in a weaker way. + Info on causal analysis
	Descriptive statistical analysis: Common features of quantitative analysis are graphical representations of statistically analysed data. Here you use descriptive statistical analysis to indicate, for example, the quantities, frequencies, distributions or classifications of phenomena. This “transversal” form of analysis often forms a basis for a more detailed approach to the phenomena studied, such as correlation or causality analysis. Open source tools like Raw Graphs can be useful for this type of “visual” analysis. + Info on descriptive statistical analysis
	Classification analysis: You may use classification when the data consists of a large group of research objects. For this you typically outline and divide the group into classes of objects (sharing similar qualities or resemblances), so you can explain and describe the composition and essence of each group. Variations in classification can vary in terms of degree of logic or similarities, sliding between exact and imprecise. + Info on classification analysis
	Network analysis: Network analysis usually aims to explore and explain social structures and the interdependence of social phenomena. Networks are somehow “everywhere” and can be understood as informative and define relationships between objects and phenomena. The focus of this type of analysis is usually an agent, such as people, organizations, events and other networked processes. The analysis does not aim to explore and explain all the characteristics or quality of the phenomena but the network of relationships around or inside it. + Info on network analysis

If you prefer to explore and use other data analysis methods, you can look for more here.

Research logistics

Finally, another order of things to consider in your research has to do with very practical tasks and skills needed for making it possible. From dissemination to data management, and as something especially important in the way these could be more transparent and coherent with open science principles and practices.

	Contact participants: Once you have defined the data collection methods, you will need to define the best strategy for reaching out to the people if you want to engage external participants in the research process (as “subjects” of study or co-researchers, following citizen science principles). This could require using emails, forum messages, social media or other channels. Also, this implies to define a clear and succinct explanation of the research aim and who is part of it, the intended use of personal data, etc.
	Project information: In relation to the previous point, you may need to set a basic document or website summarising the project research once it is under development, like who is behind it, the research objectives and ways to engage with it, or how to get more information about the process. All this, again, needs some communication skills, but it is also an important part of researching things openly (and an effort that would already help you to write down the purpose, background and results of the research for later on).
	Data management: Once you start to collect data and analyse it, it could be needed to establish a good strategy for storing it, as well as for sharing it openly online as open science “in the making”, depending on the type of content you work with. Also, because collaborating in teams can usually result in problems for finding the right document or information, especially when needed, if some data management practices have not been properly considered beforehand.
	Programming / coding: Depending on the data gathering or analysis methods you want to apply, having someone in the team with good programming or software development skills could be also important. For example for the mentioned data gathering methods of web scraping (if they need to be very elaborated) or for specific types of network analysis.
Other needs	Economic, logistic, ethical, bureaucratic, etc.

Below we provide an example of the Keating Memorial, a current research project under development at the CRI Peer-Produced Research Lab, which summarises in a visual way (using the same canvas as icons) the project’s alignment of research questions, concepts, methods and some of the “logistics” required. The main aim of the Keating Memorial project is understanding individual processes when doing personal citizen science, as well as the group dynamics taking place in these communities of practice and the role of technological infrastructure during the process. As the base of participants and sample population, it’s focused on self-researchers engaged in the Open Humans community.

As you can see above, two of the main research questions addressed in that project are: (RQ1) “What do individuals learn when participating in personal science and citizen science, and what are their motivations?”; (RQ2) “How are community interactions experienced by people engaged in personal science?”. The methods CRI researchers are currently applying in this case are content analysis of web interactions and semi-structured interviews, previous to a community survey, to get a typology of participants and relevant categories according to the three mentioned concepts (motivations, learning processes and peer support). As mentioned, the research diagram above is for the purpose of an example at this stage of the course, but based on a long-term project with many other specifics (if you want to know more, you can find additional information and related details of the research project on GitHub).

Another example could be self-research done by two members of the Open Humans community regarding the effects of the first covid lockdown at the individual level, specifically regarding productivity and physical activity (for more info see Paula Neonova’s blog and Bastian Greshake Tzovaras’ blog). Although slightly different, the concrete aim of both projects (which we summarize as a single process below) was driven by the challenges derived from personal situations when people were forced to drastically change their daily routines, usually shifting to a home office approach.

The concrete research questions of both self-research projects was to study measurable effects of the confinement in the researcher's behavior and physical activity. The data collection methods used were mainly self-tracking wearables and mobile apps, followed by data visualizations and comparisons.

A research protocol or proposal is a document describing the objectives, design, methodology, statistical considerations and organization of a research project. The research protocol also covers how you will ensure the integrity of the data collected. Here you can find an example of one of the research protocols from the Peer Produced Lab, for the Transbiome project about the exploration of the microbial diversity in the neovaginal microbiome.

At this stage, it is important that derived from your research diagram and the research questions (as well as additional material like references, overall challenge, data management, etc), you complete the following protocol document template with the detailed description of your project, prior to starting the data collection process and rest of research tasks.

Study title (One sentence)
1. Context and rationale for research	Rational presenting the context and hypotheses of research. It also includes key concepts, references to literature and previous studies or state of the art. (Two pages maximum, including references)
2. Objectives and evaluation criteria	2.1 Primary objective: (One sentence) Context: (Half page maximum) Research question(s): (Only the question) 2.2 Secondary objectives: (If needed) Context: Research question(s):
3. Organisation of the study	3.1 Description of the study design Recruitment of participants: (One paragraph) Description of the data gathering tool(s): (One paragraph) Data access, storage and processing: (One paragraph) 3.2 Methodology Method #1: (One paragraph) Method #2: (If needed) 3.3 Data analysis (One paragraph per methodology)

As the next stage of your research project, you should consider the different supports, communication strategies and dissemination channels for getting participants to provide data that you can analyse afterwards. For this, the following sections invite you to consider important elements of content, as well as tools needed for the best possible outreach and deployment of the study.

Project title and outline

Rather than the detailed approach needed for the research protocol, in this section you should elaborate a plain, easy to understand short text explaining the general purpose or challenge of the research (but avoiding as much as possible details which can induce bias or affect the expected results). Also add information about the project team, some contact details and mentioning anonymization in data sharing. In case of specific requirements or characteristics for the type of participants needed for the study, this should also be clearly stated in this section. This information will be used for the landing page of your project (at the end of this stage), with a consistent URL at the CRI projects page.

Project title and outline:
Write one or two concise paragraphs maximum

Project background image

The landing page of your project at the CRI directory allows for a representative image as background, for that more “creative” part of the layout you should consider images from repositories like Creative Commons or Wikimedia Commons with copyleft licenses.

Project background image:
Link to the selected image(s)

Data gathering tool

Whereas a survey or other required tool for data gathering, right after the project description and invitation to participate, you should include the link where participants can provide the data. For surveys you could consider easy and usable tools like LimeSurvey, Google Forms or Typeform. The header text there can also repeat some of the basic information provided on the project's landing / info page, as well as an estimation of approximate time required for filling in the requested information.

Data gathering tool:
Link to the data gathering tool, platform, etc

Project dissemination

This section refers to all the communication strategies and channels you can consider for reaching out and getting participants to visit your project landing page (providing the specific link). The following list is a first suggestion of possible ones, but feel free to add more or avoid some according to your plan. You can check first as an example of possible content and style the communication templates used for the Covid open survey project.

Project dissemination plan:
Tool / channel	Considerations	Content
Individual emails	Template message which can be personalized if needed (adding name)	Add template text here
Message to forums, mailing lists, etc	Similar message as above, but oriented to an audience (third person plural) instead of individuals.	Add template text here
Social media	For online channels like Twitter, Facebook  or similar, you should consider a very short but informative text inviting users to know more by visiting the link.	Add template text here
Other	For other platforms like Instagram (with a relevant image), or Whatsapp (adapted to specific groups or contacts) or shared videos, add details here.	Add template text here

Following the principle of peer support in the context of the course, as an initial data gathering process we invite the following 3 “clusters” of student teams to fill in the surveys (and other methods, when applied) of each other’s projects:

• Teams 1-4-9
• Teams 2-3-5
• Teams 6-7-8

For this, you have to access the course main document to find the title and links to each project landing page (or survey instead) on the project’s table, where these clusters are also indicated on the right column. Please first make sure you check and update the basic information per each team.

This part of the research process should allow you to start answering the research questions and confirming (or not) your initial hypothesis. As a non-linear process on many occasions, this phase of the research can be started (or “tested”) while the data gathering is still underway, so you can have some initial insights and preliminary results.

Regardless of the stage of data gathering for your project, at this stage of the course, we invite you to start doing some preliminary visualizations below about your progress and possible approaches to analyse your data. In relation to the different alternatives, you can check again the section above “Methods for data analysis” and use one of these main possible tools:

• Google Forms - How to make a graph in Google Sheets
• Raw Graphs (open source) - For more elaborated visualizations
• Gephi (open source) - Even more sophisticated

Prototyping visualizations

In the form below, reflect at least 3 possible approaches to visualize the data gathered through your survey and the selected tool, adding some text for each visualization to reflect its main value or related results.

Important: In case you have not yet gathered enough data via your survey or other methods, for the purpose of this part of the course (and corresponding session) you can also “simulate” the data, in a way that even if you cannot still derive real initial results, you are exploring the possible visualizations and analyses.

	Graph screenshot and short explanatory text
Visualization #1	Paste or upload the visualization and descriptive text here...
Visualization #2	Paste or upload the visualization and descriptive text here...
Visualization #3	Paste or upload the visualization and descriptive text here...

In this part of the course, we have a session dedicated exclusively to present each team’s results, followed by a round of comments and questions after each project.

Link to your presentation / document with preliminary results:

Below you will find the first feedback from the teacher’s team for the preliminary results presentation session.

Reviewers
Date

Comments:

This part of the research process refers to the necessary steps to publish and share your research, following a series of standard practices and formats in Academia (and also new possible open ones). For this, once you have completed the previous stages above, and discussed possible approaches within your team, the best strategy is to start drafting a “manuscript” that puts together all the previous elements you have worked with.

The following is a common format in academic papers (the IMRD model) which we invite you to follow as the last part of your research process. Here, instead of starting things from scratch, you will mostly need to consider the previous sections and content (research questions, state of the art, protocol, visualizations, etc) and “reuse” it in a coherent, easy to follow order.

Here’s a great guide to follow as much as you can: A framework for scientific papers, by Devin Jindrich.

Title (Up to 16 words)

• Your title should be a clear premise, carrying the main result of your research and context, strictly confined to what your data supports.
  • Bad example: Gender of teachers determined career outcome
  • Good example: Higher professional esteem of men over women when using identical course narrative by European graduate students

Abstract (Up to 120 words)

• Carries the crux of your paper. Must include clearly defined phrases on:
  • Framing the context
  • Hypothesis
  • Methodology
  • Main results
  • Conclusion
  • Perspective
• Here’s an example of how to develop a Nature introductory paragraph
• We invite you to add a visual abstract (you can also consider that possibility, as an additional form of open dissemination) as often requested in leading journals.

Introduction

• This section should contain:
  • A) WHY is the area of research important? (1 Paragraph)
  • B) WHAT is the GAP in scientific understanding? (2-4 paragraphs)
  • C) Your hypothesis and HOW do the proposed General and Measurable hypotheses FILL the gap in understanding? (1-3 paragraphs)
  • D) Overall plan (e.g., to this end we set…. And find that…)

Methods

• Clearly describe your methodology and its choice (including references to precedent use)
• Include protocol (with link to the actual survey etc)

Results

• Analysed data in table/graph format
• Link to raw data

Discussion

• Interpretation of results (What does it mean?)
• Any shortcoming
• Direct implications for the research question at hand

Speculation

• Free non-peer-reviewed paragraph where you could suggest wider implications to the field and speculate further about the validity and implication of your results in other contexts

Link to your manuscript draft:

The following questions and sections are for you as a team to reflect your main impressions and learning or findings regarding the course process. Since you developed as students a research project together, from different backgrounds and previous experiences, try to answer the following from a perspective of “students as researchers”, and in order of priority for each section.

What have been the three more challenging things for you as a team when developing your research project?

How have you experienced openness and collaboration during the process? Please provide three short examples:

How would you improve your research process if you could redo it from scratch? Please provide three concrete ideas:

Please consider completing the following information with each author’s initials at the end of your manuscript. Reflecting author and researcher contributions is another open science practise that allows to understand the development of a project in more detail, who to address in case of doubts, and have a more precise way to attribute contributions. Feel free to remove types of contributions which don’t apply to your case, or add additional ones if you consider it necessary for your specific research project (like dissemination, translation, testing, etc).

Author contributions:
Conceptualization, ; Data curation, ; Formal analysis, ; Investigation, ; Methodology, ; Software, ; Visualization, ; Figures: ; Writing—original draft, ; Writing—review and editing, ; Project administration, .

After your research process and teamwork during the course, as part of this “infinite play” of doing (open) science, we would like you to take a good look at the following manual (and leave at any moment your impressions if you want accordingly!).

Here’s a great book to read carefully: Caron, B. R. (2020). Open Scientist Handbook

Impressions from members of this research team on the handbook? :)

“Rereading” the generic description and details of the research project proposed by the team can help them to progressively produce a description that is understood in the same way by the rest of course participants, as well as different people and education actors (in this specific research topic case), even if they come from different backgrounds. The objective of the reviews or feedback is to support team members in a benevolent way to help them improve the description and development of their open science research, by pointing out elements that could be improved, modified or could need more clarification. Especially, to signal and discuss opportunities for “opening up” each stage of the research process.

The templates in this section are to be used in specific parts of the course by facilitators in parallel to each research process and stages. Except when indicated, the review should be done by people outside the team, and these cycles should be done by different reviewers to maximize the chances that the text will be assessed by people from diverse backgrounds. Feedback templates (for course facilitators) This additional section contains two samples of the feedback forms ideated for course facilitators, which are used in some sections above but can also be personalized and adapted to give specific feedback on other parts of the research documentation of every student's team.

Add an X to “yes” or “no” to the different questions. If the answer to any of the questions is “no,” add comments directly to the description text to help co-authors improve it. The point is not to say whether what has been written is "good" or "bad" but to kindly help the team members as co-authors to clarify what needs to be improved.

Reviewers:	Yes	No
Date:
After reviewing the list of research questions, seems the selected one the best option? Otherwise, indicate with comments the other possible ones or improvements that can be applied to the selected one.
Can the research question be inspiring for other learners and/or teachers? The approach should be clear and relevant enough that other people in the CRI learning community can relate to their own context.
Is the research challenge explained in a clear way? Other participants and learners should understand this without difficulty.
Does this research question and approach have the potential to be developed under open science practices? Consider all the options related to open sharing of materials, data, contributions, etc. under clear open licenses or tools.
General comments:

Reviewers:	Yes	No
Date:
Does the proposed diagram and protocol correspond to a research design? A research design must start from a defined problem or concern and refer to an explicit and detailed research question and/or hypothesis to be achieved, as well as the methods used.
Does the research protocol have the potential to be developed under open science practices? Consider all the options related to open sharing of materials, data, contributions, etc. under clear open licenses or tools.
Is the connection between the different elements (concepts, population, methods and logistics) clear enough? Other participants and researchers in general should understand this without difficulty.
Are the selection of data collection and data analysis methods coherent and doable? Other researchers and participants should understand this without difficulty.
General comments:

1. For editable version of this document (.odt format) and PDF version you can check the corresponding Zenodo files.
2. The list of research questions and the publishable versions of the different projects can be shared afterwards by participants under a CC BY-NC-SA license. In this sense, you should not share personal or sensitive data (on this and the derived documents) about participants or the people you work with.
3. The writing traces left in this document (different versions, drafts, comments, etc.) could be used by the organisation developing the course and by the rest of participants and students for research purposes, in a completely anonymous manner.
4. The materials reflected in this document contain icons (uploaded to Wikiversity with the corresponding copyleft attributions) from Kyle Miller, Iconstock, sevgenjory, Vectors Point, Adrien Coquet, Siipkan Creative, DinosoftLab, Nithinan Tatah, Arif Arif, Misbahul Munir, Eucalyp, ProSymbols, H Alberto Gongora, Nanda Ririz, ProSymbols, vigorn, Alvaro Cabrera, developed for TheNounProject under a Creative Commons license.
5. The text for research methods listed in section 4.1.2 has been adapted in some examples, as indicated, from the original content on "Guide to research and research methods for Master’s (M.A.) students", developed at the Faculty of Humanities of University of Jyväskylä (Finland), licensed under a Creative Commons Attribution - NonCommercial - ShareAlike 3.0 Unported License.