Methodology

Data collection started in November 2023 on the first component - a large-scale survey to peers across the world to investigate their reviewing practices.

 

Sampling

​

Following a four-stage stratified sampling (countries, universities, departments, faculty members), we have collected emails from scholars working in social sciences departments across the world.

​

1. Countries 

A sample of N=49 countries was identified using the following criteria to ensure inclusive geographical coverage: For each of the main geographical regions, the three most populous countries are selected; Additional countries from each region are selected among those with average and low population size, to ensure a greater diversity; Excluded from the selection are countries with a population of fewer than 200,000 inhabitants, and active totalitarian regimes; Are also excluded countries in which no universities are ranked in the Time Higher Education (THE) ranking, used for the identification of higher institutions (see below).

 

The following countries are included in the final sample: Australia, Benin, Brazil, Canada, Chile, China, Colombia, Costa Rica, Croatia, Czech Republic, Denmark, Egypt, Ethiopia, France, Germany, Ghana, Hong Kong, Hungary, India, Indonesia, Iran, Israel, Italy, Japan, Kazakhstan, Kenya, Lebanon, Lithuania, Malta, Mexico, Netherlands, New Zealand, Nigeria, Norway, Peru, Philippines, Poland, Russia, Singapore, South Africa, Spain, Switzerland, Tanzania, Tunisia, Turkey, Uganda, United Kingdom, United States, Uruguay (Botswana was initially included, but had to be dropped due to the lack of reliable information found about universities; the final selection is thus of N=49 countries.)

​

2. Institutions

In each country, a sample of N=10 institutions (universities, research centers) is identified as follows: The sample includes the 5 most important institutions (established by the position in the 2022 Times Higher Education (THE) ranking), as well as the institutions ranked 11-15 in that same ranking; Selected institutions need to have a working website address. If this is not the case, the university is replaced by another university in the THE top 15 ranking; If fewer than 15 institutions are listed for a given country in the THE ranking, the first 10 are selected; If fewer than 10 institutions are listed for a given country in the THE ranking, the list is completed by a Google search ("[country] university") with as many universities as possible. The number of institutions in the USA is oversampled (N=20), using a similar logic (10 most important institutions in the THE ranking, plus institutions between ranking 21-30)

​

3. Departments/units

In each institution (university, research center), all departments, teaching units and research clusters related to the main research disciplines covered (“social sciences”, broadly defined) are manually identified, and their websites listed in a masterfile. Approximately 3600 such departments have been identified.

​

4. Faculty members (respondents)

The list and contact details (name, surname, email address) of all members of the departments/units identified in the previous step are scraped (automated algorithm, matched with manual checks). This last step generates the initial contact list.

​

This multi-stage sampling procedure yielded a dataset including 157,439 unique emails. After removing 2,000 respondents randomly drawn for pilots run in July and August of 2023, the final sample includes 155,439 respondents. Pilot studies allow us to expect a response rate around 4%; we expect the final dataset to include approximately 6000 respondents.

​

​

More details about procedures, survey, measurement, and estimation strategies can be found in the Projects's OSF page. The data, when available, will also be stored there.

 

​