Supervisors’ experiences of doctoral supervision in times of change

Erika Löfström (Faculty of Educational Sciences, University of Helsinki, Helsinki, Finland)
Lotta Tikkanen (Centre for University Teaching and Learning, Faculty of Educational Sciences, University of Helsinki, Helsinki, Finland)
Henrika Anttila (Centre for University Teaching and Learning, Faculty of Educational Sciences, University of Helsinki, Helsinki, Finland)
Kirsi Pyhältö (Centre for University Teaching and Learning, Faculty of Educational Sciences, University of Helsinki, Helsinki, Finland and Centre for Higher and Adult Education, Faculty of Education, Stellenbosch University, Stellenbosch, South Africa)

Studies in Graduate and Postdoctoral Education

ISSN: 2398-4686

Article publication date: 31 August 2023

Issue publication date: 2 January 2024

663

Abstract

Purpose

Empirical evidence on how supervisors have perceived the changes and the implications of the COVID-19 pandemic on their supervision is scarce. This paper aims to examine how the changing landscape of doctoral education has affected supervision from the supervisors’ perspective.

Design/methodology/approach

This survey addressed change, challenges and impact in supervisory responsibilities due to COVID-19 pandemic. The survey was completed by 561 doctoral supervisors from a large multi-field research-intensive university in Finland.

Findings

Results show that supervisors estimated that their supervision had been negatively affected by the pandemic, but to a lesser extent than their doctoral candidates’ progress and well-being. In the changed landscape of supervision, the supervisors grappled with challenges related to recognising doctoral candidates’ need of help. Supervisors’ experiences of the challenges and the impact of changed circumstances varied depending on the field and the position of the supervisor, whether they supervised part- or full-time candidates, and the organisation of supervision.

Practical implications

The slowed-down progression and diminishing well-being of doctoral candidates reported by supervisors is likely to influence supervision in a delayed way. Supervisors may be anticipating some issues with stalled studying and stress, but the question is the extent to which they are prepared to handle these as they emerge in supervision encounters. The fact that the experiences varied across field, position, organisation of supervision and the type of candidates (full or part time) suggests that support provided for supervisors to overcome challenges needs to be tailored and engineered.

Originality/value

This study contributes to the literature on doctoral supervision by exploring the impact of transitioning to online supervision and the rapid changes in doctoral supervision as a consequence of the recent global pandemic.

Keywords

Citation

Löfström, E., Tikkanen, L., Anttila, H. and Pyhältö, K. (2024), "Supervisors’ experiences of doctoral supervision in times of change", Studies in Graduate and Postdoctoral Education, Vol. 15 No. 1, pp. 34-48. https://doi.org/10.1108/SGPE-01-2023-0004

Publisher

:

Emerald Publishing Limited

Copyright © 2023, Erika Löfström, Lotta Tikkanen, Henrika Anttila and Kirsi Pyhältö.

License

Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial & non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


The pandemic resulted in an almost global shift to online or distance supervision. This has inevitably affected the way supervisors and candidates work, including their expectations, roles and responsibilities (Huet and Casanova, 2020). This shift has challenged supervision and supervisors in new ways. Hence, it is no surprise that supervisors have reported experiences of having been left alone to face the challenges, and consequently, feedback may have been delayed or rushed, lacking expressions of care and empathy and coming across as overly judgemental despite good intentions and the many effective strategies (Mullen, 2021). Supervisors may not have felt supported in providing the kind of pastoral care that they had anticipated the doctoral candidates would need (UK Council for Graduate Education, 2021). While the impact of the pandemic on academic work has gained much attention, there have been few peer-reviewed empirical studies published on supervision compared to the numbers of position and reflection papers relying on anecdotal evidence (see Lokhtina et al., 2022). Most studies on the effects of the pandemic on researchers and academic work have focused on doctoral candidates’ experiences (Else, 2021). Research in the sciences showed that doctoral candidates have been identified as being one of the groups most affected by the pandemic (Myers et al., 2020). The pandemic influenced doctoral candidates’ progress in terms of research design, access to resources, workload, mental health and finances (Donohue et al., 2021).

Doctoral supervisors play a key role in buffering the impact of negative influences on doctoral candidates (Lokhtina et al., 2022). However, the changed supervision landscape also created new demands on supervisors by forcing them to adapt to new technologies and remote modes of supervision, finding new ways of organising viva voces and supporting doctoral candidates during lockdowns. Evidence about the influences of the pandemic on supervisory practices is partly mixed: while doctoral candidates have reported receiving pastoral support, help in modifying their research plans and help with institutional procedures from their supervisors during the pandemic (Lambrechts and Smith, 2020), they have also reported less frequent access to supervision compared to pre-pandemic times making their experiences predominantly negative (Pyhältö et al., 2022). Running out of funding prematurely while research progress has been delayed has been a hazard for many doctoral candidates (Donohue et al., 2021; Pyhältö et al., 2022). Accordingly, it is no surprise that expectations have been voiced to senior academics to devote more time to applying for grants that would help early-career researchers (Guintivano et al., 2021). Supervisory responsibilities are reported to have increased (UK Council for Graduate Education, 2021). Scholars who have had teaching and supervision duties have particularly reported lower work productivity (Myers et al., 2020). Supervisors have found themselves in situations in which it has been difficult to advise doctoral candidates on what to do considering the many unknowns related to the future (Lasater et al., 2021). Supervisors have also been tasked with broader assessment of the doctoral candidates’ overall situation. Supervisors may have been requested to write and retain notes on adjustments to the doctoral candidates’ plans and any agreements, and on the candidates’ situations, concerning matters such as school closures and home schooling of children, which may have affected the candidates’ performance (Lambrechts and Smith, 2020).

Yet, empirical evidence on how supervisors have perceived the changes and the implications of these on their supervision is scarce (Pyhältö et al., 2022). We contribute by examining how the changing landscape of doctoral education affected supervision from the supervisors’ perspective.

Online supervision environment

The supervisory environment has changed drastically due to transitioning to online teaching (Le, 2021), with little time to prepare (Lambrechts and Smith, 2020). The changes were experienced as being disruptive (Guintivano et al., 2021) and a source of increased workload and stress (Else, 2021). Teaching and supervising online were reported to increase workload (Herman et al., 2021). At the same time, the supervisor’s own access to scholarly networks and resources provided by their institutions also suffered, further amplifying the challenge. Yet, remote work and using communication technologies also provided an important means for coping with new demands (Adarmouch et al., 2020), transition to online supervision was experienced as smooth (Torka, 2021) and online supervision experienced as equally personalised as face-to-face supervision (Mullen, 2021). For those doctoral candidates who had their examination during the pandemic, and for their supervisors and examiners, the experience may have entailed both positive and negative connotations. For instance, doctoral candidates reported feeling comfortable being at home, while at the same time both doctoral candidates, their supervisors and examiners reported experiencing anxiety and worry regarding technology (Wisker et al., 2022; for reassuring guidelines regarding online examinations, see UK Council for Graduate Education, 2023).

Because of fully remote supervision, ad hoc informal supervision encounters in the hallways and over a cup of coffee disappeared leading to a substantial reduction in the “amount” of supervision and a lack of spontaneous supervisory interactions. Doctoral candidates reported a lack of immediate access to supervisors as one of the consequences of remote work (Lambrechts and Smith, 2020). In turn, supervisors reported increases in e-mail requests from supervisees, sometimes surpassing manageability (Lasater et al., 2021). At the same time, doctoral candidates may have been reluctant to share their problems with a supervisor via email during the pandemic, especially if they had previously been used to communicating face-to-face (Kumar et al., 2020). Yet again, identifying doctoral candidates’ circumstances may have been a new element introduced into supervision during the pandemic. An Australian comparison of the policies of various universities in support of doctoral candidates (Le, 2021) shows that supervisors had to consider doctoral candidates’ situations in terms of evaluation and extensions of study, and they were also given leeway for making these considerations. However, it may have been difficult for supervisors to estimate the compound effects of the pandemic on doctoral candidates, especially as candidates may not have revealed the full scale of their stress and burden to their supervisors (Le, 2021). Investing in understanding doctoral candidates’ situations and circumstances has been identified as a useful practice in supervision during the pandemic (Kumar et al., 2020).

Other useful online supervision practices identified include setting aside time for a regular monthly meeting (Kumar et al., 2020). Online supervision can be more structured than face-to-face supervision because of the awareness of the limited time resource (Torka, 2021). However, useful practices also include setting aside unstructured and informal meeting times between supervisors and doctoral candidates (Wang and DeLaquil, 2020). Yet, access to online supervision has also come to mean access to people’s private spheres of both doctoral candidates and supervisors. Recognising and acknowledging the tensions and the exposedness that the merging of private and professional spheres bring, is a prerequisite to finding “a pedagogical homeliness” in the remote supervision (Wisker et al., 2021, p. 618). Restraints and affordances of space (in this case home) in doctoral study had previously been addressed as a marginal experience (Burford and Hook, 2019). “Pedagogical homeliness” has been reported as affecting academics (Deznabi et al., 2021) and women especially (Burk et al., 2021; Minello et al., 2021). This situation is challenging for supervisors because of the blurred lines between work, family and leisure demanding constant productivity and connectivity. Elliot and Makara (2021) demonstrated how supervising from home, that is, accepting the “homely” features of the contexts in which everyone involved is physically located in, can be turned into an affordance in building relationship and community. Finally, research shows that it has also been possible to support professional community through online supervisor development by opening a space for learning and reflection (Vaughan et al., 2021).

While the literature suggests that fully online supervision due to the pandemic has had an impact on supervision and supervisors concerning practices and environments, few empirical studies have focused specifically on the experiences of doctoral supervisors. The aim of this study is to gain a better understanding of the influences of the rapidly changed landscape of doctoral supervision and identify the supervisory activities that were particularly challenged. In addition, we examined the variation between the supervisors in terms of the impact and challenges experienced. We posed the following research questions:

RQ1.

How do doctoral supervisors perceive the changes in supervision and doctoral candidates’ progress and study well-being, and how do they think that the challenges they experienced influenced supervisory responsibilities?

RQ2.

Are there differences in perceived impact and challenges to supervision based on discipline, positions at the university, gender, supervision of candidates working full-time cf. part-time, supervision of candidates engaging in a research group cf. working mainly on their own cf. working both in a group and on their own and supervision of candidates writing a monograph cf. an article-based dissertation?

Before explaining how we approached these questions, we describe the context in which the research was carried out.

Doctoral education in Finland

Finland has one of the highest proportions of doctoral degree holders per capita in Europe (OECD, 2014). It has a nationwide graduate school system: all doctoral candidates belong to a doctoral school at their university, and to one of the university’s doctoral programmes. Finnish doctoral education is more research-intensive rather than course-centred, although steps towards a more structured system have been taken. Doctoral candidates start their research at the very beginning of studying for their doctorate (Andres et al., 2015). They can choose whether to write their dissertation as a monograph or as a set of three to five accepted or published articles including a summary of approximately 70 pages. Most doctoral candidates now undertake the latter.

A doctoral candidate generally has two supervisors. Typically, the main supervisor is a full professor having a permanent position at the university and the second supervisor may be less experienced. Universities offer training for supervisors, and participation is recommended, but voluntary. Supervision courses are among the more popular courses offered to academic staff. Tenure tracks now exist alongside a traditional system of open competition to full professorship. In both cases, supervision experience is viewed favourably. The doctoral examination proceeds in three stages. Two external examiners pre-examine the dissertation. A public defence follows the pre-examination. After this, the faculty grants the PhD degree. The target time for completing a doctorate is four years studying full-time, but the average time for degree completion is 5–6 years (Pyhältö et al., 2022). There are no tuition fees, but the universities, projects or foundations do not automatically provide funding to support doctoral study.

Method

Next, we describe the group of participants, how we attended to research ethics, and the methods of data collection and data analysis.

Participants

This survey was completed by 561 doctoral supervisors (275 women, 50%; 266 men, 49%; 6 nonbinary, 1%) from a large multi-field research-intensive university in Finland. The response rate was 16%. The distribution of men and women were representative of the doctoral supervisor at the university. The supervisors were from Arts (n = 74), Agriculture and Forestry (n = 48), Biological and Environmental Sciences (n = 49), Educational Sciences (n = 29), Law (n = 8), Medicine (n = 99), Pharmacy (n = 15), Science (n = 83), Social Sciences (n = 62), Theology (n = 10) and Veterinary Medicine (n = 15). Of the supervisors, 44% were professors or research directors, 38% were university lecturers or university researchers, 9% were university instructors or postdoctoral fellows and 9% were tenure track professors. Half (51%) had supervised doctoral candidates for more than 12 years (mean 15 years; SD = 9.7 years). On average, the supervisors supervised four doctoral candidates at the time of the survey, and the number of doctoral degrees supervised to completion was seven despite considerable variation (0–150 with most placing between 0 and65). Professors and research directors had typically supervised a higher number of completed doctoral dissertations and had more experience of supervising than university lecturers, university researchers, postdoctoral fellows and university instructors. Professors, research directors and tenure track professors also reported a higher number of doctoral candidates under their supervision than supervisors in other positions. The supervisors reported that their supervisees typically wrote an article-based dissertation (82%) in English (91%) and were engaged in a research group (77%). Most of the supervisors (68%) reported supervising primarily full-time doctoral candidates.

Data

The data were collected between August and September 2021 online by using a modified version of a supervisory experience survey validated in previous studies (Pyhältö et al., 2015). For the present study, the following measures were used:

The change in supervisory responsibilities scale was measured on a five-point Likert scale (Significantly decreased/somewhat decreased/no change/somewhat increased/significantly increased). The impact scale used a one- to seven-point Likert response scale, where 1 = strongly disagree … 7 = fully agree (changes), and where value 4 is the mid-scale value representing a view in which neither positive nor negative experiences dominate over the other. The challenges were assessed with a seven-point scale ranging from 1 = not at all challenging to 7 = highly challenging. The data also included the supervisor’s faculty affiliation, position at the university (full-professor/research director; tenure-track professor; university lecturer/university researcher; postdoctoral fellows/university instructor) and gender (women/men/other), the typical study status of their supervisees (full-time cf. part-time), research group status of their supervisees (studying in a research team/alone/both alone and in a team) and dissertation format typically supervised (monograph cf. article-based dissertation).

Research ethics

In Finland, national policy does not require an ethics review for studies with fully informed, volunteer adult participants on a topic with no risk to participants (Finnish National Board on Research Integrity, 2019, p. 19). Participation in the study was voluntary. Information about the research, including the purpose of the research, the fact that no identifying data were to be captured, and data storage, and the link to the supervisory experience survey were sent via e-mail to the participants by using the Doctoral Schools’ supervisor mailing lists. No incentives were offered.

Analysis

To analyse doctoral supervisors’ experiences, means and standard deviations were calculated. The independent samples t-tests were used to analyse differences in perceptions between supervisor groups based on dichotomous background variables (i.e. gender, and supervisees’ study status and thesis format). Differences in experiences between disciplines, supervisors with different positions at the university and supervisees’ research group status were analysed using one-way ANOVA. For pairwise comparisons, we used the Games Howell test and Tukey’s HSD test. We used Eta-squared to determine effect size with 0.01 indicating a small, 0.06 a medium and 0.14 a large effect size (Stevens, 1996).

Results

We first report on supervisors’ perceptions of the impact of the changing supervision landscape based on differences in discipline, position, gender, candidate status, supervision setting and dissertation type. After this, we report on supervisors’ perceptions of the challenges caused during the pandemic, and their estimation about how the challenges experienced have influenced supervisory responsibilities.

Supervisors’ perceptions of the impact of the changing supervision landscape

Most supervisors (66%) reported that their supervisory responsibilities had not changed due to the outbreak of COVID-19. However, about a quarter of the supervisors (24%) reported that their supervisory responsibilities had increased, while 10% thought that their supervisory responsibilities had decreased. In general, the supervisors perceived a negative impact on their doctoral candidates’ progress and well-being, while their own supervision had been influenced to a lesser extent (see Table 1).

There were disciplinary differences in supervisors’ perceptions of the influences on their work as supervisors [F(10, 478) = 2.51, p < 0.01, η2 = 0.051]. More specifically, the supervisors in Educational Sciences perceived that the pandemic had influenced their work as a supervisor significantly less often than supervisors in Agriculture and Forestry (p < 0.05), Biological and Environmental Sciences (p < 0.01), Medicine (p < 0.05) and Science (p < 0.01) (see Table 2 for means and standard deviations among disciplines). Depending on their position, the supervisors differed in their perceptions of impact on supervision [F(3, 482) = 2.92, p < 0.05, η2 = 0.018] and candidates’ study progress [F(3, 482) = 4.21, p < 0.01, η2 = 0.026] and study well-being [F(3, 479) = 6.07, p < 0.001, η2 = 0.037]. More specifically, the tenure track professors perceived that the pandemic had negatively influenced their work as a supervisor more often than postdoctoral fellows/university instructors (p < 0.05). They also perceived more often that there had been hindrances in the progress of their doctoral candidates than professors/research directors (p < 0.05), and postdoctoral fellows/university instructors (p < 0.05). Tenure track professors also thought that the pandemic had a negative impact on their candidates’ well-being more often than professors/research directors (p < 0.01), university researchers/university lecturers (p < 0.05) and postdoctoral fellows/university instructors (p < 0.001). However, no gendered differences were detected in supervisors’ perceptions of impact.

The factors related to the supervisees also played a role in the supervisors’ perceptions of the impact of COVID-19 on supervision. The doctoral candidates’ research group status [F(2, 541) = 8.32, p < 0.001, η2 = 0.030] was related to the supervisors’ perceptions of negative influence. More precisely, those supervising doctoral candidates engaged in a research group perceived that there had been a negative impact on their work as supervisors more often than those whose supervisees were working on their own (p < 0.01) or both on their own and in a group (p < 0.01). The supervisors who typically supervised full-time candidates reported more negative impacts on their own work as a supervisor [t(543) = 5.16, p < 0.001] and on the supervisees’ progress [t(545) = 2.55, p < 0.05] and well-being [t(307) = 4.92, p < 0.001] compared to those who supervised part-time candidates. However, no differences based on the candidates’ thesis format were detected.

Supervisors’ perceptions on the challenges caused by the changing supervision landscape

Supervisors believed that some of their supervisory activities had been challenged. They experienced that recognising when doctoral candidates needed help and supporting the candidates, such as managing their well-being, had become more challenging in the new realities they were facing. In turn, supervising remotely, preparing online defences or helping candidates to change their focus were not perceived as particularly challenging (Table 1). Further investigation showed that there were differences between supervisors. The supervisors’ perceptions of how challenging it was to supervise online/remotely varied between the disciplines [F(10, 477) = 4.27, p < 0.001, η2 = 0.082]. The supervisors in Educational Sciences reported challenges in supervising online/remotely to a lesser extent than their colleagues in Agriculture and Forestry (p < 0.01), Arts (p < 0.05), Biological and Environmental Sciences (p < 0.01), Medicine (p < 0.01) and Science (p < 0.001). The supervisors in Social Sciences reported being less challenged by online supervision than their colleagues in Sciences (p < 0.01) and Medicine (p < 0.05) (see Table 3 for the means and standard deviations by the disciplines). Postdoctoral fellows/university instructors perceived supervising online/remotely (p < 0.05) and supporting the doctoral candidates to be significantly less challenging than supervisors in other positions at the university (p < 0.05). Tenure track professors perceived more challenges in supporting the candidates than professors/research directors (p < 0.05). Postdoctoral fellows and university instructors perceived fewer challenges in recognising when supervisees needed help compared to tenure track professors (p < 0.01) and university researchers and university lecturers (p < 0.05). No gender differences were detected in supervisors’ perceptions of challenges.

Supervisees’ study status was related to supervisors’ perceptions of the challenges. The supervisors who supervised full-time candidates reported higher levels of challenge in online/remote supervision [t(544) = 4.12, p < 0.001], helping doctoral candidates to change the projects/focus [t(536) = 2.74, p < 0.01], in recognising when candidates needed help [t(319) = 4.42, p < 001] and supporting them [t(311) = 4.11, p < 0.001] than those supervising part-time candidates. Moreover, candidates’ research group status was related to supervisors’ perceptions of challenges in supervising remotely [F(2, 542) = 14.02, p < 0.001, η2 = 0.049] and recognising when a supervisee needed help [F(2, 541) = 3.65, p < 0.05, η2 = 0.013]. The supervisors who supervised candidates involved in research groups perceived remote supervision as being more challenging than those who supervised candidates working on their own (p < 0.001) and candidates working both on their own and in a group (p < 0.001). They also reported difficulties in recognising when a supervisee needed help due to remote supervision more often than those supervising candidates working on their own (p < 0.05). Depending on their position, supervisors differed from each other in how challenging they perceived remote supervision [F(3, 480) = 4.08, p < 0.01, η2 = 0.025], supporting the candidates [F(3, 479) = 6.15, p < 0.001, η2 = 0.037] and in recognising when a supervisee needed help [F(3, 480) = 4.62, p < 0.01, η2 = 0.028]. More specifically, the postdoctoral fellows and university instructors perceived less challenges in online/remote supervision than professors, research directors or tenure track professors. Furthermore, postdoctoral fellows and university instructors experienced supporting their candidates as being less challenging than supervisors in other positions. Professors and research directors, as well as postdoctoral fellows and university instructors, felt more confident being able to recognise when a supervisee needed help than tenure track professors. No differences based on supervisees’ thesis format were detected in the supervisors’ perceptions of challenges.

Discussion

In general, the supervisors perceived that there had been a negative impact on their doctoral candidates’ progress and well-being, while they thought that their supervision had been influenced to a lesser extent. This result supports the interpretation according to which early-career researchers have been more affected by the pandemic than their more seasoned peers (Myers et al., 2020). It does not corroborate the results of those studies identifying additional tasks and responsibilities for supervisors during the pandemic (Le, 2021), which is likely to result from contextual differences: in Finland, candidates’ study time extensions do not require a formal procedure involving documentation and evaluation. There are recommendations as per the time it should take to complete a doctorate, and progress is monitored through an online system, but in the end, the schedule is discussed and agreed between the candidate and the supervisor, and study time is not limited. Accordingly, the supervisor’s workload was not inflated by additional assessments. Instead, lockdowns have pruned down informal supervisory activities, which might have at least temporarily reduced supervisors’ workloads. Although the pandemic hit higher education institutions globally, the above result highlights the importance of considering the contextual dimensions, which can vary highly depending on the local institutional procedures and practices influencing individual experiences of workload differently.

However, the results also reveal field-related differences, which we anticipate may bear greater similarity among fields internationally, while exhibiting greater inter-country variance. The results show that the supervisors’ experiences of the impact on supervision and related challenges is not the same for supervisors across fields. The supervisors within the field of education perceived that the pandemic had influenced their work as a supervisor less and they experienced fewer challenges with online/remote supervision compared to their colleagues in the sciences. Sciences that use labs and specific equipment and infrastructure have been challenged during the pandemic, also complicating the situation of doctoral candidates (Maranda and Yakubovich, 2020). In a similar vein, medicine involves extensive practical bench-side training and learning and moving to remote supervision is a considerable change. Demonstrations and hands-on assistance may be challenging online (Gill et al., 2020). The use and development of online learning tools and environments is an integral part of the scope of education, which may explain why supervisors in the field of education experienced remote supervision to be less challenging than their colleagues in other fields. In general, faculty in the field of education are likely to have competencies related to supporting learning processes, which equips them for supervision.

We also identified position-related differences in the supervisors’ experiences. Compared to postdoctoral fellows, tenure track professors perceived that the pandemic had negatively influenced their work as a supervisor and hindered the progress of their doctoral candidates. They had experienced challenges in supporting the candidates and in recognising when their supervisees needed help. Tenure track professors also perceived a negative impact on their candidates’ well-being more often compared to postdoctoral fellows. A systematic review of eleven empirical studies on the impact of the pandemic on early-career researchers showed that their research activities, researcher development, career prospects and well-being were affected (Lokhtina et al., 2022). Similarly, Herman et al. (2021) concluded that early-career researchers had been hit hard by the pandemic. However, while both post docs and tenure track professors in the early stages of the track may have grappled with these challenges, individuals on the tenure track are under substantial pressure to perform (Miller et al., 2011), which may have been reflected in their supervision activities. The research work of doctoral candidates and the supervision kudos that comes with doctoral dissertations supervised to completion may be particularly important for the tenure track supervisors. Postdoctoral fellows may not have felt the same pressure as they probably had a senior co-supervisor. The experience of more seasoned supervisor colleagues may have proved helpful in recognising supervisee needs and responding to challenges. Postdoctoral fellows/university instructors perceived supervising online/remotely and supporting the doctoral candidates as being significantly less challenging than supervisors in other positions at the university. Postdoctoral fellows may represent a younger generation that is more comfortable with the technological tools that are available. University instructors are likely to have used online platforms at least to some extent, as the university had been using Moodle as a core tool from before the pandemic, and many have substantial teaching experience helping them in making the online shift. On the other hand, they are likely to have fewer doctoral candidates to supervise compared to more senior supervisors, implying lower supervisory workload. They are also more likely to act as co-supervisors in a junior role.

The results show that the impacts have been more severe and the supervision more challenging for those supervising full-time candidates compared to supervising part-time candidates. Supervisors perceived that there were more negative impacts on their full-time doctoral candidates’ well-being compared to their part-time candidates. Challenges included helping doctoral candidates to change the project or the research focus, recognising when candidates needed help and supporting them. Indeed, previous research has shown that full time doctoral candidates have suffered more from restricted access to facilities and encountered more challenges with well-being compared to their part-time peers (Lambrechts and Smith, 2020; Pyhältö et al., 2022). Thus, it is not surprising that the challenges that these candidates experience are carried over into the supervision and reflected in the work of the supervisor. If studying and research stall, it is only logical that the effects accumulate for those studying full time. Part time candidates may instead have been able to concentrate more on their doctoral research if their other activities had come to a halt.

The supervisors who supervised candidates involved in research groups perceived that there had been a negative impact on their work as supervisors, especially in terms of remote supervision, and difficulties in recognising when doctoral candidates needed help due to remote supervision more often than those supervising candidates working on their own did. This result may reflect difficulties in managing the research processes and related challenges of several individuals or the group compared to addressing challenges on a one-to-one basis. At the same time, there are benefits and strengths in collective supervision and the teamwork format if the various competencies in the group and opportunities for providing and receiving feedback are used (Pyhältö and Keskinen 2012; Pyhältö et al., 2015). It appears that the online modality challenges supervision by impeding the sustaining of relationships (Wang and DeLaquil, 2020; Jung et al., 2021), unless concerted efforts are made to the contrary (Johnson and Lock, 2020; for two narratives demonstrating such efforts see Palmer and Gillaspy, 2021).

Supervisors grappled with challenges related to supervisory responsibilities, more specifically, to recognising when doctoral candidates needed help and to supporting the candidates in terms of well-being. This is in line with survey results indicating increased work in supporting candidates’ well-being and refocussing of their research (UK Council for Graduate Education, 2021).

No differences based on the doctoral candidates’ dissertation format were detected in the supervisors’ perceptions of challenges or in the transition to remote supervision. This suggests that challenges in supervision are located elsewhere than in supervising the writing processes per se. Likewise, gendered differences were not detected in supervisors’ perceptions. The results are in line with other studies showing no evidence of significant gender effects because of the pandemic (Camerlink et al., 2021). However, prior research also points to women academics having experienced more pandemic-related challenges (Aubry et al., 2021; Guintivano et al., 2021). The results of our study do not rule out gendered experiences in other areas of academic life, but it suggests that supervision in particular may be an activity in which the experiences and challenges depend on aspects other than gender.

We acknowledge that there are limitations in the study. The response rate was relatively low, but the sample size was sufficiently large to enable statistical analyses to be performed. It is possible that supervisors who have experienced more challenges were more motivated to reply than their peers who had neutral experiences. The design of the survey was more focused on negative impact and challenges than positive outcomes, which may have triggered more responses from supervisors with negative experiences compared to those with neutral or positive experiences. A survey was used to collect data for this study. Qualitative data collection methods may have provided more context and insight that would have contributed with explanations to shed light on the experiences of the supervisors. While the research is a single-institution study, the university is a large, internationally oriented institution, providing diversity in terms of fields and research cultures. While institutional procedures and practices may vary internationally, there may also be dimensions that bear more universal features, such as field-related practices related to research and supervision.

Implications

The slowed-down progression and diminishing well-being of doctoral candidates reported by supervisors is likely to influence supervision in a delayed way. Supervisors may be anticipating some issues with stalled studying and stress, but the question is the extent to which they are prepared to handle these as they emerge in supervision encounters. Indeed, it had been anticipated that doctoral candidates would have more issues with well-being because of the pandemic (UK Council for Graduate Education, 2021). The fact that the experiences varied across field, position of the supervisor, the type of candidates (full or part time) and the way in which supervision was organised suggests that support provided for supervisors to overcome any challenges needs to be tailored and engineered, at least to some extent. Based on a literature review of supervisors’ professional learning (Huet and Casanova, 2022), dialogic and self-reflective approaches to learning appear to support supervisors the best. These angles on learning may provide a fruitful starting point for critical scrutiny of institutional restraints and affordances, as well as increased understanding of changes in ones’ own professional identity. In addition, there needs to be space and time facilitating the dialogic and self-reflective approaches, and the opportunities must be contextually sensitive and meaningful.

Challenges in supervision may be part of a larger conglomeration including pressure to perform to fulfil tenure expectations, and a lack of experience of managing projects and people. The supervisors in tenure-track positions and supervisors of full-time candidates, and candidates receiving supervision in research groups may benefit from targeted support. The recovery of supervisors in tenure-track positions and supervisors of full-time candidates to full supervision capacity poses, indeed, a theme for further research. Another theme warranting further research involves the accuracy of supervisors’ estimation of their supervisees’ levels of stress, exhaustion and adverse impact of the pandemic. To research this, a design in which data from supervisors and doctoral candidates are matched is needed. This matter is important as a realistic sense of doctoral candidates’ experiences will help supervisors to respond with adequate support.

Doctoral supervisors’ experiences of the impact of COVID-19 and challenges induced by it

Item N Mean SD
Perceived impact of the COVID-19a
The COVID-19 pandemic has negative impact on my work as supervisor 549 3.78 1.84
The COVID-19 pandemic has hindered the progress of my doctoral candidates 551 4.60 1.82
The COVID-19 pandemic has impacted negatively on the well-being of my doctoral candidates 547 4.87 1.57
Challenges induced by the COVID-19b
Supervising online/remotely 550 3.52 1.78
Preparing and conducting online defences 480 3.56 1.55
Helping doctoral candidates change projects/focus 542 3.56 1.68
Recognising when someone I supervise needs help 549 4.67 1.63
Supporting my doctoral candidates, including managing their well-being 547 4.64 1.59
Helping doctoral candidate(s) with data collection/analysis 537 3.55 1.67
Notes:

aMeasured with seven-point Likert scale (1 = strongly disagree … 7 = fully agree); bMeasured with seven-point scale (1 = not at all challenging … 7 = highly challenging)

Source: Created by authors

Doctoral supervisors’ experiences of the impact of COVID-19 by discipline, position and gender and attributes related to supervisees

Variables The COVID-19 pandemic has negative impact on my work as supervisor
M (SD)
The COVID-19 pandemic has hindered the progress of my doctoral candidates
M (SD)
The COVID-19 pandemic has impacted negatively the well-being of my doctoral candidates
M (SD)
Faculty
Agriculture and Forestry 3.96 (1.74) 4.38 (1.76) 4.81 (1.80)
Arts 3.65 (1.85) 4.61 (1.96) 5.09 (1.55)
Educational Sciences 2.59 (1.62) 3.93 (1.62) 4.00 (1.61)
Biological and Environmental Sciences 4.19 (1.51) 4.94 (1.60) 5.13 (1.36)
Law 3.00 (1.77) 3.88 (1.89) 4.00 (1.93)
Pharmacy 3.33 (1.54) 4.40 (2.10) 4.80 (1.21)
Medicine 3.85 (1.95) 4.80 (1.95) 4.72 (1.65)
Science 4.12 (1.82) 4.43 (1.79) 5.04 (1.50)
Social Sciences 3.44 (1.71) 4.55 (1.61) 4.89 (1.37)
Theology 3.70 (2.21) 4.00 (2.06) 5.00 (1.33)
Veterinary Medicine 3.27 (1.75) 4.87 (2.00) 4.67 (1.50)
Position at the university
Professor/Research director 3.81 (1.89) 4.52 (1.82) 4.83 (1.56)
Tenure track professor 4.41 (1.72) 5.26 (1.51) 5.73 (1.23)
University researcher/University lecturer 3.81 (1.70) 4.78 (1.68) 4.97 (1.45)
Postdoctoral researcher/University instructor 3.30 (1.83) 4.05 (2.19) 4.45 (1.72)
Gender
Women 3.71 (1.92) 4.59 (1.95) 4.93 (1.68)
Men 3.80 (1.75) 4.56 (1.70) 4.77 (1.47)
Supervisees’ study status
Full-time 4.06 (1.80) 4.74 (1.78) 5.10 (1.47)
Part-time 3.21 (1.79) 4.32 (1.88) 4.38 (1.68)
Supervisees’ research group status
On their own 3.48 (1.87) 4.41 (1.95) 4.92 (1.64)
In a research group 4.25 (1.76) 4.89 (1.71) 5.01 (1.46)
Both on their own and in a group 3.60 (1.83) 4.49 (1.82) 4.74 (1.62)
Supervisees’ thesis format
Monograph 3.41 (1.84) 4.35 (1.86) 4.74 (1.62)
Summary of articles 3.86 (1.84) 4.64 (1.81) 4.89 (1.56)

Source: Created by authors

Supervisors’ perceptions of the challenges of supervision induced by COVID-19

Variables Supervising online/remotely
M (SD)
Preparing and conducting online defences
M (SD)
Helping doctoral candidates to change projects/focus
M (SD)
Recognising when someone I supervise needs help
M (SD)
Supporting my doctoral candidate(s) including managing their well-being
M (SD)
Helping doctoral candidates with data collection/analysis
M (SD)
Faculty
Agriculture and Forestry 3.75 (1.78) 3.40 (1.58) 3.79 (1.69) 4.63 (1.63) 4.77 (1.74) 3.69 (1.50)
Arts 3.43 (1.70) 3.80 (1.62) 3.58 (1.73) 4.56 (1.73) 4.68 (1.68) 3.32 (1.72)
Educational Sciences 2.21 (1.54) 3.08 (1.50) 2.93 (1.53) 3.90 (1.45) 4.11 (1.34) 3.36 (1.50)
Biological and Environmental Sciences 3.83 (1.69) 3.29 (1.38) 3.90 (1.64) 4.69 (1.63) 4.67 (1.53) 3.52 (1.52)
Law 3.13 (1.96) 3.38 (1.77) 2.88 (1.81) 4.38 (1.85) 4.25 (1.49) 4.00 (1.85)
Pharmacy 3.80 (1.57) 3.08 (1.44) 2.79 (1.53) 4.60 (1.40) 4.80 (1.61) 3.00 (1.71)
Medicine 3.81 (1.74) 3.86 (1.61) 3.60 (1.57) 4.67 (1.67) 4.64 (1.64) 3.72 (1.66)
Science 4.06 (1.73) 3.50 (1.48) 3.80 (1.76) 5.04 (1.44) 4.84 (1.40) 3.24 (1.79)
Social Sciences 2.82 (1.64) 4.08 (1.47) 3.37 (1.70) 4.52 (1.80) 4.61 (1.71) 3.61 (1.69)
Theology 3.40 (1.96) 3.20 (1.87) 3.10 (1.29) 4.80 (1.23) 4.60 (1.17) 3.50 (1.27)
Veterinary Medicine 3.13 (1.77) 3.27 (1.56) 3.53 (1.46) 5.00 (1.46) 4.33 (1.50) 3.50 (1.65)
Position at the university
Professor/Research director 3.67 (1.78) 3.62 (1.64) 3.58 (1.72) 4.64 (1.61) 4.72 (1.53) 3.59 (1.65)
Tenure track professor 3.80 (1.77) 3.63 (1.48) 3.78 (1.54) 5.30 (1.23) 5.25 (1.10) 3.41 (1.66)
University researcher/University lecturer 3.58 (1.76) 3.66 (1.50) 3.76 (1.66) 4.83 (1.64) 4.80 (1.51) 3.61 (1.72)
Postdoctoral researcher/University instructor 2.70 (1.47) 3.00 (1.31) 3.02 (1.61) 4.09 (1.71) 3.91 (1.83) 3.45 (1.73)
Gender
Women 3.42 (1.79) 3.54 (1.54) 3.56 (1.70) 4.72 (1.68) 4.67 (1.66) 3.61 (1.68)
Men 3.61 (1.74) 3.58 (1.55) 3.56 (1.66) 4.61 (1.59) 4.60 (1.53) 3.47 (1.63)
Supervisees’ study status
Full-time 3.73 (1.77) 3.52 (1.51) 3.69 (1.70) 4.89 (1.56) 4.84 (1.51) 3.55 (1.68)
Part-time 3.07 (1.71) 3.63 (1.63) 3.27 (1.61) 4.22 (1.70) 4.22 (1.68) 3.52 (1.66)
Supervisees’ research group status
On their own 3.20 (1.80) 3.66 (1.62) 3.45 (1.77) 4.34 (1.76) 4.51 (1.70) 3.44 (1.75)
In a research group 4.10 (1.63) 3.56 (1.57) 3.67 (1.65) 4.86 (1.58) 4.80 (1.51) 3.66 (1.59)
Both on their own and in a group 3.27 (1.78) 3.50 (1.51) 3.54 (1.67) 4.70 (1.59) 4.58 (1.59) 3.52 (1.68)
Supervisees’ thesis format
Monograph 3.49 (1.80) 3.70 (1.78) 3.56 (1.68) 4.29 (1.69) 4.39 (1.59) 3.43 (1.62)
Summary of articles 3.53 (1.78) 3.53 (1.50) 3.57 (1.69) 4.75 (1.61) 4.68 (1.58) 3.56 (1.67)

Source: Created by authors

References

Adarmouch, L., Sebbani, M. and Amine, M. (2020), “Research activity among academic medical staff during the covid-19 pandemic in marrakesh”, Education Research International, Vol. 2020, doi: 10.1155/2020/6648406.

Aubry, L.M., Laverty, T.M. and Ma, Z. (2021), “Impacts of covid‐19 on ecology and evolutionary biology faculty in the United States”, Ecological Applications, Vol. 31 No. 2, p. 2265, doi: 10.1002/eap.2265.

Andres, L., Bengtsen, S.E., Gallego Castaño, L., Croussouard, B., Keefer, J. and Pyhältö, K. (2015), “Drivers and interpretations of doctoral education today: national comparisons”, Frontline Learning Research, Vol. 3 No. 3, pp. 1-18, doi: 10.14786/flr.v3i3.177.

Burford, J. and Hook, G. (2019), “Curating care-full spaces: doctoral students negotiating study from home”, Higher Education Research and Development, Vol. 38 No. 7, pp. 1343-1355.

Burk, B.N., Pechenik Mausolf, A. and Oakleaf, L. (2021), “Pandemic motherhood and the academy: a critical examination of the leisure-work dichotomy”, Leisure Sciences, Vol. 43 Nos 1/2, pp. 225-231, doi: 10.1080/01490400.2020.1774006.

Camerlink, I., Nielsen, B.L., Windschnurer, I. and Vigors, B. (2021), “Impacts of the covid-19 pandemic on animal behaviour and welfare researchers”, Applied Animal Behaviour Science, Vol. 236, p. 105255, doi: 10.1016/j.applanim.2021.105255.

Deznabi, I., Motahar, T., Sarvghad, A., Fiterau, M. and Mahyar, N. (2021), “Impact of the covid-19 pandemic on the academic community. Results from a survey conducted at University of Massachusetts Amherst”, Digital Government: Research and Practice, Vol. 2 No. 2, pp. 1-12, doi: 10.1145/3436731.

Donohue, W.J., Lee, A.S.-J., Simpson, S.Y. and Vacek, K. (2021), “Impacts of the covid-19 pandemic on doctoral students’ thesis/dissertation progress”, International Journal of Doctoral Studies, Vol. 16, pp. 533-552, doi: 10.28945/4818.

Elliot, D.L. and Makara, K.A. (2021), “An online community of international scholars: enabling spaces for reciprocal academic and psychological support”, Innovations in Education and Teaching International, Vol. 58 No. 6, pp. 693-703, doi: 10.1080/14703297.2021.1991424.

Else, H. (2021), “Sleeplessness and anxiety: PhD supervisors on toll of covid pandemic”, Nature, doi: 10.1038/d41586-021-03374-w.

Finnish National Board on Research Integrity (2019), “The ethical principles of research with human participants and ethical review in the human sciences in Finland”, available at: https://tenk.fi/en/ethical-review (accessed 29 December 2021).

Gill, S.D., Stella, J., Blazeska, M. and Bartley, B. (2020), “Distant supervision of trainee emergency physicians undertaking a remote placement: a preliminary evaluation”, Emergency Medicine Australasia, Vol. 32 No. 3, pp. 446-456, doi: 10.1111/1742-6723.13440.

Guintivano, J., Dick, D. and Bulik, C.M. (2021), “Psychiatric genomics research during the covid‐19 pandemic: a survey of psychiatric genomics consortium researchers”, American Journal of Medical Genetics Part B: Neuropsychiatric Genetics, Vol. 186 No. 1, pp. 40-49, doi: 10.1002/ajmg.b.32838.

Herman, E., Nicholas, D., Watkinson, A., Rodríguez-Bravo, B., Abrizah, A., Boukacem-Zeghmouri, C., Jamali, H.R., Sims, D., Allard, S., Tenopir, C., Xu, J., Świgoń, M., Serbina, G. and Cannon, L.P. (2021), “The impact of the pandemic on early career researchers: what we already know from the internationally published literature”, El Profesional De La Información, Vol. 30 No. 2, doi: 10.3145/epi.2021.mar.08.

Huet, I. and Casanova, D. (2020), “Exploring the professional development of online and distance doctoral supervisors”, Innovations in Education and Teaching International, Vol. 58 No. 4, pp. 430-440, doi: 10.1080/14703297.2020.1742764.

Huet, I. and Casanova, D. (2022), “Exploring the professional development of doctoral supervisors through workplace learning: a literature review”, Higher Education Research and Development, Vol. 41 No. 3, pp. 774-788, doi: 10.1080/07294360.2021.1877629.

Johnson, C. and Lock, J. (2020), “Virtual writing groups: Collegial support in developing academic writing capacity”, in Brown, M., Mhichil, M.N.G., Beirne, E. and Costello, E. (Eds), Proceedings of the 2019 ICDE World Conference on Online Learning, Dublin City University.

Jung, J., Horta, J. and Postiglione, G.A. (2021), “Living in uncertainty: the covid-19 pandemic and higher education in Hong Kong”, Studies in Higher Education, Vol. 46 No. 1, pp. 107-120, doi: 10.1080/03075079.2020.1859685.

Kumar, S., Kumar, V. and Taylor, S. (2020), “‘A guide to online supervision’, UK council for graduate education”, available at: https://supervision.ukcge.ac.uk/resources/ukcge-guide-to-online-supervision/ (accessed 3 January 2022).

Lambrechts, A. and Smith, K. (2020), “Impact of the covid- 19 pandemic crisis on doctoral researchers in the UK”, White Rose Research Online, available at: http://eprints.whiterose.ac.uk/168633/ (accessed 1 January 2022).

Lasater, K., Smith, C., Pijanowski, P. and Brady, K.P. (2021), “Redefining mentorship in an era of crisis: responding to covid-19 through compassionate relationships”, International Journal of Mentoring and Coaching in Education, Vol. 10 No. 2, pp. 158-172, doi: 10.1108/IJMCE-11-2020-0078.

Le, A.T. (2021), “Support for doctoral candidates in Australia during the pandemic: the case of the University of Melbourne”, Studies in Higher Education, Vol. 46 No. 1, pp. 133-145, doi: 10.1080/03075079.2020.1859677.

Lokhtina, I., Castelló, M., Lambrechts, A., Löfström, E., McGinn, M.K., Skakni, I. and van der Weijden, I. (2022), “The impact of the COVID-19 pandemic on early career researcher activity, development, wellbeing and careers: the state of the art”, Studies in Graduate and Postdoctoral Education, Vol. 13 No. 3, pp. 245-265.

Maranda, V. and Yakubovich, E. (2020), “The biomedical lab after covid-19: cascading effects of the lockdown on lab-based research programs and graduate students in Canada”, FACETS, Vol. 5 No. 1, pp. 831-835, doi: 10.1139/facets-2020-0036.

Miller, A.N., Taylor, S.G. and Bedeian, A.G. (2011), “Publish or perish: academic life as management faculty live it”, Career Development International, Vol. 16 No. 5, pp. 422-445, doi: 10.1108/13620431111167751.

Minello, A., Martucci, S. and Manzo, L.K.C. (2021), “The pandemic and the academic mothers: present hardships and future perspectives”, European Societies, Vol. 23 No. sup1, pp. S82-S94, doi: 10.1080/14616696.2020.1809690.

Mullen, C.A. (2021), “Online doctoral mentoring in a pandemic: help or hindrance to academic progress on dissertations?”, International Journal of Mentoring and Coaching in Education, Vol. 10 No. 2, pp. 131-157, doi: 10.1108/IJMCE-06-2020-0029.

Myers, K.R., Tham, W.Y., Yin, Y., Cohodes, N., Thursby, J.G., Thursby, M.C., Schiffer, P., Walsh, J.T., Lakhani, K.R. and Wang, D. (2020), “Unequal effects of the covid-19 pandemic on scientists”, Nature Human Behaviour, Vol. 4 No. 9, pp. 880-883, doi: 10.1038/s41562-020-0921-y.

OECD (2014), “Education at a glance 2014”, OECD Indicators: OECD Publishing, September 9, doi: 10.1787/eag-2014-en.

Palmer, C. and Gillaspy, E. (2021), “Digital shifting in doctoral supervision: different routes to the same destination”, Innovations in Education and Teaching International, Vol. 58 No. 6, pp. 672-681, doi: 10.1080/14703297.2021.1991428.

Pyhältö, K. and Keskinen, J. (2012), “Doctoral students’ sense of relational agency in their scholarly communities”, International Journal of Higher Education, Vol. 1 No. 2, pp. 136-149, 10.5430/ijhe.v1n2p136.

Pyhältö, K., Tikkanen, T. and Anttila, H. (2022), “The influence of the COVID-19 pandemic on PhD candidates’ study progress and study wellbeing”, Higher Education Research & Development, 10.1080/07294360.2022.2063816.

Pyhältö, K., Vekkaila, J. and Keskinen, J. (2015), “Fit matters in the supervisory relationship: doctoral students’ and supervisors’ perceptions about the supervisory activities”, Innovations in Education and Teaching International, Vol. 52 No. 1, pp. 4-16, doi: 10.1080/14703297.2014.981836.

Stevens, J. (1996), Applied Multivariate Statistics for the Social Sciences, Lawrence Erlbaum.

Torka, M. (2021), “The transition from in-person to online supervision: Does the interaction between doctoral advisors and candidates change?”, Innovations in Education and Teaching International, Vol. 58 No. 6, pp. 659-671, doi: 10.1080/14703297.2021.1993959.

UK Council for Graduate Education (2021), “UK research supervision survey 2021 report”, available at: www.ukcge.ac.uk/uk-research-supervision-survey.aspx (accessed 15 May 2022).

UK Council for Graduate Education (2023), “Conducting vivas online. A guide for institutions and candidates”, available at: https://ukcge.ac.uk/assets/resources/Guide-to-Conducting-Vivas-Online-UK-Council-for-Graduate-Education.pdf (accessed 8 May 2023).

Vaughan, S., Blackburn, C. and Curzon, R. (2021), “Continuing conversations: moving support for doctoral supervisors’ professional development online”, Innovations in Education and Teaching International, Vol. 58 No. 6, pp. 716-726, doi: 10.1080/14703297.2021.1991423.

Wang, L. and DeLaquil, T. (2020), “The isolation of doctoral education in the times of covid-19: recommendations for building relationships within person–environment theory”, Higher Education Research and Development, Vol. 39 No. 7, pp. 1346-1350, doi: 10.1080/07294360.2020.1823326.

Wisker, G., Highman, L., Spronken-Smith, R. and Waghorne, J. (2022), “Across time and space: examiner and candidate experiences of online doctoral vivas”, Innovations in Education and Teaching International, Vol. 59 No. 2, pp. 131-141, doi: 10.1080/14703297.2021.2022528.

Wisker, G., McGinn, M.K., Bengtsen, S., Lokhtina, I., He, F., Cornér, S., Leshem, S., Inouye, K. and Löfström, E. (2021), “Remote doctoral supervision experiences: challenges and affordances”, Innovations in Education and Teaching International, Vol. 58 No. 6, pp. 612-623, doi: 10.1080/14703297.2021.1991427.

Acknowledgements

The study was funded by the University of Helsinki. The authors thank the two anonymous reviewers for their helpful comments on earlier versions of the paper.

Declaration of conflict of interest: The authors declare no conflict of interest.

Corresponding author

Erika Löfström can be contacted at: erika.lofstrom@helsinki.fi

Related articles