Abstract
In England, approximately 1 in 6 people have a common mental health condition, with certain groups experiencing worsening mental health since the start of the COVID-19 pandemic. Therefore, improving mental health remains a key priority for policy makers and practitioners. Community-based interventions are increasingly used to improve health and reduce inequalities. Evaluation of such interventions is important to ensure they are effective and to maintain financial support for continued delivery. Hesitation to complete administratively demanding evaluative measures by service users, which may not be suited to evaluating low intensity activities, point to the need to identify acceptable, unobtrusive methods of data collection. This review focuses on identifying unobtrusive methods that have previously been used to examine service user’s perceptions of community-based interventions and their effectiveness, and the acceptability of the methods. A review of peer reviewed, and grey literature was undertaken in July 2022. Literature was identified via six databases, Google searches, and by contacting experts. Literature was included if it described unobtrusive methods to gather service users’ perceptions of an intervention and/or reported the acceptability of such methods. Literature was excluded if it described traditional methods to gather service users’ perceptions of an intervention. Our search identified 930 citations from searching databases (n = 886), Google (n = 40), and from contacting 15 experts directly, and over 300 experts indirectly via three e-mail lists (n = 4). No literature met our inclusion criteria. We report an empty review. There is no peer reviewed or grey literature that describes unobtrusive methods of data collection for mental health and wellbeing focused community-based interventions, or their perceived acceptability. The findings from this review indicate the need to develop unobtrusive methods of data collection in the field of public mental health, suitable for low intensity activities, and examine the acceptability and feasibility of such methods.
1. Introduction
In the United Kingdom (UK), the latest survey data from 2014 estimated that 1 in 6 adults over the age of 16 had a common mental health condition [1]. Poor mental health is associated with poorer life expectancy and quality of life [2], higher risk of poor physical health [3–5], and higher risk of poorer socio-economic outcomes such as unemployment and homelessness [2, 3]. The distribution of mental health conditions is not equal. Individuals from disadvantaged backgrounds (e.g., socio-economic disadvantage, physical or cognitive disability, and immigrants) are at far greater risk of experiencing poor mental health [3, 5, 6], as are individuals from minority ethnic, sexual orientation, and gender groups [3, 7, 8]. For example, a systematic review of the prevalence of mental disorders in young refugees and asylum seekers in Europe found them to be at increased risk of being affected by depression, anxiety, and emotional and behavioural problems compared to native children and adolescents [9].
The COVID-19 pandemic has contributed to a rise in mental health problems among some groups, including young adults, women, healthcare workers, those with pre-existing mental health conditions, and those living in socio-economic adversity [10–12]. For example, COVID-19 has been linked to an increase in symptoms of depression among undergraduate students. Santander-Hernández et al. [13] reported that depressive symptoms were present in 78% of their participants, a much higher estimate than 27% which was found in a meta-analysis before the pandemic [14]. Moreover, analysis of data from the English Longitudinal Study of Ageing (ELSA), which includes adults aged 52 and over, found that the prevalence of clinically significant depressive symptoms increased from 12.5% prepandemic to 28.5% in November 2020 [15]. COVID-19 has changed the way that people work and engage with each other, which may have exacerbated mental health problems due to loneliness, lack of physical activity, and an ongoing state of uncertainty [16–18]. COVID-19 also changed the provision and accessibility of community-based services designed to improve mental health [19, 20]. As a result, there has been a rise in the number of smaller community grassroots organisations starting up or turning their focus to supporting individuals in coping with COVID-19 and its impact [21].
The need to improve mental health and the negative impact that COVID-19 has had, has become a key priority for policy makers and practitioners, even more so now, given the current economic crisis. The UK’s Department of Health and Social Care are consulting on their new mental health strategy which will be published in December 2022. The National Health Service (NHS) is additionally shifting towards community-based preventative mental health work, as outlined in the community Mental Health Framework [22]. This shift is hoped to improve accessibility of services, particularly for hard-to-reach groups, and facilitate early identification and prevention of mental illness, management of conditions, and participation in the community [22–24].
Community-based mental healthcare interventions, although not consistently defined include a broad range of nonclinical programmes that operate at individual, subgroup, or wider community level; and draw on resources within communities and beyond healthcare as part of the intervention [25, 26]. It has been suggested that community-based interventions can help address health and social inequalities and improve resilience, mental health, and psychosocial circumstances of individuals in the wider community, by promoting social wellbeing and addressing structural determinants of mental health [1, 27, 28]. A 2015 Cochrane review described three core principles that underlie community-based interventions [29]. The first is an awareness of the multiple forces that exist at all social-ecological levels (i.e., individual, interpersonal, organizational/institutional, community, and policy) that facilitate or obstruct mental health [30]. The second is investment in community participation to provide resources and inform interventions, recognising expertise outside of the healthcare system. The third is prioritization of the community’s mental health.
Community-based interventions to improve mental and social wellbeing are commonplace across the UK [26]. Such interventions tend to take the form of activities delivered by frontline delivery providers, ambassadors, or volunteers. Activities include (but are not limited to), increasing awareness of available services via communications (e.g., dispensing leaflets, posters), building networks and groups (e.g., befriending groups), and providing tailored activities to groups of individuals (e.g., coffee mornings). Community-based interventions are designed to be accessible to a wide range of individuals with varying levels of cognitive, emotional, and physical capacity. However, some community-based interventions are tailored and implemented to meet the needs of specific populations (e.g., refugees; [31]).
Monitoring the services users’ perceptions of the intervention, including its acceptability and effectiveness, is important to evaluate community-based interventions, adapt them accordingly and maintain financial support for continued implementation [32]. However, hesitation to participate in research [33, 34] and complete administratively demanding evaluative measures (e.g., surveys and interviews), in addition to the need to make effectiveness monitoring accessible to a wide range of individuals with varying levels of cognitive, emotional, and physical capacity, points to the need to identify acceptable unobtrusive and/or innovative methods of data collection. Moreover, community-based interventions are often low intensity (e.g., involve participation in a one-off activity), and therefore, more traditional methods involving quantitative or qualitative measures designed to capture large effects may not be appropriate. For example, surveys, focus groups, and interviews require researchers to obtain a substantial contribution in terms of personal time and effort from participants. In addition, traditional methods of data collection require experienced researchers to collect and analyse extensive amount of data, subsequently carrying out traditional methods of data collection tends to be restricted to experienced research teams with capacity to complete the research rather making data collection and analysis accessible to community-based service providers. Observations, although less demanding of participants’ time and effort, may still be considered obtrusive as they require the researcher to be physically present, and may result in the disturbance of the observed system because of the presence of the researcher (i.e., observer effect). Furthermore, while observations offer insight into the processes of interventions, they do not capture perceptions, attitudes, or impact data that are key to evaluating public health interventions.
This review focuses on identifying unobtrusive methods that have previously been used to examine service user’s perceptions of community-based mental health interventions and their effectiveness, and the perceived acceptability of the methods. By unobtrusive we mean, light touch methods for data collection with minimal burden on participants, which are likely to be innovative, as opposed to traditional methods such as surveys, questionnaires, focus groups, and observations. An additional aim was to identify the perceived acceptability of the unobtrusive methods as it is assumed that unobtrusive methods are likely to be more acceptable to service users, more appropriate to small scale activities and more accessible to a wider demographic of service users. Consequently, the review aimed to answer the following questions:(1)What unobtrusive methods have been used previously to examine service users’ perceptions of community-based interventions and their effectiveness at improving mental health and wellbeing?(2)Are these unobtrusive methods of data collection deemed acceptable by service users?
2. Methods
2.1. Protocol and Registration
A review of peer reviewed and grey literature were conducted. Grey literature can be defined as publicly available, open-source information, which is not controlled by commercial publishers [35] such as reports, guidelines, recommendations, and theses. A protocol was developed in advance and registered with the Research Registry on the 12th of July 2022 (registration number: reviewregistry1401)
2.2. Search Strategy and Eligibility Criteria
The inclusion of grey literature was important for the review as much of the learning about how to monitor community-based intervention effectiveness with service users may have been produced by nonacademic, small organisations that do not tend to publish in academic journals. Grey literature may additionally include information on the acceptability of monitoring methods amongst service providers and users.
We searched Embase, Psycinfo, PubMed, web of science, ProQuest, dissertations, and theses global for relevant articles published in English in the last five years using the following combinations of terms: Community AND (intervention OR programme OR activities OR group OR event) AND (monitor OR capture OR measure) AND (perception OR experience OR view) AND (acceptability OR impact OR effective) AND (mental health or wellbeing) NOT (observation) NOT (focus group) NOT (survey) NOT (questionnaire) NOT (interview)
This search was supplemented using two Google searches. The first was an advanced search including all the keywords used in the databases. The second search used broader terms (community, intervention, innovative, monitoring methods, acceptability, and mental health) to ensure all resources were identified. For both Google searches, the first 50 hits (i.e., first 5 pages) were examined for relevant resources. If there were relevant results in the last 20 hits (i.e., two pages), a further 50 hits were examined.
Additional grey literature was requested from known experts in the field of community-based interventions, public involvement, public health, mental health, and methodology. An e-mail was sent out to 15 experts directly, and to over 300 experts indirectly via three mailing lists. Three experts also forwarded our e-mail request on to their colleagues. This method has been successful in identifying relevant grey literature in previous reviews [36, 37]. Experts were asked for any papers, such as reports, theses, guidelines, or similar information.
2.3. Study Eligibility and Quality Assessment
Documents were included in the review if they were any study design, including qualitative and quantitative data, as well as recommendations or guidelines that reported unobtrusive methods and their perceived acceptability. Excluded studies were those that used traditional methods such as surveys, questionnaires, focus groups, and observations (See Table 1 for complete list of inclusion and exclusion criteria).
All identified literature was stored in a database using the software ENDNote. Relevant information for the review was identified in a three-step approach based on the inclusion criteria. In the first step, publications were excluded if their titles were irrelevant based on the inclusion criteria. In the second step, publications were excluded if their abstracts indicated they did not fit the inclusion criteria. In both steps a conservative approach was used, meaning that if there was any doubt about the relevance of the information it was retained. In the third step the included publications were viewed at full text and assessed for inclusion. All three steps were conducted independently by two reviewers (CH and NL). Any disagreement at any stage of assessment was discussed and resolved by the two reviewers or by bringing in a third reviewer (JK) if necessary.
2.4. Data Analysis
Data extraction for all the relevant literature was planned to be conducted by two authors (CH and NL) using a data extraction form created by the first author (CH). Data extraction forms varied slightly according to whether the literature was retrieved from a database, online search engine, or expert but overall included information on resource type, year of publication, author, service users, description of intervention, description of methods used, and service users perceived acceptability of the methods if included. Any discrepancies in data extraction were planned to be resolved by a third reviewer (JK).
3. Results
3.1. Search Results
Figure 1 presents a study flow diagram showing the initial results obtained and screened from the following three search strategies: databases (n = 886), Google (n = 40), and the experts (n = 4). Most resources were excluded at the point of screening titles and abstracts because they were randomised controlled trials (RCTs), not concerned with mental health and involved traditional data collection methods (e.g., surveys, interviews, and focus groups). Full texts were reviewed for 12 resources and 0 were included in the review. Reasons for exclusion at the full-text assessment stage were standard methods of data collection (e.g., survey, interviews, and focus groups n = 10) or not being focused on service users' perceptions of the intervention (n = 2).

4. Discussion
This review of published and grey literature aimed to identify any unobtrusive methods that had previously been used to capture first hand perceptions of community-based interventions, their effectiveness and the perceived acceptability of the methods used. No studies met our inclusion criteria; consequently, we report an empty review. Any empty review is a review that finds no studies, documents, or resources eligible for inclusion [38].
It is significant that we were unable to identify any studies. While there are studies that implement unobtrusive or innovative methods of data collection (e.g., postcards [39] and smartphones [40]), they are not commonplace, with traditional methods such as surveys, focus groups, observations, and interviews taking precedence in the literature. Where there is literature on unobtrusive methods of data collection, according to the findings of the current review, none have been designed or implemented to gather service users’ perceptions of community-based interventions in the field of mental health and wellbeing. Examples of unobtrusive methods that have previously been employed or suggested to offer new opportunities for data collection include using the postcard technique to gather reflective and situational data [39], smartphones to gather data on emotional variation in daily life [41, 42], Microsoft Teams for gathering feedback [43], and photovoice for preventing and controlling complex health problems [44]. Unlike the traditional methods of data collection, these methods have been suggested to offer researchers data collection methods that can be more easily integrated into the participants’ lives. For example, with mobile phones being part of everyday life for billions of people, they represent a potential suitable tool for collecting data in an unobtrusive way [41, 45]. However, such methods may not be generalisable to community-based interventions focused on mental health support, due to the high proportion of service users in this field, with varying cognitive, physical, emotional, and digital literacy needs. For example, asking service users to complete postcards may require too much time and investment from both service users and/or service providers or may require a level of written English that not everyone has. Similarly, using smartphones or feeding back via Teams may not be appropriate for individuals engaged in community-based interventions, because attendees from more disadvantaged backgrounds may have limited access to mobile phones or may lack digital literacy. The unsuitability of unobtrusive methods to the context of community-based interventions could provide one explanation for the absence of literature in this area, as highlighted by the current review.
Notwithstanding this, there is potential for some of these unobtrusive methods, such as using postcards [39], photovoice [44], and Tree of Life [46], to be adapted to different contexts and individuals, to become suitable methods for gathering evaluative data for community-based interventions to improve mental health and wellbeing. These methods could potentially offer researchers and/or service providers a more inclusive way of engaging with service users. For example, photovoice and Tree of Life have previously been found to be successful data collection methods that help break down language and cultural barriers, in addition to offering opportunities to explore vulnerabilities through a creative activity [46].
Given the varying needs of service users (e.g., language/vocabulary cognitive and physical), and the increasing number of community-based interventions being implemented by local authorities, unobtrusive data collection methods are important to progress understanding of how interventions are perceived, how they can be delivered successfully, and what impacts they might have,and for whom. Identifying successful unobtrusive methods that have the potential to be adapted to be suitable evaluative measures for community-based interventions aimed at improving health and wellbeing, will also allow service providers with limited capacity and resources (e.g., local authorities and third sector organisations) to gather light touch evaluative information for themselves. The evaluation of interventions is important to improve their effectiveness by allowing service providers to make necessary adaptations to content, provision, and implementation based on participant need. It also provides vital information to enable effective community-based work to continue to bid for and receive funding. Future research on the value of community-led mental health and wellbeing initiatives should consider developing or adapting unobtrusive methods already in existence, to gather service user perceptions of effectiveness, and should also capture acceptability of such methods. This will provide important advances in the field of intervention evaluation, particularly for community-based interventions.
4.1. Strengths and Limitations
This is the first review to systematically examine published and grey literature, to identify unobtrusive methods used to measure participant perceptions of community-based interventions to support mental health and wellbeing. A limitation was that we only included papers or reports in the field of mental health and wellbeing that concerned support in the community. There may be literature available in other health fields or in other settings (e.g., clinical) that reports such methods to examine service users’ perceptions.
While it is possible that expanding the search to include outcomes other than mental health could have resulted in some hits being identified, the current review was extensive, covering six databases, Google, Google scholar, and experts. With our search resulting in over 900 hits, we are confident in our suggestion that there is a need for unobtrusive methods to be specifically implemented and reported on in terms of their acceptability and feasibility, in the field of mental health and wellbeing.
5. Conclusions
The findings from this review highlight the need to develop or adapt and implement unobtrusive methods of data collection as evaluative measures of community-based interventions to support mental health and to explore their acceptability and feasibility among service users. Unobtrusive methods of data collection could provide important advances for monitoring interventions in an inclusive way, providing longitudinal data, and allowing local authorities to continuously review the effectiveness and cost-effectiveness of their interventions.
Data Availability
The data used to support this study are available from the corresponding author upon reasonable request.
Additional Points
What is known about this topic and what this paper adds? (i) In England, approximately 1 in 6 people have a common mental health condition. (ii) Community-based interventions are increasingly used to improve health and reduce inequalities. (iii) Identifying acceptable, unobtrusive methods to gather services users’ perceptions of community-based interventions, is important for intervention evaluations and to maintain financial support. (iv) The findings from this review highlight the need to develop and implement unobtrusive methods of data collection as evaluative measures of community-based interventions and assess the acceptability and feasibility of such methods among service users. (v) Unobtrusive methods could provide important advances for monitoring interventions, providing longitudinal data, and allowing local authorities and third sector organisations to continuously review the effectiveness of their interventions.
Disclosure
Current address for N. Leonard: MRC Life Course Epidemiology Centre, University of Southampton, Southampton, UK.
Conflicts of Interest
The authors declare that have no conflicts of interest to declare.
Authors’ Contributions
CH, NL, and JK performed conceptualisation of the review, design of the review, data extraction, screening, and synthesis. CH and NL performed the title, abstract, and full text screening. CH wrote an initial draft and all authors reviewed and edited the manuscript. All authors read and approved the final manuscript.
Acknowledgments
The authors would like to thank all the experts who provided recommendations and resources for the grey literature search. This review is funded by the National Institute for Health and Care Research (NIHR) Public Health Intervention Responsive Studies Team (PHIRST/NIHR131567). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care. Funders were not involved in the review, in writing the review, or in the decision to publish the results.