‘Keeping families and children in mind’: an evaluation of a web-based workforce resourcecfs_731 192..200

Andrea Reupert*, Kim Foster†, Darryl Maybery‡, Kylie Eddy§ and Elizabeth Fudge¶ *Senior Lecturer, Department of Rural and Indigenous Health, Monash University, Moe, Victoria, †Associate Professor,

Mental Health Nursing, University of Sydney, Camperdown, NSW, ‡Associate Professor of Rural Mental Health,

Department of Rural and Indigenous Health, Monash University & Gippsland Medical School, Moe, Victoria, and

§Workforce Development Officer, ¶Project Manager, Children Of Parents with a Mental Illness (COPMI) national

initiative, North Adelaide, South Australia, Australia

A B S T R AC T

This study outlines pilot evaluation data of the web-based training resource ‘Keeping Families and Children in Mind’, designed for clini- cians who work with families where a parent has a mental illness. The resource was developed from scoping existing workforce packages and in consultation with consumers, carers, researchers and mental- health clinicians. Preliminary evaluation data were collected from an urban and a rural site in Australia via focus group interviews and pre- and post-training questionnaires to ascertain the experiences of those who participated in the training. Additionally, training facilitators were invited to maintain journals in order to identify planning and implementation issues when using the resource. Post-training, partici- pants emphasized the need to work collaboratively with others, as well as the importance of acknowledging and working with the family members of consumers, especially children. Also, participants reported positive changes in knowledge, skill and confidence when working with families affected by parental mental illness. Facilitators highlighted technology issues and the need to work interactively with participants when using the resource. Recommendations regarding policy and future research conclude this paper.

Correspondence: Andrea Reupert, Department of Rural and Indigenous Health, Monash University, PO BOX 973, Moe, Victoria, Australia E-mail: [email protected]

Keywords: children, evaluation, families, parental mental illness, web-based workforce training

Accepted for publication: August 2010

I N T R O D U C T I O N

Mental illness is a family affair, particularly where a parent, with dependent children, has a mental illness. Several studies indicate that children where a parent has a mental illness may be at twice the risk of devel- oping a mental illness diagnosis compared to other children (Black et al. 2003; Park et al. 2003; Cunning- ham et al. 2004; Leschied et al. 2005; Edwards et al. 2006). Other studies highlight the range of behav- ioural, interpersonal, academic and other difficulties that children of parents with a mental illness might face (Rutter & Quinton 1984; Farahati et al. 2003; Maughan et al. 2007; Reupert & Maybery 2007). An

epidemiological study has estimated that between 21 and 23% of all families have, or have had, at least one parent with a mental illness (Maybery et al. 2009). Thus, given the prevalence of families affected by parental mental illness and the potential difficulties they face, it is important that the mental-health work- force is appropriately skilled at identifying and subse- quently intervening with children and their parents. This paper describes a web-based training resource, ‘Keeping Families and Children in Mind’ designed for the mental-health workforce, and the results of a pilot evaluation of the resource.

While there is ample evidence highlighting the need for early intervention, children living in families where

doi:10.1111/j.1365-2206.2010.00731.x

192 Child and Family Social Work 2011, 16, pp 192–200 © 2010 Blackwell Publishing Ltd

a parent has a mental illness have been described as ‘hidden’, because clinicians are often unaware that consumers are parents with dependent children (Fudge & Mason 2004). In the USA, it is suggested that agencies take a categorical approach and focus on either the child (e.g. in terms of child protection) or the adult (e.g. for his or her mental-health needs) (Nicholson et al. 2001). Maybery & Reupert (2006) found that while many Australian adult mental-health clinicians want to work with all family members, they report clear skill and knowledge limitations, a finding also confirmed in a Finnish study with psychiatric nurses (Korhonen et al. 2008). Slack & Webber (2008) found that even though many adult mental-health workers favour supporting children of consumers, they did not necessarily consider it their role to do so. Maybery & Reupert (2009) summarize workforce barriers in terms of (i) policy and management; (ii) inter-agency collaboration; and (iii) clinician attitude, skill and knowledge.

Notwithstanding these barriers, acknowledging and working with family members has been shown to be beneficial to the consumer, his or her children and other family members (Glynn et al. 2006; Beardslee et al. 2008). For example, it has been found that a family-focused intervention was effective in reducing the exacerbations in schizophrenia, improving medi- cation compliance and reducing or eliminating sub- stance abuse (Glynn et al. 2006). Family-sensitive practice is beneficial to the consumer as well as other family members, by reducing a family’s subjective burden of care and increasing their level of self-care and emotional functioning (Glynn et al. 2006). Fur- thermore, acknowledging and working with children

of parent consumers improves family functioning and children’s understanding of their parent’s disorder as well as a reduction in children’s internalizing symp- toms (Beardslee et al. 2008). Given the efficacy of a family-sensitive approach, it is imperative that training is designed and developed in ways that addresses the current skill and knowledge gaps found in the workforce.

In response to the training needs of the mental- health workforce, a resource ‘Keeping Families and Children in Mind’ was developed by the Australian National COPMI (Children of Parents with a Mental Illness) initiative through scoping existing workforce packages and then identifying main themes and issues across these packages (Reupert et al. 2009). Addition- ally, the resource was developed using a Delphi process (see Note 1) with 14 experts consisting of consumers, carers, researchers and mental-health clinicians (Whitman et al. 2009). These experts responded to questions about curriculum content and teaching processes in three Delphi ‘rounds’ until con- sensus was reached. In this process, experts were asked to summarize themes (generated from previous Delphi rounds) that resulted in the final six core modules of the resource (see Table 1).

The resulting Keeping Families and Children in Mind: COPMI Mental HealthWorker Education Resource deliv- ers an interactive, audio and video material using Web 2 technology. Clinicians might focus only on those modules that are of interest and/or need or undertake all six modules. The resource includes a variety of educative web pages and links, and video and audio inserts of families describing what it is like to live with parental mental illness as well as clinicians reporting

Table 1 The six modules of ‘Keeping Families and Children in Mind: COPMI Mental Health Worker Education Resource’

1 Mental Health and Families – introduces a family where a parent experiences mental illness. Information is also provided by parents, children and workers about factors that contribute to mental health and illness, stigma and mental illness and family support.

2 The Parent – introduces a second family. Information is provided about the impact of mental illness on parenting, the impact of parenting on mental illness and the recovery process.

3 The Child – provides an opportunity to reflect on the experiences of children in two families where a parent experiences mental illness. Information is provided on risk and protective factors that influence child well-being, including the impact of parental mental illness on child development. There are demonstrations of how talking to children can assist in their understanding of what is happening at home.

4 The Family – highlights the importance and influence of the family unit in the recovery of a parent who experiences mental illness, including influences on family functioning, family resilience and working with families using a strengths-based approach.

5 Carers – presents the perspective of family carers with a particular emphasis on the issues faced by young carers and grandparents who provide care for children of parents who experience mental illness.

6 Putting it into Practice – provides opportunities for learners to reflect on their work practice at an individual and systems level in regards to supporting families where a parent experiences mental illness. It provides practical examples of what workers are currently doing across Australia and an extensive list of accessible resources.

‘Keeping families and children in mind’ A Reupert et al.

193 Child and Family Social Work 2011, 16, pp 192–200 © 2010 Blackwell Publishing Ltd

about their experiences working with such families. The resource also provides scenarios about fictitious families that encourage mental-health clinicians to reflect on their clinical practice. Clinicians are able to access the resource in a web-based, self-paced mode, or alternatively, attend facilitator-led training using the resource in a group format. The resource is freely accessible at http://www.copmi.net.au/worked/ index.html

At the point of writing, several Australian state mental-health services have indicated that they will incorporate this resource as part of a large ‘roll out’ of training, with similar indications from overseas researchers and trainers (personal correspondence to the authors). Consequently, as there are likely to be hundreds and perhaps thousands of clinicians who will use this resource, it is essential to report initial data about the utility of the resource. Currently, a further evaluation of the large-scale roll out is planned, although it will take some time to collect and analyse a larger data set. Thus, this paper summarizes preliminary efficacy data from the piloting of the training resource.

M E T H O D

Training was conducted at a rural and an urban site, in two Australian states (Victoria and Tasmania). The workshops were each presented by two female facili- tators with extensive backgrounds in mental health and workforce training. The training for the rural site took place over two half days with 23 participants, while training in the urban site went for a full day and included 14 participants. Training participants came from a range of services including child and adolescent and adult mental-health agencies, non- government agencies and hospitals, and from disci- plines including psychiatry, social work, education and consumer and carer groups. The purpose of the training, at both sites, was to introduce participants to the broad issues related to families where a parent has a mental illness, and to identify where to access infor- mation about mental illnesses for families and clini- cians. Participation in the training was not dependent on participation in the evaluation, and ethics approval was provided by the Monash University Standing Committee on Ethics in Research Involving Humans.

Data were drawn from three sources. First, partici- pants were invited to participate in focus group interviews to gain their views on the resource and, second, to complete anonymous pre- and post-program questionnaires to quantify short-term

changes in learning. Finally, training facilitators completed journals in order to identify planning and implementation issues.

Focus group interviews

Across the two focus groups held at the two sites, there were a total of 28 participants, eight men and 20 women. Focus group questions aimed to determine potential changes in knowledge, attitude and practice, for example:

• What did you learn, if anything, as a result of being shown the resource?

• Has the resource changed any attitudes or ways of looking at consumers you previously had, if at all?

• In what ways, if any, do you think the resource will change the way you practice? With permission, the focus groups were audiotaped

and subsequently transcribed. Data were then analy- sed using an open coding system, attaching labels to lines or paragraphs of data and then describing the data at a concrete level, before moving to a more conceptual level (Anfara et al. 2002), first within each focus group transcript and then across the two tran- scripts. This descriptive and iterative analytic process aimed to meaningfully classify codes into themes.

Questionnaires

Twenty-seven participants (20 females and seven males) completed identical pre- and post-training questionnaires that assessed change in participant confidence, knowledge and skill when working with the parents, children and families. A coding system was employed to ensure anonymity but also to allow matching of pre- and post-data for analysis. The specifically designed ‘Workforce Questionnaire’ was employed. The questionnaire was initially developed in an adult mental-health clinician sample (see Maybery & Reupert 2006) and is currently being employed across five Victorian Health regions and by researchers in British Columbia (Canada) in a large Australia-Canada benchmarking questionnaire of family-focused practice. A brief version of the ques- tionnaire, with 27 items, was employed here. The questionnaire has excellent content and construct validity and unpublished data highlight good internal reliability of subscales ranging from Cronbach alphas of 0.87 to 0.72 for relevant subscales. Participants responded on a scale from 1 (strongly disagree) to 7 (strongly agree) for each item.

‘Keeping families and children in mind’ A Reupert et al.

194 Child and Family Social Work 2011, 16, pp 192–200 © 2010 Blackwell Publishing Ltd

Facilitator journals

The four training facilitators were invited to complete various pre-determined questions in a journal at two time points, before the workshop, to capture planning issues, and then after the workshop, to highlight imple- mentation matters. Pre-training questions included:

• What issues did you encounter in planning the training? How did you overcome them? Or what was required to overcome them?

• How did you decide which aspects of the resource to use/not use?

Post-training questions included:

• What worked well when using the resource? What didn’t work so well and why?

• What, if anything, do you think needs to change about the resource?

• What facilitation skills were particularly important in using the resource in workshop format?

• What issues did you encounter when using the resource? Facilitators were instructed to not censure them-

selves, nor be concerned about spelling or grammar. They were not required to include their name on the journal. Journals were analysed using the same the- matic analysis approach employed with the focus group transcripts.

R E S U LT S

Focus groups

Three themes emerged of participants’ experiences of the training in terms of (i) views on the resource; (ii) impact of the training on attitude and practice; and (iii) possibilities for implementation.

Views on the resource

Participants expressed very positive views on the resource, identifying it as a quality production with comprehensive content and as a valuable resource for clinicians from a range of child and adult healthcare settings, particularly in rural and remote settings. The emphasis on working together with families, rather than simply focusing on diagnosis and assessment, was appreciated. A strength of the resource was its interactivity and the life-like depiction of issues expe- rienced by children and families:

I thought that the narratives and the case studies were excel-

lent around mental health and placing that in the context of

the family . . .

Participants particularly enjoyed the video clips, which they considered very effective for learning, ‘. . . it holds your attention much more than just reading . . . it brings it to life’.

Some participants considered the resource to be quite lengthy and that it took time to learn to navigate and become familiar with it. They acknowledged that working through the resource in addition to their workload would require commitment, ‘. . . if you don’t block out time, you’ll just be too busy’. However, the flexibility and ability to choose which modules and content to read were helpful, ‘. . . the advantage is that you can go in and pick the bits you need’.

In the workshops, the opportunity to interact and discuss issues with others, including consumers and carers, and listen to their insights and practices, added to the experience.

. . . one of the things that I got the most out of was having a

consumer presence in the training session. It was actually him

and his carer . . . so you were able to get the carer’s perspec-

tive, the consumer’s perspective, and that was good.

The training facilitators’ ability to identify each group’s needs and tailor the workshop accordingly was an important aspect of training. Reliable internet access was integral to its effectiveness, with broadband access and slow running of computers, a frustration for some participants.

Impact of the training on attitude and practice

The most prominent impact of the training was par- ticipants’ heightened awareness of the need to care for all family members and the importance of taking time to sit and talk with families. For some, training reaffirmed that they were on the right pathway in what they were already practising, while for others, the resource challenged their attitudes towards families.

. . . it makes you aware of what you’re bringing to the inter-

action [with families]. Am I bringing pre-conceived ideas and

biases . . . which may or may not be an accurate view?

Training highlighted the need for clinicians from a range of backgrounds and settings to collaborate in care: ‘I think what’s come out of it, very much so, was the importance of everyone who’s working with that family, working together . . .’ On their return to work, participants felt their raised awareness would encour- age them to look further into a family’s needs, and ensure that they scheduled appointments where they could see the children as well as the consumer parent.

‘Keeping families and children in mind’ A Reupert et al.

195 Child and Family Social Work 2011, 16, pp 192–200 © 2010 Blackwell Publishing Ltd

The training added to their knowledge about chil- dren’s understandings of mental illness at different developmental stages, and the need to tailor informa- tion and address issues accordingly.

. . . because I have limited experience in working with young

carers . . . it really opened my mind to some of the issues for

young carers . . .

In addition to being more mindful of children and families, participants identified the need to develop policies around family-inclusive practice in their service. They acknowledged that while the resource might impact on individual practice, unless manage- ment supported child and family-inclusive practice as core business, the impacts could be limited.

. . . the practice will change the individual, and ideally if

they’re supported by the management structure . . . it spreads

through the organisation . . .

Possibilities for implementation

Participants saw a range of possibilities for imple- menting the resource within their organization as well as in other organizations and contexts. This included using the resource in professional development ses- sions for staff over a 12-month period, including it as part of new staff inductions, and using it as a training resource for students on clinical placement:

. . . I can see the potential for it to change practice within my

organisation.We have a carer support program . . . and I’d like

to deliver [a module] to the carer support team.

Participants made a number of suggestions for improving the resource, such as adding further infor- mation on mental illness and drug and alcohol use for users who might not have a background in mental health, and adding links to relevant local and/or regional resources and services for clinicians and family members. Finally, participants suggested that the resource should include a section on collaborative practice and possible protocols for how agencies can work with each other with families.

Questionnaire

Table 2 presents participant mean scores, standard deviations and paired sample t-statistics for pre- and post-responses to items about clinicians’ confidence, knowledge and skill concerning family issues. Almost all participant responses to knowledge (see Note 2) items moved in the expected direction (note that some items are negatively worded) and just under half of the

26 items showed a significant change – again all in the expected direction. Items shown in bold are signifi- cant at 0.05 level and actual P-values are shown in the right-hand column for each item.

Significantly different variables generally reflect changes that occurred at both the rural and the urban sites.

Program facilitator journals

Planning issues

The main planning issues for facilitators were related to information technology and the knowledge to appropriately download and save different aspects of the resource. Working with others with technological expertise was or would have been useful. Facilitators indicated that having a thorough knowledge of par- ticipants’ background (profession and organization) was important in deciding which aspects of the resource to employ.They also suggested that the work- shop needed to incorporate interactive as well as didactic components.

Implementation issues

According to facilitators, the family videos were a powerful and effective way to encourage participant discussion and reflection. Problems experienced with the resource centred primarily on facilitators’ own facilitation skills (time management, co-facilitating, providing too much information, not outlining learn- ing objectives) rather than the resource per se. All agreed that the ability to work with groups and manage group discussions were important facilitation skills when using the resource. Facilitators employed small and large group discussion groups to expand on key learnings when using the resource. Handouts, case notes, and, in particular, local and national services, were used in conjunction with the resource. Overall, modules were used in a theme-based way, related to the needs of training participants and their particular clientele.

D I S C U S S I O N

Overall, pilot evaluation data of the ‘Keeping Families and Children in Mind’ resource demonstrated high participant satisfaction and significant self-reported changes in knowledge, skill and confidence. Thus, results demonstrated the utility of the resource as a training tool for clinicians working with families,

‘Keeping families and children in mind’ A Reupert et al.

196 Child and Family Social Work 2011, 16, pp 192–200 © 2010 Blackwell Publishing Ltd

with both participants and training facilitators valuing the interactivity and flexibility of a web-based resource. Participants, including those from adult mental health, highlighted the need, post-training, to work with all family members including children. This is an important finding given that previous research has highlighted clinicians’ reluctance to work with the children of parent consumers (Slack & Webber 2008). Participants also stressed the necessity to collaborate with other agencies when working with

families, another substantial result, due to the lack of inter-agency and inter-sectoral collaboration in this area (Darlington et al. 2005). While the resource was not compared in terms of delivery (that is, self-paced mode vs. a group setting), the focus group data high- light the significance for clinicians in being trained alongside others, including consumers and carers, in order to share insights, experiences and practices.

The quantitative questionnaire data tentatively shows an improvement in participants’ knowledge,

Table 2 Scores and paired sample t-statistics for pre- and post-responses to items regarding clinicians’ confidence, knowledge and skill when working on family issues

Questionnaire item Pre Post M

t PM (SD) M (SD) diff

I am knowledgeable about how parental mental illness impacts on children and families.

5.65 (0.85) 6.04 (0.77) -0.39 -2.30 0.03

I am not confident working with families of consumer-parents. 3.00 (1.87) 2.32 (1.68) 0.68 1.71 0.10 I am not knowledgeable about the key parenting issues for consumer-parents. 3.04 (1.68) 3.00 (2.04) 0.04 0.12 0.90 I am knowledgeable about the key things that consumer-parents could do to

maintain the well-being (and resilience) of their children. 4.96 (1.28) 5.38 (1.65) -0.42 -1.15 0.26

I am knowledgeable about the role of family carers and their influence on recovery for consumers.

5.38 (1.06) 5.81 (1.17) -0.42 -1.23 0.23

I am not knowledgeable about the role of young carers in families where parents experience mental illness

3.27 (1.66) 2.23 (1.56) 1.04 2.78 0.01

I am not confident working with children of consumer-parents. 2.92 (1.77) 2.50 (1.84) 0.42 1.17 0.25 I am knowledgeable about how the role of parenting impacts mental illness. 5.36 (1.04) 5.84 (0.80) -0.48 -2.39 0.03 I am not confident working with consumer-parents about their parenting skills. 3.35 (1.79) 2.58 (1.79) 0.77 2.08 0.05 I do not have the skills to work with consumer-parents about how parental mental

illness impacts on children and families. 3.23 (1.53) 2.31 (1.49) 0.92 3.27 0.00

I provide education sessions for adult family members (e.g. about the illness, treatment).

4.63 (2.09) 4.42 (2.01) 0.21 0.41 0.69

I am skilled in working with consumer-parents regarding their parenting. 4.38 (1.84) 4.92 (1.77) -0.54 -2.01 0.06 I provide education sessions for children (e.g. about the illness, treatment). 3.94 (2.30) 4.06 (2.25) -0.12 -0.32 0.76 I am skilled in providing psychosocial-education to adult family members about

the mental illness. 4.00 (2.02) 4.86 (1.80) -0.86 -2.42 0.03

I regularly have family meetings (not therapy) with consumer-parents and their family.

4.26 (2.26) 4.74 (1.88) -0.47 -1.21 0.24

I consider information from the carer or family when diagnosing and/or treating the consumer-parents.

5.37 (1.50) 5.79 (1.47) -0.42 -2.04 0.06

I provide emotional support for family members and children. 5.48 (1.53) 6.09 (0.85) -0.61 -2.37 0.03 I do not refer children of consumer-parents to child-focused (e.g. peer

support) programs (other than child and adolescent mental health). 3.19 (1.78) 3.00 (2.05) 0.19 0.43 0.67

I do refer consumer-parents and their families for family therapy or counselling.

5.09 (1.50) 5.48 (1.34) -0.39 -1.20 0.24

I provide written material (e.g. education, information) about parenting to consumer-parents.

4.75 (1.59) 5.38 (1.47) -0.63 -3.72 0.00

I regularly provide information (including written materials) about mental health issues to the children of consumer-parents.

4.10 (2.19) 5.29 (1.76) -1.19 -4.23 0.00

I often consider if referral to parent support program (or similar) is required by consumer-parents.

5.09 (1.53) 5.65 (1.34) -0.57 -2.02 0.06

Rarely do I consider if referral to peer support program (or similar) is required by my consumer-parent’s children.

2.35 (1.47) 2.13 (1.52) 0.22 0.59 0.56

I don’t provide information to the carer and/or family about the consumer-parent’s medication and/or treatment.

3.82 (2.15) 3.77 (2.09) 0.05 0.09 0.93

I am aware of locally based resources for consumers who are parents. 4.96 (1.51) 5.76 (0.83) -0.80 -3.18 0.00 I am aware of locally based resources for families with a mentally ill parent. 5.38 (1.33) 5.85 (0.78) -0.46 -2.13 0.04 I am aware of resources available to the public about mental health and illness. 5.65 (1.13) 6.12 (0.71) -0.46 -2.07 0.05

‘Keeping families and children in mind’ A Reupert et al.

197 Child and Family Social Work 2011, 16, pp 192–200 © 2010 Blackwell Publishing Ltd

skill and confidence when working with families and children, post-exposure to the resource. Almost half of the 26 items showed a significant change, in the desired direction.

Overall, while promising, these data are primarily composed of self-reported data obtained immediately after training occurred. In addition, the quantitative findings must be considered in light of a small and limited data set. The nature of the analyses under- taken in the questionnaires promoted the likelihood of Type 1 errors. There was neither a control group nor a long-term follow-up of participants to ascertain long-term behaviour change. It might also be assumed that there was a sample selection bias, as those who attended the training were aware of and motivated to undertake family-sensitive practice. Thus, it remains to be seen whether and how behavioural changes are implemented in the workforce following exposure to the resource, and further evaluation of clinicians’ practices is required. Future studies with larger and more diverse samples of professionals, as well as lon- gitudinal collection points of data should rectify these problems. As it has been demonstrated that clinicians from different disciplines have varying training inter- ests and needs when working with families affected by parental mental illness (Whitman et al. 2009), future training and subsequent evaluations need to be mindful of these differences.

Facilitators’ journals as well as the focus group data demonstrate the need for facilitators to be rea- sonably proficient with technology when working with a web-based resource or at least to have access to others who are. Journals indicate that in a group setting, the resource needs to be carefully tai- lored to the needs and professions of participants. The need for group facilitation and managing group discussion also underlines the importance of working in an interactive manner when employing this resource.

Significantly, focus group participants highlighted the time needed to complete the training as well as actually engaging with family members, including the children of consumers. Focus group participants also stressed the need for systematic change in this area, implying that it is not sufficient to merely provi

Related Tags