Evidence based policy and practice as applied to the
treatment of children with speech and language difficulties: the difference
between knowing and believing
Presentation for the CPLOL conference – Edinburgh –
September 2003
James Law[1]
The field of speech and language
therapy, like all other clinical fields is now under the political spotlight
and it is likely that it is currently undergoing changes based on available
evidence. You might say “changes, what changes?” and it may not always be
apparent that changes are going on, but there are a number of government
initiatives in the UK that are having a bearing on the way services are
delivered. Sure Start in the UK is an example but there are many others.
But, what sort of evidence is
underpinning these sorts of judgements? This is a fair question because, while
at one level policy makers do now routinely go to the Cochrane Collaboration
Reviews, there are precious few of those available in this area and such
reviews do not always answer the questions that policy makers want to address
(Egger, Davey-Smith and Altman 2001). Nevertheless they do provide the best
available evidence and the best pointers to what we know about intervention in a given subject area and move us away
from what we believe to be the case.
This is not to say that belief is not important in clinical practice only that
belief is not sufficient to make decisions about the development of services
and should be treated very cautiously as a primary motivation for the
development of clinical practice.
This presentation considers some of
these issues with specific relation to intervention for children with speech
and language delay/disorder. Obviously I have a specific interest in this area
but the issues are comparable in other clinical fields within speech and
language therapy as they are in many other disciplines in medicine (psychiatry
and primary care are good examples) and social services.
This presentation will be in two parts. The first will
report on a systematic review and meta-analysis carried out for the Cochrane
Collaboration in the UK in the field of developmental language delay/disorder.
The second part will focus on some thoughts about the process of evaluation and
what it means for practice and policy in this field.
Title of review:- Speech
and language therapy interventions for children with primary speech and
language delay or disorder James Law, Zoe Garrett, Chad Nye[2]
It is thought that approximately 6% of children have
speech and language difficulties of which the majority will not have any other
significant developmental difficulties. Whilst most children's difficulties
resolve, children for whom their difficulties persist into primary school may
have long term problems concerning literacy, socialisation, behaviour and
school attainment that can persist into adolescence. The presentation of
primary speech and language delay/disorder can be heterogeneous affecting
either a single aspect of phonology, vocabulary, syntax, semantics or
pragmatics, or a number of different linguistic areas. The difficulties may
also be primarily expressive affecting the production of language, receptive
affecting the understanding of language, or both.
Treatment efficacy for children with primary developmental
language delay/disorder has been the subject of considerable debate in recent
years and has been the focus of a number of narrative
reviews of the literature all of which conclude that intervention is
effective (Snyder-McLean and McLean 1987, Guralnick 1988, Goldstein and
Hockenburger 1991, Enderby and Emerson 1996, Law 1997, McLean and Woods Cripe
1997, Gallagher 1998, Leonard 1998, Olswang 1998). In addition to these narrative reviews two systematic reviews (Nye, Foster and Seaman 1987, Law, Boyle,
Harris, Harkness and Nye 1998) and one “best practice” review (Yoder and
McDuffie, 2002) have been completed. These latter reviews have brought together
a range of different types of literature and have used meta-analysis to
summarise that data. These provide some support for the effectiveness of speech
and language therapy although the picture is by no means as clear as the
narrative reviews tend to suggest.
To examine the effectiveness of speech and language
interventions for children with primary speech and language delay/disorder.
The following databases were searched; The Cochrane
Controlled Trials Register (CCTR) (Cochrane Library, central: 2002/3),
Cumulative Index of Nursing and Allied Health (1982 - July 2002), EMBASE (1980
- Sept Week 4 2002), ERIC (1965 - 2002), MEDLINE (1966 - Sept Week 3 2002),
PsychINFO (1872 - 2002/10 Week 2), The National Research Register (2002/3). In
addition to this references were taken from reviews of the literature and
reference lists from articles.
Randomised control trials of speech and language
therapy interventions for children or adolescents with primary speech and
language delay/disorder were considered for the review.
Titles and abstracts were identified and assessed for
relevance, before the full text version was obtained of all potentially
relevant articles. The data was categorised depending on the control group used
in the study; no treatment, general stimulation, or traditional therapy, and
considered in terms of the effects of intervention on expressive and receptive
phonology, syntax and vocabulary. The outcomes used in the analysis were
dependent on the focus of the study with only the direct effects of therapy
being considered in this review. These were investigated at the level of the
target of therapy, measures of overall linguistic development and broader
measures of linguistic functioning taken from parent report or language
samples.
Thirty-six articles reporting a total of thirty-three
different trials were found. For twenty-five of these articles sufficient
information was obtained for their use in the meta-analysis. The results
suggest that speech and language therapy is effective for children with
phonological (SMD=0.44, 95%CI: 0.01,0.86) or expressive syntax (SMD=1.02,
95%CI: 0.04,2.01) or vocabulary difficulties (SMD=0.89, 95%CI: 0.21,1.56), but
that there is less evidence that interventions are effective for children with
receptive difficulties (SMD=-0.04, 95%CI: -0.64,0.56). No difference was shown
between parent and clinician administrators when treating children with
expressive vocabulary (SMD=0.20, 95%CI: -0.40, 0.79) or syntax difficulties
(SMD=-0.04, 95%CI: -0.56,0.48), and studies did not show a difference between
the effects of group and individual interventions (SMD=0.01, 95%CI:
-0.26,1.17). The use of normal language peers in therapy was shown to have a
positive effect on therapy outcome (SMD=2.29, 95%CI: 1.11,3.48). The review
identified longer duration (>8 weeks) of therapy as being a potential factor
in good clinical outcomes.
The review shows that overall there is a positive
effect of speech and language therapy interventions for children phonological
and expressive speech and language delay/disorder. There is a need for further
research to investigate intervention for receptive language difficulties as it
has been shown that these children have been least likely to resolve without
treatment and most likely to have long term sequelae. There is a large degree
of heterogeneity in the results, and the sources of this need to be
investigated through studies analysing the aetiology of speech and language
difficulties, and the development of greater consistency in and description of
guidelines for speech and language interventions.
What as a policy maker – or indeed a clinician would
you make of such a review? On the one hand there are a reasonable number of
studies – certainly more than there are in most such reviews. On the other hand
there is no large scale study which conclusively addresses the questions that
policy makers may want to ask. The studies, while relatively well designed, are
small and subject to random variation (both type I and type II errors) and we
have to be very cautious in our interpretation of the science at this level.
Some have even suggested that there may be real difficulties applying the RCT
methodology to speech and language therapy at all (Wertz 2002).
There is also a concern that many of these trials are pragmatic in nature in that they are
empirical studies designed to assess the value of a given intervention but are
not set up from a theoretical perspective. In other words while they may be
able to demonstrate change it is not clear precisely what has brought about
that change except at a very gross level. For example, we may be able to
conclude that parent/child interaction therapy may work with parents who opt
into such interventions but it is not clear what is working. Does this type of
work bump start a faulty language acquisition device, does the emphasis on
reciprocity simply improve the child’s attention and this then triggers word
learning which in turn has a knock on effect on the acquisition of syntax.
Alternatively one might take an affective
angle on this. The therapy may provide a way of reconnecting parent and child
who have learned over the preceding year or so to “live together in parallel”.
In part the problem occurs because there remains considerable uncertainty about
what is going wrong with these children in the first place. A functionalist and
a nativist interpretation are bound to come up with very different
interpretations of the data.
It is often assumed that intervention studies answer a
simple question – does it work or not? But, of course, there are different
interpretations of what that might mean and this affects directly the selection
of outcomes. For example most people in this audience would say that they were
interested in prevention. In fact if we are careful prevention becomes a bit of
a “motherhood and apple pie” mantra. But are we talking about primary,
secondary or tertiary prevention. Primary prevention means promoting public
health messages which affect whether children start having difficulties. A good
example here would be reducing household accidents amongst toddlers. Secondary
prevention means treating identified cases of a condition and effectively
removing the symptoms – or curing the condition. The classic example here might
be screening for phenylketonuria in infants and providing treatment which has a
clear impact on the child’s health and, indeed, life expectancy. Tertiary
prevention means the reduction or prevention of associated symptomatology.
Examples of this might be the prevention of secondary infections in someone
with a chronic condition or the reduction of behavioural symptoms in people
with learning disabilities. In the main the interventions referred to above are
framed in terms of secondary prevention but, given what we now know of the long
term outcomes of language impairment (Johnson et al. 1999), it would probably
be more correct to see them as tertiary preventions.
An interesting development of this is the work
currently going on in the UK in terms of Sure Start. This is a national
initiative designed in the long term to improve the inclusion in society of
people from socially disadvantaged backgrounds. The government has wisely
introduced a language target for all two year olds to establish whether Sure
Start is “working”. To give you some idea of the extent of the difficulty from
the perspective of evidence based practice the target form 2003/2004 is as
follows
“ An increase in the proportion of children having normal levels of communication, language and literacy at the end of Foundation Stage and an increase in the proportion of young children with satisfactory speech and language development at age 2 years.”
We will be discussing the work towards this in another
presentation in this symposium and so I will not be going into it here. My point
is that here we have a population level intervention where the outcome amounts
to primary prevention – we are trying to reduce the proportion of children
falling below a certain level at a specific point in their development.
Unfortunately out Cochrane Review does not really help us with this sort of
question.
The Cochrane model cited above relies
heavily on notions of statistical significance whether they are expressed as p
values or confidence intervals. But does this tell us the whole story? The
question remains as to whether the common statistics reported reflect enough
about real change in the children themselves. It may be the case that the
statistical significance reported in the review is not sufficiently pronounced
to be real in a clinical sense.
It is possible that statistical and
clinical significance may not map on to each other. This may be either because
statistical significance in large scale studies may be relatively easy to achieve
by virtue of the statistical power available but that such findings are
meaningless when it comes to individuals. Alternatively the use of statistical
testing may, in more modest samples, mask an effect on some individuals which
washes out in the group analysis. There may be individual learning styles, for
example, which influence the way in which certain children respond to specific
interventions, variability which may be lost in group studies. This subject has
been considered in the field of developmental language delays/ disorders (Bain,
and Dollaghan,1991) but in the main it has been the focus of other areas
concerned with other aspects of developmental change (Kazdin, 1999).
The
mutability of speech and language
In the end we need to know to what extent it is
possible to change children’s language development – what are the parameters
that we are reasonably functioning within? The data suggest that focused
environmental modifications have the capacity to change significantly both the
speech and the language development of young children. Although it is not
possible, on the basis of these results, to say to what extent these
modifications have long term downstream effects in the child’s development it
is clear that it is possible to enhance phonology and expressive language
development in the short term. It might be anticipated that there would be a
differential effect on vocabulary, which is obviously a function of the
learning process, and syntax, which to the extent that it is modular, might not
be. However, the present results do not support such a conclusion. Similarly
these studies do not make it possible to conclude whether the changes measured
in the studies concerned are systemic changes to the child’s language system or
whether they simply facilitate access to that system.
If the latter is the case it might be argued that in
the main primary language delays/disorders are really problems of access and
are therefore processing disorders rather than the result of a damaged language
system. The conclusions drawn about receptive language skills suggest that this
may in fact be the case for expressive language delays/disorders but not
receptive language delays/disorders. It may be that there are some strategies
that can be taught to facilitate the child’s listening but that beyond this
there is relatively little that can be done to remove the impairment. An
interesting extension of this is the extent to which such interventions really
should be construed as interventions at all. It might be argued that that the
process of relatively short term interventions of the type described here is
really a sophisticated form of assessment, differentiating between those that
do and those that do not respond to intervention and that the classification of
language delay/disorder should incorporate this much has been suggested in the
field of reading disorders (Law 2003, Vellutino, Scanlon, Sipay, Small, Chen,
Pratt, and Denckla, 1996).
The data picked out in the Cochrane review provide
some pretty clear evidence of the value of both speech and expressive language
interventions. It is true that the results could be demonstrated to be more
consistent, the interventions better described and the studies better designed
but I suspect that it would be possible to replicate these findings now. The
real gap concerns receptive language difficulties. This is of central
importance because we are now clear that receptive language difficulties puts
the child at increased risk of persistent problems. The data, such as they are
don’t tell us much beyond interventions don’t seem to work either if you work
with socially disadvantaged children or if you don’t provide much intervention
in the first place. These findings are hardly surprising. But where does the
evidence based practice model take us next. It looks as if large group studies
of heterogeneous populations are not the way forward. And here we need to go
back to experimental single subject designs of the type that would never turn
up in a Cochrane Review. We need to go back to basics and ask ourselves the
questions of what we know about what makes children’s receptive language
change. In fact the literature does not help us much with this. An MSc student
of mine Stephen Parsons – has recently been working on this and come up with some
very interesting results on receptive word learning.
A parallel approach to filling this evidence gap is to
look at what we know from other empirical traditions. You might ask – who has
been working on developing receptive language in young children? The answer is
that there is a tradition of research coming out of the US and attracting
interest here in the field of what is known as “reciprocal teaching”. This is
essentially a kind of structured mediated learning designed to teach children
metacognitive strategies which improve their reading comprehension. What is
more the reviews of this literature source have already been carried out
(Rosenshine and Meister 1994). And this raises a critical issue in the move
towards evidence based practice – read around the subject and make sure someone
has not already covered this. There is sometimes a temptation for therapists
working with their specific client groups to come up with sensible
interventions and to assume that people have not come up with them before.
There are many other areas which were flagged up in
the review as gaps in the evidence base. For example, we do not know about
educational interventions – increasingly the norm for these children. This it
is assumed than an inclusive environment best suits their needs but we do not
really know what effect this is likely to have on their language development.
Similarly it would be useful to examine the relationship between speech and
language interventions and behavioural outcomes, an important issue given the
tendency for language impaired children to have associated behavioural
difficulties.
Dissemination – change and
getting it out there…
Finally we move to the relationship between what we
know in a scientific sense from the literature and what actually goes on in
clinical practice. In a number of areas of medicine it is obvious that
practitioners pay scant attention to the literature and practice changes much
more slowly than the recommendations made in the reviews of the literature
would suggest. Probably unsurprisingly trawling through meta-analyses is not
most clinicians’ idea of fun.
The intention of the Cochrane review process is to
make these reviews as accessible as possible but to be honest you have to be
courageous to get through some of the detail. This issue is coming to be
recognised now and there is a move in the UK to try to distil some of the
findings for practitioners. This will mirror the work of the “What Works
Clearing House” for educational research (http://www.w-w-c.org/)
and the Campbell Collaboration for research on the criminal justice system and
in Education (http://www.campbellcollaboration.org/).
Likewise the Cochrane Collaboration itself can be a useful starting point
either in terms of the collaboration itself (http://www.cochrane.org/)
or in terms of the groups which are most likely to interest this audience Cochrane
Developmental, Psychosocial and Learning Problems Group (http://www.cochrane.org/cochrane/revabstr/g400index.htm),
the Cochrane Stroke Group (http://www.cochrane.org/cochrane/revabstr/g040index.htm)
or the Cochrane Motor Disorders group (http://www.cochrane.org/cochrane/revabstr/g230index.htm)
all of which have reviews related to speech and language therapy.
There are also other sources summarising evidence in
specific fields related to speech and language therapy. For example there are
monthly summaries of speech and language therapy literature produced by a
retired member of staff at the British Library. Perhaps a more palatable digest
of specific topics related to speech and language disorders in children is also
presented by Judith Johnston at the University of British Columbia on www.lt.audiospeech.ubc.ca.
The point is that there is a plethora of sources of
information and we do now know a great deal about speech and language disorders
and this should be feeding directly into practice. It does not always do so, of
course, because practitioners may not feel they have the time to read the
literature or because they believe they already have the right way of working
and do not need to question what they do. The important thing is that
practitioners become engaged in the process of examining the evidence,
discussing with colleagues their understanding of the literature that is
emerging. Reading groups are an ideal place to start. Likewise Action Research
helps practitioners focus on research issues relating to their own practice. It
may be possible for therapists to become involved in developing their own
studies either because of opportunities emerging at a local level or because
they are completing their own studies through Undergraduate, Masters or PhD
project work. The problem with the Cochrane message is that the very stringent
criteria for the inclusion of studies makes it looks as if this is the only
evidence that is of any use – if you are not contributing to a systematic
review or an RCT you are not involved in evidence based practice. This is
simply not true. Systematic reviews and RCTs are only stages in the hierarchy
of evidence. Evidence starts with a
belief in the value of an activity and moves towards a real knowledge of the
nature of that activity and its outcomes. The belief in itself can only be a
starting point. The important message is that the more people are involved in
research at whatever level the more likely it will be that meaningful changes
to practice will be introduced.
Bain, B.A., Dollaghan, C.A., (1991) Clinical Forum:
Treatment efficacy – The notion of clinically significant change Language Speech and Hearing Services in
Schools 22, 264-270.
Egger, M., Davey Smith, G., Altman, D. (2001). Systematic reviews in health care:
Meta-analysis in context. London: BMJ.
Enderby, P., & Emerson, J. (1996). Speech and language
therapy - does it work? British Journal
of Medicine, 312, 1655-1658.
Gallagher, T., (1998) Treatment research in speech,
language and swallowing: lessons from child language disorders Folia Phoniatr Logop 50, 165-182
Goldstein,H., Hockenburger, E., (1991). Significant
progress in child language intervention: An 11 year retrospective Research into Developmental Disabilities 12,
401-424
Guralnick, M. J. (1988). Efficacy in early childhood
intervention programs. In S. J. Odom, & M.B. Karnes (Eds.), Early Intervention for Infants and Children
with Handicaps. Baltimore, MD: Paul H. Brookes.
Johnson, C.J., Beitchman, J.H., Young, A., Escobar, M.,
Atkinson, L., Wilson, B., Brownlie, E.B., Douglas, L., Taback, N., Lam, I.
& Wang, M. (1999). Fourteen-year follow-up of children with and without
speech/language impairments: speech/language stability and outcomes. Journal of Speech, Language and Hearing
Research, 42, 744-761.
Kazdin, A.E., (1999). The meanings and measurement of
clinical significance Journal of
Consulting and Clinical Psychology 67,3,332-339.
Law, J. (1997). Evaluating intervention for language
impaired children: A review of the literature. European Journal of Disorders of Communication, 32, 1-14.
Law, J. (in press)
The relationship between assessment and intervention for children with primary
language impairments. In Verhoeven L., Van Balkom H, (Eds) The Classification of Language Disorders. Chichester:
Lawrence Erlbaum.
Law, J., Boyle, J., Harris, F., Harkness, A., Nye, C. (1998). Screening for Speech and
Language Delay: A Systematic Review of the Literature. Health Technology Assessment. 2(9) 1-184.
Law J, Garrett Z, Nye C. (2003) Speech and language
therapy interventions for children with primary speech and language delay or
disorder (Cochrane Review). In: The Cochrane Library, Issue 3, 2003. Oxford:
Update Software.
Leonard, L., (1998). The nature and efficacy of treatment.
In Laurence B. Leonard (Ed). Children
with specific language impairment (pp 193-210) Cambridge, MA: MIT Press.
Mclean, L. K., & Woods Cripe, J. W. (1997). The
effectiveness of early intervention for children with communication disorders.
In M. J. Guralnick (Ed.), The
Effectiveness of Early Intervention. Baltimore, MD: Paul H. Brookes.
Nye, C., Foster, S. H., &
Seaman, D. (1987). Effectiveness of language intervention with
language/learning disabled children Journal
of Speech and Hearing Research 52, 348-357.
Parsons, S., Law, J., Gascoigne, M.
(in press) Teaching Receptive Vocabulary to Children with Specific Language
Impairment: a Curriculum Based Approach Child
Language Teaching and Therapy.
Rosenshine, B., Meister, C. (1994)
Reciprocal Teaching: A review of the literature Review of Educational Research 64,4,479-530.
[1] Professor James Law, Department of Language and Communication Science, City University, Northampton Square, London EC1V OHB, UK e: j.c.law@city.ac.uk
[2] The review can be read on full on the Cochrane
Collaboration website; Law J, Garrett Z, Nye C. Speech and language therapy
interventions for children with primary speech and language delay or disorder
(Cochrane Review). In: The Cochrane Library, Issue 3, 2003. Oxford: Update
Software