Discourse-Level Communication Success in Aphasia: Unveiling Its Significance through Observer's Ratings

Abstract

Audrey Holland (1982) compared test scores to observers' ratings of conversational communicative success in people with aphasia (PWA). This springboarded a body of evidence employing observers to rate discourse. We review the utility of those ratings for assessing PWA's communication success. A traditional literature review identified 16 articles involving naive or trained raters assessing PWAs' communicative success across discourse genres. Another 10 articles reported ratings over time. Collectively, these studies evaluated 349 PWAs. Four studies utilized observers to rate the success of PWA's conversations. Eight studies that reported observers' ratings on other discourse genres found that multimodal communication and facilitative contexts improved success, and ratings of informativeness and comfort related to objective discourse analysis measures. Nine of 10 studies examining treatment effects found that communicative success ratings captured improvements. Observers' ratings provide social validity by reliably assessing the discourse-level communicative success of PWA. Ratings correlated with standardized diagnostic and objective discourse metrics but provided a window into factors that affect communicative success, including the degree to which communication is interactive, multimodal, and contextual. Integrating observers' ratings of discourse success at pretreatment may help identify supports or barriers to successful communication, facilitate individualization of treatments, and offer social validity of change.

Keywords

functional communication - discourse - aphasia - raters - observers

Publication History

Article published online:
03 October 2024

• References

• Anbar, S. (2022). Goddess Aphasia: A Stroke Survivor and His Dual Muse. Stillwater River Publications
• Ardila, A. (2014). Aphasia Handbook. Florida International University. Accessed August 15, 2024 at: https://aalfredoardila.files.wordpress.com/2013/07/ardila-a-2014-aphasia-handbook-miami-fl-florida-international-university2.pdf
• Armstrong, E., & Ferguson, A. (2010). Language, meaning, context, and functional communication. Aphasiology, 24(4), 480–496
• Azios, J. H., Archer, B., Simmons-Mackie, N., Raymer, A., Carragher, M., Shashikanth, S., & Gulick, E. (2022). Conversation as an outcome of aphasia treatment: a systematic scoping review. American Journal of Speech-Language Pathology, 31(6), 2920–2942
• Ballard, K. J., & Thompson, C. K. (1999). Treatment and generalization of complex sentence production in agrammatism. Journal of Speech, Language, and Hearing Research: JSLHR, 42(3), 690–707
• Bartlett, M. R., Fink, R. B., Schwartz, M. F., & Linebarger, M. (2007). Informativeness ratings of messages created on an AAC processing prosthesis. Aphasiology, 21(5), 475–498
• Behrns, I., Wengelin, A., Broberg, M., & Hartelius, L. (2009). A comparison between written and spoken narratives in aphasia. Clinical Linguistics & Phonetics, 23(7), 507–528
• Bock, K., & Levelt, W. (1994). Language production: grammatical encoding. In: M. A. Gernsbacher (Ed.), Handbook of Psycholinguistics (pp. 945–984). Academic Press
• Boyle, M., & Coelho, C. A. (1995). Application of semantic feature analysis as a treatment for aphasic dysnomia. American Journal of Speech-Language Pathology, 4(4), 94–98
• Bryant, L., Brunner, M., & Hemsley, B. (2020). A review of virtual reality technologies in the field of communication disability: implications for practice and research. Disability and Rehabilitation. Assistive Technology, 15(4), 365–372
• Busby, N., Hillis, A. E., Bunker, L., Rorden, C., Newman-Norlund, R., Bonilha, L., Meier, E., Goldberg, E., Hickok, G., Yourganov, G., & Fridriksson, J. (2023). Comparing the brain-behaviour relationship in acute and chronic stroke aphasia. Brain Communications, 5(2), fcad014
• Carlomagno, S., Santoro, A., Menditti, A., Pandolfi, M., & Marini, A. (2005). Referential communication in Alzheimer's type dementia. Cortex, 41(4), 520–534
• Casilio, M., Rising, K., Beeson, P. M., Bunton, K., & Wilson, S. M. (2019). Auditory-perceptual rating of connected speech in aphasia. American Journal of Speech-Language Pathology, 28(2), 550–568
• Clark, H. H. (2005). Coordinating with each other in a material world. Discourse Studies, 7(4–5), 507–525
• Code, C. (2008). A short history of the past and future of aphasia therapy. University of Exeter. Doi: 10.13140/2.1.3036.3840
• Copeland, M. (1989). An assessment of natural conversation with Broca's aphasics. Aphasiology, 3(4), 301–306
• Cupit, J., Rochon, E., Leonard, C., & Laird, L. (2010). Social validation as a measure of improvement after aphasia treatment: its usefulness and influencing factors. Aphasiology, 24(11), 1486–1500
• de Beer, C., Carragher, M., van Nispen, K., Hogrefe, K., de Ruiter, J. P., & Rose, M. L. (2017). How much information do people with aphasia convey via gesture? American Journal of Speech-Language Pathology, 26(2), 483–497
• Dell, G. S., Chang, F., & Griffin, Z. M. (1999). Connectionist models of language production: lexical access and grammatical encoding. Cognitive Science, 23(4), 517–542
• Dietz, A., Vannest, J., Maloney, T., Altaye, M., Holland, S., & Szaflarski, J. P. (2018). The feasibility of improving discourse in people with aphasia through AAC: clinical and functional MRI correlates. Aphasiology, 32(6), 693–719
• Doedens, W. J., & Meteyard, L. (2022). What is functional communication? A theoretical framework for real-world communication applied to aphasia rehabilitation. Neuropsychology Review, 32(4), 937–973
• Doyle, P. J., Goldstein, H., & Bourgeois, M. S. (1987). Experimental analysis of syntax training in Broca's aphasia: a generalization and social validation study. The Journal of Speech and Hearing Disorders, 52(2), 143–155
• Doyle, P. J., Tsironas, D., Goda, A. J., & Kalinyak, M. (1996). The relationship between objective measures and listeners' judgments of the communicative informativeness of the connected discourse of adults with aphasia. American Journal of Speech-Language Pathology, 5(3), 53–60
• Edmonds, L. A., Nadeau, S. E., & Kiran, S. (2009). Effect of verb network strengthening (VNeST) on lexical retrieval of content words in sentences in persons with aphasia. Aphasiology, 23(3), 402–424
• Fink, R. B., Bartlett, M. R., Lowery, J. S., Linebarger, M. C., & Schwartz, M. F. (2008). Aphasic speech with and without SentenceShaper: two methods for assessing informativeness. Aphasiology, 22(7-8), 679–690
• Gallée, J., Volkmer, A., Whitworth, A., Hersh, D., & Cartwright, J. (2024). Applications of the R.A.I.S.E. Assessment framework to support the process of assessment in primary progressive aphasia. American Journal of Speech-Language Pathology, 1–11. Doi: 10.1044/2024_AJSLP-24-00085
• Garcia, L. J., Rebolledo, M., Metthé, L., & Lefebvre, R. (2007). The potential of virtual reality to assess communication in aphasia. Topics in Language Disorders, 27(3), 272–288
• Gilmore, N., Dwyer, M., & Kiran, S. (2019). Benchmarks of significant change after aphasia rehabilitation. Archives of Physical Medicine and Rehabilitation, 100(6), 1131–1139.e87
• Goodglass, H., Kaplan, E., & Barresi, B. (2001). Boston Diagnostic Aphasia Examination (BDAE-3) (3rd ed.). Pro-Ed
• Goral, M., & Kempler, D. (2009). Training verb production in communicative context: evidence from a person with chronic non-fluent aphasia. Aphasiology, 23(12), 1383–1397
• Groenewold, R., Bastiaanse, R., Nickels, L., & Huiskes, M. (2014). Perceived liveliness and speech comprehensibility in aphasia: the effects of direct speech in auditory narratives. International Journal of Language & Communication Disorders, 49(4), 486–497
• Haley, K. L., Jacks, A., Jarrett, J., Ray, T., Cunningham, K. T., Gorno-Tempini, M. L., & Henry, M. L. (2021). Speech metrics and samples that differentiate between nonfluent/agrammatic and logopenic variants of primary progressive aphasia. Journal of Speech, Language, and Hearing Research: JSLHR, 64(3), 754–775
• Harmon, T. G., Jacks, A., Haley, K. L., & Faldowski, R. A. (2016). Listener perceptions of simulated fluent speech in nonfluent aphasia. Aphasiology, 30(8), 922–942
• Hartwigsen, G., Saur, D., Price, C. J., Baumgaertner, A., Ulmer, S., & Siebner, H. R. (2013). Increased facilitatory connectivity from the pre-SMA to the left dorsal premotor cortex during pseudoword repetition. Journal of Cognitive Neuroscience, 25(4), 580–594
• Henderson, V. W. (1992). Sigmund Freud and the diagram-maker school of aphasiology. Brain and Language, 43(1), 19–41
• Hengst, J. A. (2020). Understanding Everyday Communicative Interactions: Introduction to Situated Discourse Analysis for Communication Sciences and Disorders. Taylor & Francis Group
• Hickey, E. M., Bourgeois, M. S., & Olswang, L. B. (2004). Effects of training volunteers to converse with nursing home residents with aphasia. Aphasiology, 18(5–7), 625–637
• Hickey, E. M., & Rondeau, G. (2005). Social validation in aphasiology: Does judges' knowledge of aphasiology matter? Aphasiology, 19(3–5), 389–398
• Hickok, G., & Poeppel, D. (2007). The cortical organization of speech processing. Nature Reviews. Neuroscience, 8(5), 393–402
• Hickok, G., & Poeppel, D. (2015). Neural basis of speech perception. Handbook of Clinical Neurology, 129, 149–160
• Hogrefe, K., Ziegler, W., Weidinger, N., & Goldenberg, G. (2012). Non-verbal communication in severe aphasia: Influence of aphasia, apraxia, or semantic processing? Cortex, 48(8), 952–962. https://doi.org/10.1016/j.cortex.2011.02.022
• Holland, A., Fromm, D., Forbes, M., & MacWhinney, B. (2017). Long-term recovery in stroke accompanied by aphasia: a reconsideration. Aphasiology, 31(2), 152–165
• Holland, A. L. (1980). Communicative Abilities in Daily Living (CADL): A Test of Functional Communication. University Park Press
• Holland, A. L. (1982). Observing functional communication of aphasic adults. The Journal of Speech and Hearing Disorders, 47(1), 50–56
• Holland, A. L. (2010). Lessons from a clinical life in aphasia. Topics in Stroke Rehabilitation, 17(1), 13–19
• Holland, A. L., Fromm, D., & Wozniak, L. (2018). CADL3 Communication Activities of Daily Living (3rd ed.). Pro-ed. Accessed August 15, 2024 at: https://www.proedinc.com/Products/14650/cadl3-communication-activities-of-daily-livingthird-edition.aspx
• Holland, A. L., & Ramage, A. E. (2000). Thinking outside the (black) box. Brain and Language, 71(1), 93–95
• Hopper, T., Holland, A., & Rewega, M. (2002). Conversational coaching: treatment outcomes and future directions. Aphasiology, 16(7), 745–761
• Hula, W., McNeil, M., Doyle, P., Rubinsky, H., & Fossett, T. (2003). The inter-rater reliability of the story retell procedure. Aphasiology, 17(5), 523–528
• Jacobs, B. J. (2001). Social validity of changes in informativeness and efficiency of aphasic discourse following linguistic specific treatment (LST). Brain and Language, 78(1), 115–127
• Kagan, A. (1999). Supported Conversation for Adults with Aphasia: Methods and Evaluation. University of Toronto
• Kagan, A., Black, S. E., Duchan, F. J., Simmons-Mackie, N., & Square, P. (2001). Training volunteers as conversation partners using "supported conversation for adults with aphasia" (SCA): a controlled trial. Journal of Speech, Language, and Hearing Research: JSLHR, 44(3), 624–638
• Kertesz, A. (2007). Western Aphasia BatteryTM-Revised (WAB–RTM). Pearson. Accessed August 15, 2024 at: https://www.pearsonclinical.ca/en/products/product-master/item-176.html
• Kim, H., & Wright, H. H. (2020). Concurrent validity and reliability of the core lexicon measure as a measure of word retrieval ability in aphasia narratives. American Journal of Speech-Language Pathology, 29(1), 101–110
• Kong, A. P.H., Linnik, A., Law, S.P., & Shum, W. W.-M. (2018). Measuring discourse coherence in anomic aphasia using Rhetorical Structure Theory. International Journal of Speech-Language Pathology, 20(4), 406–421
• Kong, A. P.H., & Wong, C. W.Y. (2018). An integrative analysis of spontaneous storytelling discourse in aphasia: relationship with listeners' rating and prediction of severity and fluency status of aphasia. American Journal of Speech-Language Pathology, 27(4), 1491–1505
• Leaman, M. C., & Edmonds, L. A. (2019). Conversation in aphasia across communication partners: exploring stability of microlinguistic measures and communicative success. American Journal of Speech-Language Pathology, 28(1S), 359–372
• Leaman, M. C., & Edmonds, L. A. (2020). "By the way"… how people with aphasia and their communication partners initiate new topics of conversation. American Journal of Speech-Language Pathology, 29(1S), 375–392
• Leaman, M. C., & Edmonds, L. A. (2021). Assessing language in unstructured conversation in people with aphasia: methods, psychometric integrity, normative data, and comparison to a structured narrative task. Journal of Speech, Language, and Hearing Research: JSLHR, 64(11), 4344–4365
• Leaman, M. C., & Edmonds, L. A. (2024). Pilot results for ECoLoGiC-Tx: a new conversation-level intervention improving language in people with moderate to severe aphasia. American Journal of Speech-Language Pathology, 33(1), 153–172
• Leonard, C., Rochon, E., & Laird, L. (2008). Treating naming impairments in aphasia: Findings from a phonological components analysis treatment. Aphasiology, 22(9), 923–947. https://doi.org/10.1080/02687030701831474
• Levelt, W. J. M. (2001). Spoken word production: a theory of lexical access. Proc. Natl. Acad. Sci. USA, 98(23), 13464–13471
• Linebaugh, C. W., Shisler, R. J., & Lehner, L. (2005). CAC classics: cueing hierarchies and word retrieval: a therapy program. Aphasiology, 19(1), 77–92
• Lustig, A. P., & Tompkins, C. A. (2002). A written communication strategy for a speaker with aphasia and apraxia of speech: treatment outcomes and social validity. Aphasiology, 16(4–6), 507–521
• Macwhinney, B., Fromm, D., Forbes, M., & Holland, A. (2011). AphasiaBank: methods for studying discourse. Aphasiology, 25(11), 1286–1307
• Martin, N., & Dell, G. S. (2019). Maintenance versus transmission deficits: the effect of delay on naming performance in aphasia. Frontiers in Human Neuroscience, 13, 406
• Mayer, J. F., Madden, E. B., Mozeiko, J., Murray, L. L., Patterson, J. P., Purdy, M., Sandberg, C. W., & Wallace, S. E. (2024). Generalization in aphasia treatment: a tutorial for speech-language pathologists. American Journal of Speech-Language Pathology, 33(1), 57–73
• McNeil, M. R., & Prescott, T. E. (1978). Revised Token Test. PRO-ED
• Plowman, E., Hentz, B., & Ellis, C., Jr. (2012). Post-stroke aphasia prognosis: a review of patient-related and stroke-related factors. Journal of Evaluation in Clinical Practice, 18(3), 689–694
• Poeppel, D., Guillemin, A., Thompson, J., Fritz, J., Bavelier, D., & Braun, A. R. (2004). Auditory lexical decision, categorical perception, and FM direction discrimination differentially engage left and right auditory cortex. Neuropsychologia, 42(2), 183–200
• Porch, B. E. (1967). Porch Index of Communicative Ability. Consulting Psychologists Press
• Quaranta, D., Di Tella, S., Marra, C., Gaudino, S., L'Abbate, F., & Silveri, M. C. (2022). Neuroanatomical correlates of semantic features of narrative speech in semantic and logopenic variants of primary progressive aphasia. Brain Sciences, 12(7), 910
• Ramsberger, G., & Rende, B. (2002). Measuring transactional success in the conversation of people with aphasia. Aphasiology, 16(3), 337–353
• Rose, M. L., Mok, Z., & Sekine, K. (2017). Communicative effectiveness of pantomime gesture in people with aphasia. International Journal of Language & Communication Disorders, 52(2), 227–237
• Ross, K. B., & Wertz, R. T. (1999). Comparison of impairment and disability measures for assessing severity of, and improvement in, aphasia. Aphasiology, 13(2), 113–124
• Saldert, C., Fors, A., Ströberg, S., & Hartelius, L. (2010). Comprehension of complex discourse in different stages of Huntington's disease. International Journal of Language & Communication Disorders, 45(6), 656–669
• Schuell, H., Shaw, R., & Brewer, W. (1969). A psycholinguistic approach to study of the language deficit in aphasia. Journal of Speech and Hearing Research, 12(4), 794–806
• Schwartz, M. F. (2013). Theoretical analysis of word production deficits in adult aphasia. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 369(1634), 20120390
• Schwartz, M. F., Kimberg, D. Y., Walker, G. M., Brecher, A., Faseyitan, O. K., Dell, G. S., Mirman, D., & Coslett, H. B. (2011). Neuroanatomical dissociation for taxonomic and thematic knowledge in the human brain. Proc. Natl. Acad. Sci. USA, 108(20), 8520–8524
• Simmons-Mackie, N., Savage, M. C., & Worrall, L. (2014). Conversation therapy for aphasia: a qualitative review of the literature. International Journal of Language & Communication Disorders, 49(5), 511–526
• Stark, B. C., & Dalton, S. G. (2024). A scoping review of transcription-less practices for analysis of aphasic discourse and implications for future research. International Journal of Language & Communication Disorders; doi : 10.1111/1460-6984.13028
• Suting, L. B., & Mozeiko, J. (2021). Analysis of real-world language use in a person with Wernicke's aphasia. Perspectives of the ASHA Special Interest Groups, 6(3), 553–565
• Thompson, C. K., & Shapiro, L. P. (2005). Treating agrammatic aphasia within a linguistic framework: treatment of underlying forms. Aphasiology, 19(10-11), 1021–1036
• Togher, L., Douglas, J., Turkstra, L. S., Welch-West, P., Janzen, S., Harnett, A., Kennedy, M., Kua, A., Patsakos, E., Ponsford, J., Teasell, R., Bayley, M. T., & Wiseman-Hakes, C. (2023). INCOG 2.0 Guidelines for cognitive rehabilitation following traumatic brain injury, Part IV: cognitive-communication and social cognition disorders. The Journal of Head Trauma Rehabilitation, 38(1), 65–82
• Tompkins, C. A., Scharp, V. L., & Marshall, R. C. (2006). Communicative value of self cues in aphasia: a re-evaluation. Aphasiology, 20(7), 684–704
• Ulatowska, H. K., Freedman-Stern, R., Doyel, A. W., Macaluso-Haynes, S., & North, A. J. (1983). Production of narrative discourse in aphasia. Brain and Language, 19(2), 317–334. https://doi.org/10.1016/0093-934x(83)90074-3
• van Middendorp, J. J., Sanchez, G. M., & Burridge, A. L. (2010). The Edwin Smith papyrus: a clinical reappraisal of the oldest known document on spinal injuries. European Spine Journal, 19(11), 1815–1823
• Webster, J., & Morris, J. (2019). Communicative informativeness in aphasia: Investigating the relationship between linguistic and perceptual measures. American Journal of Speech-Language Pathology, 28(3), 1115–1126. https://doi.org/10.1044/2019_AJSLP-18-0256
• Wilson, S. M., Entrup, J. L., Schneck, S. M., Onuscheck, C. F., Levy, D. F., Rahman, M., Willey, E., Casilio, M., Yen, M., Brito, A. C., Kam, W., Davis, L. T., de Riesthal, M., & Kirshner, H. S. (2023). Recovery from aphasia in the first year after stroke. Brain, 146(3), 1021–1039