https://doi.org/10.4081/ripppo.2023.687
Validation of the client satisfaction questionnaire: a pilot psychometric analysis in Portuguese routine practice
Submitted: February 8, 2023
Accepted: June 4, 2023
Published: July 27, 2023
Accepted: June 4, 2023
Abstract Views: 1915
PDF: 515
HTML: 57
HTML: 57
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.
Authors
Treatment satisfaction has been found to have good therapeu- tic results in psychotherapy, and the 18-item version of the client satisfaction questionnaire (CSQ-18) is one of the most widely used measures to evaluate it. This study sought to carry out a pilot analysis of the psychometric’s properties and factorial structure, as well as validate the CSQ-18’s applicability to the Portuguese population within the context of routine clinical practice. It also sought to explore the association that treatment satisfaction has with general symptoms, therapeutic alliance, and expectations at an early stage of psychotherapy. The sample comprised 98 clients who were undergoing psychotherapy in a routine practice context. All these clients completed self-reported measures for symptom assessment and therapeutic relationship, namely the clinical out- come routine evaluation-outcome measure, working alliance in- ventory, and credibility/expectancy questionnaire, in addition to the CSQ-18. The semi-confirmatory factorial analysis demon- strated that the CSQ-18 has good psychometric properties and re- vealed an association between treatment satisfaction and therapeutic alliance. The results corroborate the findings of other versions of the measure and present a good adjustment model for the semi-confirmatory factorial analysis.
Attkisson, C. C., & Zwick, R. (1982). The client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcome. Evaluation and Program Planning, 5(3), 233-237. doi: 10.1016/0149-7189(82)90074-X.
DOI: https://doi.org/10.1016/0149-7189(82)90074-X
Baglin, J. (2014). Improving your exploratory factor analysis for ordinal data: a demonstration using FACTOR. Practical Assessment, Research and Evaluation, 19(5). doi: 10.7275/dsep-4220.
Barkham, M., Bewick, B., Mullin, T., Gilbody, S., Connell, J., Cahill, J., Mellor-Clark, J., Richards, D., Unsworth, G., & Evans, C. (2013). The CORE-10: a short measure of psychological distress for routine use in the psychological therapies. Counselling and Psychotherapy Research, 13(1), 3-13. doi: 10.1080/14733145.2012.729069.
DOI: https://doi.org/10.1080/14733145.2012.729069
Blenkiron P., Hammill C.A. (2003). What determines patients' satisfaction with their mental health care and quality of life? Postgraduate Medical Journal, 79(932):337-40. doi: 10.1136/pmj.79.932.337. PMID: 12840123; PMCID: PMC1742729.
DOI: https://doi.org/10.1136/pmj.79.932.337
Buffini, K. B., & Gordon, M. (2015). One-to-one support for crisis intervention using online synchronous instant messaging: evaluating working alliance and client satisfaction. British Journal of Guidance and Counselling, 43(1), 105-116. doi: 10.1080/03069885.2014.987723.
DOI: https://doi.org/10.1080/03069885.2014.987723
Brown, T. (2015). Confirmatory factor analysis for applied research. New York, Guilford Publications.
Carlson, M. J., & Gabriel, R. M. (2001). Patient satisfaction, use of services, and one-year outcomes in publicly funded substance abuse treatment. Psychiatric Services, 52(9), 1230-1236. doi: 10.1176/appi.ps.52.9.1230.
DOI: https://doi.org/10.1176/appi.ps.52.9.1230
Connell, J. & Barkham, M. (2007). CORE-10 user manual, version 1.1. Rugby, CORE System Trust & CORE Information Management Systems Ltd.
Constantino, M. J., Arnkoff, D. B., Glass, C. R., Ametrano, R. M., & Smith, J. Z. (2011). Expectations. Journal of Clinical Psychology, 67(2), 184-192. doi: 10.1002/jclp.20754.
DOI: https://doi.org/10.1002/jclp.20754
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Practical Assessment, Research and Evaluation, 10. doi: 10.7275/jyj1-4868.
Devilly, G. J., & Borkovec, T. D. (2000). Psychometric properties of the credibility/expectancy questionnaire. Journal of Behavior Therapy and Experimental Psychiatry, 31(2), 73-86. doi: 10.1016/S0005-7916(00)00012-4.
DOI: https://doi.org/10.1016/S0005-7916(00)00012-4
Donker, T., Bennett, K., Bennett, A., Mackinnon, A., Van Straten, A., Cuijpers, P., Christensen, H., & Griffiths, K. M. (2013). Internet-delivered interpersonal psychotherapy versus internet-delivered cognitive behavioral therapy for adults with depressive symptoms: Randomized controlled noninferiority trial. Journal of Medical Internet Research, 15(5), e82. doi: 10.2196/jmir.2307.
DOI: https://doi.org/10.2196/jmir.2307
Donovan, D. M., Kadden, R. M., DiClemente, C. C., & Carroll, K. M. (2002). Client satisfaction with three therapies in the treatment of alcohol dependence: results from Project MATCH. The American Journal of Addiction, 11, 291-307. doi: 10.1080/1055049029008809.
DOI: https://doi.org/10.1080/10550490290088090
Draper, M., Cohen, P., & Buchan, H. (2001). Seeking consumer views: what use are results of hospital patient satisfaction surveys? International journal for quality in health care, 13, 463-468.
DOI: https://doi.org/10.1093/intqhc/13.6.463
Egede, L.E., Acierno, R., Knapp, R.G., Walker, R.J., Payne, E.H., & Frueh, B.C.(2016) Psychotherapy for Depression in Older Veterans Via Telemedicine: Effect on Quality of Life, Satisfaction, Treatment Credibility, and Service Delivery Perception. Journal of Clinical Psychiatry, 77(12):1704-1711. doi: 10.4088/JCP.16m10951. PMID: 27835713.
DOI: https://doi.org/10.4088/JCP.16m10951
European Foundation for Quality Management. (2013). EFQM Excelence Model 2013. Brussels: EFQM.
Falkenström, F., Hatcher, R. L., Skjulsvik, T., Larsson, M. H., & Holmqvist, R. (2015). Development and validation of a 6-item working alliance questionnaire for repeated administrations during psychotherapy. Psychological Assessment, 27(1), 169-183. doi: 10.1037/pas0000038.
DOI: https://doi.org/10.1037/pas0000038
Field, A. (2017). Discovering Statistics Using IBM SPSS Statistics (5ºed). London: SAGE Publications.
Ferrando, P. J., & Lorenzo-Seva, U. (2017). Program FACTOR at 10: origins, development and future directions. Psicothema, 29(2), 236-240. doi: 10.7334/psicothema2016.304.
Hanson, K., Yip, W. C. & Hsiao, W. (2004) The impact of quality on the demand for outpatient services in Cyprus. Health Economics, 13 (12), 1167-1180. doi: 10.1002/hec.898.
DOI: https://doi.org/10.1002/hec.898
Horvath, A. O. and Greenberg, L. S. 1989. Development and validation of the Working Alliance Inventory. Journal of Counseling Psychology, 36: 223–233.
DOI: https://doi.org/10.1037/0022-0167.36.2.223
Howard, P.B., Rayens, M.K., El-Mallakh, P., & Clark, J.J. (2007). Predictors of satisfaction among adult recipients of medicaid mental health services. Archives of psychiatric nursing 21(5):257-69. doi: 10.1016/j.apnu.2007.06.004. PMID: 17904483.
DOI: https://doi.org/10.1016/j.apnu.2007.06.004
Hundt, Na., Armento, M. E. A., Porter, B., Cully, J., Kunik, M. E., & Stanley, M. (2013). Predictors of treatment satisfaction among older adults with anxiety in a primary care psychology program. Evaluation and Program Planning, 37, 58-63. doi: 10.1016/j.evalprogplan.2013.01.003.
DOI: https://doi.org/10.1016/j.evalprogplan.2013.01.003
Hutchison, M., Russell, B. S., Gans, K. M., & Starkweather, A. R. (2022). Online administration of a pilot mindfulness-based intervention for adolescents: feasibility, treatment perception and satisfaction. Current Psychology, 1-13. doi: 10.1007/s12144-022-03025-x.
DOI: https://doi.org/10.1007/s12144-022-03025-x
Jenkins-Guarnieri, M. A., Pruitt, L. D., Luxton, D. D., & Johnson, K. (2015). Patient perceptions of telemental health: systematic review of direct comparisons to in-person psychotherapeutic treatments. Telemedicine and E-Health, 21(8), 652-660. doi: 10.1089/tmj.2014.0165.
DOI: https://doi.org/10.1089/tmj.2014.0165
Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.
Keleher, B., Oakman, J. M., Capobianco, K., & Mittelstaedt, W. H. (2019). Basic psychological needs satisfaction, working alliance, and early termination in psychotherapy. Counselling Psychology Quarterly, 32(1), 64-80. doi: 10.1080/09515070.2017.1367271.
DOI: https://doi.org/10.1080/09515070.2017.1367271
Kooistra, L. C., Ruwaard, J., Wiersma, J. E., van Oppen, P., van der Vaart, R., van Gemert-Pijnen, J. E. W. C., & Riper, H. (2016). Development and initial evaluation of blended cognitive behavioural treatment for major depression in routine specialized mental health care. Internet Interventions, 4, 61-71. doi: 10.1016/j.invent.2016.01.003.
DOI: https://doi.org/10.1016/j.invent.2016.01.003
Larsen, D. L., Attkisson, C. C., Hargreaves, W. A., & Nguyen, T. D. (1979). Assessment of client/patient satisfaction: development of a general scale. Evaluation and Program Planning, 2(3), 197-207. doi: 10.1016/0149-7189(79)90094-6.
DOI: https://doi.org/10.1016/0149-7189(79)90094-6
Lorenzo-Seva, U., & Ferrando, P. J. (2006). FACTOR: a computer program to fit the exploratory factor analysis model. Behavior Research Methods, 38(1), 88-91. doi: 10.3758/bf03192753.
DOI: https://doi.org/10.3758/BF03192753
Machado, P., & Horvath, A. (1999). Inventário de aliança terapêutica – WAI. In M. R. Simões, M. M. Gonçalves & L. S. Almeida (Eds.), Testes e Provas Psicológicas em Portugal (Vol. 2, pp. 87-94). Braga: APPORT/SHO.
Matsubara, C., Green, J., Astorga, L. T., Daya, E. L., Jervoso, H. C., Gonzaga, E. M., & Jimba, M. (2013). Reliability tests and validation tests of the client satisfaction questionnaire (CSQ-8) as an index of satisfaction with childbirth-related care among Filipino women. BMC Pregnancy and Childbirth, 13, 235. doi: 10.1186/1471-2393-13-235.
DOI: https://doi.org/10.1186/1471-2393-13-235
Miller, L., & Weissman, M. (2002). Interpersonal psychotherapy delivered over the telephone to recurrent depressives: a pilot study. Depression and Anxiety, 16(3), 114-117. doi: 10.1002/da.10047.
DOI: https://doi.org/10.1002/da.10047
Palacios, J. E., Richards, D., Palmer, R., Coudray, C., Hofmann, S. G., Palmieri, P. A., & Frazier, P. (2018). Supported internet-delivered cognitive behavioral therapy programs for depression, anxiety, and stress in university students: open, non-randomised trial of acceptability, effectiveness, and satisfaction. JMIR Mental Health, 5(4), e11467. doi: 10.2196/11467.
DOI: https://doi.org/10.2196/11467
Priebe S, Golden E, McCabe R, Reininghaus U. (2012). Patient-reported outcome data generated in a clinical intervention in community mental health care – Psychometric properties. Biomed Central Psychiatr, 12, 113. doi: 10.1186/1471-244X-12-113.
DOI: https://doi.org/10.1186/1471-244X-12-113
Roberts, R. E., Atrkisson, C. C., & Mendias, R. M. (1984). Assessing the client satisfaction questionnaire in English and Spanish. Hispanic Journal of Behavioral Sciences, 6(4), 385-396. doi: 10.1177/07399863840064004.
DOI: https://doi.org/10.1177/07399863840064004
Roe, D., Dekel, R., Harel, G., Fennig, S., & Fennig, S. (2006). Clients’ feelings during termination of psychodynamically oriented psychotherapy. Bulletin of the Menninger Clinic, 70(1), 68-81. doi: 10.1521/bumc.2006.70.1.68.
DOI: https://doi.org/10.1521/bumc.2006.70.1.68
Rosenvinge, J. H., & Klusmeier, A. K. (2000). Treatment for eating disorders from a patient satisfaction perspective: a Norwegian replication of a British study. European Eating Disorders Review, 8(4), 293-300. doi: 10.1002/1099-0968(200008)8:4<293::AID-ERV346>3.0.CO;2-4.
DOI: https://doi.org/10.1002/1099-0968(200008)8:4<293::AID-ERV346>3.0.CO;2-4
Rumpold, G., Doering, S., Smrekar, U., Schubert, C., Koza, R., Schatz, D. S., Bertl-Schuessler, A., Janecke, N., Lampe, A., & Schuessler, G. (2005). Changes in motivation and the therapeutic alliance during a pretherapy diagnostic and motivation-enhancing phase among psychotherapy outpatients. Psychotherapy Research, 15(1–2), 117-127. doi: 10.1080/10503300512331327092.
DOI: https://doi.org/10.1080/10503300512331327092
Sabourin, S., Pérusse, D., & Gendreau, P. (1989). Psychometric properties of the French-Canadian version of the client of Mental Health Services Satisfaction Questionnaire (QSC-8 et QSC-18B). Canadian Journal of Behavioural Science, 21(2), 147-159. doi: 10.1037/h0079853. [Article in French].
DOI: https://doi.org/10.1037/h0079853
Säilä, T., Mattila, E., Kaila, M., Aalto, P., & Kaunonen, M. (2008). Measuring patient assessments of the quality of outpatient care: a systematic review. Journal of Evaluation in Clinical Practice, 14(1), 148-154. doi: 10.1111/j.1365-2753.2007.00824.x.
DOI: https://doi.org/10.1111/j.1365-2753.2007.00824.x
Sales, C. M. D., Moleiro, C. M. de M., Evans, C., & Alves, P. C. G. (2012). The Portuguese version of the CORE-OM: translation, adaptation, and preliminary study of psychometric properties. Revista de Psiquiatria Clínica, 39(2), 54-59. doi: 10.1590/S0101-60832012000200003. [Article in Portuguese].
DOI: https://doi.org/10.1590/S0101-60832012000200003
Schulte, S. J., Meier, P. S., & Stirling, J. (2011). Dual diagnosis clients’ treatment satisfaction - a systematic review. BMC Psychiatry, 11. doi: 10.1186/1471-244X-11-64.
DOI: https://doi.org/10.1186/1471-244X-11-64
Smeets, R. J. E. M., Beelen, S., Goossens, M. E. J. B., Schouten, E. G. W., Knottnerus, J. A., & Vlaeyen, J. W. S. (2008). Treatment expectancy and credibility are associated with the outcome of both physical and cognitive-behavioral treatment in chronic low back pain. Clinical Journal of Pain, 24(4), 305-315. doi: 10.1097/AJP.0b013e318164aa75.
DOI: https://doi.org/10.1097/AJP.0b013e318164aa75
Smith, D., Roche, E., O’Loughlin, K., Brennan, D., Madigan, K., Lyne, J., Feeney, L., & O’Donoghue, B. (2014). Satisfaction with services following voluntary and involuntary admission. Journal of Mental Health, 23(1), 38-45. doi: 10.3109/09638237.2013.841864.
DOI: https://doi.org/10.3109/09638237.2013.841864
Silva, S., Barbosa, E., Salgado, J., & Cunha, C. (2021). Portuguese validation of the credibility/expectancy questionnaire in routine practice. Research in Psychotherapy: Psychopathology, Process and Outcome, 24(1). https://doi.org/10.4081/ripppo.2021.495
DOI: https://doi.org/10.4081/ripppo.2021.495
Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological Methods, 16(2), 209-220.
DOI: https://doi.org/10.1037/a0023353
Vázquez, F. L., Torres, Á., Otero, P., Blanco, V., & Clifford Attkisson, C. (2019). Psychometric properties of the Castilian Spanish version of the client satisfaction questionnaire (CSQ-8). Current Psychology, 38(3), 829-835. doi: 10.1007/s12144-017-9659-8.
DOI: https://doi.org/10.1007/s12144-017-9659-8
Warnecke, I., Reiss, N., Werner, A., Luka-Krausgrill, U., & Rohrmann, S. (2020). Treatment satisfaction and therapeutic alliance in three different treatment groups for test anxiety. Journal of Cognitive Psychotherapy, 34(2), 131-147. doi: 10.1891/JCPSY-D-18-00027.
DOI: https://doi.org/10.1891/JCPSY-D-18-00027
World Health Organization (2007). People-Centred Health Care: A policy framework. Geneva, Switzerland: WHO Document Production Services.
How to Cite
Silva, S., Basto, I., Salgado, J., & Cunha, C. (2023). Validation of the client satisfaction questionnaire: a pilot psychometric analysis in Portuguese routine practice. Research in Psychotherapy: Psychopathology, Process and Outcome, 26(2). https://doi.org/10.4081/ripppo.2023.687
Copyright (c) 2023 the Author(s)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
PAGEPress has chosen to apply the Creative Commons Attribution NonCommercial 4.0 International License (CC BY-NC 4.0) to all manuscripts to be published.
Similar Articles
- Maria Eugenia Moneta, Horst Kaechele, A theoretical and clinical perspective of an embodied view in psychotherapy of somatic symptoms disorders , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 26 No. 2 (2023)
- Bruno Faustino, António Branco Vasco, Giancarlo Dimaggio, Ana Nunes da Silva, Sophie Seromenho, Self-assessment of patterns of subjective experience: development and psychometric study of the States of Mind Questionnaire , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 23 No. 3 (2020)
- Emanuela Saita, Martina Tramontano, Navigating the complexity of the therapeutic and clinical use of photography in psychosocial settings: a review of the literature , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 21 No. 1 (2018)
- Chiara Manfredi, Gabriele Caselli, Federica Pescini, Martina Rossi, Daniela Rebecchi, Giovanni Maria Ruggiero, Sandra Sassaroli, Parental criticism, self-criticism and their relation to depressive mood: an exploratory study among a non-clinical population , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 19 No. 1 (2016): Special issue on Qualitative and Quantitative Research in Child and Adolescent Psychotherapy: part 2
- Sarah Abraham, Elisabeth Edginton, David Cottrell, Sandy Tubeuf, Measuring health-related quality of life measures in children: lessons from a pilot study , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 25 No. 1 (2022)
- Antonino Carcione, Marta Santonastaso, Francesca Sferruzza, Ilaria Riccardi, Esoteric power, useless, useful: considerations about dreams in cognitive-behavioural therapy , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 24 No. 2 (2021): SPECIAL ISSUE "Working on dreams, from psychotherapy to neuroscience"
- Gretchen Beiza, Claudia Capella, Denise Dussert, Loreto Rodrìguez, Daniela Aguila, Carolina Gutiérrez, Ximena Lama, Institutionalized adolescents in therapy: narratives of psychotherapy and healing from sexual abuse , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 18 No. 2 (2015): Special issue on Qualitative and Quantitative Research in Child and Adolescent Psychotherapy: part 1
- Eunice Barbosa, Maria Amendoeira, Tiago Ferreira, Ana Sofia Teixeira, José Pinto-Gouveia, Joào Salgado, Immersion and distancing across the therapeutic process: relationship to symptoms and emotional arousal , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 20 No. 2 (2017)
- Marianna Liotti, Alice Fiorini Bincoletto, Fabiola Bizzi, Marta Tironi, Simone Charpentier Mora, Donatella Cavanna, Guido Giovanardi, Elliot Jurist, Anna Maria Speranza, Vittorio Lingiardi, Annalisa Tanzilli, The catcher in the mind: validation of the brief-mentalized affectivity scale for adolescents in the Italian population , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 26 No. 3 (2023)
- Michael J. Lambert, Gianluca Lo Coco, Simple methods for enhancing patient outcome in routine care: Measuring, monitoring, and feedback , Research in Psychotherapy: Psychopathology, Process and Outcome: Vol. 16 No. 2 (2013)
<< < 5 6 7 8 9 10 11 12 13 14 > >>
You may also start an advanced similarity search for this article.
