In our review, we identified recently published (2009 forward) publications dealing with novel instruments developed and validated to measure health literacy. The review followed two main objectives. In the first place, we examined how the measurement of health literacy proceeded in recent years particularly emphasizing whether novel instruments consider existing recommendations of the scientific community on features an instrument measuring health literacy should cover. In addition, we analyzed the reporting quality of the identified papers dealing with the development of health literacy measurement tools.
Our analysis resulted in six major findings, which extend the prior knowledge on health literacy measurement.
First of all, we examined an increasing use of multidimensional constructs to measure health literacy. Especially instruments with a subjective measurement format address numerous domains of health literacy such as patient-provider encounter; interaction with the health care system; rights and responsibilities; health information-seeking; understanding, processing, and using healthcare information as well as communication with healthcare professionals [8, 16, 36, 37, 45, 48, 50]. In this regard, earlier critiques towards the one-dimensional measurement modes usually used in health literacy measurement are taken into consideration when developing novel instruments [12]. This in turn allows a more in depth and comprehensive operationalization of the dynamic construct “health literacy” and helps to improve the measurement.
Furthermore, we found that almost all instruments apply a multidimensional measurement of health literacy by principally assessing print literacy and numeracy and in some cases adding oral literacy. Previous reviews dealing with health literacy measurement tools emphasized the lack of instruments integrating communication skills (oral literacy) in the health literacy construct [17]. To fill this gap, three novel instruments containing oral literacy were developed and validated in the search period of our review (2009 forward) [16, 36, 50]. This result further indicates that newly developed instruments take the recommendations of the academic circle into consideration.
In addition we identified that there is a trend towards the use of objective (task based) and subjective (self-report based) measurement approaches in a mixed manner. Scholars using this mixed measurement approach often apply already existing health literacy screeners (e.g. SBSQ; NVS) and develop additional item batteries [8, 56, 59]. Principally the mixed measurement approach offers advantages by broadening the health literacy concept and enabling researchers to address multiple skills. However, studies using this approach in our review found a weakness of coherence between the underlying constructs measured by the different approaches. This subsequently results in limited correlation between the measurement approaches [8, 52, 53]. Consequently, these results should be taken into consideration when using the mixed measurement approach.
A further striking finding is that regardless of the used measurement approach, scholars do not sufficiently explain why they are choosing a certain type of measurement. According to Abel, the first issue in the circle of instrument development is to determine the purpose of the instrument by answering the “what for” question. As soon as the given theoretical context and setting is clear, ideas on the way of measurement can be developed systematically [62]. If the reason for a certain approach is not clearly determined, the development of a structured and comparable procedure to measure health literacy will be hard to achieve.
Finally, there is an extensive use of assessment formats modeled on already existing instruments such as the REALM or the TOFHLA inserting mostly straightforward additions [37, 38, 40, 44, 47]. Since these instruments have many weaknesses, researchers are calling for the development and use of new measurement approaches to avoid stagnation [17].
The appraisal of the reporting quality of publications dealing with the development and validation of health literacy indices has yielded mixed findings. Some domains such as the description of the article background and presentation and interpretation of results are reported thoroughly, while other domains addressing methodological properties have received less consideration. Overall, the identified papers included in the review demonstrate a lack of compliance with reporting guidelines especially for methodological issues such as psychometric properties of the developed instruments, sample selection strategy and response rate presentation. These findings are in line with previous research stating that key survey characteristics in health care literature in general [63, 64] and in health literacy research in particular [13] are often underreported. Although Jordan and colleges had already identified these weaknesses in their review considering measurement tools published between 1990 and 2008 [13] only few improvements are noticeable. Especially the reporting on the psychometric properties (reliability, validity) of the instruments is still not appropriate in nearly one third of all instruments. Additionally more than two third of the articles neither mention the issue of instrument scoring nor discuss the significance of non-response in the study setting. These findings demonstrate potential for further enhancements in improving health literacy research.
From an overall perspective, almost all identified instruments apply a multi-dimensional measurement (often print and numeracy literacy) and the majority utilizes a mixed measurement approach (objective and subjective measurement) with a multidimensional construct enhancing the comprehensiveness of tools measuring health literacy. Nevertheless, there is no clear indication of the demanded “consensus” on health literacy measurement. This is mainly because there have been only minor developments among the measurement formats, as can be seen in the increased use of earlier instruments, even though the academic world is calling for new instruments [17].
To continuously advance the field of health literacy measurement work should proceed on several fronts. Though there is currently a huge effort to improve the more comprehensive measurement of health literacy, the format of measurement generally relies on already existing approaches such as the cloze technique (used in the “The test of functional health literacy in adults” (TOFHLA)) or word recognition (used in “Rapid estimate of adult literacy in Medicine” (REALM)) [37, 38, 40, 44, 47]. Therefore, future health literacy research should strongly emphasize the development of new measurement approaches such as skill-based concepts with a generic approach [36, 50]. Here, the use of vignettes assessing ones abilities in a daily life setting could be an innovative step towards an approach that is already being used for measuring mental health literacy [65]. Consideration of measurement formats used in the field of information literacy could be also of great interest as they focus on the handling of information [66, 67]. Of course, these need to be tailored to the capacity of lay people.
Apart from the issue of originality, it would be necessary to reflect more closely on the combination of objective and subjective measurement instruments, thus current studies show less coherence. Though the limited reporting guideline compliance of health literacy instruments was identified by Jordan and colleges before [13] our analysis displays similar findings. Especially the poor reporting of the scoring methods and the weaknesses among the currently used procedures to determine construct validity need to be improved. Thus, construct validity is most often measured by comparing the instrument with screeners assessing functional literacy derived from standardized literacy tests without taking into account that health literacy is a dynamic and comprehensive construct and therefore not comparable with tests. The described procedure does not contribute to the qualitative improvement of health literacy indices but increases a path dependency. The consequences are recognizable among newly developed instruments in European countries often simply translating literacy based screeners developed in English speaking countries [32, 33] without considering cultural and institutional differences.
In considering such recommendations, certain limitations should be noticed regarding our review. Although we followed the PRISMA guidelines when performing our systematic review and used MESH terms and key words, we may have missed relevant literature. Furthermore, there was no reporting guideline available that provided a scoring scheme for the reporting quality. As a consequence we could not grade the reporting quality of the identified articles resulting in a descriptive description of the results. Finally, the appraisal of health literacy instruments was limited as the item batteries and scoring methods were not always available despite a direct request to the authors.
Apart from this, our review exhibits certain strengths such as the compliance to guidelines when performing the literature search, data selection, analysis and appraisal of the reporting quality of the identified articles.
