Using structured and open-ended procedures for eliciting data on learners': metacognitive knowledge : a qualitative comparative study

Abstract

This chapter reports on findings from an exploratory study which sought to analyze the concurrent validity of various structured and unstructured procedures as methods for collecting data on metacognitive knowledge (MK), commonly known as beliefs. Four self-directed learners (SDL) were administered a set of both, structured and open-ended instruments within a period of 4 weeks, including: an autobiography, an interview, a structured questionnaire on self-concept and language learning, a speaking questionnaire, a scenario exercise and a speaking strategies checklist administered after undertaking two oral tasks. Each of the structured instruments was followed by focused interviews. A qualitative mixed approach was adopted to code and analyze beliefs, which compared profiles obtained with each method, and subsequently, percentages of agreement between the beliefs gathered with different instruments were also calculated. The resulting comparisons suggested coincident beliefs across instruments was fairly consistent, which was taken as evidence of the concurrent validity of the structured instruments. While some of the inconsistencies were to be attributed to the instruments used, others seemed to relate to the nature of MK itself. Advantages and drawbacks associated with each instrument are discussed in light of the results obtained, which has implications for both instructional and research practices.