Jonker, Elferink-Gemser, de Roos, and Visscher (2012) investigated the effect of self-reported reflection (using the reflection subscale of the SRL-SRS questionnaire) and attainment (national vs international level) 2.5 years later in 52 elite youth athletes. Results indicated that those who became international level athletes scored higher on reflection at baseline compared with those who achieved national status in their senior years. The authors concluded with a statement referring to the value of reflection when seeking to attain senior international status later in an athlete’s development.
Looking closely at the participant characteristics, the coefficients of variation (CV) for reported sport experience (years) in those reaching international level was 25.99 and for national level achievers was 25.14. The CV reported training hours per week were 64.31 and 66.97 for international achievers and national level achievers respectively. The self-reported reflection total score junior international-senior international (4.26±0.47; CV: 11.03; n=14), Junior international-senior national (4.08±0.39; CV:9.56; n=5), Junior national-senior international (4.40±0.43; CV: 9.77; n=12) and Junior national-senior national (4.04±0.26; CV: 6.44; n=21). The junior national-senior national group was the only group demonstrating a statistical significant difference in mean total score which was lower than the 3 other groups (range of difference= 0.04-0.36). P-values for group differences were not reported limiting analytical interpretation.
The p-value is defined as ‘the probability of the observed result, plus more extreme results, if the null hypothesis were true’ (Goodman, 2008). Therefore, the rejection of the null hypothesis in the statistically significant result does not mean that the alternative hypothesis is true, as there may be more fitting hypotheses that model the data more accurately (the alternative hypothesis in this case would have been: ‘there is a difference in total reflection scores between group 4 and the remaining groups. The only logical outcome from the analysis is that the null hypothesis can be rejected i.e. ‘there was no difference between group 4 and the remaining groups’. This rejection does not lead us to the opposite conclusion as other factors play a part in the concluding statement e.g. the noise in the data could have prevented clear data analysis, the power of the analysis could have been insufficient (more participants required), the group sizes were inconsistent (true in this study) which requires either statistical controls or standardisation of group numbers. Additionally, the practical significance of the results is difficult to determine (e.g. what does a 0.04-0.36 difference in total reflection score mean in behavioural terms? Is it important? It could be argued that a qualitative or mixed methods approach would have been able to contextualise and bridge the perception-behaviour gap more accurately.
Consequently, further research is required to ascertain the effects of reflective practice on the development of athletic performance. Ideally practical significance will be better achieved if we can ascertain the effects of reflective practice on learning quality/effectiveness. The current study investigated the effect of self-reported reflection (a measure open to self-report biases) on performance level attainment. The demographic data demonstrates inconsistencies in training hours per week and years involved in the sport. Subsequently, physical factors, talent selection biases (relative age effect), logistical and personal factors could have influenced the outcome of the individuals involved in this study. Subsequently, a more critical approach to ascertaining the cause of attainment level is required to generate more accurate conclusions.
- Although this study has discovered a difference in self-reported reflection between groups, the chosen measure is open to report biases. Additionally, the practical significance of the 0.04-0.36 difference in self-reported reflection total scores needs further clarification to be meaningful.
- Further research should investigate the possibility of designing measures of the quality of reflection to improve future understanding within this area. For example, the quality of reflection could be longitudinally measured throughout the development of an athlete using a mixed method approach to both contextualise numerical data and draw awareness to other potential confounding variables. More specifically, understanding the lived experience of the participants may help researchers understand the reasons for lack of achievement when the quality of reflection is high. Utilising a targeted case study approach to further contextualise unusual numerical findings may assist the researcher in fully understanding the underlying causes. Using the analogy of an ice-berg: ‘through targeted qualitative research we may deepen our perception of what lies beneath’
Goodman, S. (2008). A dirty dozen: twelve p-value misconceptions. Paper presented at the Seminars in hematology.
Jonker, L., Elferink-Gemser, M. T., de Roos, I. M., & Visscher, C. (2012). The role of reflection in sport expertise. The sport psychologist, 26(2), 224-242.