NUSTEM’s 5 Year Impact Study: Primary Schools
2014 – 2019 Findings
The Study
NUSTEM has evaluated the long-term impact of sustained engagements with partner primary schools during our first 5 years of operation, 2014 – 2019.
Our research study in primary schools measured children’s interest and self-efficacy in science, perceptions of scientists, and aspirations for STEM jobs.
Summary
Our study has shown:
- It is possible to broaden the range of STEM careers children would consider and reduce the gendered nature of those career choices, but changing aspirations takes time. After 5 years we have made some changes to aspirations for some physical science careers such as engineer, but less impact on health science careers, and there is still a gap between the aspirations of different genders.
- It is possible to reduce stereotypical views of scientists, shown by changing perceptions of scientists among girls. Again it takes time to make these changes. Changing perceptions of scientists is important as the more closely these perceptions match the girls’ descriptions of themselves (self-identity as kind and hardworking) there is an increased likelihood that they can see themselves as a scientist in the future.More research is needed to explore why perceptions of scientists did not change amongst the boys in the study.
- We found it is more difficult to improve enjoyment and self-efficacy in science in the current educational climate, and we didn’t see expected changes in this area. That children are not confident in their science abilities is a concern to us. We would like to explore how this compares to confidence in other subject areas.
- And lastly, we have also found it is difficult to disentangle the impact of NUSTEM’s work from the impact of many other influences on children in schools (teachers, school policy changes, curriculum policy changes, other school programmes/interventions etc). Having a Theory of Change which describes the intended change is important in making sense of these results, along with the flexibility to adapt if we find evidence that an intervention is not working as intended.
Our Recommendations
From our 5-year study NUSTEM makes the following recommendations to help increase in the diversity of STEM.
- Teachers should talk about the characteristics that people need to work in STEM, not just about the job they do. They should also help children to see that they have these characteristics as well – strengthening their self-concept as a ‘science person’. The NUSTEM STEM Person of the Week resources can be used throughout the school and focus on STEM characteristics.
- School staff, including teachers and teaching assistants, should develop an awareness of their own unconscious biases, and explore how choices they make in terms of language and what they say to children may effect childrens’ decisions about subject choice and careers as they grow up.
- School curricula and schemes of learning should include examples of different available STEM careers that can be regularly linked directly into science and maths lessons. The NUSTEM Primary Careers Tool gives over 100 different examples, and a simple model, for teachers to do this.
Methodology and broad findings
In 2015 we collected baseline data from children in Years 3 and 5 in six primary partner-schools (research schools). We collected repeat data from children in the same year groups and schools in 2017 and 2019. Data analysed was collected from 308 pupils in 2015, 312 pupils in 2017 and 254 pupils in 2019.
Data were collected using specially designed research tools for younger children including a simple Science and You questionnaire (collecting demographic data and data on children’s enjoyment of science and self-efficacy in science), Most Like Me/Most Like a Scientist sorting game (collecting data on children’s perceptions of self alongside their perceptions of scientists) and the STEMKat sorting game (collecting data on children’s knowledge of, and aspirations, for STEM jobs).
The Year 5 cohort in 2015 who had received no intervention at the time of data collection was used as a comparison group for the two Year 5 cohorts in 2017 and 2019 who had participated in 2+ years of NUSTEM activities.
Data were coded and analysed using the statistical analysis software SPSS. Chi Square and Fishers Exact tests were used to determine whether there were statistically significant relationships between the comparison groups. Significance is reported where the p value is less than 0.5.
Broadening Children’s Aspirations for STEM Careers
The Theory for Change
NUSTEM’s approach to STEM engagement is strongly linked to careers. Most interactions with children, and their families, link science (and STEM) topics with a related career. In nursery and reception young children can ‘try out’ different STEM careers using loans boxes, for older children classroom-based workshops introduce the characteristics of people working in different STEM careers, whilst resources such as the ‘Primary Careers Tool’ make it easy for teachers to include STEM careers within each science or maths lesson.
The research study finds evidence the NUSTEM intervention had broadened primary school children’s aspirations for STEM jobs, especially in engineering and the physical sciences, and particularly amongst girls. Compared with the baseline cohort, children in schools which had received 2 or more years of NUSTEM intervention were more likely to consider:
- The 13 STEM related jobs within STEMKat tool (+12%) (2015=37%, 2017=49%, 2019=41%), with an increase of 12% among girls, and 8% among boys;
- Jobs in the physical sciences (+14 %) (2015=37%, 2017=50%, 2019=44%), with an increase of 17% among girls and 12% among boys;
- Careers in engineering (+27%) (2015=37%, 2017=63%, 2019=44%), with an increase of 31% among boys (p=.001) and 23% among girls (p=.001).
- Careers in digital technology (+17%) (2015=44%, 2017=61%, 2019=53%), with an increase of 20% among girls and 9% among boys.
Our baseline data in 2015 showed that more boys had aspirations for STEM jobs than girls and they were more focused on physical sciences and technology careers. Girls with STEM aspirations were more inclined towards health and life sciences jobs. The 2019 data, shows this is still largely still the case, with statistically significant differences between the genders aspirations for many STEM Jobs. However, the gap between the gender’s aspirations for some STEM jobs (engineer, game tester and technician) has reduced.
Research Instrument
We collected data using our ‘STEM Knowledge and Aspiration Tool’ (STEMKat). STEMKat asks children to sort a list of 30 jobs. Children first sort the jobs into those that are ‘known’ or ‘not known’. The jobs the children know are then sorted into those they would like to do, would not like to do, or aren’t sure about. The 30 jobs included in the STEMKat tool are a broad mix of STEM and non-STEM jobs.
The STEM jobs included in STEMKat were: astronaut, banker, doctor, engineer, farmer, game tester, mechanic, nurse, pilot, surveyor, technician, vet, zoologist.
Improving Childrens’ Perceptions of Scientists
The Theory for Change
Children can have stereotypical perceptions of scientists and engineers, and so NUSTEM resources challenge these stereotypes and introduce positive attributes to describe scientists and engineers. We achieve this by using diverse role-models and examples within our school outreach, and through specific targeted interventions such as STEM Person of the Week.
The research finds evidence that in partner primary schools which had received 2 or more years of NUSTEM interventions girls perceptions of scientists have changed, particularly in reducing the perception that scientists need to be ‘very clever’. Girls were:
- Less likely to rank scientists as clever (-24%) than children of the same age in the baseline cohort (2015=72%, 2017=45%, 2019=48%). This difference was statistically significant (p=.016).
- More likely to rank scientists as kind (+17%) than children of the same age in the baseline cohort (2015=4%, 2017=10%, 2019=21%.) This difference was statistically significant (p=.001).
- More likely to rank scientists as sensible (+19%) than children of the same age in the baseline cohort (2015=41%, 2017=48%, 2019=60%). This difference was statistically significantly different (p.015).
In each case changes in boys’ attitudes was not statistically significant.
In 2015 our baseline data show that most children believed scientists to be hardworking (70%), clever (68%), hardworking and creative (48%).
In 2019 sensible (57%) replaced creative, with hardworking (74%) and clever (58%) still the most common characteristics.
In 2015 we found that more girls reported a perception of scientists as clever than boys (boys:62%, girls:73%). In the 2019 data this is reversed with more boys reporting the perception that scientists are clever than girls (boys:68%, girls:48%). This change is attributed a reduction in the perception among girls (-25%), while boys perceptions have largely stayed the same.
Research Instrument
Our research tool ‘Most Like a Scientist’ asked children to rank nine attributes into a ‘diamond nine formation’ to identify attributes they thought were most like, and least like, a scientist.
The nine attributes were clever, cool, creative, friendly, fun, hardworking, kind, sensible and strange.
Enjoyment of Science & Self-Efficacy
The Theory for Change
NUSTEM works to foster childrens’ natural interests in science and engineering, and enable them explore STEM topics and work scientifically with confidence.
However our research shows that despite NUSTEM’s intervention children’s enjoyment of science and belief in their science ability declined slightly during the 5 years. Children who has 2 or more years of intervention were:
- less likely to report to ‘like science’ than children of the same age in the baseline. Enjoyment of science declined by 8% during the intervention period (2015=77%, 2017=76%, 2019=69%) but this decline was not found to be statistically significantly different.
- less likely to report to ‘be good at science’ than children of the same age in the baseline. Children’s science self-efficacy declined by 7% during the intervention period (2015=63%, 2017=59%, 2019=56%) but this decline was not found to be statistically significantly different.
Research Instruments
Our ‘science and you’ worksheet asked children to respond to the statements ‘I like science’ and ‘I am good at science’ on a 5-point Likert scale.
Find Out More
If you to know more about NUSTEM’s research activities, we have produced a report which explores how the NUSTEM Theory of Change allows us to evaluate the impact that our activities have had in a range of areas: NUSTEM Impact Report 2014 – 2020.
You may also like to take a look at our research publications on our Publications and Dissemination page.
