How Do You Measure Engineering Learning?

Posted by Cynthia Berger on 3/15/16 11:00 AM

The new Next Generation Science Standards (NGSS) are bringing engineering into elementary classrooms. They're opening the door to new ways of learning, but these standards also pose a challenge when it comes to assessments. Engineering is a team effort, but most assessments are designed to be taken by individual students.

2015.05.05_E4.jpgWhen kids work in a group, what can you say about each individual student with respect to level of engagement? Performance? Learning progress? EiE is developing new assessments that address these questions as component of E4, an NSF-funded study that compares the effectiveness of two elementary engineering curricula.

 EiE's E4 Study Looks at Classroom Impacts

The E4 study followed thousands of students (from more than 500 classrooms in 150 schools) over the course of three years. "The EiE research team has developed many assessment tools over the year, but for E4 we created a special instrument we call 'EEPA,' short for elementary engineering performance assessment," says Cathy Lachapelle, EiE's director of research.

Like traditional assessments, the EEPA is a pencil-and-paper document. But it’s also like an EiE lesson in that it presents a scenario, calls on the student to solve a problem by designing a technology, and perhaps most importantly, asks the student to explain why they think their design will work.

Kids Explain Their Engineering Design Choices

To understand how this works, consider the EiE unit “Thinking Inside the Box: Designing Plant Packages.” It starts with a storybook about a child who wants to send a plant as a gift. Students work in teams to design their own sturdy packages to protect a live plant while it sits on a store shelf or is shipped through the mail.

As part of the design process, kids explain their design ideas to fellow team members: "We could make holes to allow for air flow," "We could add transparent windows to let in sunlight," "We could use absorbent material to keep the roots moist," and so on.

The EEPA also asks students to come up with design ideas, and it poses a scenario to create context—in fact, three scenarios are offered, to be sure students can chose a topic they find interesting. These scenarios are short (a single paragraph instead of an entire storybook).

2015.05.05_EEPA_Scenario_EiEIn the example on the right, the engineering challenge is to design an “animal lifter,” or a device that that will safely move a pet up onto the exam table, for a veterinarian. Students draw a diagram of their design (they don’t actually build it), then they justify their decisions about design choices to whoever reads the assessment.

2015.05.05_EiE_animal_lifter-1.jpg“We see strong evidence that the EEPA is valid for its intended use in the E4 project, which is measuring students’ ability to explain and justify design elements,” says Lachapelle.

So the assessment tool shows promise as a way to compare the effectiveness of different engineering curricula. But it also raises another question: Will an instrument designed for research be valid for classroom assessments? "We hope to extend our research to answer this question too," says Lachapelle.

We'll be sharing more on what we learned with this new engineering assessment at NARST 2016, an international conference on research in science education.

Engineering is Elementary is a project of the National Center for Technological Literacy at the Museum of Science, Boston.

This article was first published on the EiE Blog on 5/5/2015.

Topics: Engineering Habits of Mind, Assessment