(Illustrations by Celyn)

 
Earn as you learn

An ambitious economic field experiment studies how financial incentives for students, teachers, and parents affect academic performance.

 

On 16th Place in Chicago Heights, an industrial south-suburban town that feels haunted by the ghosts of vanished jobs, a school blends into its worn neighborhood. The tan, brown, and red brick buildings that make up the Washington-McKinley elementary and junior high schools could be the offices of any local business or government agency without the budget to update its facilities. On one wall, though, there’s a vibrant contrast: a bright red, blue, yellow, and green apple, the logo for the Griffin Early Childhood Center, an experimental preschool within the school, a field laboratory for education research.

The colorful sign represents a $10 million grant from the Kenneth and Anne Griffin Foundation for what Bloomberg Markets Magazine called “one of the largest field experiments ever conducted in economics.” Recruiting 650 families each year, the study tests financial-incentive strategies for improving academic performance.

In a community suffering from economic problems that infect its schools—an unemployment rate above 15 percent and almost 80 percent of students who qualify for free or reduced lunch—the investment offers a two-fold opportunity: research and outreach. Now in its second academic year, the project already has shown benefits for the children involved as the researchers try to identify long-term solutions for schools in other distressed districts like Chicago Heights. “It’s a really good representation of the urban education problems we have in the United States,” says University of Chicago economist John List, who developed the project with Kenneth and Anne Griffin, colleague Steven Levitt, and Harvard’s Roland Fryer.

Among the attempts to solve education problems, financial incentives for teachers have become more common in recent years. The concept of merit pay has been controversial; opponents argue that home and community factors that affect student performance make it impossible to implement a fair system. Linking teacher bonuses to standardized-test scores has been especially contentious (see, "Failed Tests").

Motivating students with money has also been on the rise. In 2008 USA Today reported that at least a dozen states planned to pay students—most with money from corporate and philanthropic donations—for meeting classwork or test-score standards. Fryer, who at 30 became the youngest African American to receive tenure at Harvard in 2008 and now directs its Education Innovation Laboratory, leads education research projects nationwide. Fryer’s experiments have found that paying students not for meeting performance expectations but for fundamental actions common to academic success—good behavior, regular attendance, homework completion—produces better results.

Two years before they started the Griffin experiment, Chicago economists List and Levitt were involved in another financial-incentive study at a Chicago Heights high school. List, who calls field experiments “my passion,” has fanned out around the world to conduct research on sports memorabilia, environmental regulation, and charitable giving. A 2008 New York Times Magazine story about his philanthropy research caught the attention of Chicago Heights physician William Payne, who contacted List to discuss ways his expertise could be put to use to help the local community. At about the same time, Chicago hedge-fund manager and philanthropist Kenneth Griffin met with List to discuss ways to examine the effects of education incentives. They set out to study those effects in Chicago Heights.

A Griffin-funded experiment during the 2008–09 school year tested cash incentives for ninth graders. Payne described a major dropout problem, telling List that as many as half of Chicago Heights high-school students failed to graduate, with most leaving between their first and second years. The researchers set out to patch that wound. In one part of the study, called the Chicago Heights Miracle, students who maintained a C average and met attendance and behavioral requirements received $50 a month. Others qualified for a $500 end-of-year lottery. Another part of the study focused on parents, tracking the effect of those incentives on mothers and fathers who received the money or a place in the lottery if their children achieved the standards.

Because of those financial incentives, List estimates the $400,000 experiment kept about 25–40 potential dropouts on track to earn a diploma this spring. “Per student, that’s pretty expensive,” he says. A scene in the 2010 Freakonomics movie shows List and Levitt discussing the results with Chicago graduate student Sally Sadoff, AM’08, PhD’10, who wrote her dissertation on the study.

As the researchers analyzed the data, they realized that, although the experiment stopped some bleeding, it did not treat the festering infection. By ninth grade the students were so far behind—reading, on average, at a third-grade level—that it was too late for the incentives to make a meaningful difference. “It’s like asking someone who’s never had a math class to solve a second-order linear partial differential equation and saying, ‘I’ll give you a million dollars if you solve it.’” List says. “They really want to solve it, but they just can’t.”

Edie Dobrez, executive director of the Griffin Early Childhood Center, loves how the Freakonomics scene ends. Rather than letting the results discourage them about whether behavioral economics can influence educational progress, the researchers decide to be more ambitious, recalibrating their hypothesis to focus on much younger children and, of particular interest to List, their parents. “They said, ‘That’d be cool, let’s do that,’” Dobrez says during a tour of the center that grew out of that conversation. “That’s the innovative spirit and the courage it takes to do something like this. That’s why we’re here.”

"Here" is a hallway and part of the basement at Washington-McKinley. There are five Griffin Center preschool classrooms in the building, each with 15 students aged 3 to 5, a teacher, and an assistant. The students follow a traditional reading and math curriculum called Literacy Express. At Highlands, another Chicago Heights school, five more Griffin preschool classrooms use a curriculum called Tools of the Mind that focuses on social and emotional skills linked to later academic success: self-confidence, self-control, assertiveness. The idea is to measure, among the 150 students enrolled every year in the tuition-free, all-day preschool, the effect of each method on later academic success.

List believes that a key factor in promoting educational achievement does not receive enough attention: the role of parents. “We have too many eggs in the kid basket,” he told Bloomberg. “We need to spend much more time and many more resources on helping parents.”

Families who sign up for the Griffin program are assigned by lottery to one of three tracks—the preschool, a parent academy, or a control group that receives no intervention. The children of the parent-academy group do not attend the Griffin preschool. Instead there are 18 sessions for moms and dads, complete with homework assignments, designed to teach “how to be a good parent in the academic sense,” List says. Parents in the academy receive payments, up to $7,000 a year, for attending the sessions and completing assignments. Their sons and daughters, although not in the preschool, also undergo assessments to measure the effects of the parent intervention on their development.

Once the children reach kindergarten, they enter the mainstream Chicago Heights school system. At that point, the researchers no longer set the curriculum, but the monitoring continues and the experiments on financial incentives expand to include students and teachers, in addition to parents. “We have little idea how best to incentivize those components,” List says, “and how those components might act as complements or substitutes.”

Identifying the proper balance requires solving a sort of multiple-choice story problem. To List, basic behavioral-economic questions apply to each facet: “how should I induce effort, and how does that map onto student outcomes?” When Ron Huberman, AM’00, MBA’00, ran the Chicago Public Schools in 2009–10, he sought List’s advice: “John, I have $50 million to spend, and I want to know the best way to spend it to advance student achievement.” List felt embarrassed that he didn’t know—he had been trying to answer exactly that question with the Chicago Heights ninth graders, but he did not have enough data for an informed recommendation.

With the early-childhood project, List has taken an aggressive step toward compiling that data. As Clancy Blair, a New York University applied psychologist who studies cognitive development in young children, told Bloomberg, it’s a “crazy idea” of astonishing scope.

On the surface, it looks like a typical preschool: one November morning, a handful of students at Washington-McKinley gather at one end of their classroom wearing plastic fire helmets and reflective vests. Others color and play with blocks, alone or in small groups, while the teacher and an assistant move around to support and supervise.

Both the Tools of the Mind and Literary Express approaches improved overall preschool performance during the study’s first year. Students entered the preschool program at about the 30th percentile among their peers nationwide, based on skills related to the curriculum they would follow. By June, they were in the 55th percentile. “Of our nine-month intervention,” List says, “we moved them about 17 academic months.”

In other early-childhood academic studies, immediate improvement tends to evaporate over time. By the third grade, List says, the kids who benefit from research intervention typically slip back to about the level of the control group.

The Griffin researchers want to find out how much educating (and paying) parents will help prevent that kind of decline. In a cavernous room in the basement of Washington-McKinley, Griffin parent-academy director Ty Jiles displays slides to a group of six mothers and fathers. The group sits in folding chairs arranged in a semicircle, dressed casually and interacting like friends or colleagues, asking questions and interjecting their own experiences as Jiles delivers the day’s lesson.

She emphasizes topics from the Griffin preschool curriculum, focusing on how parents can help impart those skills to their children, amplifying the bullet points on the screen. Use your child’s interests to connect to learning, she says, such as counting the number of times they throw a ball in a game of catch. Encourage them based on what they already know, rather than directing attention to what they don’t—praise their ability to recognize letters as an inspiration to learn the sounds they make.

At home the parents not only have to put the advice into practice; they also have to record how they handle specific situations. An assignment, due at the next session two weeks later, requires parents to describe an action they wanted their child to master. They’re instructed to watch the child doing the activity alone, determine how they would help based on those observations, and then write about the experience.

The parent academy offers two types of financial incentives. Half the parents receive direct payments. The other half earns deposits into a college fund. “We’re trying to figure out the optimal way to get people to invest in their children,” List says.

Early results show performance improvements above the control group for the children whose parents receive either type of payment, but neither made as much progress as the preschool students. The students whose parents receive the college-fund investments, though, had the best skill-retention rates over the summer among all the groups studied, preschool included.

The research in Chicago Heights will theoretically last forever, monitoring students enrolled as three-year-olds for as long as they live. List hopes that his grad students’ grad students will still be gathering data decades from now. As the Griffin Center’s executive director Dobrez puts it, “The researchers want to understand life outcomes, not just educational outcomes.” They will track the adult subjects’ jobs, earnings, and criminal records as closely as they tracked their grades, test scores, attendance, and behavior as students.

At the preschool level, only the parent group receives monetary incentives, but from the time the children enter kindergarten, students and teachers also have a financial stake. In a two-month after-school training program, students spend one hour, four days a week, working on computer-based tests that measure either cognitive or noncognitive skills. Results of individual and team competitions determine who receives weekly awards worth $7–$10. The team competitions measure peer effects on student performance.

There is also a “teacher empowerment program,” says Sadoff, the Chicago graduate who’s now an assistant professor of management and strategy at the University of California, San Diego. In the empowerment program, teachers select two students each week and establish the criteria they must meet to earn a reward worth about $15. The idea, Sadoff says, is to assess whether teachers “can incentivize students more effectively than policy makers can.”

Teachers also receive their own financial rewards in the Chicago Heights experiment. All K–eighth grade math and reading teachers in the school system, along with fourth- and seventh-grade science instructors, were eligible for the study. Those who signed up were randomly assigned into the incentive or control groups.

The incentive group receives bonus pay at the beginning of the year because, List says, the promise of extra money at the end does not inspire the same effort as an upfront bonus. “People have a really hard time thinking about months into the future, even adults,” he says. “We tend to discount [the value of a later payment] at a rate that’s way too high.”

To test that, two-thirds of the teachers in the Griffin study receive an extra $4,000 when school starts, an advance they have to repay if their students fail to meet expectations. “We leverage loss aversion and bring the reward to the very front,” List says. “Our teachers do sign contracts, and if their students do not achieve, they give us some or all of the money back.” The early results indicate that prepaid incentives yield student gains five to ten times greater than performance bonuses that are withheld from teachers until the end of the year.

The payoff of the Griffin experiment can seem far off and far reaching. Any immediate effects might be difficult to identify, considering the longevity and complexity of the research—and the depth of the problems it addresses. But Dobrez, who chats with moms and dads as they leave their class session and hugs preschoolers who call her “Miss Edie,” sees the effects on a personal level, in the children’s development and in the parents’ commitment, which the first year’s progress underscored.

“This is to the benefit of these families now,” she says, “and the families can also feel like they’re benefitting future generations” with answers to the complicated—and controversial—questions about whether financial incentives elevate the value of education.