(Photography by Maya Dukmasova)

Proving ground

UChicago’s Urban Labs turn promising ideas for helping cities into hard evidence of what works.

On a sunny Tuesday morning in early June, with the end of the school year already palpable, three girls were hard at work in a Chicago Vocational Career Academy classroom. The Chicago Public Schools high school is a sprawling Art Deco building on the southernmost edge of Avalon Park, on the South Side of Chicago. About 98 percent of its students are African American, and nearly 94 percent come from low-income families.

The school opened in 1940 to serve 6,000 students, but today fewer than 1,000 flow through its cavernous hallways as the population of the surrounding neighborhood declines. CVCA is still largely a vocational school, with majors in carpentry, cosmetology, and culinary arts, and only about half its graduating seniors are college bound.

In 2011 the school’s test scores and graduation rates had gotten so low that CPS intervened, overhauling the staff and introducing new programs to help students stay on track to graduate. CVCA now has an intensified science, technology, engineering, and mathematics curriculum that offers college credit, but many students who arrive as freshmen fell behind years earlier and still haven’t mastered middle school math.

While most of their peers were in traditional classrooms—one teacher, 25 students—the three girls sat face to face with personal tutors in the Math Lab. They attended the elective in addition to their normal math class. Their tutors were recent college graduates spending a year doing public service: Nichole Jannah, a neuroscience major from the College of William and Mary, and Amelia Hansen, AB’14, a UChicago biology major. That afternoon they were doing division with unknown variables on small dry-erase boards. “Number 23 is a little curveball but I bet you can do it,” Jannah encouraged her student.

Veronica, a freshman, started the year with a D in math. Walking into class, she says, felt “horrible.” She began working daily with her tutor this year, reviewing foundational concepts in arithmetic and algebra to catch up to the material being covered in her math class. Her grade by June was a high B, three points shy of an A.

Sarah, also a freshman, transferred to CVCA in the middle of the school year. Her math grade rose from a C to an A with the help of her tutor. She hadn’t felt good in math class since sixth grade. After a few months of tutoring, she’s confident: “When I go into math class, I fly through work,” she said, snapping her fingers.

The tutoring Veronica and Sarah are getting could alter the course of their lives. Foundational math classes are a “key gatekeeper” for high school diplomas in many urban school districts, says Jens Ludwig, the McCormick Foundation Professor in Social Service Administration, Law, and Public Policy. Succeeding in those classes makes it much more likely that the girls will graduate. Graduating, in turn, will give them better prospects as adults in everything from their earning power to their health.

Their experience is also helping UChicago researchers learn how tens of thousands more students citywide, and beyond Chicago, can improve their chances of graduating. CVCA’s Math Lab, one of 14 across CPS, is part of a large policy experiment being conducted by the Education Lab—part of the University of Chicago Urban Labs. The Education Lab’s scholars, analysts, and practitioners model studies like this one after the randomized controlled trials that provide gold-standard evidence in medicine. Participants in the Math Lab are chosen randomly from a pool of low-performing students thought to be at high risk for dropping out. Researchers can compare their gains to the academic progress of a similar group of CPS students who have access to the city’s regular support services, but not the tutoring program.

The five Urban Labs focus on education, health, energy and the environment, poverty, and crime, using rigorous social science methods to evaluate potential social interventions. The idea, says Ludwig, is to produce data about the effectiveness of programs and policies to help kids stay in school, for example, or reduce shootings, or provide better health care. When there’s strong evidence of positive impact, the Urban Labs can use that data to mount powerful arguments for city or state governments to administer the programs on a larger scale, and can work with the labs’ web of partners to influence policy. The labs then support cities in scaling up the most effective and cost efficient, ultimately contributing to social change on a large scale.

Ludwig cofounded the Crime Lab in 2008 and continues to serve as its director. The first of the Urban Labs, the Crime Lab branched out last winter to add an office in New York City, embedded in the Mayor’s Office of Criminal Justice. Its success helped spur the creation of the other four labs this April, each led by a senior faculty member. Ludwig also codirects the Education Lab with Northwestern professor Jonathan Guryan and UChicago’s Timothy Knowles, the Pritzker Director of Urban Labs.

The five labs forge partnerships with government agencies and nonprofits “to target the most critical challenges” for cities, says Knowles, who is also the chairman of the Urban Education Institute and John Dewey Clinical Professor in the Committee on Education. Bringing Math Lab to CPS students, for instance, was a joint effort between Urban Labs, CPS, the City of Chicago, and Boston-based Match Education, where the tutoring program originated.

When philanthropy focuses on direct support, Ludwig explains, “The people you help are the only people you help.” But having hard data and acute analysis of a program’s success and cost efficiency means being able to make a case for governments to adopt such programs, scale them up, and reach many more people.

“Nationwide, the US spends $200 billion per year on the criminal justice system and $500 billion on K–12 education,” Ludwig says. “If you’re a philanthropist, anything that you can give to provide direct services to kids is just a drop in the bucket. Our theory of change is if you can show the government how to spend its $200 billion better, or its $500 billion better, that’s how you start to have a really big impact.”

Some of that impact can already be seen. A 2012–13 Crime Lab study, for example, determined that Chicago’s dropout and violence prevention program Becoming a Man reduced violent crime arrests by 44 percent among participants during the program year. Becoming a Man, which offers in-school and after-school sessions to build boys’ social-cognitive skills, now serves about 2,000 students in Chicago. The results of the study also helped inspire President Barack Obama’s $200 million My Brother’s Keeper initiative, announced in 2014.

The labs, Ludwig says, are “trying to help the government spend its money in a way that generates more social good per dollar spent.” Or, as Knowles puts it, to “make smarter choices to improve the human condition, at scale.”

In any school system, differences in students’ academic skills grow larger and larger as they progress through the grade levels. The effect can be particularly pronounced in systems serving many students from economically disadvantaged backgrounds. Middle schools and high schools in such systems are trying to educate students who are “all over the map in terms of what they need academically,” Ludwig says.

“Everything in education policy right now is about getting teachers to do a better job teaching grade-level material,” he adds. “Let’s fire the crummy teachers, let’s hire the better teachers, let’s incent teachers with accountability systems that hold their feet to the fire.” But no amount of pressure on high school teachers to teach algebra better will help their students working at third-grade level who haven’t yet mastered multi-digit arithmetic.

One solution to this problem has been recognized for a long time: individualized instruction. The Math Lab, Ludwig points out, follows the same educational model that has been used at the University of Oxford for centuries. “What the people who founded Oxford knew was that one-on-one instruction, or two-on-one instruction, is the most effective way to teach anybody anything.” The challenge has been not about solving a pedagogical problem so much as an economics problem: how to give Oxford-style instruction at CPS prices?

Match Education may have found a way. “The key a-ha moment that Match had,” says Ludwig, “was to realize that teaching one or two kids is fundamentally different from teaching 25 or 30 kids. What you need to be able to do to be a good tutor is massively different from what you need to be a good classroom teacher.”

Being able to successfully teach in the classroom involves years of practice and training in pedagogy and classroom management. Several studies, Ludwig says, have shown that most teachers perform quite badly in their first couple of years. But the steep learning curve is worth it for a classroom teacher who makes a lifelong commitment.

To get results as a tutor, he says, requires only knowledge of the material, good rapport with people, and commitment, “so you can expand the pool of people who tutor, lower the costs, and thereby solve the key economic problem at the heart of all this.” Because a tutor doesn’t need special training or years of on-the-job learning to become good at it, Match can recruit recent college graduates or older career switchers who are willing to work for a modest salary for a year as a public service, in the same spirit in which new college graduates join Teach for America or the Peace Corps. The Match tutors—about 85 across the test schools—work full time, teach 12 to 14 students each, and are paid $17,000 for the year plus benefits.

If the Education Lab and its partners can persuade cities to invest in tutoring on a large scale, thousands of students who have fallen far behind could catch up to grade level, reengage with regular classroom instruction, and begin to have real hope for a diploma. Ludwig calls the tutoring a safety net for those students, and a potential solution to what’s been an intractable systemic challenge for urban school districts across the country.

When principal Douglas Maclin arrived at CVCA four years ago, just 44 percent of the freshman class was “on track” for graduation—that is, passing all their classes. Math and English classes were particular obstacles. After Maclin’s first year that rate improved into the low 70s. Now, since the Match tutoring program arrived at the school in fall 2013, the freshman on-track rate has risen to more than 86 percent, and the sophomore rate to 87 percent. “The only thing different in our school is Match,” says Maclin. The program has helped the most lagging freshmen and sophomores so much that this past summer, for the first time, CVCA did not need to provide any credit-recovery classes for failing students. Instead the school focused on offering higher-level math and honors courses.

Tutor Marian McElroy, a retired attorney, works with a total of 10 students. Tutors are required to contact parents or guardians at least once a week, and she talks with some of the parents almost every day. That’s crucial, she says, “so that you’re not just always giving them the bad news. Nobody wants the bad news all the time.” One of her students, Michael, was in a special needs math classroom until this year. Now his performance has reached the level of his peers, and he’s even participated in a math competition at the school. He shyly reported that his mother bought him a cake after McElroy called to tell her that he had won two rounds of the contest.

Michael diligently attends his daily tutoring sessions with McElroy and has gotten into the habit of eating his lunch in the Math Lab classroom. Felipe Alaniz, the site director of the Match program at CVCA, works to create an environment where students have a second home. The boys’ Math Lab (separate from the girls’ classroom because Urban Labs is tracking their data separately) is decorated with posters showing every tutor’s picture, educational background, and favorite foods and hobbies. Chicago native Miss Cooper recently graduated from Syracuse University and loves chicken Alfredo. Mr. Cooman studied biochemistry at IIT. Miss Richards went to Northwestern University and likes to knit.

Each 50-minute Match session starts with a five-minute silent written quiz. This assesses the student’s skill level on the concept he is studying in his regular math class, or that he will learn in that day’s Match session. No one, not even the tutors or Alaniz, can speak in the classroom during this time. At the end of the session a second five-minute assessment informs the tutors of their students’ progress and allows Alaniz to collect data on the tutors’ effectiveness.

Alaniz jokes and laughs easily with the students, but he can become serious quickly when holding them accountable. He uses what he called “success rhetoric” to motivate the students, addressing each as Mister or Miss and using their last names even when speaking about them to colleagues. The main reason some students continue to perform poorly despite Match, he says, is missing school frequently. Alaniz tries to get to know each student’s family and personal situation so he can stay attuned to their moods and help when they get discouraged.

“A lot of students are going through a lot,” he says. “Sometimes they’ll come in and I can already tell, because you get to know their ways and their mannerisms. That’s really my role. I want my tutors to have all their authority; they rule their tables. But I’m that support where, if they have two students at a time and one student just needs to vent or talk, I’m the one that will take them and talk to them outside.”

About halfway through the study, the students in CVCA’s Math Lab doubled the amount of math they would have been expected to learn without tutoring, and many caught up with grade level. Data from across the 15 test schools tells a similar story. Math Lab students fail math courses half as often as students not in the program, and fail other courses 23 percent less often. They also closed the black-white test score gap by 30 percent—an improvement previously considered unlikely at the high school level. The results are particularly heartening to the Education Lab given the scale of the study, with more than 1,000 students participating last year.

In the education policy world, Ludwig says, it has been widely thought that helping more at-risk youth from low-income backgrounds stay and succeed in high school is nearly impossible. Many experts have suspected that adolescence might already be too late to substantially improve the academic outcomes of children in poverty. But the Match study’s early results indicate that teens like Veronica, Sarah, and Michael still have a chance.

Many more students may get that chance. After seeing the first year’s results, CPS took over stewardship of the program for 2015–16 and hopes to expand it in coming years. And after the New York Times covered the program last winter, cities across the United States contacted Match about implementing the model, including New York City. Ludwig and Knowles hope that eventually the program will draw government funds now supporting programs with less evidence of effectiveness, multiplying the number of students who benefit.

At CVCA, Math Lab continues this fall, growing to include 120 students from 80 last year. Again it serves freshmen and sophomores who are at the highest risk for dropping out due to poor academic achievement. Asked if they wanted to continue with the tutoring program next year, all the students interviewed for this story said, without hesitation, yes.

Urban vision

What makes the University of Chicago Urban Labs’ approach distinct, according to Pritzker Director Timothy Knowles:

  • Rigorous research that aspires to improve the human condition at large scale
  • Close partnerships with leading government and not-for-profit agencies to maximize the chances that encouraging results translate into social impact
  • A commitment to helping promote broad-scale adoption of promising solutions 

Maya Dukmasova is a freelance writer and photographer based in Chicago. She writes about issues of social justice and social inequality for local and national publications.