Additional Scholarly Literature
This page summarizes data from evaluations of Urban Debate Leagues (UDLs). The data suggest that UDL participants are
moving in the right direction across several indicators (notably: literacy, attendance rates, graduation rates, college matriculation,
communication skills, and risk-taking behavior). The available evidence, despite some attempts to generate rigorously
collected data, is, however, incomplete.
The Edna McConnell Clark Foundation has outlined a "hierarchy of knowing about program quality" that describes the degree of confidence one can have in evidence with regard to the quality and effectiveness of youth development programs. This page attempts to provide a brief outline of the quality of the evidence generated by previous evaluations of the UDL approach.
Urban Debate Leagues have yet to establish proven quality through the use of a random assignment, experimental research design. Still, a range of studies and efforts to document the effect of urban debate participation have (I) demonstrated the effectiveness , or (II) suggested the apparent effectiveness , of Urban Debate Leagues. Finally, (III) the UDL approach is aligned with existing social scientific research about what improves outcomes in low performing schools.
I. Studies that Demonstrate Effectiveness
Based on systematically collected data comparing Urban Debate League participants with similar students who did not participate in debate, several studies have concluded that urban youth benefit from participation. These various data collection and evaluation efforts share the following aspects in their methodological approaches: the evaluator reviews the UDL program and services, collects outcome data for one group of program participants, and compares it to information and data on another group of comparable students who are not UDL participants. Studies vary in the degree to which the evaluator represents an independent, external agency and in the level of rigor employed in selecting comparison groups. Studies that demonstrate effectiveness include:
- Arguments for Success: A Study of Academic Debate in the Urban High Schools of Chicago, Kansas City, New York, St. Louis, and Seattle (2004). This study assessed debate's impact on reading, self-esteem, and risk-taking behaviors within a treatment group of 209 debaters and a comparison group of 212 urban high school students. As the title suggests, the sample was drawn from urban high school students in five cities across the country. A standardized reading test, the Scholastic Reading Inventory, was administered in a pre-test/post-test format, along with a self-report survey of risk-taking behavior. The study concluded that "academic debate improves performance at statistically significant levels on reading test scores, diminishes high-risk behaviors, and improves academic success and student attitudes towards higher education." Linda Collier at the University of Missouri, Kansas City, led the research team. Data analysis was conducted by Elaine Maag of the Urban Institute and Edward Harris of the Congressional Budget Office. Notable findings include:
- There was a 25% increase in debater (n=209) reading scores compared to the increase in the control (entire n=212) category scores. Debaters increased test scores 18% more than the honors controls (n=64).
- "The fall results combined with the very significant difference in the arts results predicts positive academic results and lower behavior risk for debaters.
- "The spring scores reflect a statistically significant increase in GPA by debaters over controls."
- Linda Collier suggests that "this study shows that debate uses competition to motivate increased reading — a basic and essential building block of education — in a powerful way that will help students for the rest of their lives."
- The Effect of the Minnesota Urban Debate League on the Academic and Social Development Outcomes of Students, First Year Report (2005). Quantitative and qualitative methods were used to gather data for this evaluation using a pre-test/post-test protocol. To more accurately isolate the impact of debate, researchers recruited a set of comparison students to participate in the testing alongside the debater cohort. Scholastic Reading Inventory instruments were used alongside a survey seeking to capture background characteristics: students' report of academic performance, school attendance, involvement in academic and debate activities; attitudes, such as interest in school and intention to attend college; locus of control; self-esteem; reasons for joining the UDL; and the Adolescent Risk Taking Scale. (These instruments were administered in the fall of 2004 and spring of 2005). School records provided students' GPAs, English classes taken, academic progress (age for grade, school promotion), school attendance, and eligibility for free/reduced-price lunch. The findings include:
- Debaters scored 36% higher on the reading post-test than on the pre-test. This improvement is 61% greater than improvements among the comparison group.
- 80% of debaters reported no attendance problems compared to 49.02% with no reported attendance problems among the comparison group.
- Debaters averaged 15% higher self-esteem than the comparison group, and this boost in self-esteem was positively correlated with the duration of debate participation: the longer he/she debated, the wider the differential.
- By the end of their first year of debate, 100% of the debaters reported an increased interest in their classes.
- Compared to the comparison group, 87% of debaters were better able to analyze information.
- On a 4.0 scale, the gross average of debaters' 2006 GPAs was 2.97, compared to 2.5875 among the comparison group. Returning debaters averaged a 0.13 increase in their GPAs, while returning comparison group members lost an average of 0.10 points.
- 100% of Minneapolis UDL debaters were unlikely to engage in negative risk behavior (drug use, early pregnancy, and alcohol). Debaters scored the highest possible score on this indicator.
- Chicago Urban Debate League. The Director of the Chicago Debate League collected data over the course of two years. The first set of data was collected from spring 1997 to spring 1998. The sample of 76 urban debaters and 37 comparison group students was drawn from five Chicago Public Schools high schools and represented students across all four years in school. GPAs were obtained from student transcripts. The findings suggest that during the study period, urban debaters improved their grades by 7.6% (average GPA went from 3.17 to 3.41), compared to a comparison group whose grades fell 1.3% (average GPA went from 2.98 to 2.94). A second batch of data was collected from spring 1998 to spring 1999. This sample consisted of 59 urban debaters and 34 comparison group students, again drawn from five Chicago Public Schools high schools and representing students across all four years in school. This study concluded that urban debaters improved their grades from spring 1998 to spring 1999 by 6.6% (average GPA went from 3.31 to 3.52), compared to a comparison group whose grades went down 0.7% (average GPA went from 3.03 to 3.01).
As is apparent, none of the previous three studies adequately controls for potential self-selection bias.
II. Evidence Suggesting Apparent Effectiveness
The second set of data on UDLs constitutes systematically collected data that suggest urban youth are benefiting as intended from participating in UDLs. These data sets consist of self-reports from UDLs assessing which students access UDLs, what programs they participate in, and what outcomes they achieve. Students were given unique identifiers and Leagues collected basic demographic data. In some instances, UDLs collected baseline and post-test data to capture changes in students' academic performance over time. Data sets that suggest apparent effectiveness include the following:
- Atlanta Urban Debate League (2005). The self-report data of the middle school urban debate program show a statistically significant increase in GPA among UDL participants. In addition, at-risk middle schoolers reduced their disciplinary referrals by 50% after one year of participation in urban debate. Disciplinary referrals included all in-school detentions, out-of-school suspensions, or mandatory teacher/parent conferences. At the end of the pilot year, the number of referrals for the student body remained relatively constant, with a small decline of only 1%, while referrals for urban debaters fell 50%.
- Baltimore Urban Debate League (2006). The Baltimore UDL conducts extensive college-access programming with College Summit. They report a 100% graduation rate and a 91% college matriculation rate.
- Center for Applied Research, or Learning What You Can, Building from What You Learn: Assessing the Role of the Baltimore Urban Debate League on Academic and Social Outcomes of Students, a Two Year Quantitative Analysis (2003). This study employed a sample of 235 students and 25 coaches involved in the Baltimore UDL. Researchers from the Center for Applied Research collected student data from school records. They also conducted observation of debate training and debate tournaments, and used focus group discussions to collect students' perceptions of the benefit of participating in debate. Findings include the following:
- During the focus group sessions, 44.2% of debaters reported increased reading comprehension.
- The most significant finding "is the potential of (debate) to shift the students' locus of control (i.e., to empower) ... a higher debating level increases the odds of empowerment among students four-fold; a high level of participation in tournaments increases the odds of an internal locus of control three-fold." The degree and intensity of participation thus seems correlated with certain outcomes.
- Kansas City Urban Debate League. Self report data suggests that in Kansas City, urban debaters have a 99% high school graduation rate (2002, 2003).
III. Alignment with Research on Improving Low Performing Schools
A body of rigorous empirical social scientific research suggests that certain features and design principles, embodied in the UDL approach, improve educational outcomes in urban public school systems. Read more here.