Brett Keller, MPA
Back in September the New York Times reported on an unexpected finding from a clinical trial: “A promising but expensive device to prop open blocked arteries in the brain in the hope of preventing disabling or fatal strokes failed in a rigorous study.” Many promising medical innovations fall short when they finally reach clinical trials, but this story was unusual because the stents had already been approved by the FDA under a so-called humanitarian exemption. The FDA approved the stents to reduce the risk of stroke, but those who received it had twice as many strokes.
How did this happen? The Times chronicled experts’ puzzlement: “Researchers said the device seemed as if it should work.” And Joseph Broderick, a prominent neurologist, is quoted as saying “Quite frankly, the results were a surprise.” Researchers are delving into this case to discover why the stent failed, but policymakers from all fields should take it as a valuable lesson. This is one more argument for testing policies whenever possible: not only does expert opinion sometimes get things wrong, but without good data there is often no way to really know when they are right.
Similar lessons can be gleaned from the history of surgical response to breast cancer. In The Emperor of All Maladies (2010), a new history of cancer, oncologist Siddhartha Mukherjee chronicles the history of such failed interventions as the radical mastectomy. Over a period of decades this brutal procedure – removing the breasts, lymph nodes, and much of the chest muscles – became the tool of choice for surgeons treating breast cancer. In the 1970s rigorous trials comparing radical mastectomy to more limited procedures showed that this terribly disfiguring procedure did not in fact help patients live longer at all. Some surgeons refused to believe the evidence – to believe it would have required them to acknowledge the harm they had done. But eventually the radical mastectomy fell from favor; today it is quite rare. Many similar stories are included in a free e-book titled Testing Treatments (2011).
As a society we’ve come to accept that medical devices should be tested by the most rigorous and neutral means possible, because the stakes are life and death for all of us. Thousands of people faced with deadly illnesses volunteer for clinical trials every year. Some of them survive while others do not, but as a society we are better off when we know what actually works. For every downside, like the delay of a promising treatment until evidence is gathered properly, there is an upside – something we otherwise would have thought is a good idea is revealed not to be helpful at all.
Under normal circumstances most new drugs are weeded out as they face a gauntlet of tests for safety and efficacy required before FDA licensure. The stories of the humanitarian-exemption stent and the radical mastectomy are different because these procedures became more widely used before there was rigorous evidence that they helped, though in both cases there were plenty of anecdotes, case studies, and small or non-controlled studies that made it look like they did. This haphazard, post-hoc testing is analogous to how policy in many other fields, from welfare and education, is developed. Many public policy decisions have considerable impacts on our livelihoods, education, and health. Why are we note similarly outraged by poor standards of evidence that leads to poor outcomes in other fields?
A recent example from New York City helps illustrate how helpful good evidence can be in shaping policy. A few years ago Mayor Michael Bloomberg rolled out a massive program that seemed to make a lot of sense: pay teachers bonuses based on their students’ performance. The common sense proposal was hailed as “transcendent” and gained the support of the teachers’ union. It cost $75 million, and it didn’t work. How do we know? The program was designed from the beginning as a pilot where schools were randomly assigned to the program or to a control group, and the research showing that the program had no effect on outcomes was subsequently published. What would have happened if this policy had been put in place without an effective evaluation plan? In all likelihood New York officials would now be touting its success at conferences and urging other cites to implement similar programs. Instead it was quietly shelved. That this particular program did not have the intended effect is disappointing, but it is much better than if we believed it worked and continued on unaware.
The pros and cons of randomized trials have been discussed here on 14 Points before – see recent posts by Jake Velker and Shawn Powers. The cases I presented here are ones where the results were not “no-brainers” at all, and without systematic evaluation bad policies would have been or tragically were put in place. While good evidence does not have to come from randomized trials, there are still many areas where they are underused. In areas where they are feasible (i.e. not macroeconomics) such evidence should be the norm, and those who implement policies with great optimism but without planning for thoughtful evaluation should be panned. Even without random assignment of the treatment, the best policy evaluations should involve a serious attempt to estimate the counterfactual: what would have happened in the absence of the intervention. Moving beyond arguments over specific programs and whether they work, policymakers can move us towards better outcomes by creating a culture where strong evidence is valued. After all, the clinical trial as we know it in medicine is a 20th century innovation; it hasn’t always been this way.
Showing posts with label education. Show all posts
Showing posts with label education. Show all posts
Friday, December 16, 2011
Thursday, December 1, 2011
A Real Remedy for Youth Unemployment in Saudi Arabia: Scrap "Saudization" and emphasize employment education
Mary Svenstrup, MPA
The Kingdom of Saudi Arabia has been relatively unaffected by the Arab Awakening. So far, the government has been able to maintain stability by cracking down on protests while simultaneously providing generous handouts to appease its citizens. Saudi youth, however, are growing increasingly dissatisfied with their government because they cannot find employment. And as recently seen in Egypt, Tunisia, and Yemen, a dissatisfied youth population has proven to be an important factor contributing to instability – and, ultimately, regime change.
Employment has significant cultural implications in Saudi Arabia and the Arab world: it is an essential prerequisite for marriage and transition to adulthood. Without employment, youth are stuck in social purgatory, feeling restless and unsuccessful. Yet Saudi Arabia has the highest youth unemployment rate in the Middle East and North Africa region excluding Iraq.
The Saudi government’s answer to youth unemployment is “Saudization,” a policy that sets employment requirements for Saudi nationals. This policy has been ineffective because it does nothing to address the underlying issue that Saudi graduates do not have the skills demanded by private companies. Furthermore, this policy disincentivizes non-hydrocarbon sector growth, which is critical to create more jobs in the Kingdom. The Saudi government should scrap Saudization and instead focus on education that will build skills for employment.
Structural Barriers to Employment in Saudi Arabia
Saudization is an artificial employment requirement that does not address the structural problems with the Saudi economy that contribute to youth unemployment, such as an overreliance on the hydrocarbon sector, a constantly growing social transfer system, and insufficient private sector growth.
Saudi Arabia’s economy is largely based on petroleum, but the hydrocarbon sector is not a reliable source of job creation. Oil and gas comprise about 45% of fiscal revenues, 55% of GDP, and 90% of export revenue – yet, the national oil company employs less than 1% of Saudi labor force. The hydrocarbon sector does create a source of revenues for the government’s generous social transfer programs; these programs, however, may not be sustainable, crowed out other social expenditures, and most of all, do nothing to address unemployment. As an example, in lieu of real social reforms, the Saudi government introduced new fiscal initiatives on February 23 and March 18, 2011 to quell domestic protests. Although these types of social transfers may help to immediately pacify the population, the IMF notes that these programs will require oil prices sustained higher than $90 per barrel for the next several years. That may not be sustainable and puts a huge burden on the Saudi government to control oil prices.
Given the problems created by reliance on the hydrocarbon sector for employment, the non-hydrocarbon sector of the Saudi economy is critical. This sector, however, has not been able to create enough jobs for Saudis. Over the next five years, the IMF estimates that private sector non-oil GDP will need to grow by 7.5% annually to create a sufficient number of employment opportunities for the domestic population. While Saudization addresses the issue that most private sector jobs are being allocated to more qualified expatriate workers, the policy increases the operating costs of private companies in the Kingdom, thereby reducing incentives for investment and hindering non-hydrocarbon growth. For example, a recent equity research report by EFG Hermes suggested that Saudi companies will meet Saudization requirements in the near term by hiring more Saudis rather than reducing the number of expatriates, given the skills mismatches of Saudi workers. Basically, companies are being forced to increase personnel costs simply to satisfy a legislative mandate.
Furthermore, both the appeasement tax and Saudization may have a feedback loop creating more pressure on the government. As Saudis become wealthier and more connected to the rest of the world, their expectations for employment and inclusion in the economy will continue grow. Higher expectations combined with growing dissatisfaction with unfulfilling employment opportunities will further increase the government’s cost of appeasement. Therefore, creating sustainable economic opportunities for its citizens will mitigate the long-run fiscal burden of providing appeasement handouts and remove the need for Saudization, as long as Saudis have the skills necessary to be competitive employees.
Education for Employment
The underlying cause of youth unemployment is that, even with a postsecondary degree, Saudi graduates lack the right skills for jobs in a modern, knowledge-based economy. The Saudi education system itself has flaws, but a main problem is that students choose to study subjects that have no direct linkages to labor markets. For example, in 2008, 40% of university students in Saudi Arabia were concentrating in arts and humanities (versus averages of 20% and 17% in Asia and Latin America, respectively), while only 24% chose to study science or engineering. Furthermore, there is a social stigma against technical and vocational training, and any type of university degree, even one that is very unlikely to lead to employment, is socially viewed as superior.
To reduce youth unemployment—and the risk of social instability in the Kingdom—the government ought to at least address skills mismatches by orienting the education system toward private sector employment opportunities. Ideally, Saudi Arabia should scrap Saudization and instead focus on making their graduates competitive employees. Forcing graduates to compete for jobs will ensure that they choose education tracks that are conducive to employment. Additionally, technical and vocational programs should be associated with prestigious universities and fellowships, which would alleviate some of the social stigma of choosing this track. Lastly, the government should adopt a national quality assurance framework to regulate private education companies doing business in the Kingdom to ensure that degrees and certificates are uniform across the country. These changes, paired with continued investments in upgrading the overall education sector, should help to link skill-based post-secondary education with employment opportunities.
Encouraging youth to select education for employment will result in more qualified workers that will naturally increase the demand for Saudi workers and ultimately reduce the cost of doing business in the country. The government and the royal family should act quickly on this issue, for their own sake and for the sake of the growing youth population in their kingdom.
The Kingdom of Saudi Arabia has been relatively unaffected by the Arab Awakening. So far, the government has been able to maintain stability by cracking down on protests while simultaneously providing generous handouts to appease its citizens. Saudi youth, however, are growing increasingly dissatisfied with their government because they cannot find employment. And as recently seen in Egypt, Tunisia, and Yemen, a dissatisfied youth population has proven to be an important factor contributing to instability – and, ultimately, regime change.
Employment has significant cultural implications in Saudi Arabia and the Arab world: it is an essential prerequisite for marriage and transition to adulthood. Without employment, youth are stuck in social purgatory, feeling restless and unsuccessful. Yet Saudi Arabia has the highest youth unemployment rate in the Middle East and North Africa region excluding Iraq.
The Saudi government’s answer to youth unemployment is “Saudization,” a policy that sets employment requirements for Saudi nationals. This policy has been ineffective because it does nothing to address the underlying issue that Saudi graduates do not have the skills demanded by private companies. Furthermore, this policy disincentivizes non-hydrocarbon sector growth, which is critical to create more jobs in the Kingdom. The Saudi government should scrap Saudization and instead focus on education that will build skills for employment.
Structural Barriers to Employment in Saudi Arabia
Saudization is an artificial employment requirement that does not address the structural problems with the Saudi economy that contribute to youth unemployment, such as an overreliance on the hydrocarbon sector, a constantly growing social transfer system, and insufficient private sector growth.
Saudi Arabia’s economy is largely based on petroleum, but the hydrocarbon sector is not a reliable source of job creation. Oil and gas comprise about 45% of fiscal revenues, 55% of GDP, and 90% of export revenue – yet, the national oil company employs less than 1% of Saudi labor force. The hydrocarbon sector does create a source of revenues for the government’s generous social transfer programs; these programs, however, may not be sustainable, crowed out other social expenditures, and most of all, do nothing to address unemployment. As an example, in lieu of real social reforms, the Saudi government introduced new fiscal initiatives on February 23 and March 18, 2011 to quell domestic protests. Although these types of social transfers may help to immediately pacify the population, the IMF notes that these programs will require oil prices sustained higher than $90 per barrel for the next several years. That may not be sustainable and puts a huge burden on the Saudi government to control oil prices.
Given the problems created by reliance on the hydrocarbon sector for employment, the non-hydrocarbon sector of the Saudi economy is critical. This sector, however, has not been able to create enough jobs for Saudis. Over the next five years, the IMF estimates that private sector non-oil GDP will need to grow by 7.5% annually to create a sufficient number of employment opportunities for the domestic population. While Saudization addresses the issue that most private sector jobs are being allocated to more qualified expatriate workers, the policy increases the operating costs of private companies in the Kingdom, thereby reducing incentives for investment and hindering non-hydrocarbon growth. For example, a recent equity research report by EFG Hermes suggested that Saudi companies will meet Saudization requirements in the near term by hiring more Saudis rather than reducing the number of expatriates, given the skills mismatches of Saudi workers. Basically, companies are being forced to increase personnel costs simply to satisfy a legislative mandate.
Furthermore, both the appeasement tax and Saudization may have a feedback loop creating more pressure on the government. As Saudis become wealthier and more connected to the rest of the world, their expectations for employment and inclusion in the economy will continue grow. Higher expectations combined with growing dissatisfaction with unfulfilling employment opportunities will further increase the government’s cost of appeasement. Therefore, creating sustainable economic opportunities for its citizens will mitigate the long-run fiscal burden of providing appeasement handouts and remove the need for Saudization, as long as Saudis have the skills necessary to be competitive employees.
Education for Employment
The underlying cause of youth unemployment is that, even with a postsecondary degree, Saudi graduates lack the right skills for jobs in a modern, knowledge-based economy. The Saudi education system itself has flaws, but a main problem is that students choose to study subjects that have no direct linkages to labor markets. For example, in 2008, 40% of university students in Saudi Arabia were concentrating in arts and humanities (versus averages of 20% and 17% in Asia and Latin America, respectively), while only 24% chose to study science or engineering. Furthermore, there is a social stigma against technical and vocational training, and any type of university degree, even one that is very unlikely to lead to employment, is socially viewed as superior.
To reduce youth unemployment—and the risk of social instability in the Kingdom—the government ought to at least address skills mismatches by orienting the education system toward private sector employment opportunities. Ideally, Saudi Arabia should scrap Saudization and instead focus on making their graduates competitive employees. Forcing graduates to compete for jobs will ensure that they choose education tracks that are conducive to employment. Additionally, technical and vocational programs should be associated with prestigious universities and fellowships, which would alleviate some of the social stigma of choosing this track. Lastly, the government should adopt a national quality assurance framework to regulate private education companies doing business in the Kingdom to ensure that degrees and certificates are uniform across the country. These changes, paired with continued investments in upgrading the overall education sector, should help to link skill-based post-secondary education with employment opportunities.
Encouraging youth to select education for employment will result in more qualified workers that will naturally increase the demand for Saudi workers and ultimately reduce the cost of doing business in the country. The government and the royal family should act quickly on this issue, for their own sake and for the sake of the growing youth population in their kingdom.
Thursday, October 27, 2011
What Do We Do With Data Soup?
Katherine DiSalvo, MPA
As policy professionals, we’re likely to encounter messes of contradictory findings more and more throughout our careers.
There is contradictory data on important issues like the real level of US poverty, whether moving people out of a neighborhood of concentrated poverty improves their chances in life, the success of charter schools, or the effectiveness of giving away free bed-nets to combat malaria.
Do you know what to do with data soup? At the Woodrow Wilson School, I don’t think we students learn this sufficiently.
According to R. Kent Weaver, in Ending Welfare as We Know It (Brookings, 2000), the 1980s and 1990s saw a “multiplication” of policy research with “differing assumptions and conclusions.” Simultaneously, interest groups were adopting social science techniques and creating “a welter of conflicting findings.” In a separate article Weaver and a colleague assert that this may result in the “devaluation of the currency” of policy research. Weaver argues that it may “cause legislators to simply dismiss all evidence that does not fit their personal or constituency preferences.”
Devaluation of policy research is becoming commonplace. Even in the era of “data-driven” education leadership, Brenda Welburn, the head of the National Association of State Boards of Education (NASBE), recently told a WWS workshop team researching “School Choice and Impacts on Cities” that State Boards of Education members don’t know whose data to trust. As board members attempt to make state education policy and funding decisions, sometimes they don’t know how to do it with facts. “We [at NASBE] are dealing with perceptions, often,” Welburn said.
I don’t think it’s easy to digest data soup, and I think the Woodrow Wilson School needs to do more to help its students develop this ability. You may scoff and tell me you know how to wade through the stew. You know statistics! You know what research methods matter!
I don’t think any policy professional can rely on statistical prowess alone. The statistics program at the Wilson School is strong, and its decision to expand statistics requirements was a good one. However, with our limited time we students (not to mention professionals) cannot dig into data sets, look at assumptions, and evaluate every conclusion we read for ourselves. While some such analysis might be possible before an important policy decision or publication, we consume too much information to scrutinize it all.
The best proof that policy students won’t always use technical skills to sort through conflicting data professionally is that Woodrow Wilson students don’t always do so here! When I encounter conflicting data in classes, I’m too often told we students should dig deeper and decide who’s right…later.
We can’t rely exclusively on the “the gold standard” professors teach us to love: data generated by randomized control trials (RCTs). This creates an easy top tier of information on too few topics. Additionally, all the emphasis we hear on the “gold standard” may lead us to trust in RCT-based research too easily. The best part of the WWS course on data-based decision making is hearing Professor Lorenzo Moreno talk about how complicated it can be to do the right thing in the evaluation field. All that glitters…
We policy students need more practice criticizing questionable research. We need more practice wading through data mess and taking and defending a stand – not on politics, as we do in the introductory 501 course, Politics and Public Policy, but a stand on what we think is the truth. We need more sophisticated conversations about what data to trust and about how to evaluate vendors of policy research when we cannot evaluate each product. We need more shorthand than one “gold” standard.
We also need to talk about making policy in a world where different “facts” are consumed by different constituencies, and the truth is always up for debate. It’s the world in which we live, and it’s likely to get worse. If the Woodrow Wilson School could prepare us to digest data soup and to help change these cooking trends, that would truly be in the nation’s service and in the service of all nations.
As policy professionals, we’re likely to encounter messes of contradictory findings more and more throughout our careers.
There is contradictory data on important issues like the real level of US poverty, whether moving people out of a neighborhood of concentrated poverty improves their chances in life, the success of charter schools, or the effectiveness of giving away free bed-nets to combat malaria.
Do you know what to do with data soup? At the Woodrow Wilson School, I don’t think we students learn this sufficiently.
According to R. Kent Weaver, in Ending Welfare as We Know It (Brookings, 2000), the 1980s and 1990s saw a “multiplication” of policy research with “differing assumptions and conclusions.” Simultaneously, interest groups were adopting social science techniques and creating “a welter of conflicting findings.” In a separate article Weaver and a colleague assert that this may result in the “devaluation of the currency” of policy research. Weaver argues that it may “cause legislators to simply dismiss all evidence that does not fit their personal or constituency preferences.”
Devaluation of policy research is becoming commonplace. Even in the era of “data-driven” education leadership, Brenda Welburn, the head of the National Association of State Boards of Education (NASBE), recently told a WWS workshop team researching “School Choice and Impacts on Cities” that State Boards of Education members don’t know whose data to trust. As board members attempt to make state education policy and funding decisions, sometimes they don’t know how to do it with facts. “We [at NASBE] are dealing with perceptions, often,” Welburn said.
I don’t think it’s easy to digest data soup, and I think the Woodrow Wilson School needs to do more to help its students develop this ability. You may scoff and tell me you know how to wade through the stew. You know statistics! You know what research methods matter!
I don’t think any policy professional can rely on statistical prowess alone. The statistics program at the Wilson School is strong, and its decision to expand statistics requirements was a good one. However, with our limited time we students (not to mention professionals) cannot dig into data sets, look at assumptions, and evaluate every conclusion we read for ourselves. While some such analysis might be possible before an important policy decision or publication, we consume too much information to scrutinize it all.
The best proof that policy students won’t always use technical skills to sort through conflicting data professionally is that Woodrow Wilson students don’t always do so here! When I encounter conflicting data in classes, I’m too often told we students should dig deeper and decide who’s right…later.
We can’t rely exclusively on the “the gold standard” professors teach us to love: data generated by randomized control trials (RCTs). This creates an easy top tier of information on too few topics. Additionally, all the emphasis we hear on the “gold standard” may lead us to trust in RCT-based research too easily. The best part of the WWS course on data-based decision making is hearing Professor Lorenzo Moreno talk about how complicated it can be to do the right thing in the evaluation field. All that glitters…
We policy students need more practice criticizing questionable research. We need more practice wading through data mess and taking and defending a stand – not on politics, as we do in the introductory 501 course, Politics and Public Policy, but a stand on what we think is the truth. We need more sophisticated conversations about what data to trust and about how to evaluate vendors of policy research when we cannot evaluate each product. We need more shorthand than one “gold” standard.
We also need to talk about making policy in a world where different “facts” are consumed by different constituencies, and the truth is always up for debate. It’s the world in which we live, and it’s likely to get worse. If the Woodrow Wilson School could prepare us to digest data soup and to help change these cooking trends, that would truly be in the nation’s service and in the service of all nations.
Monday, October 17, 2011
D.C. Public Schools: 1 AMR (After Michelle Rhee)
Jeff Ross, MPA
One year ago this month, Michelle Rhee stepped down as chancellor of the District of Columbia Public Schools (DCPS). Anyone who has lived in the District knows that it is an odd mix of national and local priorities, Redskins fans and Cowboys fans, transplanted individuals and born-and-raised lifers. But no matter how connected or disconnected one is with the goings on of local DC issues and politics, nearly everyone who lived in the District from 2007-2010 had an opinion about “that Korean lady who was running the schools.”
Ms. Rhee, selected as chancellor by Mayor Adrian Fenty in 2007 shortly after the D.C. City Council voted to establish direct mayoral control of the public school system, was young for an urban superintendent and had a non-traditional background for a superintendent of a large urban school district. She taught elementary school in Baltimore for three years before founding the non-profit The New Teacher Project, an organization that has worked with urban school districts to recruit or hire over 43,000 teachers. At the time, DCPS had long been considered one of the worst performing school districts in the nation, a system that spent one of the highest amounts of dollars per student yet got some of the lowest academic outcomes in the country. Leadership had been a constant revolving door, with six different superintendents in the previous ten years.
In the three plus years of Ms. Rhee’s controversial tenure, she brought a sense of urgency coupled with significant change aimed at improving the school system as quickly as possible. Washington Post writer Bill Turque, who covered the D.C. Public School system since relatively early in Ms. Rhee’s tenure, recently wrote a review of Ms. Rhee’s record. For the most part, I think it paints a fair picture of the successes (improvements in student outcomes and test scores, operational improvements in central office functions and food service, increased focus and importance on education as a civic issue) and challenges (opposition from some parents and community members, continued high principal turnover) faced by Ms. Rhee during her tenure, with a few caveats:
1) While the statistics surrounding D.C. Public Schools prior to Ms. Rhee’s tenure mentioned earlier are well-known and oft-cited, I think people often forget just how much agreement there was in the unacceptable state of affairs in DCPS at that time. The landslide 9-2 vote to enact mayoral control coupled with the unanimous vote to approve Ms. Rhee as chancellor are but two markers of the universal call for change at the time.
2) A noticeable omission from the article was the focus Ms. Rhee and DCPS put on ensuring every school had an arts, music, and physical education program. While parents and teachers alike often complain of the edging out of more robust educational opportunities in some schools (particularly those in urban areas) in favor of more practice with tested subjects (reading and math), DCPS under Ms. Rhee invested significant resources to ensure these options were available to all students for the first time.
3) Turque mentions that parents and community members felt that Ms. Rhee moved forward with school improvement efforts – including but not limited to closing underutilized school buildings – despite their stated objections. Indeed, Ms. Rhee faced decreasing approval ratings from D.C. residents during her tenure. However, it is important to note that while residents disapproved of Ms. Rhee personally in increasing numbers, Washington Post polls found that residents were in fact more satisfied with the levels of safety and overall satisfaction with public schools under Ms. Rhee, suggesting that much of the discontent was communication or personality related.
As a former employee of DCPS under Ms. Rhee, I certainly think that there were things that could have been done better. While I’m sympathetic to the position of Ms. Rhee and others that when it comes to student outcomes, change and improvement cannot happen fast enough, I think that it is crucial to present such efforts in as fair and collaborative manner as possible. While some would call that a subtle distinction, such a mindset shift would potentially have large effects on the number of new programs created, the rollout of vital new systems like the IMPACT teacher evaluation system, and communication with elected officials and community members. That’s why I’m confident in the future of DCPS under the leadership of current DCPS chancellor Kaya Henderson, who brings the same unwavering passion and commitment to improved student outcomes with a more inclusive style. Nevertheless, I think it’s important to take stock of where things were just a short time ago, and to have the understanding that such meaningful and positive change wouldn’t have occurred without the efforts of Chancellor Michelle Rhee.
One year ago this month, Michelle Rhee stepped down as chancellor of the District of Columbia Public Schools (DCPS). Anyone who has lived in the District knows that it is an odd mix of national and local priorities, Redskins fans and Cowboys fans, transplanted individuals and born-and-raised lifers. But no matter how connected or disconnected one is with the goings on of local DC issues and politics, nearly everyone who lived in the District from 2007-2010 had an opinion about “that Korean lady who was running the schools.”
Ms. Rhee, selected as chancellor by Mayor Adrian Fenty in 2007 shortly after the D.C. City Council voted to establish direct mayoral control of the public school system, was young for an urban superintendent and had a non-traditional background for a superintendent of a large urban school district. She taught elementary school in Baltimore for three years before founding the non-profit The New Teacher Project, an organization that has worked with urban school districts to recruit or hire over 43,000 teachers. At the time, DCPS had long been considered one of the worst performing school districts in the nation, a system that spent one of the highest amounts of dollars per student yet got some of the lowest academic outcomes in the country. Leadership had been a constant revolving door, with six different superintendents in the previous ten years.
In the three plus years of Ms. Rhee’s controversial tenure, she brought a sense of urgency coupled with significant change aimed at improving the school system as quickly as possible. Washington Post writer Bill Turque, who covered the D.C. Public School system since relatively early in Ms. Rhee’s tenure, recently wrote a review of Ms. Rhee’s record. For the most part, I think it paints a fair picture of the successes (improvements in student outcomes and test scores, operational improvements in central office functions and food service, increased focus and importance on education as a civic issue) and challenges (opposition from some parents and community members, continued high principal turnover) faced by Ms. Rhee during her tenure, with a few caveats:
1) While the statistics surrounding D.C. Public Schools prior to Ms. Rhee’s tenure mentioned earlier are well-known and oft-cited, I think people often forget just how much agreement there was in the unacceptable state of affairs in DCPS at that time. The landslide 9-2 vote to enact mayoral control coupled with the unanimous vote to approve Ms. Rhee as chancellor are but two markers of the universal call for change at the time.
2) A noticeable omission from the article was the focus Ms. Rhee and DCPS put on ensuring every school had an arts, music, and physical education program. While parents and teachers alike often complain of the edging out of more robust educational opportunities in some schools (particularly those in urban areas) in favor of more practice with tested subjects (reading and math), DCPS under Ms. Rhee invested significant resources to ensure these options were available to all students for the first time.
3) Turque mentions that parents and community members felt that Ms. Rhee moved forward with school improvement efforts – including but not limited to closing underutilized school buildings – despite their stated objections. Indeed, Ms. Rhee faced decreasing approval ratings from D.C. residents during her tenure. However, it is important to note that while residents disapproved of Ms. Rhee personally in increasing numbers, Washington Post polls found that residents were in fact more satisfied with the levels of safety and overall satisfaction with public schools under Ms. Rhee, suggesting that much of the discontent was communication or personality related.
As a former employee of DCPS under Ms. Rhee, I certainly think that there were things that could have been done better. While I’m sympathetic to the position of Ms. Rhee and others that when it comes to student outcomes, change and improvement cannot happen fast enough, I think that it is crucial to present such efforts in as fair and collaborative manner as possible. While some would call that a subtle distinction, such a mindset shift would potentially have large effects on the number of new programs created, the rollout of vital new systems like the IMPACT teacher evaluation system, and communication with elected officials and community members. That’s why I’m confident in the future of DCPS under the leadership of current DCPS chancellor Kaya Henderson, who brings the same unwavering passion and commitment to improved student outcomes with a more inclusive style. Nevertheless, I think it’s important to take stock of where things were just a short time ago, and to have the understanding that such meaningful and positive change wouldn’t have occurred without the efforts of Chancellor Michelle Rhee.
Sunday, October 9, 2011
Seeing Beyond Tomorrow: The scourge of extreme poverty and finally ending it
Ayokunle Abogan, MPA
Can we end extreme poverty within the next three decades?
This question was posed in an article I read while in Nigeria, my home country. In trying to answer it, I cannot help but view the problem from a personal angle. Herein I share Modupe’s story.
Modupe is a woman I met during a volunteer project created to eradicate poverty in Nigeria. She is a Nigerian woman, likely in her mid-thirties, although she can only guess. AIDS (contracted from her now-dead husband), poverty, and hunger have taken a devastating toll on her—she looks more like 60. Does Modupe worry whether her six children also have AIDS? No. She doesn’t have time to worry. She’s focused solely on daily survival. Her mother, who lives with her, needn’t worry about AIDS—she’s already dying of tuberculosis.
Modupe scavenges for scrap paper at the rubbish dump to sell to market vendors. If Modupe is lucky, she can make as much as 60 cents a day. When luckier, she finds discarded dregs of produce, meat and dairy. Most days Modupe is not lucky. She averages three to four meals in a week. Land surrounds her leaf-and-mud hut but the adjacent factory’s chemical wastes have rendered the land toxic, infertile. It doesn’t matter. Dying of AIDS, Modupe can barely scavenge, never mind farm, competing alongside scores of others scrabbling for scraps. They suffer, too.
I know Modupe. I know many like her. Too many.
Nearly 1.2 billion people worldwide—one-sixth of the world’s population—suffer from extreme poverty. No clean water, sanitation, or electricity. The numbers are staggering. Illiteracy ensures that they continue to suffer. Some regions with entrenched cycles of poverty, death, and inequity, helplessly pass them from one generation to another. In my continent, Africa, more than half of us live in extreme poverty. Come 2040, nearly 30 years from now, the world’s population is forecast to increase to 8.8 billion, with more than 70% living in so-called developing countries. If we can’t manage poverty now, how will we manage it then on such a greater scale?
To cite statistics here, however, is to intellectualize a crisis that one must feel viscerally. Ironically, society today is now inured to others’ pain while being simultaneously, due to technological advances, close enough to observe it. We witness yet remain detached, isolated. But if you experience directly what I have experienced, the more critical question becomes: “Can we really afford to wait 30 years?”
International organizations including the World Bank and the UN emphasize improving income levels. That doesn’t work. It benefits only a small percentage, the educated, who better grasp how to improve living standards. The illiterate do not.
Basic needs must be met first. How can people educate themselves if they don’t even have food or water? If disease is everywhere around them? Surviving today isn’t just a means to an end; it becomes the end itself. Resolving basic needs will then naturally segue into health services, education and improved housing.
These are the core necessities we must provide our starving brothers and sisters:
Modupe doesn’t have 30 years. Neither do we.
Can we end extreme poverty within the next three decades?
This question was posed in an article I read while in Nigeria, my home country. In trying to answer it, I cannot help but view the problem from a personal angle. Herein I share Modupe’s story.
Modupe is a woman I met during a volunteer project created to eradicate poverty in Nigeria. She is a Nigerian woman, likely in her mid-thirties, although she can only guess. AIDS (contracted from her now-dead husband), poverty, and hunger have taken a devastating toll on her—she looks more like 60. Does Modupe worry whether her six children also have AIDS? No. She doesn’t have time to worry. She’s focused solely on daily survival. Her mother, who lives with her, needn’t worry about AIDS—she’s already dying of tuberculosis.
Modupe scavenges for scrap paper at the rubbish dump to sell to market vendors. If Modupe is lucky, she can make as much as 60 cents a day. When luckier, she finds discarded dregs of produce, meat and dairy. Most days Modupe is not lucky. She averages three to four meals in a week. Land surrounds her leaf-and-mud hut but the adjacent factory’s chemical wastes have rendered the land toxic, infertile. It doesn’t matter. Dying of AIDS, Modupe can barely scavenge, never mind farm, competing alongside scores of others scrabbling for scraps. They suffer, too.
I know Modupe. I know many like her. Too many.
Nearly 1.2 billion people worldwide—one-sixth of the world’s population—suffer from extreme poverty. No clean water, sanitation, or electricity. The numbers are staggering. Illiteracy ensures that they continue to suffer. Some regions with entrenched cycles of poverty, death, and inequity, helplessly pass them from one generation to another. In my continent, Africa, more than half of us live in extreme poverty. Come 2040, nearly 30 years from now, the world’s population is forecast to increase to 8.8 billion, with more than 70% living in so-called developing countries. If we can’t manage poverty now, how will we manage it then on such a greater scale?
To cite statistics here, however, is to intellectualize a crisis that one must feel viscerally. Ironically, society today is now inured to others’ pain while being simultaneously, due to technological advances, close enough to observe it. We witness yet remain detached, isolated. But if you experience directly what I have experienced, the more critical question becomes: “Can we really afford to wait 30 years?”
International organizations including the World Bank and the UN emphasize improving income levels. That doesn’t work. It benefits only a small percentage, the educated, who better grasp how to improve living standards. The illiterate do not.
Basic needs must be met first. How can people educate themselves if they don’t even have food or water? If disease is everywhere around them? Surviving today isn’t just a means to an end; it becomes the end itself. Resolving basic needs will then naturally segue into health services, education and improved housing.
These are the core necessities we must provide our starving brothers and sisters:
- Enhanced food production. Food is fuel; we don’t run without it. Farmers comprise 60+ percent of the world’s extremely poor. Why not teach subsistence farming techniques for that 60 percent? A simple application of the “give a man a fish and he eats for a day; teach him to fish and he eats for a lifetime” philosophy. Governments must invest in responsible farming techniques, tools, storage, and irrigation, and also develop suitable transit of farm products to outlying marketplaces.
- Basic Infrastructure and Amenities. Clean water supply, electricity, and basic sanitation are taken for granted yet are all but unknown to the impoverished. The technology exists! Waste recycling, management, education and facilities will cut disease. Healthcare facilities decrease malaria and HIV/AIDS and preventable death. Rainwater harvesting, water wells, and hand pumps when appropriate, can provide additional water—substantial hours are spent daily traveling to obtain water; local water quality inspections limits typhoid and other water-related problems. Constructing micro-hydroelectric plants to boost electricity supply can funnel power to those outside centralized grid sources. Basic sanitation systems eradicate health risks, lessen water source pollution, and enhance human dignity.
- Education. In addition to lifestyle education, developing human capital leads to better jobs, wages, and living conditions. The educated make informed decisions concerning healthcare, reproduction, employment, and economic equality. Attendance at school until a legally-employable age, for men and women, and vocational training/skills improvement for adults lacking education are a must.
- Debt Relief. Developed countries not only consume most of the world’s resources but also have technology to improve their economies. With debt relief, struggling countries can focus their resources to address national poverty. Fluctuating food prices and high energy proces make it more difficult for poor people to afford enough food to eat. Food and energy represent 60 percent of impoverished household expenditure. Even the US, an affluent nation, has seen much of its middle and lower classes forced into poverty by rising food and energy costs while battling unemployment and foreclosure in an economic crisis. The Middle East continually faces riots due to spiraling food costs. Mitigating the devastating price swings and economic slowdowns in developing countries is critical.
Modupe doesn’t have 30 years. Neither do we.
Making Butter Without the Cream of the Crop: Overcoming the bimodality of urban school choice
Drew Haugen, MPA
In the education reform world, charter schools have been garnering a lot of attention. Schools like KIPP (Knowledge is Power Program), started in Houston in 1994, have achieved tremendous gains with students from underserved communities and have taken their model nation-wide.
Charters like KIPP also usually endure a barrage of criticism and scrutiny. Common critiques range from charges that they take the “cream of the crop” of available students to accusations of being quick to expel students with behavior, language, or disability issues. Others argue that their model is financially unsustainable.
My critique of charter schools like KIPP is through a different lens—what I call the Commitment Dilemma.
Commitment to a school like KIPP, with its longer days, Saturday school, longer academic year, hours of homework every night, and a rigorous behavioral culture, is a hefty commitment for a child of any ability level to make. Colorado State Senator Mike Johnston, my boss this summer and former principal of a high school that dealt with at-risk students, explained the Commitment Dilemma to me using the following analogy.
Let’s say, hypothetically, that all students have 100 “Free-time Commitment Points.” These 100 points represent the available free-time a student has for the activities to which they can devote their energies outside of mandatory requirements (sleeping, the seven-hour school day required by law, etc.).
Students can allocate their Free-time Commitment Points however they like, divvying them up between activities like studying, family time, organized sports, time with friends, and so forth.
For the purposes of this example let’s say that to graduate in the middle of her class at a KIPP school the average low-income urban student, Shirley, must allocate 80 of her total 100 Free-time Commitment Points to KIPP and the additional time, activities, and homework KIPP requires.
In contrast, Shirley’s local school, which is not as academically rigorous and does not yield the same high probability of college acceptance for Shirley as KIPP, is nonetheless a safe and moderately performing school that many of Shirley’s friends attend. To graduate from this school in the middle of her class requires 40 of Shirley’s Commitment Points.
Now let’s pause for a moment and inject some self-reflection into this analogy. Say I give you two options for the undergraduate institution you will attend: Vanderbilt or CalTech. You must choose one of these two options.
If my intuition is correct, a number of my capable and intelligent readers will choose Vanderbilt and a number will choose CalTech.
Applying our Free-time Commitment Points analogy, my guess is that readers who want to devote a large portion of their Points to academic extracurriculars like course reading, studying, and so forth in exchange for a more rigorous academic experience will choose CalTech. I’d also guess that readers who want to spend more of their Free-time Commitment Points on activities not directly related to school, such as social events, sports, and so on, will choose Vanderbilt.
This is a rough analogy, but the gist is this: students that are equally capable and intelligent (my readership) will choose different schools for different reasons and some of these reasons have little or nothing to do with academic pursuits.
Luckily for us, CalTech and Vanderbilt are both academically rigorous institutions that yield intellectual development and professional readiness for their students and a baseline of required academic proficiency in order to receive a diploma.
The same cannot be said of Shirley’s choices in our example. Even if Shirley is an above-average student with 60 Points to commit to academic enrichment, she falls short of the extraordinary commitment required to succeed at KIPP and will most likely end up in her neighborhood school. Shirley becomes a cautionary tale. This is the Commitment Dilemma.
It is true that KIPP achieves extraordinary results. But KIPP also enjoys extraordinarily committed administrators, teachers, parents, and students. The portion of our society willing to devote this much free-time and energy to schooling is a small minority.
If we can assume normality of student free-time commitment levels, then ideally there should exist a normal distribution of school options for students like Shirley to pick from. It would be a relatively easy endeavor for Shirley to find a school to match, or come close to, her 60-Commitment Point level.
All schools in this distribution would require a baseline academic proficiency of their students equivalent to a high school diploma or GED. As schools increase their academic enrichment activities (longer days, school years, more homework, AP and IB courses, advanced diplomas, etc.), so would their Free-time Commitment requirement.
Near the top of this distribution, we would find schools that prepare students for entry into elite higher education institutions. Near the bottom of this distribution, we would find schools that prepare students for success in community and junior colleges and entry-level four-year institutions.
Unfortunately, the reality of what most poor urban students encounter is a bimodal distribution of school options. Their first option is a low-commitment school, usually a failing public school that graduates them unprepared for success at a community college. The other option is a high commitment school, usually a rigorous charter school that prepares them for success at a mid-range four-year college.
The education reform movement must devote more energy to “building out the middle” of this currently bimodal distribution of school options. The development of rigorous and challenging schools for administrators, teachers, parents, and students of all commitment levels must be a much stronger priority.
A fitting model is the California higher education system, which was reorganized under the 1960 Master Plan for Higher Education.
The Master Plan organized California’s higher education system into tiers. Tier 1 is comprised of the state’s marquee institutions—the University of California (UC) system. The California State University system makes up Tier 2, and the state’s community and junior college system rounds out Tier 3.
In California, the top 12.5% of graduating seniors are guaranteed a spot at one of the UC schools. The top 33% is guaranteed a spot at a Cal State, and California Community Colleges are to admit “any student capable of benefiting from instruction.”
This diversification of higher educational opportunities in California has yielded tremendous results: enrollment has increased ten-fold since 1960, while the California population has only approximately tripled. What’s more, there are avenues for advancement and enrichment between tiers: Santa Monica Community College in Los Angeles is the #1 institution for transfers to UCLA and UC-Berkeley.
By providing a similarly wide array of school options for public K-12, all of which afford students rigorous opportunities for academic proficiency and enrichment, our school system will yield higher retention levels, better academic fits for students, and more robust achievement on a large scale. Our education system must adapt itself to make butter with all types of milk—not just the cream.
In the education reform world, charter schools have been garnering a lot of attention. Schools like KIPP (Knowledge is Power Program), started in Houston in 1994, have achieved tremendous gains with students from underserved communities and have taken their model nation-wide.
Charters like KIPP also usually endure a barrage of criticism and scrutiny. Common critiques range from charges that they take the “cream of the crop” of available students to accusations of being quick to expel students with behavior, language, or disability issues. Others argue that their model is financially unsustainable.
My critique of charter schools like KIPP is through a different lens—what I call the Commitment Dilemma.
Commitment to a school like KIPP, with its longer days, Saturday school, longer academic year, hours of homework every night, and a rigorous behavioral culture, is a hefty commitment for a child of any ability level to make. Colorado State Senator Mike Johnston, my boss this summer and former principal of a high school that dealt with at-risk students, explained the Commitment Dilemma to me using the following analogy.
Let’s say, hypothetically, that all students have 100 “Free-time Commitment Points.” These 100 points represent the available free-time a student has for the activities to which they can devote their energies outside of mandatory requirements (sleeping, the seven-hour school day required by law, etc.).
Students can allocate their Free-time Commitment Points however they like, divvying them up between activities like studying, family time, organized sports, time with friends, and so forth.
For the purposes of this example let’s say that to graduate in the middle of her class at a KIPP school the average low-income urban student, Shirley, must allocate 80 of her total 100 Free-time Commitment Points to KIPP and the additional time, activities, and homework KIPP requires.
In contrast, Shirley’s local school, which is not as academically rigorous and does not yield the same high probability of college acceptance for Shirley as KIPP, is nonetheless a safe and moderately performing school that many of Shirley’s friends attend. To graduate from this school in the middle of her class requires 40 of Shirley’s Commitment Points.
Now let’s pause for a moment and inject some self-reflection into this analogy. Say I give you two options for the undergraduate institution you will attend: Vanderbilt or CalTech. You must choose one of these two options.
If my intuition is correct, a number of my capable and intelligent readers will choose Vanderbilt and a number will choose CalTech.
Applying our Free-time Commitment Points analogy, my guess is that readers who want to devote a large portion of their Points to academic extracurriculars like course reading, studying, and so forth in exchange for a more rigorous academic experience will choose CalTech. I’d also guess that readers who want to spend more of their Free-time Commitment Points on activities not directly related to school, such as social events, sports, and so on, will choose Vanderbilt.
This is a rough analogy, but the gist is this: students that are equally capable and intelligent (my readership) will choose different schools for different reasons and some of these reasons have little or nothing to do with academic pursuits.
Luckily for us, CalTech and Vanderbilt are both academically rigorous institutions that yield intellectual development and professional readiness for their students and a baseline of required academic proficiency in order to receive a diploma.
The same cannot be said of Shirley’s choices in our example. Even if Shirley is an above-average student with 60 Points to commit to academic enrichment, she falls short of the extraordinary commitment required to succeed at KIPP and will most likely end up in her neighborhood school. Shirley becomes a cautionary tale. This is the Commitment Dilemma.
It is true that KIPP achieves extraordinary results. But KIPP also enjoys extraordinarily committed administrators, teachers, parents, and students. The portion of our society willing to devote this much free-time and energy to schooling is a small minority.
If we can assume normality of student free-time commitment levels, then ideally there should exist a normal distribution of school options for students like Shirley to pick from. It would be a relatively easy endeavor for Shirley to find a school to match, or come close to, her 60-Commitment Point level.
All schools in this distribution would require a baseline academic proficiency of their students equivalent to a high school diploma or GED. As schools increase their academic enrichment activities (longer days, school years, more homework, AP and IB courses, advanced diplomas, etc.), so would their Free-time Commitment requirement.
Near the top of this distribution, we would find schools that prepare students for entry into elite higher education institutions. Near the bottom of this distribution, we would find schools that prepare students for success in community and junior colleges and entry-level four-year institutions.
Unfortunately, the reality of what most poor urban students encounter is a bimodal distribution of school options. Their first option is a low-commitment school, usually a failing public school that graduates them unprepared for success at a community college. The other option is a high commitment school, usually a rigorous charter school that prepares them for success at a mid-range four-year college.
The education reform movement must devote more energy to “building out the middle” of this currently bimodal distribution of school options. The development of rigorous and challenging schools for administrators, teachers, parents, and students of all commitment levels must be a much stronger priority.
A fitting model is the California higher education system, which was reorganized under the 1960 Master Plan for Higher Education.
The Master Plan organized California’s higher education system into tiers. Tier 1 is comprised of the state’s marquee institutions—the University of California (UC) system. The California State University system makes up Tier 2, and the state’s community and junior college system rounds out Tier 3.
In California, the top 12.5% of graduating seniors are guaranteed a spot at one of the UC schools. The top 33% is guaranteed a spot at a Cal State, and California Community Colleges are to admit “any student capable of benefiting from instruction.”
This diversification of higher educational opportunities in California has yielded tremendous results: enrollment has increased ten-fold since 1960, while the California population has only approximately tripled. What’s more, there are avenues for advancement and enrichment between tiers: Santa Monica Community College in Los Angeles is the #1 institution for transfers to UCLA and UC-Berkeley.
By providing a similarly wide array of school options for public K-12, all of which afford students rigorous opportunities for academic proficiency and enrichment, our school system will yield higher retention levels, better academic fits for students, and more robust achievement on a large scale. Our education system must adapt itself to make butter with all types of milk—not just the cream.
Sunday, October 2, 2011
When it Comes to Colleges, Spare Us the Rankings (But Keep the Numbers)
Dan Fichtler, MPA
Arguably the most relied-upon college admissions advice for high school seniors comes not from a guidance counselor or an admissions officer or even a parent, but rather from the pages of a well-known magazine. The editors of U.S. News & World Report (USNWR) have just released the 2012 version of their annual college rankings (Editor’s note: Princeton #1!), and like in recent years, many soon-to-be applicants took another step towards higher education by consulting this publication to navigate the difficult decisions that lie ahead.
As has also become something of an autumn tradition, this year’s USNWR rankings were taken out to the woodshed by a wide variety of educators and pundits. The rankings have existed for nearly three decades now, and the charges laid against them are fairly predictable. Critics often cite the use of input-driven data (SAT scores and class rank of incoming students) rather than output-driven data (post-graduation salaries and percentages of alumni pursing or holding advanced degrees). Another common complaint is the ability of colleges and universities to manipulate data by changing their practices; that is, allocating resources towards those characteristics that the USNWR editors deem important, at the expense of the many other crucial elements of a college education. Anyone who has ever wondered why so many college courses are capped at the unusual level of 19 students, for example, can likely find an answer within the USNWR rankings formula.
But perhaps most objectionable is the idea that colleges and universities can be ranked in such a neat and simplistic way – or that they can be ranked at all. This criticism has more merit than any other that is regularly piled onto the USNWR editors (more on this point in a bit).
So how does USNWR determine these rankings? Well, it assigns each college and university an overall score, which is calculated via a relatively simple process. Data on a series of school characteristics are compiled, computed into common units, and summed, with different characteristics holding different weights. The schools are then ranked based upon their overall scores.
USNWR makes no attempt to hide its methodology[1], and yet anecdotal evidence suggests that very few high school seniors or their parents (or really anyone else for that matter) understand how the scores are calculated. And while an informal survey certainly cannot substitute for more rigorous hypothesis testing techniques, I sought to observe this anecdotal evidence. I asked about a dozen well-educated individuals to estimate the weight given to a school’s acceptance rate in the USNWR rankings. Answers ranged from a low of 8% to a high of 100%, with most falling in the range of 15-20%. The correct answer: 1.5%.
The table below provides the factors (and weights) used to determine the USNWR overall scores.
Perhaps what appears to be systematic overestimation of the importance of acceptance rates comes from perceptions encouraged by the USNWR critics. Stories in the media that demonize elite universities for lowering their acceptance rates to game the rankings may lead readers to believe that this single variable plays an important role in the rankings formula. Of course, there are plenty of reasons to be concerned about ever lower acceptance rates. There are also, however, plenty of reasons why schools might seek to lower their acceptance rates, USNWR aside. To directly or indirectly attribute causal effects to USNWR for this trend without considering these other factors represents nothing more than lazy analysis.
As the critics know, it is not difficult to find flaws in this system, or at least to question the validity of the specific weights assigned to various factors. What the critics often fail to acknowledge, however, is the valuable service that the USNWR editors provide. That service is the pure provision of information in a market that has otherwise been difficult for consumers to navigate.
Choosing which colleges to apply to, and eventually which to attend, are important life decisions. Future careers, lifelong friends, and even spouses are discovered during these years for many individuals, which therefore raises the stakes on the importance of making good decisions. If we believe that high school students (and parents and counselors) are best served by having access to information on many characteristics of colleges and universities, then we must applaud those who make this information widely available and easy to process. That group includes not only USNWR, by the way, but also the editors of rival rankings systems, who created their rankings in part to rectify what they felt were flaws in the USNWR system.
The creation of new, widely circulated systems suggests that college rankings will not be disappearing anytime soon. Forbes and Washington Monthly have entered the rankings game in recent years, and the college rankings edition of USNWR continues to sell remarkably well. And despite the valid criticisms of the specific USNWR methodology, the larger question remains: can simple rankings of colleges and universities across many variables help consumers?
For example, is #12 Northwestern really an incrementally better university than #13 Johns Hopkins? And is #2 Amherst really an incrementally better liberal arts college than #3 Swarthmore? They surely are not – at least not categorically. Ranking schools on this type of continuous scale therefore makes very little sense. For some students, Northwestern is a better option than Johns Hopkins; for others, the opposite is true. The same goes for Amherst and Swarthmore.
So how do students make these decisions? They could use the wide array of data that USNWR puts forth, comparing schools on the characteristics that are most important to them, whether those are class size or graduation rate or academic reputation. Some students certainly do this. Many others, however, fall victim to the “rankings-as-gospel” syndrome. Swarthmore may be a better fit for their personality and interests, but they cannot get beyond the fact that Amherst is ranked higher, and a mismatch occurs. The solution to this dilemma is quite simple: remove the rankings, keep the numbers.
And so the critics are correct – USNWR seems to be at least partially to blame for mismatching and suboptimal decision-making. But USNWR is also at least partially responsible for the good decisions of those students who are able to look past simple rankings and delve deeper into the data. Since 1983, USNWR has provided crucial information to students in an easy-to-digest manner. For that, they deserve our (partial) thanks.
Arguably the most relied-upon college admissions advice for high school seniors comes not from a guidance counselor or an admissions officer or even a parent, but rather from the pages of a well-known magazine. The editors of U.S. News & World Report (USNWR) have just released the 2012 version of their annual college rankings (Editor’s note: Princeton #1!), and like in recent years, many soon-to-be applicants took another step towards higher education by consulting this publication to navigate the difficult decisions that lie ahead.
As has also become something of an autumn tradition, this year’s USNWR rankings were taken out to the woodshed by a wide variety of educators and pundits. The rankings have existed for nearly three decades now, and the charges laid against them are fairly predictable. Critics often cite the use of input-driven data (SAT scores and class rank of incoming students) rather than output-driven data (post-graduation salaries and percentages of alumni pursing or holding advanced degrees). Another common complaint is the ability of colleges and universities to manipulate data by changing their practices; that is, allocating resources towards those characteristics that the USNWR editors deem important, at the expense of the many other crucial elements of a college education. Anyone who has ever wondered why so many college courses are capped at the unusual level of 19 students, for example, can likely find an answer within the USNWR rankings formula.
But perhaps most objectionable is the idea that colleges and universities can be ranked in such a neat and simplistic way – or that they can be ranked at all. This criticism has more merit than any other that is regularly piled onto the USNWR editors (more on this point in a bit).
So how does USNWR determine these rankings? Well, it assigns each college and university an overall score, which is calculated via a relatively simple process. Data on a series of school characteristics are compiled, computed into common units, and summed, with different characteristics holding different weights. The schools are then ranked based upon their overall scores.
USNWR makes no attempt to hide its methodology[1], and yet anecdotal evidence suggests that very few high school seniors or their parents (or really anyone else for that matter) understand how the scores are calculated. And while an informal survey certainly cannot substitute for more rigorous hypothesis testing techniques, I sought to observe this anecdotal evidence. I asked about a dozen well-educated individuals to estimate the weight given to a school’s acceptance rate in the USNWR rankings. Answers ranged from a low of 8% to a high of 100%, with most falling in the range of 15-20%. The correct answer: 1.5%.
The table below provides the factors (and weights) used to determine the USNWR overall scores.
Methodology Used by U.S. News & World Report in its 2012 College Rankings[2] | |
Component | Weighting (out of 100%) |
Undergraduate Academic Reputation | 22.5% |
Academic peer assessment | 15% |
High school counselor assessment | 7.5% |
Retention | 20% |
Six-year graduation rate | 16% |
Freshman retention rate | 4% |
Faculty Resources | 20% |
Average faculty salary (including benefits) | 7% |
Proportion of classes with fewer than 20 students | 6% |
Proportion of faculty with highest degree in their field | 3% |
Proportion of classes with 50 or more students | 2% |
Student-faculty ratio | 1% |
Proportion of faculty who are full-time employees | 1% |
Student Selectivity | 15% |
SAT and ACT scores of entering students | 7.5% |
Proportion of freshman graduating in top 10% of HS | 6% |
Acceptance rate | 1.5% |
Financial Resources | 10% |
Graduate Rate Performance | 7.5% |
Alumni Giving Rate | 5% |
As the critics know, it is not difficult to find flaws in this system, or at least to question the validity of the specific weights assigned to various factors. What the critics often fail to acknowledge, however, is the valuable service that the USNWR editors provide. That service is the pure provision of information in a market that has otherwise been difficult for consumers to navigate.
Choosing which colleges to apply to, and eventually which to attend, are important life decisions. Future careers, lifelong friends, and even spouses are discovered during these years for many individuals, which therefore raises the stakes on the importance of making good decisions. If we believe that high school students (and parents and counselors) are best served by having access to information on many characteristics of colleges and universities, then we must applaud those who make this information widely available and easy to process. That group includes not only USNWR, by the way, but also the editors of rival rankings systems, who created their rankings in part to rectify what they felt were flaws in the USNWR system.
The creation of new, widely circulated systems suggests that college rankings will not be disappearing anytime soon. Forbes and Washington Monthly have entered the rankings game in recent years, and the college rankings edition of USNWR continues to sell remarkably well. And despite the valid criticisms of the specific USNWR methodology, the larger question remains: can simple rankings of colleges and universities across many variables help consumers?
For example, is #12 Northwestern really an incrementally better university than #13 Johns Hopkins? And is #2 Amherst really an incrementally better liberal arts college than #3 Swarthmore? They surely are not – at least not categorically. Ranking schools on this type of continuous scale therefore makes very little sense. For some students, Northwestern is a better option than Johns Hopkins; for others, the opposite is true. The same goes for Amherst and Swarthmore.
So how do students make these decisions? They could use the wide array of data that USNWR puts forth, comparing schools on the characteristics that are most important to them, whether those are class size or graduation rate or academic reputation. Some students certainly do this. Many others, however, fall victim to the “rankings-as-gospel” syndrome. Swarthmore may be a better fit for their personality and interests, but they cannot get beyond the fact that Amherst is ranked higher, and a mismatch occurs. The solution to this dilemma is quite simple: remove the rankings, keep the numbers.
And so the critics are correct – USNWR seems to be at least partially to blame for mismatching and suboptimal decision-making. But USNWR is also at least partially responsible for the good decisions of those students who are able to look past simple rankings and delve deeper into the data. Since 1983, USNWR has provided crucial information to students in an easy-to-digest manner. For that, they deserve our (partial) thanks.
Wednesday, April 6, 2011
Rediscovering discovery: Investing in public school science labs
Jacob Hartog, MPA
Here’s a Jeopardy clue for IBM's supercomputer, Watson:
“This educational institution has produced more Nobel Prize-winning graduates than Princeton, MIT, Caltech, Oxford, or Yale.”
The question, surprisingly, is “What is the New York City Public School system?”
From the 1930s through the 1970s, the New York schools (and not just that one high school in the Bronx) produced legions of future scientists and engineers, including many who later made pathbreaking discoveries: from superconducting materials to mating bacteria, from Arrow's Impossibility Theorem to the design of the atomic bomb.
Where should we assign the credit for this astonishing productivity? Some of it, undoubtedly, is due to the particular mix of immigrants whose children filled the schools. Some of it is also due to the culture of those times, which celebrated science and discovery, which knew you could free the world from Fascism, stave off Sputnik, and make it to the moon, if you just got the differential equations right.
But part of the credit must go to the science labs.
At the same time as it was teaching long division to future laureates like Richard Feynmann and Robert Solow, New York City built and rehabilitated thousands of schools. On each floor of every one of the intermediate and high schools, they put science labs—rows of lab benches with gas lines and sinks, rows of cabinets full of chemical supplies, and a huge demonstration bench across the front of the room with ring stands and clamps, for showing off what happens when you drop pure sodium into a beaker of water or mix hydrogen peroxide with liquid soap and then add potassium iodide. (Watson, take note: the former explodes, the latter turns to foam.)
I began my teaching career in such a room, in a school built in the South Bronx in 1960, in the immediate aftermath of Sputnik’s blinking passage across the sky. Of course, by the time I got there, the gas lines had long ago been permanently shut off. The faucets were hacked off at the base, leaving jagged pieces of metal coming out of the lab benches. The sinks were choked with debris and, in some cases, emptied not into plumbing but directly into the cabinet below, so that when young Jose and Arthur tried out their model volcano, a flood of baking soda-and-vinegar lava coursed across the floor. The chemical cabinets were empty, except for decades of old dittos and worksheets, several nests of mice, and one unstoppered bottle of concentrated hydrochloric acid, the fumes of which had gradually turned the locker that held it into an ocean of rust.
The South Bronx has had some troubled years, no doubt. But this scene of decay is not unique to poor, inner-city schools. A trip through a middle-class suburban school might also show you sinks that don’t work and gas lines that no one knows how to turn on, microscopes with broken lenses and chemical cabinets with nothing in them but some aged litmus paper that no longer changes color in acid and base. Science class in such rooms can feel less like a trip to the future than a sojourn at an archeological dig.
What happened, to make us disinvest so thoroughly in science education?
Competing priorities within education deserves some of the blame. Although teachers’ salaries in American schools are lower than average for developed countries, American schools employ many more people than they once did, especially in special education and among specialists employed outside the classroom. And increasing regulation (and unnecessary asbestos abatement) has made even modest overhauls of school buildings prohibitively expensive.
Computer technology, meanwhile, has become a tantalizing investment for principals and grant agencies, at the expense of science supplies and equipment. Even science teachers themselves, some of them uncomfortable with the risks and inconvenience of doing real experiments, are often all too willing to pass out the laptops or textbooks and give their students something to do that, unlike hands-on labs, is quiet and indisputably safe. But staring at a screen is not conducting an experiment, and no amount of individualized support can make up for a classroom experience that, regrettably, involves filling out worksheets instead of blowing things up.
The “Sputnik Moment” that President Obama asserted was upon us, in which our relative educational attainment in science and math seems to fall further every year, cannot be solved with any single government intervention. But one thing we can take away from the original Sputnik moment is the importance of investing in high quality science labs and equipment, so that along with lots of Oobleck, Green Slime, and Elephant Toothpaste, the next generation’s Nobel Prize winners can also be made in the public schools.
Here’s a Jeopardy clue for IBM's supercomputer, Watson:
“This educational institution has produced more Nobel Prize-winning graduates than Princeton, MIT, Caltech, Oxford, or Yale.”
The question, surprisingly, is “What is the New York City Public School system?”
From the 1930s through the 1970s, the New York schools (and not just that one high school in the Bronx) produced legions of future scientists and engineers, including many who later made pathbreaking discoveries: from superconducting materials to mating bacteria, from Arrow's Impossibility Theorem to the design of the atomic bomb.
Where should we assign the credit for this astonishing productivity? Some of it, undoubtedly, is due to the particular mix of immigrants whose children filled the schools. Some of it is also due to the culture of those times, which celebrated science and discovery, which knew you could free the world from Fascism, stave off Sputnik, and make it to the moon, if you just got the differential equations right.
But part of the credit must go to the science labs.
At the same time as it was teaching long division to future laureates like Richard Feynmann and Robert Solow, New York City built and rehabilitated thousands of schools. On each floor of every one of the intermediate and high schools, they put science labs—rows of lab benches with gas lines and sinks, rows of cabinets full of chemical supplies, and a huge demonstration bench across the front of the room with ring stands and clamps, for showing off what happens when you drop pure sodium into a beaker of water or mix hydrogen peroxide with liquid soap and then add potassium iodide. (Watson, take note: the former explodes, the latter turns to foam.)
I began my teaching career in such a room, in a school built in the South Bronx in 1960, in the immediate aftermath of Sputnik’s blinking passage across the sky. Of course, by the time I got there, the gas lines had long ago been permanently shut off. The faucets were hacked off at the base, leaving jagged pieces of metal coming out of the lab benches. The sinks were choked with debris and, in some cases, emptied not into plumbing but directly into the cabinet below, so that when young Jose and Arthur tried out their model volcano, a flood of baking soda-and-vinegar lava coursed across the floor. The chemical cabinets were empty, except for decades of old dittos and worksheets, several nests of mice, and one unstoppered bottle of concentrated hydrochloric acid, the fumes of which had gradually turned the locker that held it into an ocean of rust.
The South Bronx has had some troubled years, no doubt. But this scene of decay is not unique to poor, inner-city schools. A trip through a middle-class suburban school might also show you sinks that don’t work and gas lines that no one knows how to turn on, microscopes with broken lenses and chemical cabinets with nothing in them but some aged litmus paper that no longer changes color in acid and base. Science class in such rooms can feel less like a trip to the future than a sojourn at an archeological dig.
What happened, to make us disinvest so thoroughly in science education?
Competing priorities within education deserves some of the blame. Although teachers’ salaries in American schools are lower than average for developed countries, American schools employ many more people than they once did, especially in special education and among specialists employed outside the classroom. And increasing regulation (and unnecessary asbestos abatement) has made even modest overhauls of school buildings prohibitively expensive.
Computer technology, meanwhile, has become a tantalizing investment for principals and grant agencies, at the expense of science supplies and equipment. Even science teachers themselves, some of them uncomfortable with the risks and inconvenience of doing real experiments, are often all too willing to pass out the laptops or textbooks and give their students something to do that, unlike hands-on labs, is quiet and indisputably safe. But staring at a screen is not conducting an experiment, and no amount of individualized support can make up for a classroom experience that, regrettably, involves filling out worksheets instead of blowing things up.
The “Sputnik Moment” that President Obama asserted was upon us, in which our relative educational attainment in science and math seems to fall further every year, cannot be solved with any single government intervention. But one thing we can take away from the original Sputnik moment is the importance of investing in high quality science labs and equipment, so that along with lots of Oobleck, Green Slime, and Elephant Toothpaste, the next generation’s Nobel Prize winners can also be made in the public schools.
Subscribe to:
Posts (Atom)