Empowering Evaluation: Looking beyond the numbers

Elizabeth Hoody, MPA

As a former grant-maker, I was glad to see that 14 Points talked about the challenge of evaluating anti-domestic violence work in a post by Payal Hathi last May. In my own experiences working in a women’s rights foundation, I saw just how difficult it can be to quantify the impact of women’s rights organizing in numerical terms. While numbers can tell an important story (such as how many young women receive sexual and reproductive health education), they often leave out what for me is most compelling about a group’s work. This might be the reflections of an individual young woman who now feels that she can talk to her partner about contraceptives or the story of a group of girls who decided to form their own anti-trafficking student organization after participating in a prevention workshop. So while there are many valid and pressing questions about how to “get the numbers right” in program evaluation, my bigger concern these days is how to evaluate impact beyond the numbers…and then again how to aggregate and share this type of evaluation in a way that donors, policymakers, and peer organizations can easily understand.

In the past year, I have come across several creative examples of evaluation strategies that do attempt to move beyond the numbers. Many of these strategies attempt to articulate, verbally or visually, the systemic impact of an organization’s work. One example is a map that was released by the Global Fund for Women just this past week, which captures the Fund’s impact around the world using bright spots. In the Global Fund’s words, this map “explores where a relationship between Global Fund for Women and grantee groups is more likely to yield a higher movement building impact.” While the map does rely on a series of numerical indicators, the visual analysis tells a bigger story about the collective impact of Global Fund for Women grants on women’s rights movements around the world.

The second example is the Gender At Work Framework, which “helps organizations see their work from new perspectives by combining best practices in organizational development with feminist thought.” One tool that the framework uses is a graph where civil society organizations can plot the different types of social changes they are addressing through their work. The graph places the continuum of “individual versus systemic” change on vertical axis and “formal vs. informal” changes on the horizontal axis, resulting in four quadrants of change:

• Women’s access to resources (quadrant I)
• Women’s and men’s consciousness (quadrant II)
• Informal cultural norms and exclusionary practices (quadrant III)
• Formal institutions, laws, and policies (quadrant IV).

Women’s organizations use the tool to visually represent the changes that they are trying to impact through their work. The graph is also a good indicator of what programs will be easier to quantitatively evaluate (such as programs that work for specific policy changes) and those that will be difficult to track through traditional metrics (how do you measure changes in men’s and women’s consciousness?).

What I like most about the Gender At Work Framework is that it encourages grassroots organizations to define evaluation framework themselves at the beginning of their planning processes. When this happens, evaluation shifts from being a chore for donors to being an effective way of tracking and reflecting on an organization’s progress towards its goals. In my personal, unquantifiable opinion, the resulting story more often than not contains richer analysis, more honest reflections, and genuine learning.

For the past year, I have been working on the Advisory Committee of FRIDA – The Young Feminist Fund, which is a newly formed foundation that is run for and by young feminist activists. As FRIDA prepares for its first grant-making round, these questions about evaluation are on my mind. How do we tell our story to our first funders and how do we empower the first FRIDA grantees to take ownership over evaluation of their work? Can we be accountable and collaborative in evaluation? As a starting point, we are planning to develop the first grant evaluation process in partnership with FRIDA’s first round of grantees. As we begin to hash out exactly what the process will look like, I am grateful for these new tools that push us towards more creative, dynamic ways of articulating our goals, visions, and impacts.

21st Century Slavery: The scourge of human trafficking and how we can fight it

Elina Sarkisova, MPA

Human trafficking, although not a new phenomenon, has experienced exceptional growth in the last quarter of the 20th century, becoming one of the fastest growing criminal enterprises in the world. According to the State Department, an estimated 600,0000 to 800,000 men, women, and children are trafficked across international borders each year, the majority being women and children. When accounting for internal trafficking, the number of victims jumps to 12.3 million. Most states have taken steps, both on an individual and collective level, to try to reverse this trend; most noticeably in the form of international legal instruments and national laws aimed at migration controls, crime prevention, and victims’ assistance programs. However, these efforts have proved largely ineffective and are even to blame, in some circumstances, for making matters worse.

Why human trafficking is on the rise
The competing and often conflicting forces of globalization and national sovereignty help explain the increase in the overall volume of human trafficking. While globalization has brought about unprecedented opportunities for economic growth and prosperity, the results of economic liberalization (a key feature of globalization) have been mixed for developing countries. The swift spread of free markets has led to increased levels of inequality both within and between states and widespread economic instability, thus fostering strong “push” factors in migrants’ countries of origin. Meanwhile, increased demand for cheap labor in developed countries (characterized by a growing ratio of elderly to working-age people), declining birth rates, and high cost of labor, fosters strong “pull” factors.

On the other hand, governments have stepped up efforts to limit the movement of people across their borders, even when such policies are highly inconsistent with national labor market demands. This is usually the result of strong societal resistance to immigration and/or political pressures. Such restrictive immigration policies have the effect of increasing undocumented migration flows. However, trafficking flourishes where an additional factor is present: few opportunities to overcome barriers to illegal migration without the help of a third party. Most trafficking cases start out as voluntary transactions between a potential migrant and a smuggler – an exchange that is usually initiated by the migrant. However, the situation quickly deteriorates upon arrival at the destination country, where the victim often finds himself in a situation of forced labor.

National responses to human trafficking and why they haven’t worked
States tend to perceive human trafficking through the lens of national security, best addressed through migration controls, crime prevention and victims’ assistance programs. The United States, for instance, has increased border control enforcement along its 2,000-mile border with Mexico over the last two decades. However, strong migratory pressures render such efforts useless at best and counterproductive at worst, contributing to an increasing death toll in the border area and driving increasing numbers of economic migrants to seek the services of unscrupulous migration intermediaries intent on exploiting the gap between labor supply and demand. Law enforcement (e.g. police raids) has also become a key tool in countries’ toolbox for combating human trafficking. However, such efforts simply drive the trade further underground, where victims become more and more isolated and vulnerable to abuse, especially in the case of prostitution. Lastly, while some countries have initiated victims’ assistance programs, efforts have largely been limited to “rescue” operations that offer little in the way of long-term solutions and conditional assistance programs that overlook underlying incentives.

A way forward
The human security framework can offer a new, more useful way of conceptualizing an old problem – namely, from the standpoint of the driving forces behind the trade, the mechanisms that facilitate the trade, and the threat that such a trade poses to society. Broadly, policy response should focus on prevention (targeting underlying causes of supply and demand specific to each country), prosecution (law enforcement), and protection (victim assistance programs). Policy options should include, at a minimum:

• Women, children, and minority empowerment programs: Past projects aimed at the economic empowerment of women have, for instance, tended to be components of broader economic development programs rather than anti-trafficking strategies. Efforts need to be streamlined and incorporated into a broader anti-trafficking strategy.

• Immigration reforms: Destination countries should create temporary guest worker programs, perhaps in a more targeted way than some countries have already done, as part of a broader anti-trafficking agenda.

• Anti-corruption programs among border patrol officials: Trafficking would not be as widespread were it not for the help of corrupt government officials, especially in border patrol areas, who are complicit in this trade.

In the end, improving the track record of international anti-trafficking programs will require both structural changes in the manner in which the issue is conceptualized by states and operational changes in the manner in which programs are implemented. Only then could we realistically attempt to end this scourge of the 21st century.

Attention Women: RUN!

Christina Henderson, MPA


Back in the day when Michele Bachmann was leading in the Republican presidential primary (i.e., this summer), I was doubled over with grief at the idea that she could be first. Call me crazy, but I hope whoever achieves the honor of becoming the first woman to win the presidential nomination of major political party does not have to be convinced that the Founding Fathers did not in fact outlaw slavery.

Now that Bachmann’s star has fallen, I am concerned that neither political party has a deep enough bench to field a viable female candidate for president in the next three years. Currently, there are only six women serving as governors, 17 women in the United States Senate, and 72 in the House of Representatives. If we control for age, experience, name recognition, likability, alternative ambitions, and public displays of wackiness (yes, this is applies to male candidates too because wackiness knows no bounds), the numbers look even bleaker. We have got to get more women off the sidelines and into the field!

A 2009 study conducted by Stanford University and the University of Chicago found that on average, women in Congress introduce more legislation, attract more co-sponsors, and secure more resources for their districts and states than their male counterparts. In general, women in government are known to get things done and not take no for an answer. This is not to diminish the impact of men, but there are times when the fortitude and deep abiding commitment of women is stronger than that of said counterparts. And at this time in our nation’s history, it is needed.

In the last ten months, we’ve witnessed an unbelievable war on women across the country. From "forcible rape" to Planned Parenthood, from the assault on public employees (the majority of whom are female) to allowing hospitals to deny pregnant women life-saving care, can we please call off the vultures?! Now, I am not expecting every woman who decides to run for office to be progressive and pro-choice (it would be nice, but I’m realistic). However, when a city decides to decriminalize domestic violence over a budget dispute with its county counterparts, I would hope all the women could come together as a collective and say: “Are you serious?!”

We need more women running and serving in public office. Now, I would be remiss not to mention the difficulties involved in running for office. As a veteran campaign staffer, I know campaigns are not for the fainthearted. They are tough—physically, mentally, and spiritually. And for women—Democrat or Republican—it’s harder in ways men cannot even fathom. You will get frustrated, you may want to quit, and yes, it is possible that you may even lose your race. But know that hopefully because of your efforts the next time a woman runs, it will be a little easier that go around. When you are woman seeking to break political glass ceilings, the process is as much about you as the next five who follow. (Hillary to Bachmann: “You’re welcome.”) Don’t sit this opportunity out. Join the other incredible women in our nation’s history who have dared to define the role women play in politics on their own terms. Your country needs you.

Farewell to Robertson: Thinking beyond metrics

Payal Hathi, MPA


I met with a woman this week that came into my office with huge bruises on her arms and face. She has been married to the man who gave her those bruises for 16 years, and they have two children together. She believes that she deserved what happened, that although she called the police for help in making the violence stop, she couldn’t actually tell them the truth about what was happening in her home; that if she just stays quiet and stays in the basement, her husband won’t get angry; and that all she needs to do is wait for 5 more years until her daughter is old enough to go to college. Then it will all be over.

Violence against women is not something that comes up often in policy circles – domestic violence in particular is often considered a family issue, or one that is generally addressed by activists. But there is certainly a place for policy to play a role. Societal portrayals and beliefs of women – from the media to childhood socialization to educational and job opportunities – all play a role in allowing the feminine to be seen as the inferior. In addition, the general attitude of blaming the victim and the cultural stigma often attached to women who “can’t keep their families together” perpetuates not only intergenerational cycles of violence, but also silence and shame around the issue.

A major part of the problem is that in the world of metrics and quantitative measures of success that we live in today, it often seems that there is little space or value given to work that cannot be counted using numbers. When I am asked how many women with whom I work have finally left their partners, found a job, or have secured housing for themselves, it seems much of the rest of the work that I do is invalidated because I cannot quantify what it means to get a woman to believe that she has the right to live free of fear or that she deserves to earn a living wage for her work. Of course the importance and power of statistics and clear, concise, and concrete information is critical in making policy issues relevant to people. But having worked in the violence against women movement for over a year now, it is clear that an additional system of evaluation is necessary if we as policymakers are to make the world a safer and more just place.

I came into graduate school with a background in quantitative evaluation work, hoping to gain the skills to enable small non-profits to incorporate evaluation as a routine practice to improve their own work. Through my work this past year with women facing human rights and economic justice violations in almost every aspect of their lives, I have become acutely aware that as policymakers, it is not enough to think about the efficiency with which different ideas can be put into practice, or the cost-effectiveness of one approach versus another. Rights, and the way in which they are put into practice in our families and daily interactions—while not easy to measure—must form the foundations of our policy frameworks in order to make meaningful impacts in the communities we serve. As I graduate, I recognize that as we look towards more sophisticated ways to evaluate the work of service providers, we cannot diminish the work that instills in people an awareness of their rights and gives them the tools to exercise them.

Payal Hathi worked at Sakhi for South Asian Women from 2010 to 2011. If you or someone you know has an issue with domestic violence, please go to Sakhi’s website http://sakhi.org or call their helpline at (21) 868-6741.