April 2021

“Deep Dive” Case Study of Women’s Voice and Leadership Ukraine

Formative Evaluation of the Women’s Voice and Leadership Program

Terms of Reference  

“Deep Dive” Case Study of Women’s Voice and Leadership Ukraine

 

1. Background

Women’s Voice and Leadership

Announced in June 2017, Women’s Voice and Leadership (WVL) is one of the flagship initiatives of Global Affairs Canada’s Feminist International Assistance Policy (FIAP). The program was created to support women’s organizations that are based in developing countries to achieve their goals of advancing gender equality, addressing women and girl's empowerment and promoting and protecting human rights. WVL aims to meet these organization’s needs by supporting them to be better equipped to work towards making changes in policy, legislation, services, and systems, as well as social beliefs and practices that are discriminatory and violate the human rights of women and girls. This multi-country initiative supports women’s rights organizations (WROs) through 33 projects in 31 countries and sub-regions to improve their capacity, effectiveness and networking in the pursuit of gender equality and women’s empowerment.

While each WVL project’s design varies, most include four support modalities for participating WROs: i) multi-year funding; ii) fast, responsive funding for discrete activities and short projects that respond to immediate needs; iii) institutional capacity-building support; and iv) network and alliance strengthening to build the women’s movement in countries and across regions.

Through this program, the department works with international and/or national partners to:

  • improve the management and sustainability of local and regional women’s rights organizations,
  • strengthen the capacity of women’s rights organizations to program and advocate to advance gender equality and the empowerment of women and girls, and
  • increase the effectiveness of local and national women’s rights platforms, networks, and alliances to affect policy, legal and social change.

More information on WVL is available on Global Affairs Canada’s website.

Women’s Voice and Leadership in Ukraine

WVL Ukraine is being implemented by the Ukrainian Women’s Fund, a locally-based women’s fund that provides financial and capacity support to civil society organizations. The project is intended to run for five years, having begun in 2019. The project is valued at $5M CAD. WVL Ukraine is being implemented in collaboration with two local organizations, and supports almost 30 regional and thematic partners throughout the country.  More information can be found on UWF’s website.

WVL Formative Evaluation

Global Affairs Canada has launched a program-wide evaluation of WVL, which will take place over the course of 2021. The evaluation is a corporate formative evaluation that integrates the feminist evaluation principles of empowerment, inclusivity, and participation. It is conducted by a team of evaluators at Global Affairs Canada.  

The evaluation covers the entirety of the Women’s Voice and Leadership program, from its announcement in 2017 until March 2021. The evaluation focuses on three main components:

  1. The organizational structures, policies, systems, processes, practices and capacities of Global Affairs Canada as they relate to Women’s Voice and Leadership
  2. WVL as a program overall, including general design characteristics, implementation modalities, governance, coordination, technical guidance and support, and the monitoring, evaluation and learning (MEL) strategy
  3. WVL projects: While all of the 33 projects will be covered by the evaluation, there will be five (5) in-depth case studies which will examine in greater detail specific projects with high learning potential. WVL Ukraine has been selected as one of these case studies

The purpose of this evaluation is to provide the department and other key stakeholders, including implementing partners (IPs) and funded women’s rights organizations (WROs), with evidence to support policy, program, and operations improvement, as well as horizontal learning. The evaluation will generate and share knowledge and learnings within the department and with external stakeholders on how to best support WROs and women’s movements through feminist programming.

Potential uses for the evaluation’s evidence include the department adjusting its current programming, informing future programs and strengthening the department’s systems and capacities for feminist programming. Other potential users extend to implementing partners and WROs who can use the evidence gathered to support their own learning, planning, project design, advocacy, policy dialogue and resource mobilization efforts.  

The evaluation has three objectives:

  1. To determine if and to what extent the department is “fit for purpose” to support feminist programming that directly benefits WROs, in particular WVL.
  2. To determine if WVL’s design features and implementation modalities are relevant and appropriate to address the diverse needs of women’s rights organizations and movements, and how it has adapted to COVID 19 challenges.
  3. To determine the extent to which WVL’s interventions have made progress towards results and what factors have supported or hindered progress to date.

See the Annex for the evaluation questions.

2. General approach to the Deep Dives

Objectives

The “Deep Dives” are in depth case studies of 5 selected WVL projects.

The deep dives have four intended objectives:

  1. Provide rich evidence and examples to feed into the broader program-level evaluation. This evidence and examples aim to reflect the voices of the different actors involved and will span across the diversity of the WVL program. The Deep Dives will constitute one of the lines of evidence of the evaluation and will provide information to answer evaluation questions as per the evaluation matrix.
  2. Generate findings and learnings on the specific issues that are of interest to each of the selected projects, for their own use, including improvements to current projects, design of new initiatives, advocacy, policy dialogue, and resource mobilization. 
  3. Generate learnings on key issues that are of interest across WVL projects. These learnings will include what works, what doesn’t work, and why. These learnings aim to be shared across WVL projects as part of the program’s overall learning strategy and more broadly.
  4. 4.      Provide an opportunity to model feminist evaluation principles and learn from the process.

Focus and Questions

The evaluation team and key project stakeholders will determine the specific focus and questions of each Deep Dive, to ensure that each Deep Dive reflects the appropriate knowledge needs. They will also align where relevant with the broader evaluation questions, sub-questions and indicators as per the evaluation matrix. ().

Illustrative examples of potential foci for a Deep Dive include:

  • gaining a better understanding of how WVL projects have been designed and implemented in different contexts and by different actors,
  • understanding what is working well, and what is not working as well and why, when it comes to reaching and providing direct support and capacity strengthening to diverse WROs that work in different contexts, 
  • understanding what factors support or hinder progress in various contexts,
  • documenting emerging contributions of the WVL projects towards:
    • strengthening the organizational capacities of participating WROs.
    • enhancing their programming and advocacy.
    • strengthening WROs’ platforms, networks and alliances.
  • exploring how WVL feminist principles have been operationalized, what has been challenging if anything, and what has changed because of these principles.  

Methodology

The deep dives will be designed and implemented using a feminist approach. Specifically, each deep dive will be:

  • Co-designed by the evaluation team (Global Affairs Canada evaluators and local evaluator) and the project level stakeholders (including Implementing partners, participating WROs, Global Affairs Canada staff) to make sure that they reflect their learning needs.
  • Managed (led and facilitated) by the local evaluator who brings experience on feminist approaches to allow for contextualized knowledge.
  • Participatory in its evaluation methods and approaches, Methods could include storytelling, Most Significant Change, Outcome Harvesting, Outcome Mapping, and draw on techniques such as photo-voice, video-diaries, and Sensemaker. The selection of the most appropriate method for each Deep Dive will be done as part of the co-design process. There will be no one-size-fits-all approach for all Deep Dives. Methods will be adapted to the specific focus and questions of each Deep Dive as well as to the context and the actors involved.
  • Inclusive of participatory sense-making/interpretation activities facilitated by the local evaluator. 
  • Inclusive of a capacity-strengthening dimension on feminist research or evaluation for the stakeholders involved.
  • Inclusive of a knowledge sharing strategy with learning products that can take the form of a report, poster, pamphlet, video, etc.

All deep dives will include a preparatory desk component and a field component.

The preparatory stage will involve an in-depth document review and key informant interviews with Global Affairs Canada Staff at HQ and in-country. The evaluation team and local researchers may conduct both activities jointly. 

The field component will include a diversity of evaluation approaches, methods and techniques (see above). The local researcher and the project stakeholders will collaboratively design the field work during the Deep Dive co-design phase. The evaluator/researcher must use mixed methods that are feminist and participatory for this Deep Dive.  

All case studies will collect information from: implementing partner staff, Global Affairs Canada staff at HQ and in-country, technical experts associated with the projects (e.g., external monitors, gender specialists), and participating WROs. Information will also be collected from WROs constituents and other key stakeholders (e.g., government partners, other local WROs not participating in the WVL Program) when and if relevant and feasible.  

3. Scope of Work

Services required

Given these terms of reference, the International Assistance Evaluation Division at Global Affairs Canada seeks the services of a consultant (or team of consultants) based in Ukraine to conduct the Deep Dive of the WVL Ukraine Project, by adapting the general approach described above to the WVL Ukraine project. The services required include (but are not limited to):

  • participate in initial training and orientation by the Global Affairs Canada evaluation team,
  • review relevant documents and participate in or conduct preparatory desk phase activities as needed (e.g., key informant interviews),
  • lead the co-design process of the deep dives,
  • lead the field-level data collection and data analysis process,
  • co-design and co-facilitate sense-making activities,
  • lead report writing and other product development,
  • share findings back with the evaluation team,
  • co-facilitate knowledge sharing activities,
  • provide capacity building in relation to feminist evaluation and research, as needed, to implementing partners and WROs,
  • maintain on-going communications with the evaluation team.

Activities, deliverables and deadlines

The selection of evaluators will take place during April - May 2021. The contracting process for the selected candidate(s) will be concluded during May 2021.

The selected candidate will take appropriate steps to ensure that all people have equal opportunities to participate (e.g. identifying and providing accommodations for persons with disabilities, creating safe spaces for at-risk communities). This requires proactive effort to identify and accommodate those with different needs.

All evaluation deliverables will be produced in English and submitted in MS Word, MS Excel, or MS PowerPoint format, based on the guidelines shared by the evaluation team and/or as decided during the co-design phase.

The consultant will conduct the following activities according to the proposed schedule:

 

 

Activities

Dates

Deliverables

1

Scoping/planning:

  • Participate in an introductory meeting with the evaluation team before beginning the deep dive design
  • Conduct an in-depth review of relevant documents (project documents, lit. review, etc.) and participate in or conduct preparatory desk phase activities as needed (e.g., key informant interviews).
  • Working with the WVL Ukraine stakeholders, develop an approach for the co-design process of the Deep Dive, and share this with the evaluation team. This short document will indicate which activities will be carried out to co-design the Deep Dive, who will be involved and how, and the timelines.
  • Lead the Deep Dive co-design process jointly with the evaluation team, and the project level stakeholders.
  • Develop a draft Deep Dive design document and validate with the GAC evaluation team and project level stakeholders. The design document will include the following information: objectives, scope, questions, methodology, expected deliverables, roles and responsibilities, capacity building and knowledge sharing approach, work plan and calendar.
  • Develop final Deep Dive design.

June 2021

Co-design approach (1 week after signature of the contract)

Deep Dive design document: (draft and final version completed at the latest 4 weeks after signature of the contract).  

2

Data collection:

  • Develop tools and guidelines for data collection, if relevant in collaboration with selected project stakeholders
  • Develop and apply data collection protocols to ensure protection of privacy, confidentiality of stakeholders involved; -do no-harm. Prepare appropriate consent forms (for evaluation participation and photo taking).
  • Validate the documents with the project stakeholders and the GAC evaluation team and make needed revisions and adjustments.
  • Pilot the tools as needed.
  • Conduct data collection activities as per the methodology agreed upon during co-design phase.
  • Ensure stakeholders’ participation in the data collection process.
  • Provide updates to the evaluation team on status of the data collection activities via email or online meetings.

Capacity Building

  • Implement activities to strengthen capacity in feminist evaluation and research for project stakeholders, as outlined in the co-design phase.

June -   August 2021

Data collection tools (6 weeks after signature of contract)

3

Data analysis

  • Conduct data analysis.
  • Prepare preliminary documentation for sense-making activities.

Sense-making

  • (Co)-design and (co) facilitate participatory sense-making activities with project stakeholders to identify/validate emerging findings.

Report Writing

  • Draft the synthesis report according to guidelines that will be provided by the evaluation team. The goal of the report will be to share the Deep Dive findings with the evaluation team so that they can be integrated into the formative evaluation at the program level.  
  • Share a draft with the evaluation team and revise if necessary

August- October 2021

Preliminary documents for sense-making (week before sense-making activities)

Synthesis report including the raw data (draft by end of September, final version by mid-October)

 

4

Knowledge sharing and learning products:

  • Co-design and co-develop learning products for the Deep Dive with the project stakeholders. The goal of these products (eg. reports, videos, pamphlets, infographics, etc. based on what was decided during the Deep Dive design) will be to share the findings of the Deep Dive with the stakeholders, in keeping with their specific learning needs.
  • Co-design and co-facilitate knowledge sharing activities for the Deep Dive,
  • Review and provide feedback/input into evaluation products and participate in evaluation knowledge sharing activities as needed.

October –December 2021

Learning and knowledge sharing products

COVID-19 Adaptations

The selected consultant will be expected to adhere to local public health regulations related to COVID-19, particularly with regard to travel restrictions and face-to-face interactions. The consultant will adapt their approach to the current restrictions and monitor any changes and adapt the approach as necessary based on health and safety guidelines from local authorities, the WHO, and the Canadian Embassy. The consultant will inform the evaluation team of any necessary changes. In the event of ongoing or new restrictions on travel and gatherings, the consultant will be expected to conduct data collection, participatory interpretation, and knowledge sharing activities via online communication platforms.

4. Required Profile

The evaluation team is seeking candidates with the following experience, either as an individual consultant or as a team:

  • Experience with feminist research and/or feminist evaluation approaches, and/or experience using feminist methods based on gender equity and/or other similar participatory approaches
  • Experience (at least 5 years’ experience for the principal consultant) in the design and conduct of all phases of an evaluation/research process (design, development of tools, data collection, data analysis, report writing, knowledge sharing)
  • Knowledge and experience with the design and facilitation of participatory evaluation/research processes, particularly with the application of participatory evaluation methods/approaches like Most Significant Change, Outcome Mapping, Outcome Harvesting, as well as other techniques like photo voice, photo journals, and Sense Maker among others.
  • Understanding of intersectionality and how to apply it in an evaluation process
  • Knowledge and understanding of gender equality and the women’s rights movement in Ukraine
  • Experience in implementing participatory learning processes and the development and sharing of knowledge products
  • Strong writing skills, as well as facilitation and presentation skills in English.
  • Prior experience working with women’s rights organizations is an asset
  • Excellent knowledge of the Ukrainian language is a requirement

5. Selection Process

How to apply

Interested candidates are asked to submit the following information:

  • A short technical proposal, explaining
    • The candidate’s interest in performing this work
    • Their understanding of the assignment and their proposed approach
    • The limitations posed by COVID-19 and related mitigation strategies
    • Relevant prior experience
    • For teams of consultants, the proposed roles and responsibilities
  • A financial proposal, including the estimated level of effort, the daily rates and a complete budget
  • CV(s)
  • 2 – 3 references
  • A relevant work sample, in particular of previous evaluations/studies that have been guided by feminist, gender-based, or similar participatory approaches.

Complete proposals must be submitted to Bénite Mandaka at benite.mandaka@international.gc.ca

The deadline for submissions is May 2nd, 2021 at midnight (Eastern Standard time, UTC-05:00). Successful applicants will be invited for an interview between May 3 and 6.

Selection Criteria

Candidates will be evaluated in a fair and transparent manner based on the following criteria, and based on the best value for money for the Government of Canada.

Understanding of the assignment and proposed approach

40%

Capacity (including qualifications, relevant experiences, and quality and relevance of the work sample)

40%

Financial proposal

20%

Annex: Evaluation Questions

 

Evaluation Issue

Questions

Sub-issues

Global Affairs Canada fit-for-purpose for feminist programming 

(Relevance, efficiency*)

Q1.           How has Global Affairs Canada - as an organization - operationalized feminist values and principles in support of WVL?

1.1 Policy environment

1.2 Corporate understanding of feminist programming

1.3 Corporate systems and processes for feminist programming

1.4 Corporate capacities, roles and responsibilities for feminist programming

1.5 WVL program guidance, coordination and oversight

Relevance and appropriateness of program/project design and implementation

(Relevance, coherence, sustainability*)

Q2.           To what extent are WVL’s design features and implementation modalities addressing the funding and capacity gaps of the diverse range of targeted WROs in different contexts?

2.1 Alignment to feminist principles and international good practice

2.2 Coherence with other initiatives

2.3 Relevance to WRO needs

2.4 Response to COVID 19

2.5 Fostering and supporting innovation

Progress towards results

(Effectiveness, sustainability*)

Q3.           What early progress has been made by WVL projects in different contexts towards achieving intended results, and how has that happened? 

3.1 Early progress towards intermediate outcome 1100 and related factors

3.2 Early progress towards intermediate outcome 1200 and related factors

3.3 Early progress towards intermediate outcome 1300 and related factors

3.4 Unintended results

3.5 Examples of innovations

The Deep Dives will focus on Questions 2 and 3. A complete version of the evaluation matrix will be shared with the selected consultants.

 

See also

December 2020
This document is the result of focus groups, consultations and analytical work that began in July 2020 thanks to the project of the Ukrainian Women's Fund (UWF) “Women's movement at the frontline of response to post-COVID-19 challenges in Ukraine” supported by the European Union through the Rapid Response Mechanism for civil society organisations in the Eastern Partnership, and as part of its COVID-19 support to Ukraine.
November 2020
“The quota has worked, but it has proved to be less effective than expected. Thus, according to the results of the previous local elections, women received only 15 percent of seats in oblast councils (nearly twice less than now) and 20 percent in city councils. The 2015 local elections saw 30-32 percent of women on electoral lists, while now there are 43 percent of women. But let’s keep in mind that women constitute 54 percent of Ukraine’s population. Women are the ones who do a bigger share of work during elections as observers and members of election commissions. However, there are no women among the leaders of electoral lists, neither are they well represented in decision making bodies.”