Skip to main content
Concordia University
Concordia University

The SHIFT Centre for Social Transformation

The SHIFT Centre for Social Transformation

Working group – Program evaluation

During SHIFT's pilot year, this group played a key role in evaluating the SHIFT Funding and Support program, proposing improvements and suggesting potential pathways forward.

Mandate:

  • Monitor and evaluate the impact of the SHIFT Funding and Support Program, taking into account questions of clarity and effectiveness of the funding model, transparency and fairness in application and selection processes, relevance and impact of accompaniment services, and overall alignment with SHIFT’s values and ways of working
  • Evaluate the relative effectiveness of the funding program in comparison to other SHIFT activities, comparing the investment of resources with the resulting impact on social transformation
  • Provide recommendations on the SHIFT project support and funding program, and potentially a redistribution of priorities for SHIFT activities, in year 2 and beyond
  • Propose metrics and processes to facilitate SHIFT’s ongoing impact evaluation and self-reflective learning
 

Approach:

After exploring serveral possible paths to evaluation through research and discussion, the Program Evaluation Working Group members decided to approach their mandate using a Developmental Evaluation approach.  They focused their efforts on three questions:

  • What about the SHIFT’s Funding and Support program worked well during SHIFT’s Pilot Year? What could be improved?
  • What elements should staff and Steering Committee members keep their eye on? What are the best practices out there?
  • In what direction should SHIFT’s Funding and Support program go in Year 2?


Data collection strategies included:

  • Formal interviews (with successful applicants)
  • Informal interviews (with staff members, Steering Committee members and other key informants)
  • Survey circulated to all applicants to the Round 1 Funding Program
  • Observation and readings
  • Research and analysis of 9 other funding programs operating in Montreal and beyond, including approaches to funding allocation and evaluation

Findings & Recommendations

The Program Evaluation Working Group proposed recommendations for SHIFT's Funding and Support program at a variety of scales, from individual application questions to organizational priorities. Here below are 6 key recommendations, based on their research and analysis:

 

Define and clarify information about in-kind support. There is a mutual lack of understanding between applicants and SHIFT about non-financial support measures.  However, these supports have the potential to bring important benefits to community partners in the context of project-based funding. 

Transform the application process in a multiple-step process with increased opportunities for back-and-forth communication between applicants and SHIFT staff before their applications are submitted to jury members for selecton. 

Support funded project teams in implementing evaluation tools that are relevant for their projects.  In order to evaluate the net impact of the program, SHIFT must create systems which enable an initial evaluation of project teams and compare it with a second evaluation of where the project team is at when the funding and support has concluded. SHIFT should provide support to teams to develop their own internal evaluation processes, which will be particular to each project.

Foster the creation of a learning community in which knowledge and expertise is shared between SHIFT staff and project teams, as well as between projects and the broader SHIFT community

Prioritize evaluation processes, data collection and information feedback loops throughout SHIFT during Year 2 by creating a working group or committee explicitly assigned with these tasks

Prepare the groundwork for an efficiency evaluation to be conducted at the end of Year 2 by building in evaluation tools and metrics throughout program development.  There is generally insufficient data at this time to assess whether or not the cost of administering the Funding and Support Program is justifiable, or whether other programming could offer more efficient progress towards SHIFT’s mission and vision.

Read full report

Working Group Lead:

Rose-Anne St Paul was our Program Evaluation Advisor and led the Program Evaluation working group throughout their mandate.

Rose-Anne's goal is to work with the Program Evaluation Working Group to evaluate and measure the impact of the SHIFT funding program, in order to make sure that SHIFT's activities are aligned with its values and serving social transformation in the most effective way possible. 

You can review the job description that outlines Rose-Anne's responsibilities and mandate.

 

Members:

Thy Anne Chu Quang, Project Manager at Naskapi Nation of Kawawachikamach

Emma Haraké, Coordinator and Community Facilitator at Concordia's Center for Oral History and Digital Storytelling (COHDS)

Laïty Fary Ndiaye, Community worker, evaluation consultant and lecturer at University St-Paul

Maria Stawnichy, Concordia undergraduate student in Human Relations and intern at Dean's Office of the Faculty of Arts and Science

A snapshot from the first virtual meeting of the Program Evaluation Working Group.
 
Back to top

© Concordia University