Samson Community Wellness – Ekaya Pihtwaw Tobacco Cessation Project (March 2017)

Williams Consulting was contracted by Samson Community Wellness as project evaluators and helped launch the Ekaya Pihtwaw Tobacco Cessation Project to coincide with World No Tobacco Day 2015. The planned project activities include: advocating for smoke-free homes; promoting awareness campaigns regarding the dangers of second-hand smoke; holding tobacco awareness workshops; working with parents/families to to increase awareness and reduce access to tobacco products for children and youth; holding peer mentoring workshops on smoking cessation for youth; having school-based education and awareness programs to educate youth on the harmful side effects of tobacco products; holding community workshops and presentations by elders on the cultural use of tobacco for ceremonial purposes; and having weekly smoking cessation support groups.

The Samson Community Wellness Program received funding from the First Nation and Inuit Health to deliver a tobacco cessation program entitled “Ekaya Pihtwaw” to area First Nations. The three main objectives are: to prevent the use of non-traditional tobacco among young people and adults; to protect from exposure to environmental tobacco smoke; and to promote tobacco cessation among smokers.

Year 1 Evaluation Status Report: Sioux Lookout First Nation Health Authority HSIF PROJECT – Far North Specific Integrated Public Health Service Delivery System Public Health Service Integration (July 2013)

Williams Consulting was contracted to evaluate progress made on achieving the HSIF deliverables for this public health project.

Update the SCRAP-TB Resource Tool (January 2011)

The First Nations and Inuit Health Branch (FNIHB) in partnership with First Nations peoples, Tuberculosis (TB) experts and community health care providers developed a resource to enhance community capacity to develop local Tuberculosis programs in 2004. The objective of this project was to develop a more “user-friendly” electronic version of the Strategic Community Risk Assessment and Planning for Enhanced Tuberculosis Planning Resources (SCRAP-TB Resource) as well as a communication strategy.

Chiefs of Ontario Prescription Drug Abuse Strategy (May 2010)

Williams Consulting was contracted by Chiefs of Ontario to work with an Advisory Panel to produce a report that establishes a strategic action plan for negotiation with the Federal and Provincial Government to address the Prescription Drug Abuse epidemic in Ontario First Nation communities.  The Vision of the Prescription Drug Abuse Strategy is that First Nations communities and leadership will eliminate prescription drug abuse by working together to support the continued development of healthy communities. The Prescription Drug Abuse Strategy builds upon existing research and practices as well as community promising practices to formulate a solution based action plan for the prevention, reduction and elimination of Prescription Drug Abuse in Ontario First Nation communities.  The four key areas of the strategy include approaches to address: health promotion, healthy relationships, reducing the supply and supporting a continuum of care.

First Nations Community Public Health Pilot Projects Formative Evaluation (March 2010)

The First Nations Community Public Health Pilot Project is intended to help inform future directions in the delivery of public health on-reserve, leading over time to improved health outcomes and increased First Nations control over delivery of their own public health services. The pilot project is expected to be in place for up to five years, predicated on tripartite (First Nations, federal and provincial) collaboration and partnership. The Assembly of First Nations (AFN) is responsible for coordinating and supporting the three pilot projects (which are currently underway) based on collaborative partnerships among First Nations, federal and provincial departments.

Williams Consulting assisted the three pilot projects in conducting a formative evaluation for year two of the initiative. The final evaluation report consisted of four components: a formative evaluation report from each of the three pilots and a single analysis by the Williams Consulting study team of overall observations.

Evaluation of the Strategic Community Risk Assessment and Planning for Enhanced Tuberculosis Program (March 2010)

The First Nations and Inuit Health Branch (FNIHB) in partnership with Aboriginal peoples, Tuberculosis (TB) experts and community health care providers developed a resource to enhance community capacity to develop local Tuberculosis programs. The resource was piloted in 2004, the key recommendations from the pilot evaluation were incorporated into the final product and it was released for use by First Nation communities across Canada through their respective FNIH Regional TB Programs.

The First Nations and Inuit Health Branch is undertaking an evaluation of the SCRAP-TB resource.  The evaluation will seek to answer how communities were made aware of the resource, whether it is being used in First Nation communities and how useful it has been as a tool to support and enhance TB programming. Nancy has provided administrative and research support on all aspects of this contract from design of the tools, to note-taking during key informant interviews, report writing and control of communications.

FNIH-OR Public Health Unit Strategic Planning (September 2008- March 2009)

Williams Consulting provided facilitation services and technical support to First Nations and Inuit Health Ontario Region (FNIH-OR) Public Health Unit for the development of their strategic plan.

Public Health Agency of Canada – Support for engagement of Aboriginal Public Health Stakeholder Activities across the Ontario Region (December 2008 – June 2009)

Williams Consulting is contracted by the Public Health Agency of Canada (PHAC) Ontario/Nunavut Regional office to provide a summary report that outlines Aboriginal public health stakeholder activities across the Ontario region.  The report will provide a general demographic overview of First Nations, Inuit and Métis in Ontario, distinguishing between on-reserve and off-reserve Aboriginal populations.  Where available, the report will clearly and concisely outline the mandates, funding source(s), populations served, key activities (including detailed public health activities), key priorities (including detailed public health priorities), political affiliations and key linkages with other government and community stakeholders, for identified organizations with whom our regional office will be meeting.  The report is intended to inform PHAC regional staff prior to meetings with stakeholders.

Nation-wide comparative analysis of Addictions/Mental Health Services by Urban

Aboriginal Health Centres (April – September 2008)

This study undertook a comprehensive comparative analysis (formative evaluation) of addictions treatment, prevention and mental health services provided in three urban Aboriginal Health Centres.

The report outlined the existing situation, needs and services in three centres and documented the promising practices for treatment and prevention in meeting the needs of urban First Nations and Inuit people.

FNIH-OR Public Health Unit Strategic Planning (September 2008)

Williams Consulting provides facilitation services and support to First Nations and Inuit Health Ontario Region (FNIH-OR) Public Health Unit for the development of their strategic plan.

First Nations Public Health Relationship Framework      (October 2007 – March 2008)

Chiefs of Ontario contracted Williams Consulting to support the rollout of an important new initiative designed to improve the coordination of public health services on-reserve and guide Canada, Ontario and First Nations in delivering public health services on-reserve.  Ms Williams and team members have been supporting the establishment of a tripartite advisory committee that will ultimately be responsible for designing and implementing a First Nations/Public Health Relationship Framework that will address jurisdictional, governance, resourcing and infrastructure issues that currently impede public health service delivery in Ontario First Nations.  Williams Consulting has conducted a gap analysis, researched models of coordinated service delivery and facilitated strategic planning of the launch of this initiative.

Ontario Independent First Nations (IFN) Health Report of First Nations Health Status, Initiatives, Strengths and Challenges

(March – August 2007)

Williams Consulting was contracted to prepare an Independent First Nation (IFN) Health Report of First Nations Health status, initiatives and priorities.  The Independent First Nations took part in two separate Health Status reporting and assessment processes.  The first template is known at the Upstream Investment initiative.  The purpose of this tool is to assist communities in identifying existing resources and services.  Communities would also identify how these resources can be enhanced with new funding to address unmet needs or gaps. The second activity was known as Community Asset Mapping.  For communities who followed the provided template, they took part in exercises designed to use asset mapping as a means of examining assets or strengths as a visioning exercise in the community health planning process and then focused planning around assets and gaps.

The Report of First Nations Health status analyzed the needs, priorities and trends for health planning.  This information will be provided to the IFN Leadership to inform the development of recommendations for the implementation of health initiatives with independent First Nations.

Williams Consulting team members were responsible for working with the client, data analysis and report writing.

First Nations and Public Health Environmental Scan and Literature Review for Ontario (February – March 31, 2006)

Williams Consulting was subcontracted to develop the tools, data collection and report writing for an environmental scan for the Chiefs of Ontario titled First Nations and Public Health Environmental Scan and Literature Review for Ontario.  This methodology includes key informant interviews (telephone) with Public Health Units, the Regional Medical Officer (FNIHB) and Health Directors with the Provincial/territorial organizations, questionnaires to each Ontario First Nation and an annotated bibliography/literature review.  The project authority for this contract was the Chiefs of Ontario.  Williams Consulting was responsible for project management, data collection, analysis, report writing, and successful financial management.

Williams Consulting facilitated two training/consultation sessions and one meeting between federal government representatives and First Nation representatives in order to effectively obtain Aboriginal input into decision-making.  Williams Consulting developed the teaching/curriculum aids and material and wrote the synthesis report.

The Chiefs of Ontario Community Support Environment Scan

(November 2005 – March 1, 2006)

The Chiefs of Ontario and the Ministry of Health and Long-Term Care were seeking advice and recommendations about the allocation of $6 million to expand community support services in First Nations in Ontario.  They were undertaking an environmental scan to explore the various issues involved. The methodology included the design of a survey tool and conducting two focus groups, designed to determine the following: Projected First Nation population growth over the next five years; impact and extent of First Nations people who return to their First Nation when they retire; geographic mobility trends within First Nations (how many First Nations community members are moving on and off reserve; The community support resources that are available to each First Nation community; How the needs of First Nations communities would be best met through expanded community support services; Best/promising practices in service integration; and Barriers First Nations people encounter in accessing mainstream services such as Community Care Access Centres (CCAC).  Andrea Williams was project manager, senior analyst and responsible for data analysis and report writing.

Chiefs of Ontario, Public Health Initiative:

Date (Month/Year): February 2006 – March 31, 2008

Client Name and Telephone Number: Tracy Antone, Health Coordinator, Chiefs of Ontario, (416) 597-1266

This initiative took place in three stages:

Stage 1: First Nations and Public Health Environmental Scan and Literature Review for Ontario (February – March 31, 2006)

Williams Consulting was subcontracted to develop the tools, data collection and report writing for environmental scan for the Chiefs of Ontario titled First Nations and Public Health Environmental Scan and Literature Review for Ontario.  This methodology includes key informant interviews (telephone) with Public Health Units, the Regional Medical Officer (FNIHB) and Health Directors with the Provincial/territorial organizations, questionnaires to each Ontario First Nation and an annotated bibliography/literature review.  This contract was with the Chiefs of Ontario.  Andrea Williams was project manager, senior analyst and responsible for financial management and report writing.  Once again, Williams Consulting facilitated two focus groups and one meeting between federal government representatives and First Nation representatives in order to effectively obtain Aboriginal input into decision-making.  Williams Consulting wrote the synthesis report.

Due to threats of potential pandemic outbreaks, there has been an increasing urgency for First Nations to develop Public Health Strategies across all Regions in Canada.  Recently the Assembly of First Nations (AFN) commenced work on the development of a national First Nations Public Health Framework.

Chiefs of Ontario (COO), in conjunction with its health advisory technical body, the Health Coordination Unit (HCU), examined ways to best prepare for a made-in-Ontario approach to future Public Heath programs and service delivery activities, as required.  The Chiefs of Ontario undertook an environmental scan to gain a better understanding of the jurisdictional realities and issues with respect to Public Health and First Nations in Ontario. Williams Consulting was subcontracted to undertake that work.

The environmental scan consisted of four data collection tools; namely a literature review, a survey distributed to Ontario First Nations by the Chiefs of Ontario office, key respondent interviews, and a focus group facilitated by Williams Consulting.

These environmental scan tools asked questions specific to:

• The prominent issues and challenges faced regarding Public Health and First Nations.
• The successes and challenges in developing and maintaining relationships with other governing bodies/agencies.
• Preparation in the field of pandemic awareness and development of an Emergency Response Plan (ERP).
• How public health services for First Nations could be improved.
• Developing approaches for a Made-in-Ontario model for Public Health programs and services for First Nations.

The four general categories or theme areas emerging from this analysis are:

1.     Pandemic Preparedness;

2.     Health Surveillance;

3.     Jurisdictional Issues; and

4.     Resourcing of Public Health for First Nations.

Findings:

Pandemic Preparedness:

Although many First Nations indicated that some public health services are provided by them in their community, they feel that acute care is such a pressing need that it usually takes priority over public health. The shifting of priorities with respect to acute care services should make identifying health determinants a priority over treating diseases.

Most, if not all, First Nations have at least initiated the formation of an Emergency Response Plan. They can be in various stages of completion. A common core framework for planning for pandemic was identified as an important tool that should be provided all First Nations in Ontario. It could consist of a template and have a core of activities, duties and responsibilities that are common to all First Nations while allowing the flexibility for adaptation in the local community.

The level of awareness and appropriateness of linkages of pandemic preparedness is not consistent. Some regions and Public Health service providers seem to have a strong awareness and effective linkages with respect to pandemic preparedness while others appear to be in disarray. There is no one comprehensive framework or model from which everyone is working and initiatives seem to be very dependent on the personalities and level of engagement pursued by individuals with relation to pandemic planning. First Nations indicated there is also a need for a seamless system of pandemic preparedness where First Nations are included from the onset so that they are not considered an afterthought. Furthermore, there is a significant “rural and remote” challenge that seemingly goes on unaddressed as it historically has for Public Health for First Nation communities.

It was suggested that the federal, provincial and First Nations governments all have a role to play in supporting this planning process.  However inconsistent relationships were described with other parties involved in pandemic or emergency response planning.  Some concern was expressed that linkages with the province and the federal government need improvement in preparation for a pandemic event.

There was some confusion expressed by First Nations regarding the line of leadership responsibility in the event of a pandemic. This lack of clarity was also expressed by the province and the federal government.

Health Surveillance:

A consistent and province wide model for data collection and surveillance is a need expressed by the First Nations. It is vital to program development and service delivery and there is a need for general and community level data to run the programs properly.

First Nations are not ready to consent to the collection and utilization of data without ownership issues resolved first. There is the existing provincial legislation that stipulates that at least existing programs and services should meet provincial standards and mandatory health programs. It is exceedingly hard in the absence of current surveillance data to ensure that this is being done. The need for more staff to foster and maintain this surveillance capacity was also expressed.

A systemic re-vamping should be initiated with sound data collection and ongoing surveillance as a means to develop effective and adaptable delivery mechanisms for Public Health programs and services for First Nations people.

Jurisdictional Issues:

There are many issues that need to be addressed with respect to relationships where the provision of public health for Ontario First Nation communities is concerned.

There is a dramatic difference in the level of awareness of the programs and services provided to First Nation people. It appears that various Public Health administrators, managers and health providers have varying degrees of awareness of the many jurisdictional issues with respect to the provision of Public Health programs and services to First Nation communities. Federal and provincial participants have limited knowledge of actual programs and services and their knowledge of particular eligibility policies for First Nation people is not consistent.

Many First Nations stated that they did not receive equitable services as those received by non-First Nations people in Ontario and many identified a need for a proactive approach to identifying gaps amongst jurisdictional roles and responsibilities and addressing the issues. First Nations voiced that more clarity is required in relation to the roles and responsibilities and a more concise understanding with the federal and provincial governments was needed.

Resourcing of Public Health for First Nations:

Many First Nations felt that they have an inadequate infrastructure and funding to support public health programming.  There is an inadequate number of staff and wages are inadequate. Professional development and in-house servicing was requested to acquire, maintain and develop the necessary skills to properly and effectively engage in Public Health for First Nations in Ontario.

The importance of developing relationships and improving communication across all levels of Public Health on First Nations was repeatedly highlighted.

Made in Ontario approach

All levels of Public Health governance detailed a need for an infrastructure to be in place to adequately address the needs of First Nations people.

The participants readily admit that many of the primary determinants of health can be addressed with adequate housing, clean drinking water and access to primary health care.

Transportation is a major challenge for the provision of Public Health in Ontario as a result of the nature of Ontario’s geography and the distribution of First Nations communities.

A surveillance system to monitor trends, outbreaks and health issues would be a component of the “Made in Ontario” approach.

A flexible and adaptable model that is not rigidly linked to pre-capita provision of Public Health has to be considered.

The importance of partnerships with the community is a key factor.

In summary, a Made-in-Ontario approach would be an innovative and flexible service/program delivery model that takes into consideration the unique logistical and legislative challenges of First Nations communities. The model needs to enable us to adequately and equitably satisfy a broad range of needs of our cultural diverse First Nations communities.

The following recommendations were meant to encourage discussion and debate with respect to the provision of programs and services for First Nations Public Health in Ontario. These recommendations might aid in informing or directing the construction/compilation of a Public Health and pandemic strategy for First Nation people in Ontario.  Additional recommendations are found in Chapter  5 of this report found in Appendix G.

It was recommended that a Public Health Monitoring Group be established which would be given a specific mandate to follow up on issues raised by the preliminary findings of the environmental scan including but not limited to pandemic preparedness, health surveillance, jurisdictional issues and resourcing of Public Health for First Nations. The mandate of the Public Health Monitoring Group would also include:

• responding to the Assembly of First Nations document titled, “First Nations Public Health: A framework for improving the Health of our People and our Communities”;
• working with the federal and provincial governments on jurisdictional issues; and
• developing position papers which would include undertaking additional research on this issue.

Note:  this recommendation is now in place

It was recommended that the Chiefs in Assembly provide authority to the Chiefs of Ontario to pursue funding through the Aboriginal Health Transition Fund to assist in carrying out public health planning work.  Note:  This recommendation was successful.

It was recommended that a Public Health Specialist with a background in working with First Nations in public health issues be employed to work directly with the Chiefs of Ontario on public health issues. This Public Health Specialist would act as an Advisor to the Chiefs of Ontario to assist on such issues which would include but not be limited to pandemic preparedness, health surveillance, jurisdictional issues and resourcing of Public Health for First Nations. Note:  This recommendation was successfully implemented.

Stage 2: First Nations and Public Health Network Symposium (December 2006)

In February 2006, the Chiefs of Ontario Office (COO) began coordinating the development of a Regional First Nation Public Health Strategy in response to the broader, National Framework which is currently being spearheaded through the Assembly of First Nations (AFN).  Based on the report listed above, Williams Consulting facilitated a tripartite meeting between the federal government, provincial government and Ontario First Nation communities that are involved in Public health and pandemic preparedness.  This meeting is an example of effective Aboriginal engagement into Federal and provincial health policy decision-making. Williams Consulting also prepared the synthesis report of this two-day workshop.

Stage 3:  Funding for a First Nation Public Health Relationship Framework is secured as a result of the work undertaken in Stage 1 and 2.

First Nations Public Health Relationship Framework             (October 2007 – March 2008)

Chiefs of Ontario contracted Williams Consulting to support the rollout of an important new initiative designed to improve the coordination of public health services on-reserve and guide Canada, Ontario and First Nations in delivering public health services  on-reserve.  Ms Williams and team members have been supporting the establishment of a tripartite advisory committee that will ultimately be responsible for designing and implementing a First Nations/Public Health Relationship Framework that will address jurisdictional, governance, resourcing and infrastructure issues that currently impede public health service delivery in Ontario First Nations.  Williams Consulting has conducted a gap analysis, researched models of coordinated service delivery and facilitated strategic planning of the launch of this initiative.  This work is now underway in the capable hands of Provincial Territorial Organizations, Independent First Nations, as well as Provincial and Federal government representatives.  Our Elder Shirley Williams remains to guide them until the end of this fiscal year.

This contract demonstrates Williams Consulting’s ability to work with clients. conduct focus groups and interviews with key stakeholders, develop data collection tools within a short time frame, collect large data sets from multiple lines of inquiry, provide analysis and synthesis the material into useful evidence based reports.

• Working with Federal government, service providers, regional consultants and First Nations people
• Review of existing programs and development of performance measures
• Questionnaire design
• Key Informant interviews
• Facilitate focus groups and meetings with key stakeholders to receive feedback
• Surveying a number of different stakeholder groups including Aboriginal leadership and representative of service providers at the federal, provincial and First Nation, community levels.
• File, document and literature review;
• Professional minute taking
• Tripartite meeting facilitation
• Creation of culturally appropriate multi-media tools targeted to a First Nation audience to encourage engagement in discussions
• Subject matter expertise;
• Strategic Action Planning
• Data entry, cleaning and data analysis; and
• Report Writing.

Date (Month/Year): September 2009 – March 2011

Client Name: Dawn Walker, Public Health Special Advisor, Director Generals Office, Primary Health Care and Public Health, First Nations & Inuit Health Branch, Health Canada,

Research Subject Matter:

The First Nations Community Public Health Pilot Project is intended to help inform future directions in the delivery of public health on-reserve, leading over time to improved health outcomes and increased First Nations control over delivery of their own public health services. The pilot project is expected to be in place for up to five years, predicated on tripartite (First Nations, federal and provincial) collaboration and partnership. The Assembly of First Nations (AFN), is responsible for coordinating and supporting the three pilot projects, which are currently underway, based on collaborative partnerships among First Nations, federal and provincial departments and are scheduled to be in place for up to five years (2007/2008 – 2011/2012).

There are three pilot project sites: The File Hills Qu’Appelle Tribal Council in Saskatchewan; the Four Arrows Regional Health Authority in Manitoba; and the Kenora Chiefs Advisory in Ontario.  Initial work-plans for the three pilots focus on the following elements:

• Defining “the public health system”;
• Engaging provinces;
• Clarifying jurisdictional roles and responsibilities;
• Increasing disease surveillance through acquisition of better health data;
• Identifying funding shortfalls and programmatic gaps;
• Improving health human resources;
• Exploring enabling legislation; and
• Developing comprehensive public health programs.

The formative evaluation provides a comprehensive means to evaluate the current First Nations Community Public Health Pilots in Year 2, 3 and 4. Williams Consulting will assist the three pilot sites (the File Hills Qu’Appelle Tribal Council in Saskatchewan, the Four Arrows Regional Health Authority in Manitoba and the Kenora Chiefs Advisory in Ontario) as they conduct a formative evaluation for each year at the individual site.

The consultants will prepare the final annual evaluation report consisting of four components:

• a formative evaluation report from each of the three pilots;
• a single analysis of overall observations.

The objectives of the work of the consultant are to:

• provide the 3 pilot sites with guidance and support as they conduct a formative evaluation for years 2, 3, and 4 of the pilot projects;
• complete a final evaluation report for years 2, 3 and 4 of the pilot projects; and
• collect information and observations for the duration of the contract that will inform an anticipated summative evaluation for year 5 of the pilot projects.

An evaluative framework entitled First Nations Public Health Pilots Evaluation Framework, has been developed to guide the formative evaluation prepared by each of the pilots for years 2, 3, and 4, as well as the summative evaluation for year 5. The framework will also guide the final evaluation reports prepared by the consultant for each of these years. This evaluation framework is intended to examine the overarching project goals and objectives through lenses of: effectiveness, efficiency, partnerships, relevance, impact and lessons learned. It utilizes practical questions to ascertain data that, when taken together, will provide information regarding the overarching goals and objectives of the pilot, as well as information specific to each of the pilots. The information provided to the pilots will enable each pilot to determine how to adjust the questions used in the framework for subsequent formative evaluations and the anticipated summative evaluations for year 5.

As suggested in the paragraph above, questions and indicators in the framework are meant as a starting point and that they are to be reviewed with the stakeholders, particularly the individual pilot sites, for their relevance and utility. It is recognized that each pilot project, in collaboration with the consultant, will need to tailor the questions and determine each year which questions are most critical and most appropriate to answer. While some discussion amongst stakeholders has occurred it may be (as some have suggested) important to meet face to face to review this draft and answer other related questions that the parties might have.

The First Nations Public Health Pilots Evaluation Framework provides six broad evaluation questions with their own subset of evaluation sub-questions to aid the pilot sites in their formative evaluation report.  This Pilot Project Evaluation Chart (October 2008) provides indicators, data sources, collection methods and appropriate time of measurement (i.e. Year 2, Year 3, Year 4 and Year 5).  The evaluation questions are as follows:

1. What was done?

2. Who was involved?

3. What were the management and administrative resources?

4. What are the obvious short term impacts?

5. What are the longer term impacts?

6. What are the lessons learned?

Methodology:

Williams Consulting, prepared a qualitative and quantitative study design in response to the formative evaluation objectives. In keeping with the participatory approach and current planning practices, it was important that the primary information for this study come from the people who have the experience the researcher is seeking to understand.

The Community Public Health Pilot Project is a demonstration project intended to operate for up to five years. The evaluation framework is designed to be a process or formative evaluation with a focus on project processes and the progress made toward the 5 year project outcomes. A formative evaluation typically involves gathering information during the early stages of the program, with a focus on finding out whether efforts are unfolding as planned, uncovering any obstacles, barriers or unexpected opportunities that may have emerged, and identifying mid-course adjustments and corrections which can help insure the success of the work. The feedback from the evaluation is primarily designed to fine tune the implementation of the program and often includes information for use by program managers.

Formative evaluation is a method of judging the worth of a program while the program activities are forming or happening. By contrast, a summative evaluation is a method of judging the worth of a program at the end of the program activities; the focus is on the outcome. The purpose of a summative evaluation is to assess the quality and impact of a fully implemented program.

This formative evaluation framework is intended to examine the overarching project goals and objectives through lenses of:  effectiveness, efficiency, partnerships, relevance, impact and lessons learned. It utilizes practical questions to ascertain data that taken together provide information regarding the goals and objectives and may also provide specific individual pilot sites with information useful to their particular site. The framework will be adjusted for years 2, 3, and 4 to create the summative evaluation after year 5.

To answer these questions information will be gathered in a variety of ways including site visits, observation, document reviews, key informant interviews, focus groups, self activity reports, and individual surveys. These methodologies will increase from year to year to build on the previous year’s knowledge.

Sources of that information will include: the agency itself, community members, government and non-government partners and other community departments and related agencies.

Summary of Core Competencies:

This contract demonstrates our ability to work collaboratively with a committee, conduct focus groups and interviews with key stakeholders, develop data collection tools within a short time frame, collect large data sets from multiple lines of inquiry, provide analysis and synthesis the material into useful evidence based reports.

• Working with Federal government, service providers, regional consultants and First Nations people
• Review of existing programs and development of assessment tools
• Questionnaire design
• Key Informant interviews with a number of different stakeholder groups including First Nation leadership and representative of service providers at the federal, provincial and First Nation, community levels.
• Facilitate focus groups and meetings with key stakeholders to receive feedback
• File, document and literature review;
• Auditing of financial and administrative data
• Subject matter expertise; and
• Report Writing.

Evaluation of the Strategic Community Risk Assessment and Planning for Enhanced Tuberculosis Program

Date (Month/Year): July 2009 – January 2010

Client Name: Andrea Coady, National Tuberculosis Nurse Advisor, Communicable Disease Control Division, Primary Health Care and Public Health, First Nations & Inuit Health Branch, Health Canada,

Research Subject Matter:

The First Nations and Inuit Health Branch (FNIHB) in partnership with Aboriginal peoples, Tuberculosis (TB) experts and community health care providers developed a resource to enhance community capacity to develop local Tuberculosis programs. The resource was piloted in 2004, the key recommendations from the pilot evaluation were incorporated into the final product and it was released for use by First Nation communities across Canada through their respective FNIH Regional TB Programs.

The First Nations and Inuit Health Branch is undertaking an evaluation of the SCRAP-TB resource.  The evaluation will seek to answer how communities were made aware of the resource, whether it is being used in First Nation communities and how useful it has been as a tool to support and enhance TB programming. This contract has been awarded through a competitive process to a qualified Aboriginal Consulting firm, Williams Consulting, with extensive experience in the area of Aboriginal health.

The evaluation will utilize the existing National SCRAP-TB Evaluation Framework Document to guide this process. The framework document was originally developed to evaluate the SCRAP-TB pilots completed in 2004.

The main objectives of this evaluation are as follows:

Objective 1

Determine the SCRAP-TB resource relevance and effectiveness in addressing and enhancing TB programming for First Nations (FN) communities in Canada by answering these key questions (to be guided by the framework document methodology and evaluation tools):

• To what extent is SCRAP-TB being utilized in FN communities across Canada since its release?
• What is the communication strategy to ensure key stakeholders (community members, FN health care providers, FN leadership) are aware of the availability of the resource? Is the communication strategy effective?
• Are the SCRAP-TB resource materials (Coordinators and Participants Guides) at an appropriate language level for the intended audience?
• Are the resulting TB action plans enhancing TB performing at the community level? Do the benefits of implementing SCRAP-TB continue after the process has been completed?
• What impact does SCRAP-TB have on the capacity to promote and protect the community’s health with respect to TB?
• In communities where SCRAP-TB has been utilized, to what extent has TB knowledge, TB program partnerships and community involvement been affected?
• Are there sufficient support and fiscal resources available to support the application of SCRAP-TB at the community level?
• What are the barriers to initiating SCRAP-TB?
• What are the key challenges of working through the steps of SCRAP TB and its subsequent completion?

Objective 2

To determine whether it is an appropriate tool to share with other Indigenous populations for adaptation at the National and International levels.

• Does the resource achieve the objectives for which it was designed and intended?
• Based on the outcome of the evaluation, should the resource be recommended for use with other Indigenous populations both nationally and internationally?
• Does SCRAP-TB meet the World Health Organizations Stop TB Strategy, Objective #5 – Empower people with TB, and communities, Community Participation in TB Care?

Methodology:

Williams Consulting, prepared a qualitative and quantitative study design in response to the evaluation objectives. In keeping with the participatory approach and current planning practices, it was important that the primary information for this study come from the people who have the experience the researcher is seeking to understand.

Williams Consulting utilized a document and literature review, as well as key informant interviews with regional First Nation and Inuit Health Coordinators and First Nation community members who utilized the resource.

Summary of Core Competencies:

This contract demonstrates our ability to work collaboratively with the project authority, conduct interviews with key stakeholders, develop data collection tools, provide analysis and synthesis the material into useful evidence based reports.

• Working with Federal government, service providers, regional consultants and First Nations people
• Review of existing programs and development of assessment tools
• Questionnaire design
• Key Informant interviews with a number of different stakeholder groups including First Nation leadership and representative of service providers at the federal, provincial and First Nation community levels.
• Facilitate focus groups and meetings with key stakeholders to receive feedback
• File, document and literature review;
• Subject matter expertise; and
• Report Writing.

Date (Month/Year): February 2009 – September  2009

Client Name: Sylvia Maracle, Executive Director, Ontario Federation of Indian Friendship Centres

Research Subject Matter:

The Ontario Federation of Indian Friendship Centres (OFIFC) is a provincial Aboriginal organization representing the collective interests of twenty-seven member Friendship Centres located in towns and cities throughout the province. The OFIFC administers a number of programs which are delivered by local Friendship Centres in areas such as health, justice, family support, and employment and training. Friendship Centres also design and deliver local initiatives in areas such as education, economic development, children’s and youth initiatives, and cultural awareness.

The Vision of the Aboriginal Friendship Centre Movement is “to improve the quality of life for Aboriginal people living in an urban environment by supporting self-determined activities which encourage equal access to and participation in Canadian Society and which respects Aboriginal cultural distinctiveness”.

A culturally appropriate Code of Ethics is the cornerstone to the OFIFC’s award winning culture-based approach to communications, lobbying, training, and program support. In 1985 the OFIFC became one of the first Aboriginal organizations to establish a Twenty-Year Long Range Strategic Plan.

Friendship Centres are not-for-profit corporations which are mandated to serve the needs of all Aboriginal people regardless of legal definition. This necessitates responding to thousands of Aboriginal people requiring culturally-sensitive and culturally-appropriate services in urban communities.

Williams Consulting was contracted by the Ontario Federation of Indian Friendship Centres to conduct a gap analysis of information and data collection needs for OFIFC programs. OFIFC is a leader in collecting electronic data and aims to improve their capabilities to produce synergistic information/data collection across their programs.

Currently six of the 22 programs administered by OFIFC through the Friendship Centres have electronic databases in which to collect information.  They are designed to provide basic program information and to meet funding requirements.  Potential future uses for OFIFC information collection identified by the client include:

• Program management which is more responsive;
• Tracking clients between programs;
• Accountability to more effectively manage programs;
• Determining trends in client and participant patterns;
• Influencing policy; and
• Being responsive the emerging trends of OFIFC constituents by providing evidence-based information to obtain the necessary financial resources.

The scope of this work included:

1.     Conducting an environmental scan of existing OFIFC electronic databases (which include the OGI, Mental Health, Children’s Mental Health (AHWS), Life Long Care (LLC), Akwe:go and AHWS) from a social scientific approach.  The environmental scan documents existing data variables collected through the application; analyzes relevance and consistency of information contained in the variables; identifies gaps/inconsistencies in terminology and/or variables for comparative analysis purposes; and makes recommendations for additional program/policy variables to inform the electronic database development.

2.     Conducting an environmental scan of OFIFC programs collecting data manually (including core activities of Friendship Centres) using a social scientific approach.  The environmental scan documents information fields (variables) currently being collected manually; examines commonalities and disparities; determines the variables necessary to meet funding requirement needs; and makes recommendations regarding the addition of potential variables for program/policy development needs; and the value of adding additional programs to the OFIFC electronic database, and if so.

In addition, Williams Consulting conducted data analysis of OFIFC’s electronic databases, (OGI, Mental Health, Children’s Mental Health (AHWS), Life Long Care (LLC), Akwe:go and AHWS).  They have exported up to eleven years of data from the abovementioned electronic databases (to March 31, 2009) into SPSS and conducted data cleaning.  In addition, syntax has been developed to recode data to enable basic statistical applications on the existing core data (including exploratory data analysis, cross-tabulations, correlations, etc).  The preliminary analysis and scan of this data has informed the relevance of variables, as well as strengths and gaps in data collection.  It has also influenced any recommendations for additional program/policy variables to enhance the impact and utility of information collection.  The SPSS data files will be provided to the client so that OFIFC analysts may examine their individual program data, as well as cross-reference program data.

Methodology:

Williams Consulting, prepared a qualitative and quantitative study design in response to the environmental scan objectives. Key informant interviews and meetings were conducted at the OFIFC Director level to discuss program information needs of the OFIFC. Meetings were held with OFIFC Program Managers to determine their vision for more effective and efficient information gathering.  The purpose of these meetings was to discuss with the Program Managers their program databases and information collecting practices. Program Managers also provided annual submission, year-end reports and information collecting forms for use in the creation of this report.

Williams Consulting conducted a document review of more than 120 documents from 20 OFIFC programs.

A preliminary database review was conducted using the electronic copy of the separate application schema and table structures for the OFIFC databases provided.  This included analysis of the tombstone data, program requirement and funding requirement components of each program database.  The applications included:

• OFIFC – GREAT Initiative (OGI)
• Aboriginal Healing and Wellness Strategy (AHWS)
• Mental Health (MH)
• Children’s Mental Health (CMH)
• Akwe:go

During the implementation of the study, an environmental scan of existing OFIFC electronic databases was conducted, which include the OGI, Mental Health, Children’s Mental Health (AHWS), Life Long Care (LLC), Akwe:go and AHWS.  The analysis was conducted from a social scientific approach in order to document the current data variables.  An analysis of the relevance and consistency of information contained in the variables was then completed.

Part of the analysis of the variables was to identify gaps and inconsistencies in the terminology and/or variables used for comparative analysis purposes.  Where appropriate, the report makes recommendations for additional program and/or policy variables that may be used to inform the electronic database development.

An environmental scan of OFIFC programs collecting data manually (including core activities of Friendship Centres) was conducted using a social scientific approach.  Information fields (variables) currently being collected manually was documented to examine commonalities and disparities.  The variables necessary to meet funding requirement needs were identified as well as additional variables for program and/or policy development needs.  The report makes recommendations regarding the value of adding additional programs to the OFIFC electronic database, and if so, will identify variables to be included.

OFIFC Program Managers were very helpful in providing the information collected on their programs.  Williams Consulting requested a sample of templates on the most recent annual submissions, year-end reports and information gathering forms that are used to collect information for all the programs.  A data dictionary was developed and submitted to OFIFC documenting this information, and included the following programs:

• Aboriginal Community Mental Health
• Aboriginal Criminal Courtwork
• Aboriginal Healing and Wellness
• Aboriginal Healthy Babies Healthy Children
• Aboriginal Responsible Gambling
• Akwe:go – Urban Aboriginal Children’s Programme
• Child Nutrition
• Community Action Programme for Children (CAP-C)
• Community Justice
• Family/Combined Courtwork
• Fetal Alcohol Spectrum Disorder
• Life Long Care
• Ontario Aboriginal Health Advocacy Initiative
• OFIFC – GREAT Initiative (O-GI)
• Urban Aboriginal Healthy Living
• Urban Multipurpose Aboriginal Youth Centres Initiative
• Wasa-Nabin – Urban Aboriginal Youth
• OFIFC Core

Data analysis of the existing electronic data was conducted on the six OFIFC’s programs with an electronic database, which include the OGI, Mental Health, Children’s Mental Health (AHWS), Life Long Care (LLC), Akwe:go and AHWS.  The data was exported to PASW (formerly known as SPSS) statistical analysis software to conduct data cleaning and the development of syntax to recode data.  Basic statistical analysis on the existing core data is underway, including exploratory data analysis, cross-tabulations and correlations.

Numerous challenges were faced in importing the data to social science statistical software including obtaining permissions, access to the information as well as the cumbersome manner in which the data tables were stored.  For example, the AHWS database consists of 56 separate tables of data that range in both the number of variables it collects and the number of records found within each database.  Alternatively, the Akwe:go database consists of 58 tables of data but are labelled AHWS, Core and CMH.  It is assumed this database was developed later and that there were elements carried over from previously developed database applications.  Sorting through the various tables to determine whether they contain pertinent data further compounded the difficulties in analyzing the information found within.  For example, only seven of the 56 separate data tables in AHWS contained relevant data for analysis.  These tables have been imported separately for analysis and efforts are being made to join them in one analysis file for cross-tabulations and statistical interpretation.

The analysis and scan of the data which was collected in the abovementioned electronic databases were used to determine the relevance of variables and identify strengths/gaps in data collection.  Also, recommendations for additional program and/or policy variables were made which could enhance the impact and utility of information collection.

In this report, Williams Consulting synthesized the relevant information and findings collected from multiple lines of inquiry and offered an interpretation of the data. The environmental scan data analysis involved the triangulation of both qualitative and quantitative data.  This report discusses options to improve the synergy/harmonization of the OFIFC electronic database. The Federation’s vision for enhanced information collection is to expand the utility of information and data collection within a competitive environment.  This expansion will enhance their capacity to project trends, influence policy, be more responsive to emerging trends/needs of community members and provide evidence-based information to obtain the necessary financial resources to address the emerging trends of OFIFC constituents.  An enhanced database would potentially enable OFIFC to leverage financial resources to meet the changing community needs, and demonstrate impact of their unique and distinctive programming.