EXPRESSION OF INTEREST (EOI)

DATA ANALYTICS AND PSYCOMETRICIAN SERVICES REQUIRED

Touchstone Institute (“Touchstone”) is a non-profit corporation governed by a board of directors and receives financial support from the Government of Ontario. Touchstone Institute works closely with physicians, nurses and other healthcare providers who provide subject matter expertise to assess and improve the level of competency across individual practice. Building on its history to facilitate entry to training or practice for Internationally Educated Health Professionals, Touchstone Institute seeks to continue the work in this area while expanding to support excellence in competent practice for all who deliver health care.

VISION: Public confidence in professional competence.

MISSION: To excel in providing competency assessment and education services that meet professional and societal needs.

Touchstone is seeking Expressions of Interest (EOI) from individuals or firms for the provision of occasional data analytics and psychometrician services, on an ongoing basis.

This is a unique opportunity for individuals/firms to provide valuable insights and data-driven solutions through their expertise in Data Analytics and Psychometrics to contribute to the important work of Touchstone. Successful individuals/firms will work closely with subject matter experts and internal staff to ensure Touchstone assessments are reliable and meet their intended purpose.

Opportunities are open to qualified individuals/firms and are being advertised as a services undertaking, on a consulting basis. Individuals/firms would not be considered employees of Touchstone, but rather will be asked to perform work on an as needed basis during the term their Services Agreement. One or more individuals/firms may be selected to enter a Service Agreement with Touchstone.

All work will be done remotely.

Description of Services

1. Psychometrician Services

The individual/firm providing psychometrician services will be is responsible for contributing to the development, administration and interpretation of Touchstone assessments including knowledge tests and performance-based exams. The Psychometrician will also contribute to research projects related to Touchstone assessments.

Additionally, services could include:

Design and Development

• Consult with Touchstone Institute staff and relevant Subject Matter Experts (SMEs) to develop and operationalize clinical examinations using exam methodology that are deemed appropriate and psychometrically defensible.
• Create examination blueprints based on feedback and content provided by SMEs.
• Lead examination development inclusive of item writing, standard setting and assessor/examiner training.
• Employ appropriate methodologies to validate assessments to ensure they are fit-for-purpose.
• Consult with staff and SMEs to develop, pilot and validate workplace-based assessment programming.

Quality Assurance and Quality Improvement

• Conduct an evaluation of assessment scoring systems and methodologies to ensure ongoing psychometric integrity relating to current and future assessment programs.
• Develop quality assurance standards and procedures that align with industry best practices.
• Contribute to on-going quality improvement initiatives to ensure the continued integrity of assessments.
• Contribute to developing processes and procedures to ensure appropriate data storage in accordance with best practices in the field.
• Guide internal staff to develop new analysis tools and technologies.

Operations and Reporting

• In collaboration with senior leadership, guide the development of policy and protocols surrounding execution of assessment and exams.
• Collaborate with senior leadership and team members to further improve and refine research processes and approaches.
• Collaborate with Program Directors to develop and deploy statistics, survey and data analysis solutions and visualizations for clients and funders.
• Continually ensure that all data released, and research developed by the organization is accurate and demonstrates best practices.

Requirements

Individuals/firms should possess a superior understanding of assessment theory and methodological approaches to performing psychometric analyses, have excellent knowledge and experience with statistical analysis software such as R, SPSS or Python, knowledge of relevant health profession terminology regarding competency-based assessment, and familiarity with key concepts related to workplace-based assessments.

In addition, they should possess strong conceptual, analytical and problem-solving skills, plus strong data visualization skills, and comfort applying them in new and innovative ways.

A master's degree in Statistics, Biostatistics, Psychology, Developmental Psychology, or other equivalent degree is required, and a PhD is an asset.

2. Data Analytical Services

The Data Analytics work involves performing data maintenance for Touchstone's assessment and educational programs, conducting quantitative analyses to develop reports for internal and external stakeholders, creating datasets to support organizational research projects, and providing support and consultation to internal Touchstone Institute departments.

Specifically, services could include:

• Define, transform and structure data so it is ready for analysis (i.e., data cleaning);
• Contribute to development of systems/processes for updating and storing exam statistics generated post exam, such as exam banks etc.;
• Work alongside manager to apply best practices to complete test development, preparation and administration responsibilities that maintains validity, reliability and fairness of assessments at all times;
• conduct analyses on assessment results to generate scores and psychometric statistics and participates in processes to ensure accuracy;
• Contribute to quality assurance through understanding and interpreting basic statistics;
• Complete quantitative and qualitative analyses and provide statistics for operational and research reports, publications, and funding applications as required;
• Contribute to the development of reports for various stakeholders;
• Extract data and prepare the data for analyses by both internal staff and external stakeholders. This includes data validation, de-identification, and preparing written documentation describing data specifications;
• Complete data integrity checks (e.g., descriptive statistics) to ensure consistency with previously reported data, and reconcile any differences;
• Ensure optimized and accurate collection of data, improving the efficiency of data collection, processing, submission and retrieval to meet the objectives the organization and its funders; and
• Other relevant services.

Requirements

Individuals should have a good understanding of measurement theory and methods, be able to perform statistical analysis and interpret results and have strong analytical and report writing skills in addition, Individuals will have well-developed critical thinking skills in order to identify key issues and propose solutions to program setbacks, and have the ability to integrate data and research concepts, and exercise good judgement.

A post-secondary education in Statistics, Data analytics or a related field of study is also required, and knowledge of SPSS is preferred.

Services Agreement

The successful candidate(s)/firms shall be required to execute a Services Agreement with Touchstone as a condition of being retained to perform any of the above services. Service Agreements will be for a term of up to three years.

The Services Agreement sets out the terms and conditions which shall apply to the delivery of services described above. Touchstone reserves the right, in its sole discretion, to make minor changes to the terms and conditions when preparing the final draft Agreement.

This document, along with all appendices and candidates' information shall be attached to, or incorporated by reference into, the Services Agreement and shall form part of it.

Please note that the Services Agreements are not exclusive and there is no guarantee on the volume of work to follow. Requests for services will be requested by Touchstone as identified above on a case-by-case basis, with a clear scope of work and associated timelines. At any one time, Touchstone may engage with one or more individuals/firms to perform discrete scopes of work, as outline above.

EOI Submission Process

Interested parties should forward their EOI and curriculum vitae, electronically, to Georgina Ankrah, at [email protected] No later than 5:00PM (EST) on Friday, November 1, 2024.

Questions about the EOI or the services outlined above may be submitted to Georgina Ankrah, no later than Monday, October 28, 2024. A consolidated list of all questions and answers will be available to any party, by request, by Wednesday, October 30, 2024. To receive a copy of the consolidated list of questions and answers, please contact [email protected] .

*Note: Responses to the Expression of Interest should address all of the qualifications listed in the above Services section and include any potential or perceived conflicts of interest.