An objective independent evaluation of the effectiveness of program elements in reaching policy goals

  • Agency:
    U.S. Department of Education
  • Office:
    Office of the Secretary, International Affairs Office
  • Solution:
    Synergy developed and successfully applied an evaluation and assessment approach that included appropriate qualitative and quantitative tools, survey and interview protocols, and data analysis.

Synergy's Approach

Synergy was awarded the first Department of Education (ED)/Asia-Pacific Economic Cooperation (APEC) contract in 2005, which continues through 2015. Our subject matter experts in international education policy developed briefing papers and conducted a systematic analysis and synthesis of existing documents focused on providing access to high-quality, evidence-based knowledge related to the APEC Ministers’ five priority areas: mathematics and science education, career and technical education, language education, information and communication technologies, and systemic reform.

In 2008, Synergy performed a short-term analysis of the existing APEC—it is the premier Asia-Pacific economic forum, consisting of 21 member countries. APEC’s primary goal is to support sustainable economic growth and prosperity in the Asia-Pacific region. Synergy was selected to create a strategic plan that delineated steps to increase awareness, quality of materials, and use of Human Resources Development Working Group (HRDWG) websites throughout the APEC region. This included an analysis of the current content collection and generation methods of existing APEC websites (EDNET Portal and Knowledge Bank). Through this analysis, Synergy created a system to allow for submission and review of publicly available resources—materials, policy documents, lesson plans, and curriculum standards—from APEC members.

Synergy’s researchers continue to design, direct, and manage numerous large-scale federal and international evaluation and education policy projects. Senior researchers have delivered Keynote Presentations for APEC (e.g., at the Education Research & Policy Pre-ministerial Symposium on February 5, 2012, in Moscow, Russia) and worked with international faculty on a six-chapter research study of teacher preparation in math and science. Our senior researchers also contributed content expertise to APEC in the form of a literature review on foreign language education for APEC program staff at ED. The paper was titled “The Leading Role of APEC in Language and Culture Education” and was published in 2014 in the peer-reviewed World Language Education and Talents Cultivation under Beijing Language and Culture University Press.

The APEC Secretariat’s Steering Committee on Economic and Technical Cooperation (SCE) selected Synergy to perform an independent review of HRDWG operations to assess their efficacy and impact on the SCE and APEC objectives. The HRDWG assessment required document analysis, participant observation at appropriate HRDWG meetings, surveys, and interviews with relevant stakeholders. Our empirical research methods included document analysis, participant observation and interviews, and survey analysis. We carried out quantitative and qualitative data analyses using tools appropriate to research principles and ease of use. Synergy analyzed survey data using SPSS and extracted themes from narrative and observational data using ATLAS software. Synergy’s evaluation was well-received by the SCE and the APEC Secretariat, which accepted our recommendations and developed a schedule for adopting them.