|
Results of an
interview conducted with Dr. Deborah Hobbs of Spectrum Education
Group, the external evaluator of a study based in Huntsville,
Alabama for a TAH program entitled Community for Teaching America’s
Past (CTAP).
Please note that
responses are summaries from the interview, and do not represent
verbatim responses.
What is the CTAP program?
How many teachers are involved in CTAP?
What types of professional development activities are part of this TAH program?
What is the role of the evaluator for this TAH program?
How are you measuring changes in teacher content knowledge?
As part of the evaluation, are there other things you are measuring?
What obstacles have you faced in terms of the evaluation?
What do you see as the strengths of the evaluation?
Are there innovative or unique aspects of your evaluation that you would like to share with other grantees?
What other insights or comments do you have based on your experience as an external evaluator for a TAH program?
What is the CTAP program?
Community for Teaching America’s Past (CTAP) focuses on Alabama’s
role in American History. The two goals of CTAP are (1) to increase
the depth and breadth of teachers’ American history content
knowledge by teaching them to think historically and providing them
with extensive, dynamic and high quality content instruction and
resources and (2) to improve teachers’ ability to present American
history content to students by providing them with content-rooted
pedagogical instruction and making them citizens of a collegial
community of practice dedicated to improving instruction.
As a Fall 2005 grantee, CTAP spent its
initial year developing the CTAP experience for the first group of
teachers and 2006 as the first year of implementation of CTAP. back to top
How many teachers are involved in
CTAP? There were 30 fourth grade teachers in the project from
across seven districts in the northern Alabama area.
Dr. Hobbs, the project evaluator from
Spectrum Education Group (SEG), noted that CTAP intentionally and
specifically focused on fourth grade teachers. She mentioned that
she has had other projects which have tried to include all grade
levels from K – 12 in a single project. Dr. Hobbs has advised TAH
clients to focus on either elementary, middle school, or high school
so the content can be geared specifically to those teachers and
their learners. Dr. Hobbs noted that “You can put a fourth grade
teacher in a [professional development] institute or workshop with
an AP history teacher and the content is going to be received very
differently.” The self-contained classroom used in elementary
grades requires different content and teaching methods as contrasted
to secondary classroom settings.
back to top
What types of professional
development activities are part of this TAH program? Saturday
seminars and workshops are provided monthly to CTAP participants.
The Project Director, Pam Gothart, generally has a speaker on
content and then a speaker on pedagogy, and the pedagogy person
typically integrates content knowledge. There are also local field
trips. The participants’ tour last year was all over the state of
Alabama to tie in with their focus on the state as part of American
history. Some of the responses in the anecdotal evidence were that
teachers had talked about these sites and events for years and now
they had “breathed life into it” by visiting these locations. One
teacher noted, “My kids are so excited that they can’t wait until I
go again.” Dr. Hobbs added that professional development is an
ongoing constant need akin to any other profession; and she also
noted that she believes CTAP has had huge gains because the
professional development has been able to go in-depth into the
content areas.
During each busy school year, CTAP
employs a blended approach to professional development that combines
the best attributes of technology- and classroom-based instruction.
Dubbed “Classroom-anchored, media-based study” (CAMBS), this
approach both accommodates the time-constraints of life as a teacher
and satisfies the mandates of instructional theory. In practical
terms, CAMBS programs consist of two tightly integrated strands of
instruction, one media-based and the other delivered “live” in
classrooms:
1) The “media-based” strand delivers
the bulk of the content and can consist of courses offered over the
Internet, on DVD or CD-ROM, or on educational television, depending
on teachers’ needs. (Teachers complete media-based lessons or
sessions according to a schedule that divides the body of
instruction into topical sections, or “blocks,” and that indicates
dates by which each participating teacher should complete and submit
questions concerning each block of content.
2) The classroom-anchored strand
satisfies learners’ needs to personal, expert answers to specific
questions and consists of periodic two-hour, in-person “capstone”
sessions, one of which takes place at the conclusion of each content
block—every six weeks or so. During the first hour of each capstone
session, prominent professors from local universities (or other
qualified subject matter experts) answer content questions submitted
by teachers as they studied the preceding content block. During the
second hour, teachers participate in a collaborative workshop lead
by an accomplished educator on the pedagogical challenges of
presenting the content in question to students.
back to top
What is the role of the evaluator
for this TAH program? For CTAP, Dr. Hobbs is an active
participant in the project team. As an integral part of the team,
Dr. Hobbs provides objective feedback to the director (Pam Gothart).
She is able to make suggestions and Dr. Hobbs adds that, “Pam sees
the value of evaluation. I see Pam as an esteemed colleague, and I
think she sees me as the same. She is extremely capable, very well
organized and she listens. It makes all the difference in the
world.”
Dr. Hobbs is located in Utah and
visits the project site in Alabama three times a year to conduct
evaluation activities. She is in constant email or phone contact
whenever needed. Through a subcontract with Spectrum Education
Group, DeepWell Data Services provides a web portal through which
data is input from the project and instantly reported back to staff
to allow for Continuous Improvement Management.
back to top
How are you measuring changes in
teacher content knowledge? The CTAP evaluation plan is based on
a five-level framework specifically designed to meet the needs and
specifications of CTAP based on the work of Dr. Thomas Guskey
(2000). [See this month’s “Resource Spotlight” for more information
on Guskey’s framework]. The second level of Guskey’s framework
measures effects on participant knowledge. Specifically, the CTAP
evaluation measures teacher content knowledge through a pre/post
test instrument that was prepared by the project director in
collaboration with the University of Alabama at Huntsville. Content
tests and formative assessments are available online to participants
and the data is available immediately through the web portal. The
professional development provided throughout the year is designed to
address the gaps teachers have in their understanding based on the
initial pre-test and other feedback.
back to top
As part of the evaluation, are
there other things you are measuring? An all too common fault
of many evaluation plans is the lack of scientifically-based
measures of program implementation. CTAP uses the Concerns Based
Adoption Model (CBAM) and its several component measures to assess
teachers’ implementation of CTAP-provided content-knowledge and
pedagogical practices. CBAM components are listed and briefly
explained below.
First, the “Innovation Configuration”
document describes the innovation(s) proposed by the project in
terms of observable behaviors. In effect, IC documents describe how
classrooms should look and sound, how teachers should teach, and how
students should perform as a result of CTAP professional development
activities.
Next, the “Stages of Concern” (SoC)
survey enables evaluators to focus on the concerns of individual
teachers. It divides the continuum of concerns teachers might have
about the project into seven “stages” and measures the intensity of
their focus on concerns within each stage at regular intervals (see
Figure A at right). Concerns in the lower stages are self-related,
those in the midrange are task-related, and those in the higher
stages are impact-related. A progressive change of focus in an
individual (i.e. moving “up” through the stages) can be regarded as
evidence of participants’ increasing adoption of concepts taught
through project-supported professional development. |
