Assignment
#3: Evaluation Assessment
Centennial Collegiate History Study Tour Program
Engage Stakeholders
Who should be
involved?
Student
participants in HSTP, teachers, parents, administration, Saskatoon Public
School Division
|
How
might they be engaged?
By
participating in a study of a unique learning opportunity to determine
whether or not the structure should be adjusted to ensure student success in
all areas of study.
|
Focus
the Evaluation
What are you going to
evaluate? Describe program (logic
model).
The
History Study Tour Program (HSTP) at Centennial Collegiate. This is a unique two year program that
students can apply to enter starting in their grade eleven year at Centennial
and ending with a two and a half week tour of Europe in grade twelve. The program provides students with an inter-disciplinary
approach to learning about History through the lens of Art by considering the
connections between the two subjects through activity based investigations. The course focuses its study of history on
the 20th Century and specifically examines the two world wars, the
rise of dictatorship, the Cold War, and the impact Canadians had during this
time. The overall goal of the program
is to provide students with a more meaningful understanding of History and
expand their independence and critical thinking skills when they learn about
the world around them.
|
What
is the purpose of the evaluation?
To
determine if the program is a valuable enough learning experience to justify
pulling students away from their other classes for such a long period of
time.
Use
as a guide for future recruitment of students to Centennial.
|
Who will use the
evaluation? How will they use it?
Who/users
|
How
will they use the information?
|
Administration
|
To
determine if the length and time of tour is beneficial to student success in
all of their classes and make adjustments to the structure of the program if
needed.
|
Students
|
To
make an informed decision about their choice to enter the two year program.
|
Division
|
To
determine if the program would be successful at other schools.
|
What
questions will the evaluation seek to answer?
Is
the learning experience for students meaningful enough to justify the length
of time missed in other classes while on the European tour? If the trip was connected to a break to
avoid missing so much class time, would it aid student success?
|
Do
students fall behind in other classes as a result of participating in the
program? How do different curricular
departments view the program?
|
Is
the program only tied to Centennial and the teacher that created it? Or is this a program that should be started
at other schools to enhance student enrollment?
|
What information do
you need to answer the questions?
What
I wish to know
|
Indicators
– How will I know it?
|
How
do students feel about missing so much time in other classes as a result of
being in this program?
|
Interview
of program participants
|
Do
grades drop as a result of missing two and a half weeks of regular classes?
|
Survey
with data analysis
|
How
do teachers of other curricular courses view the effects of the program on
student success?
|
Interview
of teachers in other departments
|
Could
the tour dates be changed to avoid missing so much class time? Would the learning experience still be as
meaningful if the trip was not organized around Remembrance Day?
|
Interview
and survey of teachers and participants
|
When
is the evaluation needed?
February
2014, before course registration for the 2014-2015 school year.
|
What
evaluation design will you use?
Scriven
formative/summative with a participatory evaluation approach.
|
Collect
the information
What sources of information will you use?
Existing information:
|
Course
brochure, assignments, student blog
|
People:
|
Interview
students and teachers that participate in the program, other teachers of
curricular courses at Centennial
|
Pictorial records and observations:
|
Images
and observations from Italy 2013 trip
|
What data collection
method(s) will you use?
•
|
Survey
|
|
Document review
|
•
|
Interview
|
•
|
Testimonials
|
•
|
Observation
|
|
Expert panel
|
|
Group techniques
|
|
Simulated problems or
situations
|
|
Case study
|
|
Journal, log, diary
|
|
Tests
|
|
Unobtrusive measures
|
•
|
Photos, videos
|
|
Other (list)
|
Instrumentation: What
is needed to record the information?
Interview
questions, access to online blogs, photos from program participants, survey
(online “Poll Everywhere”)
|
|
When will you collect
data for each method you’ve chosen?
Method
|
Before
program
|
During
program
|
Immediately
after
|
Later
|
Interview
|
|
|
•
|
|
Survey
|
|
•
|
•
|
|
Observation
and Photos
|
|
•
|
|
|
Will a sample be used?
No
|
|
|
Yes
|
•
|
If
yes, describe the procedure you will use.
|
|
||
Survey
questions
|
Pilot
testing: when, where, how?
Pilot
test survey and interview questions before they are given to participants and
teachers.
|
Analyze and Interpret
How will the data be
analyzed?
Data analysis methods:
|
Survey
questions will be scored and averaged according to answers to look for
commonalities. Interview answers will
be analyzed and I will also look to see if there is consistency in responses.
|
Who responsible:
|
The
evaluator of the program (me)
|
How will the information be
interpreted—by whom?
The
information would be interpreted by the evaluator to determine whether or not
the program could be restructured to aid in student success.
|
|
What did you learn? What are
the limitations?
Limitations
could be student answers. Unless the
survey is anonymous, students may not give accurate answers.
|
There
may be bias when interviewing certain teachers (ie: the program creator, or
other department teachers who feel like the program may be taking away from
student learning in their classes)
|
Use the Information
How will the evaluation be communicated
and shared?
To whom
|
When/where/how to
present
|
Future
program participants
|
Before
they decide whether or not to apply for the program.
|
Administration
|
After
the evaluation is complete to determine whether or not they should restructure
the time frame.
|
School
Division
|
After
the evaluation is complete to determine if they should try to make the course
available at other schools.
|
|
|
•
|
Human subject’s protection
|
|
Utility
|
|
Management chart
|
•
|
Feasibility
|
•
|
Timeline
|
•
|
Propriety
|
•
|
Responsibilities
|
|
Accuracy
|
|
Budget
|
|
|
Good program theme for this assignment. You do a great job of setting up the assessment with a description of the program. The questions and how you hope to address them are also strengths. You could add a bit more detail around data collection, sample and the analysis of the results. I agree with the limitations you raise. Have there been any surprises in your initial assessment of the program? Make sure you address the management of your evaluation in the final project more thoroughly. These are crucial elements to the success and acceptance of a solid evaluation plan.
ReplyDeleteJay
Jay