Section III & IV - Evaluation Models and Performance Improvement

One model that is covered in the text is called Brinkerhoff’s Success Case Method, or SCM. This evaluation method is based on the comparison of successful cases (of a certain situation) versus unsuccessful cases (of the same situation). This method has a heavy focus on the participation of the subjects/stakeholders and their “buy-in” to the program.

The five steps that are included in this method are: 

1) Focus and plan what is going to be evaluated, the nature of the study, as well as how the stakeholders will be ultimately affected. 

2) A visual impact model will be created to visually present the basis and outcome of the study to those involved. 

3) Hold a survey research study to find the best and worst cases. This survey would use a short questionnaire that will be given to the participants non-anonymously. 

4) Hold thorough interviews that focus mainly on the successful cases and what worked well for them. Interviewing some of the nonsuccess cases is also recommended to find out some of the factors that did not work for the participants. 

5) A final report of the findings is necessary to communicate the outcomes to the stakeholders and address the six conclusions that the SCM requires, which are: 

·         What actions and results is the program helping to produce?

·         Are some parts of the program working better than others?

·         What environmental factors are helping support success and what factors are hindering progress?

·         What is the ROI of the program?

·         How much more additional value could be derived from the program?

To evaluate my own instruction, I would use the Brinkerhoff method to determine if and how my instructional methods are affecting the Hispanic ELLs (English Language Learners) in comparison to the EOLs (English Only Learners) within my classroom. Due to me being Hispanic myself, ESL certified, and a Spanish speaker, I usually involve some sort of Spanish language connection within my lessons so that my ELLs are directly involved in what is happening, rather than typically tuning out and staying quiet throughout a class. ELLs tend to be shy and non-participatory due to their apprehension toward using the English language.  As well, a lack of background knowledge in regards to the texts being read in class causes many of my ELLs to be withdrawn because it is difficult for them to connect to topics to which they have never been exposed. Using the Brinkerhoff method would help me evaluate whether or not the use of Spanish cognates and Hispanic background connections is a benefit or a hindrance to those students in relation to the EOLs in the class, who generally make English language and background knowledge connections easily. The outcome of the evaluation would determine if the grade level reading achievement of ELLs is affected in comparison to the reading achievement of the EOLs.

Another model that is explained is called Patton’s Utilization-Focused Evaluation, or U-FE. This method is complementary to other types of models and allows for the evaluators to determine if the evaluation results are being applied and to what degree the results are being utilized. If the outcomes are not being utilized, then the evaluation method is not considered valuable.

        There are nine steps included in this method: 

       1)    Conduct a readiness assessment 

       2)    Identify the “primary intended users” and develop a working relationship with them.

       3)    Conduct a situational analysis. 

       4)    Identify the “primary intended uses” 

       5)    Focus the evaluation. 

       6)    Design the evaluation. 

       7)    Collect, analyze, and interpret the evaluation data. 

       8)    Continually facilitate evaluation use. 

       9)    Conduct a metaevaluation.

These steps can be implemented alongside any of the other evaluation models. So, technically, it is an evaluation of the significance of the evaluation.

As an English teacher, this method would be somewhat more difficult to use to evaluate my instruction due to the fact that it is more complicated to evaluate the reading and writing progress of students, say, in comparison to Math, Science, or Social Studies, which can be measured somewhat more quantitatively than the writing process allows. The most plausible way I could see this method being implemented would be to break down the various writing skills and implement the evaluation during a time that specific isolated skills are being taught. This would aid in allowing the evaluation to identify if the design of my instruction is effective. This would also work for specific reading skills that could be isolated in class, such as vocabulary breakdown or context clues.

Once the intended participants are shown to make progress in their reading/writing abilities, I can build on the strategies being used and continue to evaluate said strategies for effectiveness over time. However, if the participants show no identifiable progress, the data could then be analyzed and an improvement plan can be designed. Either way once the evaluation is completed on the instructional design itself, I can then evaluate whether the data is being utilized appropriately and to what degree it is being used.

Other measures for Instructional Evaluation?

Instructional design evaluation is ultimately about whether or not students are gleaning the largest amount of usable and applicable knowledge from the instruction that is being presented to them in the amount of time they are allotted per day of their school year. So, in the end game of it all, as teachers, instructional design evaluators, and/or parents vested in their child’s overall education, we are looking at whether or not a student can take away enough usable knowledge and skills to be an effective, meaningful person in their community and/or world.

With that being said, some questions that can certainly be embedded into an instructional design evaluation would be the effectiveness of the instruction’s ability to convey the significance and/or purpose of what is being presented as something that is going to aid the participant in their educational and personal lives. If a student does not see the basic purpose of what they are doing as something that will benefit them later, the likelihood of them placing a strong emphasis on that instruction is quite low. This is a constant problem seen in many of my current and past classrooms, where students are struggling to see why they have to practice skills, because in their minds, there is not a huge importance for reading and writing in the world, especially when they do not see it in their personal/family lives. Along with significance comes the evaluation of whether or not instruction is relevant to the students who are receiving the instruction. Many students do not see importance in instruction because they do not see the relevance of it to their situation in life. Including a measure for relevancy would definitely be difficult to evaluate, given the individuality of each student, however, there could possibly be, at the very least, a measure for relevancy to real-world society as a whole.

ROI and management of resources

In regards to ROI on instructional design evaluation, I do agree that there should be return on investment in the form of higher student achievement and, at the very minimum, a higher student interest. When we are spending time and money on a program that is aimed at increasing our students’ achievement levels, there needs to be something feasible that can be seen from the effort towards improvement. Otherwise, there would be no point in implementing an evaluation on instructional design. Everyone would just do what they have been doing and everything will stay the same. When it comes to managing the resources involved, this is something that absolutely should be included in instructional design evaluations, again, due to the fact that we are spending money on resources and if they are not being implemented correctly, there is no need to continue to spend the money required for them.

Section IV

One performance problem that I, and many other teachers, have noticed is the common inability of some teachers to navigate an effective classroom management program, even after they have been teaching for a few years. When I say some teachers, I am mainly focusing on teachers who are not able to effectively run a classroom to the point where learning is actually occurring. Often, these are first-year teachers or even some older teachers who are set in their old school ways, though it can include teachers who are in their third year and beyond, due to the fact that they never learned how to run efficient classroom. I have taught in schools where I have seen teachers walk-out of classrooms, where students have threatened teachers, where students have disrupted the classroom so thoroughly that the teacher was not able to teach at all. Many times, the negative actions of the students stems from their realization that the teacher has not clue what they are doing. 

Aside from the aspect of the kids certainly being held accountable for their actions, I feel that teachers would feel much more supported when they have someone who can come in to show them how to implement a great classroom management system to help the students find a routine and minimize the foolishness that can occur when the teacher is not in full control. In this case, there really isn’t a “class” that a teacher can take to help them be a more assertive, effective teacher - they simply have to see it being done, practice what they have seen, and then pull the confidence from within to pull it off in their classroom. Informal learning can take place by watching another teacher manage their classroom as well as having a performance support system where a teacher/administrator is available to come into the classroom in certain situations to facilitate and guide (without judgment) while the main teacher gets his/her bearings. 

As well, in the case of alternative certification, many teachers do not have adequate time to practice handling a classroom full of children and then they get thrown into the trenches, sometimes quitting after the first year because they are so overwhelmed. If school districts had a stronger support system than, say, just a mentor, I feel that many more teachers would stay in the field because they would feel supported and know that they have the opportunity to learn by watching and following some sort of knowledge management system. One example of a knowledge management system could be where new teachers are given a basic comprehensive list of the most important things they need to focus on for the first year, such as but not limited to, specific grading procedures, parent contacts, getting to know the kids, and creating and sticking to routines. This could help those teachers not worry about what they don’t know, which is usually what most new teachers are worried about - what am I missing and forgetting to do?

Despite the fact that a good amount of teachers have the ability to figure out their own management systems and end up being strong, effective teachers, a system such as this would help strengthen and build up a resilient educator force necessary to take on the next generation of students.