CAEP Institutional Report

CAEP Annual Report
Years 2013-2014

Standard One: Candidate Knowledge, Skills, and Professional Dispositions

1.1 Candidate Knowledge, Skills, and Professional Dispositions

What do candidate assessment data tell the unit about candidates' meeting professional, state, and institutional standards and their impact on P-12 student learning? For programs not nationally/state reviewed, summarize data from key assessments and discuss these results.

The COE uses four (4) transition points to measure a candidate's knowledge, skills, dispositions, and impact on P-12 student learning as one moves through a given program or variant. The Transition Point Assessment Table, attached, displays the key assessments at each transition point. The first column lists each transition point. The following two columns are separated by use. The Verification column lists key assessments we use for documentation purposes. The final column lists key assessments whose data are analyzed to make program changes are discussed below.

Transition Point One: Admission
Incoming GPA: Candidate content knowledge in her/his chosen is assessed and evaluated at the time of admission to the program. This assessment entails a detailed transcript review to ensure that the applicant has met all content requirements for the endorsement. Applicants without appropriate content background must complete the appropriate content requirements before admission. In addition, candidates are required to pass a Teacher Standards and Practices Commission (TSPC) – approved required content area examination prior to their student teaching. Candidates are required to have a minimum GPA of 2.75 prior to admission.

Basic Skills Tests: Candidates are required to enter the program having already developed or established their content expertise, as assessed by undergraduate GPA. Basic Skills tests are also considered a key assessment in evaluating to evaluate candidate content knowledge in reading, mathematics, and writing. All students are required to pass one of the following prior to admission. Standardized tests approved by TSPC are: CBEST, Praxis Academic Core, EAS, and WEST-B, as well as the ORELA Protecting Student & Civil Rights in the Educational Environment (2010 - present). ORELA MSE I/II and constitute content area tests for ECE, Elementary and Middle Level authorizations. 100% of program completers met this threshold. Data note: non-passing scores, while in the data file, indicate program non-completers.

Transition Point Two: Completion of Foundational Coursework
The second transition point is the completion of foundational coursework. Though only used for verification purposes, the COE considers cumulative GPA of 3.0 as successful completion, as well as submission of TSPC documents required for field experiences.

Transition Point Three: Methods Block Completion and Entry into Student Teaching
Subject Area Tests: Candidate scores on approved content area tests provide one measure of mastery of content knowledge. This includes Initial Teaching License candidates in secondary level authorizations and Special Education candidates as well as Advanced Programs candidates adding on an endorsement. With the exception of the special educator, reading, and ESOL endorsements, we recognize that these content area test scores are a reflection of the knowledge gained prior to entry into the COE. However, this does verify our diligence in selecting only candidates who have the content and content-pedagogical knowledge of our candidates. Passing scores have differing ranges depending on the area/test type being taken. MSE I/II are considered subject matter tests for ECE/ELEM candidates, and are noted as Multiple Subjects in the tables. An additional exam that the COE uses to assess candidate strength with regard to student learning and dispositions is the ORELA Protecting Student & Civil Rights in the Educational Environment exam. Passing scores are 240 for this exam. Candidates must pass all exams prior to student teaching For each test, 100% of our program completers have met or exceeded the state minimum score. Data note: non-passing scores, while in the data file, indicate program non-completers.

Transition Point Four: Program Completion
Teacher Summary Evaluation I/II: Candidates are assessed at the midpoint of their 1st (mid-placement) authorization experience and twice (midpoint and final) during their 2nd (student teaching) authorization experience by both their mentor(s) and university supervisor. Competencies evaluated are appropriate to the mission of Pacific University and the College of Education, aligned with the conceptual framework, comply with state standards, and aligned with national professional standards (Oregon Content Standards; InTASC Standards). The attached Alignment Matrix displays our alignment to InTASC Standards. Each competency is evaluated on a 1-6 point scale across six categories: (1) candidate content knowledge is a nine-item category entitled Content Knowledge; (2&3) content-Pedagogy is assessed in a seven-item category entitled Plan for Instruction and a six-item category entitled Standards Based Teaching; (4) pedagogy is assessed in an 11-item category entitled Establish Classroom Climate; (5) professional Knowledge and Dispositions are assessed in an 11-item category entitled Professional Behavior; (6) student Learning is assessed in a four-item category entitled Assessment. Each category of the Teacher Summary Evaluation specifically address content knowledge competence and the ability to construct effective instruction. In evaluating each section, the mentor and university supervisor must agree that overall the candidate has met the competencies in each section in order for the candidate to be recommended for licensure. Analyses are conducted for the 2nd (student teaching) Teacher Summary Evaluation, as the COE considers the 1st (mid-placement) Teacher Summary Evaluation as a formative measure to assess adequate progress, and as such scores are not maintained in the database. Those not meeting the standard either repeat student teaching until the standard was met, moved to a non-licensure track, or are counseled out of the program.

Work Sample I/II: Every candidate completes two work samples to demonstrate the ability to plan, teach, and assess a unit of instruction in a content area before they are recommended for licensure, e.g., Content Knowledge, Content Pedagogy, Pedagogy, Professional Knowledge and Dispositions, and Student Learning. The first work sample is completed during the 1st authorization experience and is used formatively to identify candidate strengths and areas for improvement. The second work sample is completed during their 2nd authorization experience and is used summatively to demonstrate necessary competencies required for initial license. The work sample process is identical across authorization experiences, and thus shows growth as candidates transition from student to professional. The work samples are evaluated by the candidate's university supervisor using a scoring rubric that has been developed, tested and modified over the years. For example, Forest Grove work samples are based on a five-point scale specific to each of 18 different domains. Each section is evaluated on a 1 (Unmet) to 5 (Exceptional) scale with a minimum of 3 (Met) for acceptance. The sections of the work sample that particularly evaluate understanding of content knowledge are the Rationale for the instructional unit, the Learning Goals, the Key Points to be learned, and Content Relations. Impact on student learning is assessed in the Analysis of Summative Assessment Data for the Class as a Whole, Analysis of Summative Assessment Data for Selected Students and Summative Interpretation sections. (Complete statistics for all work sample areas are attached). Prior to 2009, Eugene used the same rubric. Beginning in 2009 through 2013, Eugene used a different work sample rubric. It contains a three-point scale, ranging from Does Not Meet Standard to Exceeds Standard in three different domains ([context] Who are the students? and What do they already know?; [unit/lesson/assessment planning] What will students learn? and How will they demonstrate their learning?; and [assessment analysis and evaluation] To what extent have I achieved my purpose? In 2014, Eugene piloted a new instrument (Teacher Inquiry Project [TIP]) that more accurately reflected Pacific University’s values and mission statement, as well as more tightly aligned to InTASC Standards (rationale discussed at the end of this report). The TIP contains a 5 domain (Context, Planning and Delivery of Instruction, Assessment, Analysis of Data on Learning Gains, and Interpretation and Reflection) and 27 item rubric on a 0 (No Evidence) to 3 (Exceeds) scale. Unlike the alignment mapping the COE underwent with the Teacher Summary Evaluation, The COE considered there to be a stronger relationship between newly adopted InTASC Standards and each work sample rubric. Thus, analysis proceeded according to InTASC Standards 1-10. 100% of our program completers met the standard. Used in conjunction with the TSE, those who did not either revised components to meet the standard, moved to a non-licensure track, or were counseled out of the program.

1.2.a Standard on which the unit is moving to the target level

  • Describe areas of the standard at which the unit is currently performing at the target level for each element of the standard.
  • Summarize activities and their impact on candidate performance and program quality that have led to target level performance.
  • Discuss plans and timelines for attaining and/or sustaining target level performance as articulated in this standard.
Candidates in our programs are expected to demonstrate in-depth content knowledge as described by a variety of standards. We have established four different transition points in our programs when we measure candidate knowledge, skills, dispositions, and impact on P-12 student learning.

Transition Point One: Admission and Content Knowledge
Incoming GPA: Our first stage assesses the quality of content knowledge of our incoming candidates. A minimum GPA of 2.75 is required for admission. Overall cumulative GPAs collected from a total of 118 candidates averaged 3.31. 11 students did not meet the initial GPA requirement across years, but were conditionally admitted, as other data sources were exceptional (e.g., interview scores, personal essay, timed writing exercise, test scores, etc.). These candidates' progress was monitored each term for satisfactory progress.

Interview Scores: As part of the admissions process, all candidates are personally interviewed by faculty. There are seven questions on a 1-5 scale. Considering the entry purpose of the interview, program data are reported in total interview scores. The overall mean (n = 622) was 4.18. Question responses ranged from 1.50 (q. 4 [peer group interaction]) to 5.00 (all questions). By program, overall interview scores ranged from 2.36 (MAT: FG: 2010) to 5.00 (across all programs and years). As the SPED interview process is different (with different scales per element [10 for Screening, 10 for Interview, and 5 for Timed Writing), data are reported separately. SPED scores ranged from 6.00 (Screening/Interview) and 2.00 (Timed Writing) to the maximum across all elements and years.

Basic Skills Tests: An additional indicator requirement assessing appropriate content knowledge of our incoming candidates is a passing score on basic skills test. These tests include (passing score/scale) CBEST (37/80), PPST (Reading 174/190, Math 175/190, Writing 171/190), EAS (240/300), or West-B (240/300); ORELA MSEI/II (240/300), MSAT (155/200); and from 2010-2012 the ORELA Protecting Student and Civil Rights in the Educational Environment (240/300) exams. Due to annual changes in testing requirements, e.g., norming, test types, cut scores, etc., data are reported by test type. The overall pass rate across tests was 100% for candidates applying for licensure. Entry tests were waived for one candidate, because s/he already possessed a Masters or Ph.D. prior to admission.

Transition Point Two: Completion of Foundational Coursework
As discussed in standard 1.1, completion of foundational coursework is verified. Candidates must maintain a cumulative GPA of 3.0, with no more than one "C" on her/his transcript. Though data on candidates who did meet this criterion across years is equivocal, the progress of these candidates was monitored each term to ensure satisfactory progress.

Transition Point Three: Methods Block Completion and Entry into Student Teaching
Subject Area Tests: Our third transition point evaluates candidate content knowledge with passing scores on subject tests required for licensure. Similar to the entry tests reported above, due to the changes in scales, numbers of tests required, passing rates, etc. data are reported in frequencies by subject area (N = 308) and means are reported by subject test (N = 163). 100% of program completers passed subject area tests.

Transition Point Four: Program Completion
Teacher Summary Evaluation I/II: Our fourth stage assesses the quality of content knowledge, skills and dispositions of our candidates during their practicum/student teaching placements. Candidates are rated by both the supervisor and mentor teacher twice throughout their in-situ experiences. Our Teacher Summary evaluations (TSE) contain 48 questions total across five domains. Candidates are rated on a scale of 1-6 with a minimum score of 4 required to pass. The Alignment Matrix displays TSE items mapped onto the 10 InTASC standards. Aggregate mean TSE scores across years across standards ranged from 2.00 (Learner Development, Assessment, Planning for Instruction, and Instructional Strategies: 2012) to 6.00 (all standards and years). All 10 standards yielded means above the threshold across programs, campuses, and years. Those candidates not meeting minimum scores were either required to repeat a placement or were not licensed at that authorization level.

Work Sample I/II: The Work Sample (WS) is an additional indicator of candidate content knowledge, teaching skill, dispositions, and impact on P-12 knowledge that is completed during candidate practicum/student teaching placements. Candidates are required to submit and pass two WSs. While constructs and questions have remained similar in content and format between Eugene and Forest Grove campuses over years (indicating shared values), administration, scoring guide, and passing thresholds differed between the sites. Over years reported here, Forest Grove campus has used the same scales and questions (1-5 scale; 3 passing). The Eugene campus has modified its WS to reflect a more concise assessment of valued constructs beginning in 2009 through 2013 (1-3 scale; 2 passing). With respect to impact on P-12 student learning, work sample data are reported by InTASC Standard #6 (Assessment). Scores on this standard ranged from 2.89 (FG WS: 2014), 2.00 (EUG WS: 2013 and 2014), and 1.00 (EUG TIP: 2014) to the maximum (6:FG, 3:EUG WS and TIP) across programs and years. As another quality check in the program, those candidates not successfully completing all elements of a work sample were either required to redo a placement, or were successfully counseled out of the licensure program.

Teacher Inquiry Project (TIP) Rationale

Throughout their time in our program and across the four transition points, candidates demonstrate their knowledge through a range of applications, indicating an ability to use inquiry, critically analyze, synthesize and apply this knowledge in their teaching with students.

The key to its application with students is the pedagogical content knowledge that draws on candidate's understanding of principles of learning, pedagogical techniques, and the specific requirements of their discipline (or in the case of multiple subjects authorizations, each discipline). It is the synthesis of all these factors into pedagogical content knowledge that must be evinced not only in the work samples, but also in the teaching observed by mentor and supervisor. These lessons are evaluated in part on the authenticity of student experience and therefore student engagement in learning. TPaCK or Technological Pedagogical Content Knowledge is also an important aspect of this teaching and is covered in our coursework as well as a special technology conference that Pacific organizes and that all candidates are expected to attend (Oregon Technology in Education Conference).

As professional and state standards and now the common core state standards are at the heart of faculty instruction and candidate lesson planning, lessons are naturally structured around these standards. The critical dimensions of universal design for learning are reflected in the flexibility candidates exhibit in considering family and community contexts.

The fundamental purpose of education is not in teaching, but in supporting student learning. This is the focus of our programs and the evidence is always to be found in feedback on formative and summative assessments as well as other indicators of engagement and learning. Therefore methods of assessment including a wide range of standardized and student-centered tools are reviewed with candidates and required to be included in their work samples (our authentic assessment tool).

We begin all our programs with a focus on our candidates as a community of learners. Throughout our time together we work to provide an atmosphere of collegiality and communal support. We believe that this is one means of instilling in our candidates the disposition of the ethic of caring for each other in what is an eminently social profession. During the first month of the program and at each transition point thereafter, candidate dispositions are assessed by faculty, later by supervisors and mentor teachers. We require all candidates to also reflect on their own dispositions and to be proactive in working to affect positive professional change.

Candidate knowledge, skills and professional dispositions are assessed at the four points detailed above, as well as at other points when necessary. In faculty meetings and through coordinator meetings special attention is paid to any candidate who may be struggling with personal or professional issues, or who is having difficulty with content. We pride ourselves that all students recommended for licensure by the COE have exceeded these minimum standards.

For many years, Pacific's College of Education has been regionally recognized for providing quality teachers. As the educational environment continues to change, and as our own understandings and abilities within the COE increase, it is critical to undertake some important revisions to the program that we provide. These revisions are coming about because the College and its faculty continue to develop our own knowledge, skills, and understandings about the educational landscape, and our belief in educating future teachers in a way that keeps them at the forefront of high teacher quality in this state.

In redesigning the MAT program, we carefully considered many factors that build upon our traditions, make use of faculty expertise, provide more program flexibility, build more program coherence, and better prepare candidates to work with students from various backgrounds and with school personnel serving in different roles. These considerations are described in this section.

We first recognize the strength of Learning Communities as the backbone of the MAT program for over a decade. This sequence of courses focuses upon identity development, diversity and equity, and examination of problems of practice during student teaching; the committee sought to preserve these vital elements. This is done in the Inquiry Sequence by teaching candidates to focus on themselves as positioned individuals learning to teach students who will not share similar backgrounds with them, and by having them focus on cultural identity markers as a critical entrée into working with others. Indeed, central questions within the Inquiry Sequence ask students to examine their own schooling experience, the ways in which their identities are/were supported or disregarded within schools, having them pay attention to their interactions with students, and focusing on student learning, as opposed to teacher performance.

A second component of this MAT program is the Sociological Teaching Core. It is within this course that students will undertake the study of cultural identity, including their own, along the major sociological identity markers, and the history and structure of schooling. In other words, the Core will continue vital aspects of Learning Communities and the outgoing course Schools and Society. The Core will expand the work of Learning Communities by calling specific attention to two populations, which the MAT program has largely neglected: special education students and English language learners. Furthermore, a key feature of the Core is involving multiple faculty members to share their expertise on specific topics through modules. This will help to convey to candidates that as a faculty, we work together in this community to educate our students and each other, that not everyone knows everything about everybody, and that knowledge is best ascertained through multiple voices and perspectives.

A third component of Learning Communities that will exist in this MAT program is the Student Teaching Seminar, a course that allows space for candidates to process and build stronger understanding of their work as teachers, the ability to revisit topics, concepts, and experiences from earlier in the program, and to strengthen the practice of professionals working together as a critical component of improving one's teaching skill and ability. This component will also be enhanced by the third course in the Inquiry Sequence. Thus, as we move to revise our curriculum, we carry forward the Learning Communities content and affect into specific courses, and its spirit into the broader program.

We also considered the value of action research completed during student teaching as a powerful experience for pre-service teachers, particularly useful in supporting pre-service teachers in learning about, negotiating, and reinterpreting the often conflicting discourses they encounter during student teaching, creating a transitional space (Phillips, 2007). Pre-service teacher action research projects are carried out while the student teacher is positioned simultaneously as "student," "teacher," and "researcher." These often-conflicting roles are situated in and among places of power, representing spaces of "formation and transformation, a scrutiny into what one is doing, and who one can become" (Britzman, 2003, p. 31). For the pre-service teacher, the authoritative and competing discourses of the school site, mentor teacher, university, and prior experiences in learning all vie simultaneously to encourage, dissuade, empower, and dictate. Each discourse portrays itself as normative, clearly defining the "good teacher;" other competing discourses resist, creating spaces of tensions where one's identities are constantly under construction and reconstruction. Such spaces of tension are places of potential changes, influencing our behavior as moral agents (Foucault, 1984; McLaren, 2002). By virtue of engaging candidates in work directly related to their practice, action research projects may serve as powerful transitional spaces (Ellsworth, 2005) in student teaching, scaffolding the process of teacher identity formation in a positive way.

As envisioned by the committee, the Inquiry Project will occur in conjunction with a unit of study taught in placements. The Inquiry Project will, in effect, take the place of the current work sample so as not to add an additional major project to the program, but to reconfigure the existing project into one that more strongly embodies principles associated with inquiring into one's own practice. The Inquiry Project will be embedded in the Inquiry Sequence, so as to scaffold candidate understanding of the process, the use of disparate knowledge and concepts from throughout the program, and the value and practice of reflection.

This redesign is being structured to provide even more opportunity for candidates to hone their knowledge, skills and professional dispositions.


1.4 Exhibits for Standard 1*

*All documents are available in pdf format.