Part III: Evidence for Meeting Each Standard

Standard Two: Assessment System and Unit Evaluation

Pacific University has a comprehensive assessment system that collects and analyzes data on applicant qualifications, candidate and graduate performance, and unit operations. The processes and results are reviewed annually, and refined as needed. The description of the assessment system provided in this section represents current implementation as well as future plans for assessment at Pacific University. The evidence documented in this section will support the claim that Pacific University meets Standard Two.

Element One: The Assessment System

The assessment system was developed with input from the professional community through joint faculty and committee meetings with input and review from our Consortium (with representatives from the P-12 community, students, alumni, and COE faculty), and reflects both the conceptual framework and the candidate proficiencies outlined in professional and state standards. The College of Education uses this information to evaluate and improve the unit and its programs. The assessment system includes a comprehensive and integrated set of evaluation measures – closely aligned with our Conceptual Framework -- that are used to monitor candidate performance and improve operations and programs. Decisions about candidate performance are based on multiple assessments made at admission, at appropriate transition points, and at program completion. These assessments are regularly reviewed to assure they are accurate predictors of candidate success. Assessment processes and results are reviewed annually, and effective steps are taken to eliminate bias in assessments and to establish fairness, accuracy and consistency in assessment.

The knowledge, skills, and dispositions that form the framework for the Candidate Assessment System are derived from the Conceptual Framework and state and professional standards. Based on accepted standards and knowledge from educational research, these competencies were determined in collaboration with the professional community – through full faculty discussions, committee work, input from the Consortium and feedback from administrators and teachers in the broader community. Candidate competence on each outcome is assessed at multiple points, in both a formative and summative manner. In order to ensure that the assessment instruments are fair, accurate, and consistent they are reviewed periodically by the COE Curriculum Committee; the aggregate results are reviewed annually by the Consortium and the COE faculty at the August faculty retreat. The assessment system assures that candidates are evaluated in terms of the state TSPC competencies, national INTASC standards, the College of Education additional competencies, and our Conceptual Framework themes.

The process we have used to seek national accreditation has pushed us to develop rigorous assessment instruments and a complete assessment system. Thus, data from candidate assessments, candidate review of their programs, feedback from alumni, and feedback from employers are gathered annually and used for assessing changes in the Initial Teaching License and Advanced Program curricula. Because we are new in this process the early feedback has been used to refine the instruments more than the curriculum itself. To begin the process of curriculum review, we examined the early results of this data at our faculty retreat in August 2007. We will be using the robust data gathered this year at an assessment summit next August at which time we will consider curriculum and program changes that might be as revealed by the data.

All COE assessments from program admission to post-graduation are aligned with the appropriate standards that facilitate the acquisition, application, and implementation of appropriate knowledge, skills, and dispositions. The COE emphasizes the importance of candidate performance on students' learning (as measured in the work sample and professionally assessed by the mentor teacher and university supervisor) and incorporates the assessment of these results into the system.

Initial Teaching License Candidate Assessments

Multiple means are used to assist candidates throughout their programs. The following table (Table 2.1.1) describes the alignment between the desired Candidates Outcome, the Conceptual Framework, and the means used to assess the knowledge, skills and dispositions. A list of key assessment instruments includes:

Work Sample Evaluationpdf (WS)
Student Teaching Midplacement Evaluationpdf (ST-M)
Student Teaching Summary Evaluationpdf (ST-S)
Dispositions Faculty Evaluationpdf (on all candidates following each course)
Candidate Disposition Self-Assessmentpdf
Student Knowledge and Skills Self-Evaluationpdf (K&S) (Special Education only)
Field Experience Evaluation by Mentor Teacherpdf

Table 2.1.1
Candidate Assessment and Evaluation

 Candidate Outcomes Knowledge/Skills/
Dispositions
Assessment and Evaluation Instruments Used
Transforming Education through Communities of Learners
Candidates are reflective practitioners. Skill
WS Evaluation
Candidates are leaders and agents of change. Skill
ST-M Evaluation
ST-S Evaluation
Candidates contribute to the learning community. Disposition
Disposition Self-Evaluation
Faculty Evaluation of Dispositions
Field Experience Evaluation by Mentor Teacher
Candidates are lifelong learners. Disposition
Disposition Self-Evaluation
Faculty Evaluation of Dispositions
K & S Self-Evaluation (SpEd)
Candidates have inquiring minds. Disposition
Disposition Self-Evaluation
Faculty Evaluation of Dispositions
Candidates are confident, energetic and healthy. Disposition
Disposition Self-Evaluation
Faculty Evaluation of Dispositions
Promoting Cultural Competence
Candidates are working toward cultural competence. Disposition
Disposition Self-Evaluation
Faculty Evaluation of Dispositions
K & S Self-Evaluation (SpEd)
Field Experience Evaluation by Mentor Teacher
Candidates believe all students can learn. Disposition
Interview
Disposition Self-Evaluation
Faculty Evaluation of Dispositions
Candidates interact constructively. Skill
ST-M Evaluation
ST-S Evaluation
Candidates are data-driven decision-makers. Skill
ST-M Evaluation
ST-S Evaluation
WS Evaluation
Creating Student-Centered Learning Environments
Candidates have both content knowledge and pedagogical content knowledge. Skill
ST-M Evaluation
ST-S Evaluation
WS Evaluation
Field Experience Evaluation by Mentor Teacher
K & S Self-Evaluation (SpEd)
Candidates promote critical thinking. Skill
WS Evaluation
Candidates promote self-regulated learning. Skill
ST-M Evaluation
ST-S Evaluation
Candidates are effective instructors. Skill
ST Evaluation
WS Evaluation
K & S Self-Evaluation
Technology Tools for Learning
Candidates use instructional technology effectively. Knowledge
ST-M Evaluation
ST-S Evaluation
WS Evaluation
K & S Self-Evaluation (SpEd)
Candidates are technology-literate. Skill
Course projects
WS Evaluation
K & S Self-Evaluation (SpEd)

At the end of their programs students are surveyed to determine how well they believe they are being prepared. The results of the survey are shared with the COE’s joint faculty and with the Consortium, leading to recommendations for curricular changes if needed. For example, comments from the program completion surveys from both campuses have resulted in strengthening the classroom management portion of the Initial Teaching License curriculum (MAT 5th, MAT Flex, undergraduate).

Table 2.1.2
Overview of Candidate Assessment Plan – Initial Teaching License

Transition Points
Knowledge and Skills
 Dispositions
Admissions
(Assessments prior to admission)
Transcript Review
Essays
Letters of Recommendation
Basic Skills Test
Interview Evaluation
Writing Samples
Subject Matter PRAXIS Tests
Candidate Dispositions Self-Evaluation (at Orientation)
Interview evaluation
Essays
Writing Samples
Foundations Block
(During/end of 1st semester)
Have passed or registered for  state-mandated standardized test
Successful Completion of Foundational Coursework
Candidate Dispositions Evaluation by Faculty
Methods Block
(During/end of 2nd semester)
Successful Completion of Methods Block coursework
Field Experience Evaluation by Mentor Teacher
Completed/scored Modified Work Sample
Additional/optional Subject Matter PRAXIS Tests
Candidate Dispositions Evaluation by Faculty
Observation Journal and Reflections
Field Experience Evaluation by Mentor Teacher
Entry/During/Exit from Student Teaching
(3rd  semester)
Observation Journal and Reflection
Completed Work Samples (2) evaluated by University Supervisor
Midplacement (formative) Student Teaching Evaluation (by Candidate, Mentor Teacher, and University Supervisor)
Summary Student Teaching Evaluation of Student Teaching (by both University Supervisor and Mentor Teachers)
Midplacement (formative) Student Teaching Evaluation (by Candidate, Mentor Teacher, and University Supervisor)
Summary Student Teaching Evaluation of Student Teaching (by both University Supervisor and Mentor Teachers)
Candidate Dispositions Evaluation by Faculty
Program Completion
Successful completion of student teaching
Final Presentation
Audit and Verification of meeting all licensure requirements:
  • Evidence of Bachelor’s degree from a regionally accredited college
  • Basic skills test passing score
  • Fingerprint clearance
  • Completed  TSPC-approved program
  • Met requirements for two work samples
  • Met requirements for student teaching
  • Content knowledge test(s) passing scores
  • Show evidence of First Aid card
Final Presentation

Table 2.1.3
Overview of Candidate Assessment Plan – Advanced Programs

Transition Points
Knowledge and Skills
Dispositions
Admissions
(Assessments prior to admission)
Transcript Review
Essays
Letters of Recommendation
Initial Teaching License
Interview evaluation
Essays
Dispositions Self-Evaluation
During Program
Completion of required coursework for MEd, CTL, endorsement, certificate, or authorization
Minimum GPA; no C- or lower
Candidate Dispositions Evaluation by Faculty
Within field experiences as required
Program Completion
Completed Work Samples
Research Project
Portfolio completion
Final Presentation
Portfolio

Table 2.1.4
Overview of Candidate Assessment Plan – Initial Teaching License
Special Education

Transition Points
Knowledge and Skills
Dispositions
Admissions
(Assessments prior to admission)
Transcript Review
Resume
TSPC Character Questions
Essay
Letters of Recommendation
Basic Skills Test
Interview Evaluation
Writing Sample
Candidate Dispositions Self-Evaluation
Interview evaluation
Essay
Writing Sample
Initial Coursework
(Beginning of 1st Summer)
Successful completion of beginning coursework
Self-assessment
Candidate Dispositions Evaluation by Faculty
Observation Journal and Reflections
Continuing Coursework
(End of 1st Summer)
Successful Completion of Coursework
Minimum GPA; no C- or lower
Fingerprint Clearance
Self-assessment
Candidate Dispositions Evaluation by Faculty
Anti-discriminatory Workshop
Candidate Dispositions Self-Evaluation
Clinical Practice
During/End of Student Teaching
(Academic Year)
Successful Completion of Coursework
Minimum GPA; no C- or lower
Formative Evaluations by Mentor Teacher
Formal Evaluations by University Supervisor
Formal Evaluations by School/District Administrator(s)
Completed Required Work Sample(s) evaluated by University Supervisor
Have passed or registered for state-mandated standardized test (ORELA Multiple Subjects)
Have passed or registered for Subject Matter PRAXIS Test
Midplacement (formative) Student Teaching Evaluation (by Candidate, Mentor Teacher, and University Supervisor)
Summary Student Teaching Evaluation of Student Teaching (by University Supervisor and Mentor Teacher)
Self-assessment
Candidate Dispositions Evaluation by Faculty
Work Sample Reflections
Formative Evaluations by Mentor Teacher, University Supervisor, and School/District Administrator(s)
Midplacement (formative) Student Teaching Evaluation (by Candidate, Mentor Teacher, and University Supervisor)
Summary Student Teaching Evaluation of Student Teaching (by University Supervisor and Mentor Teacher)
Final Coursework
Program Completion
(End of 2nd Summer)
Successful Completion of Coursework
Minimum GPA; no C- or lower
Self-assessment
Audit and Verification of meeting all licensure requirements:
  • Evidence of Bachelor’s degree from a regionally accredited college
  • Basic skills test passing score
  • Show Evidence of Valid First Aid Card
  • Completed TSPC-approved program
  • Met requirements for work sample(s)
  • Met requirements for student teaching/practicum
  • Content knowledge test passing score
Candidate Dispositions Evaluation by Faculty

Master Calendar for Assessments

Our Assessment Coordinator has the responsibility for maintaining the Master Assessment Calendarpdf, which identifies the points in each program when all assessments, surveys, and course evaluations are scheduled, as well as candidates’ opportunities to take Praxis exams.

Data in the Assessment System

Our admissions database contains all relevant data from the application and admissions process (prior GPA, test scores, rating on writing samples and on the interview), as well as demographic information. This data is entered during the admissions phase by the COE Admissions Counselor. The data is then transferred to the Assessment Coordinator who manages the input of additional data as candidates proceed through the program. This relational database is maintained on an Excel spreadsheet for each program, and allows us to generate reports that correlate, for example, course grades with prior GPAs, or interview scores with cultural competence items from the work sample. Thus far the reports have indicated 1) that we are very effective in our selection process, since 99% of candidates are retained in the programs; and 2) that the interview process along with the writing sample protocol is highly effective in determining the candidates who will be successful completers of our programs.

A very small number of candidates each year are not successful. The most common deficiencies are either due to poor grades or dispositions issues. A candidate who receives a C- or lower in any course may be given the opportunity to repeat the course, or may be counseled out of the teacher preparation program. We have become much more astute at identifying and documenting dispositions issues early so that they can be dealt with before they become a serious problem. (See Dispositions Faculty Evaluationpdf and Disposition Procedurespdf.)

When a graduate student who has completed more than half their teacher education program is considered unlikely to be able to complete the standards for an Oregon Initial Teaching License, they are ordinarily encouraged to pursue the Master of Arts in Education/Curriculum Studies (MAE/CS) degree, since the courses they have already completed can be applied to it. The MAE/CS allows students to become competent in the field of education without earning an Oregon teaching license. As a planned program, this degree is often earned by foreign students planning to return to their native country, or by community college instructors needing a Master’s degree.

Admissions Procedures – Initial Teaching License

To be admitted into a program leading to an Initial Teaching License, candidates must pass one of the basic skills tests: CBEST, PPST, or CBT. In addition, before student teaching, they must pass the appropriate subject area tests required for authorizations and/or endorsements.

Requirements for admission to an MAT Initial Teaching License program include:

  1. A baccalaureate degree from an accredited college or university.
  2. A 2.75 minimum GPA in undergraduate work or 3.00 minimum GPA in at least 8 semester hours of graduate study.
  3. Completed application
  4. Completed checklist of TSPC character questions
  5. Résumé
  6. Basic skills test passing score
  7. Official transcripts documenting coursework from each college or university attended.
  8. Three letters of recommendation
  9. Ability to write clearly and cogently as demonstrated in a brief essay on a current educational issue.

(For minor differences in the admissions procedure for the undergraduate program, see the Undergraduate Catalog.)

Admission to Advanced Programs

Teachers who have completed the Basic or Initial License programs at Pacific University are automatically accepted into the program. Those teachers must submit the following prior to being assigned an advisor:

  1. Letter of intent
  2. Current resume that includes professional work history
  3. Official transcripts from any college or university where coursework has been completed since completing the Pacific University teacher education program
  4. Copy of Oregon Basic or Initial License

Teachers who have completed licensure programs at other universities must apply to the program. Applications are accepted throughout the year. Admission is selective. Requirements for admission include:

  1. Official transcripts from each college or university attended
  2. 2.75 minimum GPA in undergraduate work or 3.0 minimum GPA in at least 8 hours of graduate study
  3. Three recommendations from school administrators, evaluators, supervisors, or peer teachers addressing the applicant’s professional teaching abilities, relationships with students and staff, and attitudes about improvement of teaching
  4. Current resume that includes professional work history
  5. Completed application
  6. Completed checklist of TSPC character questions
  7. Essay addressing an aspect of teaching on which the applicant wishes to concentrate in a professional development program
  8. Copy of Oregon Basic or Initial License
  9. Candidate Interview

Scoring Guides Used to Assess Candidate Performance

The key scoring guides used in the COE programs are the following:

Work Sample Scoring Guide
Student Teaching Midplacement Evaluation
Student Teaching Summary Evaluation
Dispositions Form and Dispositions Procedure
Advanced Programs Portfolio Assessment Scoring Guide

Elimination of Bias and Assurance of Fairness, Consistency, and Reliability

Every effort is made to be certain that assessments are fair, consistent, reliable, and unbiased. The state-mandated subject-area tests (Praxis II, ORELA) required for each endorsement are standardized and objective. TSPC periodically reviews all required tests to determine issues of fairness and applicability to state content standards. For candidates who have difficulty passing the required test, TSPC has alternative processes in place. However, if a candidate has not passed the required tests prior to student teaching, they must stop out of the program until they are able to do so.

Throughout our coursework, classes use a wide variety of instructional and assessment measures, including authentic performance tasks, group assignments, technology-based projects, etc. (See Preferred Assessment Methods Faculty Surveypdf) Both university supervisors (full-time and part-time) and mentor teachers have several opportunities each year to attend training sessions at which the development and assessment of the work samples and of student teachers are a major focus. These sessions have ensured consistency in the assessment process as well as promoted communication among all faculty members, especially those new to the College of Education. (See Training Session Agendas.)

Regular Review of Program and Unit

Data for program assessment purposes is formally collected annually from candidates, mentors, principals, and alumni.

Table 2.1.5
Program Assessment Ratings by Candidates, Alumni, Mentor Teachers, Principals and Employers
(1 = Strongly Disagree; 5 = Strongly Agree)

Program Assessment Item

The Teacher Education Program promoted growth in the candidates’ ability to:

Candidates
Mentor Teachers
Alumni
Principals/ Employers
FG
N=108
Eug
N=71
FG
N=46
Eug
N=60
FG
N=40
Eug
N=14
FG
N=40
Eug
N=25
RR*= 54%
RR= 50%
RR= 13%
RR= 25%
1. Develop a community of learners.
4.42
4.27
4.37
4.31
4.20
4.29
4.35
4.53
2. Increase awareness and knowledge in cultural competency.
4.10
4.16
4.11
4.33
4.03
4.21
4.24
4.20
3. Construct understanding of pedagogical content knowledge and skills.
4.25
4.97
4.36
4.22
3.93
4.00
4.23
4.60
4. Create a student-centered learning environment in the classroom.
4.10
4.21
4.39
4.26
4.10
4.29
4.37
4.64
5. Use technology to enhance student learning.
4.17
4.01
4.39
4.26
4.13
3.79
4.32
4.29
6. Develop skills for utilizing a variety of instructional strategies.
4.38
4.41
4.28
4.29
4.20
4.21
4.19
4.37
7. Accept and nurture a range of learning styles and intelligence.
4.29
4.33
4.11
4.33
4.13
4.21
4.22
4.36
The Teacher Education Program provides clear expectations and instructions.
4.00
3.97
4.33
4.26
4.05
4.00
4.13
4.43
The Teacher Education Program staff and faculty are organized and well-prepared.
4.26
4.10
4.33
4.47
4.10
4.21
4.14
4.54

*RR = Response Rate; for detail, see Response Rate file.

As the results indicate, all of the stakeholders determined that the Initial Teaching License programs promoted growth in the ability of teacher candidates to meet the standards for the conceptual framework and the license. In fact, all stakeholders rated each category in a range from agree (3.5-4.4) to strongly agree (4.5-5). Even then, we know that we have room to improve in clearly communicating what it takes to be a successful completer of our programs and more importantly in helping our candidates be highly successful in meeting the needs of all learners.

Table 2.1.6
Program Assessment Ratings by Advanced Program Candidates and Alumni
(1 = Strongly Disagree; 5 = Strongly Agree)

Program Assessment Item

The Teacher Education Program promoted growth in the candidates’ ability to:

Advanced Program Candidates
Advanced Program Alumni
N = 32
N = 19
1. Develop a community of learners.
3.81
3.84
2. Increase awareness and knowledge in cultural competency.
4.00
4.05
3. Construct understanding of pedagogical content knowledge and skills.
3.23
3.63
4. Create a student-centered learning environment in the classroom.
3.90
4.21
5. Use technology to enhance student learning.
3.91
4.26
6. Develop skills for utilizing a variety of instructional strategies.
3.41
4.11
7. Accept and nurture a range of learning styles and intelligence.
3.88
4.37
The Teacher Education Program provides clear expectations and instructions.
3.55
4.16
The Teacher Education Program staff and faculty are organized and well-prepared.
3.81
4.00

Comprehensive and Integrated Set of Evaluation Measures

Throughout this Institutional Report, the reviewer should find clear evidence of a comprehensive and integrated set of evaluation measures. Table 2.1.7 shows the relationships between the set of evaluation measures and the major use of each data set.

Table 2.1.7
Summary of Integrated Sets of Evaluation Measures and Assessment Applications

Level
Data Sets
Database
Use
Candidate Assessment
  • Course Level assessments that reflect the conceptual framework, SPA, state and national standards
  • Course grades and GPA
  • Field experience evaluations
  • Student Teaching Summary Evaluations
  • Work Sample evaluations
  • Dispositions assessments
  • External assessments (PRAXIS)
The relational database has tables of data on each course, by semester; it contains candidate admissions data, work sample data, and student teaching evaluation data. The relational nature of the database allows the production of necessary reports. Candidate assessments are used for:
  • feedback to individual candidates
  • feedback to faculty for decision making on candidate advancement
  • curriculum alignment and course modification
Program Evaluation
  • Each data set above is aggregated by program.
  • Candidate evaluations of courses, faculty, field placements, and clinical experiences are done on-line so that statistical results are available immediately for analysis.
  • Analysis of assessment tools is done annually for accuracy, consistency, fairness, and avoidance of bias.
  • Alignment of course assignments and work samples with the conceptual framework is regularly reviewed.
  • Candidate evaluation of Univ. Supervisor and Mentor Teachers is done on-line.
 
  • Disaggregated reports for each program are generated by the relational database.
  • Candidate evaluations of courses is reviewed by individual faculty members. The results of the self studies will drive course modifications and curriculum actions
  • Program evaluation data (See Table 2.1.5)
  • Disaggregated data are used to: drive program improvements.
  • Annually review and revise assessment tools and scoring criteria for accuracy, consistency, fairness, and avoidance of bias.
  • Annual faculty and Consortium review of effectiveness of curriculum and instruction in each program; revise when necessary.
  • Identify trends and areas for improvement in all programs
Unit Evaluation
  • Comprehensive analyses of program strengths and weakness, trends in learning outcomes, areas in need of improvement, using program-level evaluations: exit and post-graduation and employer surveys
  • Candidate complaints and their resolutions
  • Diversity data for students and the professional community
  • Faculty data
  • Recruitment and retention data
  • Reports from each program are aggregated
  • Program evaluation data (e.g., Table 1.5) compared by year, program, and site.
 
  • Alignment of curriculum and assessment with CF
  • Improvement of curriculum and instruction
  • Increase unit effectiveness
  • Continue predictor analyses
  • Analyze effectiveness of diversity initiatives
  • External reports