|
Criteria for
Assessment Plans
Guidelines for Institutions and
Evaluators
The assessment of
institutional effectiveness at a college or university, a review which
includes student learning,
examines both purpose and process issues in assessment.
Two statements about
outcomes assessment most clearly express what the Middle
States Commission on
Higher Education expects institutions to include in their assessment plans.
One statement appears in Framework for Outcomes Assessment (1996), the
Commission's guidelines which put the purposes of assessment in the
contemporary context of institutions of higher education. The other appears
in Characteristics of Excellence in Higher Education (1994), the Commission's
primary statement of standards for accreditation, which defines the process
of assessment.
While the
fundamental purpose of outcomes assessment is educational improvement...
[c]olleges and universities now face more intense pressures to demonstrate
their accountability, effectiveness, and efficiency [Framework, p. 4].
Outcomes
assessment involves gathering and evaluating both quantitative and
qualitative data which demonstrate congruence between the institution's mission,
goals, and objectives and the actual outcomes of its educational programs and
activities [Characteristics, p. 17].
These two basic documents
also identify at least five purpose and process components of
assessment programs that
lead successfully to institutional improvement and self-renewal.
Each institution's
self-study or periodic review report might consider some of the questions
listed under the criterion which are offered here as suggestions for institutions
developing self-study or periodic review reports and for evaluators or
reviewers of those reports:
 A Foundation in the
Institution's Mission,
Goals, and Objectives
The
institution's mission, goals, and objectives are the starting and ending points
of
an effective assessment plan. They link the broad purposes for which the
institution exists to the knowledge base and specific skills the institution
seeks to develop in its students.
Evaluators
might wish to include the following questions in their review of an
institution and its self-study or periodic review report:
Has
the institution linked the measures and strategies utilized in its assessment
program to the institution's mission and to its goals and objectives at both
the program and course levels-for general education and for undergraduate and
graduate studies in the major-and the goals and objectives for its services
to the campus community?
Do
the measures address a seamless chain, from the institution's mission to the
program goals and objectives?
What
system exists, what personnel are involved in ensuring relevance, and how
effectively do the components work together (i.e. to reconcile variances; to
add or eliminate measures and strategies; or to revise the mission, goals,
and objectives)?
The Support and Collaboration
of Faculty and Administration
Assessment is a joint
effort that requires genuine support and collaboration among
and between faculty and
administration. Faculty are expected to be the primary sources for guidance
on tile scope and depth of an assessment program, including the measures and
strategies best utilized for their curriculum areas. They also are expected
to be the leading participants in implementing the program and evaluating its
findings about student achievement. The
administration, on the other hand, is expected to provide technical and operational support –
including external consultants, materials, and services relating to
assessment activities.
They also are expected to
coordinate the development of a coherent program of supportive services for
students, faculty, and staff; the evaluation of the assessment program; and
the development of an institution-wide consensus on overall institutional
effectiveness.
Evaluators may ask:
What
is the evidence that both faculty and administrators participated in: ( 1)
the development of the institution's assessment plan; and (2) the
implementation of the plan?
Were
their respective leading and supporting roles clearly delineated, understood,
and agreed upon in advance of developing the plan, and to what extent did
each actually contribute to the effort as the planning and implementation
evolved?
Can
the relationship between faculty and administration on assessment issues be
fairly characterized as supportive and collaborative? If not, in what areas
might the institution strive for improvement?
Did
the faculty, administration, and governing board demonstrate skill in raising
questions about institutional effectiveness and in seeking answers?
A
Systematic and Thorough Use of
Quantitative and Qualitative
Measures
The institution has the
prerogative of selecting the particular combination of assessment instruments
and techniques it will utilize. However, the Commission expects that the
program will be distinguished by:
1.
the skill with which faculty, administration, and governing boards
"raise questions about institutional effectiveness, seek answers, and
significantly improve procedures in light of their findings"
[Characteristics, p. 16];
2.
quantitative and qualitative assessment approaches that examine the extent
and quality of student learning, effective teaching, the role of campus
climate, and where applicable, effectiveness in research and scholarship; and
that these approaches be applied
systematically and thoroughly; and
3.
the utilization of existing data and information at the institution, in
addition to gathering any new data-a process that may lead to decisions about
purpose and process that improve data gathering across the institution.
Evaluators may ask:
In
what manner and by what process does the assessment program examine:
(1)
the extent and quality of
student learning;
(2)
the effectiveness of
teaching;
(3)
the role of campus climate in the teaching and learning experience; and,
(4)
where applicable, the research and scholarship of the institution's
personnel?
Has
the institution selected an appropriate balance of the quantitative and
qualitative instruments and strategies that may apply to each particular
measurement task?
Is
the choice of measures consistent with current research, theory, and trends
in assessment?
What
evidence is there that the measures have been applied systematically and
thoroughly?
Assessment and Evaluative
Approaches
That Lead to
Improvement
Inherent
in the selection, use, and analysis of assessment measures is the assumption
that there is a teaching/learning/assessment/improvement feedback loop. That
is to say, teaching (including its institutional context) influences
learning, and outcomes assessment leads to improvements in teaching and
overall institutional effectiveness [Framework, p. 11].
Evaluators
may ask:
What
evidence is there that cumulative data from assessment contributed to student
achievement, institutional research, and planning and resource allocation?
For example:
1.
Has the institution examined how data from the assessment of students from
different cultural/ethnic backgrounds is interpreted and utilized, especially
the impact of the data on the academic choices that those students make and
on institutional decisions about programs, services, and teaching?
2.
What process was utilized to evaluate the findings, to achieve consensus on
the meaning of the findings and their applicability to the institution, and
to improve the institution's curricular and administrative decisions?
3.
How have the institution's assessment data led to changes in public policies
that subsequently have had a positive or negative impact on improvement at
the institution?
Are
there external or systemic internal barriers affecting the institution that
might inhibit the development of workable assessment initiatives or their
successful implementation?
To
what extent does the institution conduct its own assessments utilizing its
own personnel, or rely on the use of external consultants for primary
support, or participate in assessments conducted by others?
Realistic Goals and a Timetable,
Supported by Appropriate
Investment
Institutions
should design assessment initiatives that are realistic in scope, can be
accomplished within an appropriate timetable, and are consistent with the
institution's financial and other resources, in order to ensure that the
assessment program can be accomplished [Framework, pp. 30-31].
Evaluators
may ask:
Are
the goals of an institution's outcomes assessment realistic, given the
financial, human, or other resources of the institution that currently exist
or may be reasonably acquired?
Has
there been a good faith effort to implement the realistic strategies that
were outlined?
Was
the timetable adequate for implementation?
Do
the faculty, administration, and governing board invest appropriately in
assessment by allocating sufficient resources to accomplish its goals and
objectives?
Are
long-term plans being made to provide additional resources for any capital
improvements that assessment findings have indicated would be necessary to
improve teaching, learning, and overall institutional effectiveness.
An Evaluation of the Assessment
Program
Institutions
should evaluate the institution's assessment initiatives as a whole, in the
context of the preceding five criteria.
Evaluators
may ask:
To
what extent does the evaluation of assessment initiatives address the five
criteria for outcomes assessment: the relevance of the assessment program to
mission, goals, and objectives; the effectiveness of support and
collaboration; the measures selected; the use of findings for improvement;
and the realism of the assessment goals, timetable, and resources?
What
process is utilized to evaluate the assessment initiatives?
By
whom is the evaluation conducted, and who participates in providing the data
and other information that is utilized?
Over
what time period and with what frequency does the evaluation occur?
How
are the findings from assessment utilized to refine or to restructure the
assessment initiatives?
With
whom are assessment data or findings shared and for what purposes?
|
