Salisbury State University

Student Learning Outcomes Assessment Report

April 2001

 

 

 


 

 

 


presented to the

 

 

 

The Board of Regents of the University System of Maryland

&

The Maryland Higher Education Commission

 

 

 

 

 


The timing of the 2001 Student Learning Outcomes Assessment Report (SLOAR) was particularly advantageous for Salisbury State University since it coincided with the development and delivery of the University’s Five-Year Periodic Review Report to the Commission on Higher Education Middle States Association of Colleges and Schools (MSA).  Primary among the topics of interest to the MSA is evidence of on-going outcomes assessment to improve institutional effectiveness, and, in particular, student learning outcomes. 

Of the USM institutions, Salisbury State University continues to graduate the greatest proportion of entering freshmen within six years, while the University’s reputation as a quality, public comprehensive university is recognized in several national publications and by our peers.  However, these are indicators of excellence and achievement that are less important than the skills and values we effect and impart to SSU students.  Salisbury State University considers essential the core values of excellence, student‑centeredness, learning, community, civic engagement, and diversity.  These values form the nucleus of our institutional identity and are the heart of our efforts to ensure excellence in academic programming, to provide comprehensive instruction and practice in content areas, to promote skills development, to foster critical thinking proficiencies, and to cultivate a respect for the value of global, societal, and individual differences.

As such, the University validates student learning outcomes through a variety of instruments, measures, and methodologies at the course, departmental, school, and institutional level.  Retention and graduation rates, which, albeit provide an indication of success, do not communicate specific outcomes.  Instead, the University considers the following measures more indicative of student learning and institutional effectiveness:

 

·       32% of SSU graduates enroll for additional course-work within one year of graduation; of these, 82% are pursuing a Master’s degree or higher

·       the total graduate school going rate within one year of graduation is fully 26%; of these, 29% are pursuing a Master’s degree in Education, while 16% are pursuing a Master’s in Social Work

·       86% are pursuing graduate study in the same or a related major as their undergraduate degree

·       98% report that Salisbury State University prepared them for graduate study

·       83% are employed full-time

·       20% of those employed full-time are employed as teachers

·       the licensure pass rate for the Respiratory Therapy Entry Level Credential Exam of SSU students was 100%

·       the licensure pass rate for the Medical Technologist Certification Exam of SSU students was 100%

 
The University conducts course assessments or processes to measure student learning in skills and content areas.  Equally significant to SSU faculty and administration are students’ assessment of those same skills one year after graduation, and, whether or not in the opinion of the graduate, the University provided satisfactory instruction.  The University gains this information through an annual survey of alumni, and, according to the 1998-1999 graduating class:

 

·       95.7% were satisfied that they were taught to utilize a broad base of knowledge in multiple applications

·       96.6% were satisfied that they were taught to speak, read, write, and listen effectively

·       98.1% were satisfied that they were taught to identify and solve problems, think critically, and reason effectively

·       91.6% were satisfied that they were taught to use information technology in multiple applications


Several of Salisbury State University’s professional programs have earned specialized accreditation, certifying that a program meets the criteria and standards of an accrediting agency.  These agencies require periodic program assessment to measure, validate, or certify quality in higher education.  They also establish rigid criteria and standards for program accreditation and validate the success of the program through multiple measures, including student learning outcomes.  These standards are stringent, often adding substantially to the overall cost of an academic program but designed to ensure academic rigor.  Salisbury State University has earned specialized accreditation in the following programs:  Accounting, Athletic Training, Business Administration, Chemistry, Economics, Elementary Education, Environmental Health, Management, Medical Technology, Nursing, Physical Education, Respiratory Therapy, and Social Work.  Each program is able to validate in detail the success of the curriculum and student learning to the appropriate accrediting agency and to the University.

As evidenced, Salisbury State University conducts student learning outcomes assessment at both the program and the institutional level.  Throughout the academic year, any number of other activities will and do occur that support the University’s efforts to validate institutional effectiveness.  Recent activities include:

 

·       Participation in the 2000 National Survey of Student Engagement

·       Participation in the 1999-2000 Cooperative Institutional Research Program (CIRP) Survey

·       Cyclic academic program review of every institutional program once every five to seven years

·       Participation in the 2001 National Study of Instructional Costs and Productivity (the Delaware Study) and,

·       Participation by the Perdue School of Business in an International Association for Management Education (AACSB) Benchmarking Study and a Survey of Business Alumni

 

Finally, Salisbury State University has been reviewing its general education curriculum.  In fall 2000, after three and a half years of work, extensive research, and consultation with the Salisbury State University faculty, the General Education Task Force (GETF) submitted to the Provost two models suggesting curricular frameworks for the revision of general education.  Both models are assessment driven.

Salisbury State University is fully engaged in program and student learning outcomes assessment.  The University is acutely aware of the dynamic nature of assessment and the need for continuous improvement.  As a result, it is continually enhancing its institutional, program, and course assessments to validate institutional effectiveness and improve learning.  Although current processes and activities provide ample evidence of student, and therefore, institutional success, these same processes are under continuous review.


Part 1: Institutional Impact of Assessment

 

The core values of Salisbury State University are excellence, student‑centeredness, learning, community, civic engagement, and diversity.  The University’s SLOAR will broadly demonstrate the efforts to effect and impart two of these values and their related outcomes.

 

Excellence: Excellence, the standard against which all University activities and outcomes are measured, connotes the perfection and the quality for which we strive and hold ourselves accountable.  

 

Learning: Learning is fundamental to living a life with purpose in an increasingly inter‑related world and that our role, is to teach students not what to think, but how to think.  The University introduces students to a system of ideas about the nature of humanity, the universe, and the world created by art and thought.  Through active learning, service learning, international experience, and co‑curricular activities, students connect research to practice, and theory to action.

 

Expected Outcomes: Although the following is not an exhaustive list of expected outcomes, Salisbury State University’s graduates will:

 

·         possess a broad base of knowledge

·         demonstrate competence in a specific academic discipline

·         speak, read, write and listen effectively using a variety of media

·         know how to obtain, accurately assess, and present information and ideas

·         identify and solve problems, think critically, and reason effectively

 

The following activities, programs, processes, measures, and results attest to the University’s effectiveness and continuous efforts to improve student learning.  In order to communicate the scope of institutional assessment, the highlights provide a comprehensive view rather than detailed analysis.

 

¨       Academic Program Review

 

Every academic program must conduct a formal internal review every five to seven years.  The review is a self-study at the program level and each unit must submit a quantitative and qualitative summary to the Board of Regents of the University System of Maryland.  In academic year 2000-2001, Salisbury State University redefined its review format, developing more stringent criteria than were previously required while permitting each program wider degrees of freedom in providing discipline appropriate quantitative and qualitative review.  Several criteria specifically reference expanded expectations structured around outcomes assessment including:

 

Program Mission: Articulate the mission and student learning outcomes of your program.  Briefly discuss the degree to which program mission is consistent with University mission.  What, if any, are the gaps between the two mission statements?  Attach copies of any relevant documents such as departmental long-range plan and/or five-year plan.        

 

Assessment: Describe the status of the program relative to any formal or informal assessment activities or plans to implement such activities in conjunction with the expectations of external accrediting agencies.  Assessment initiatives should include both long-term and short-term plans, with the goal of improving the curriculum and thus student outcomes.

 

Further, one criterion of the new format requires each program to incorporate an external evaluation.  Serving as peer review, the evaluation must be included in the final report and provides one of many assessments upon which future program enhancements can be considered.

In AY 2000-2001, the following academic programs were required to submit an academic program review: Biology, Communication Arts, English, Mathematics, Philosophy, Physics, Political Science, and Sociology.  Additionally, although the current process is cyclic, it is expected that program assessment be continuous and that frequent monitoring and reporting be conducted at the departmental and/or course level.  This holistic strategy to excellence in teaching and learning unifies assessment and the validation of student outcomes into a seamless curricular framework.  As such, program review provides a process whereby faculty regularly assess the program against criteria that are faculty designed, discipline specific, and fosters a culture of continuous assessment and improvement.  

 

¨       Survey of Alumni

 

Salisbury State University conducts an annual survey of alumni who graduated the previous year.  The results are analyzed internally on an annual basis, and, by legislative requirement, are submitted to the Maryland Higher Education Commission (MHEC) on a biannual basis in even-numbered years.  The results of the survey provide valuable insights into the University’s graduates’ perception of their undergraduate educational experience and the degree to which that experience and the University have impacted their lives.  Several primary indicators, including graduate school attendance, employment levels, employment levels in Maryland, satisfaction ratings, and employment fields can be assessed from the “survey of alumni.”

Salisbury State University conducted its survey of 1998-1999 graduates in May 2000 and published the results in October 2000.  In addition to the survey instrument, results, and discussion, the publication includes comments solicited from the 1998-1999 graduating class.  The report contains relevant information that is used for program assessment and can help validate a number of the outcomes targeted in Managing For Results.  Some noteworthy highlights of the Salisbury State University Report on the Year 2000 Alumni Survey of 1998-1999 Baccalaureate Degree Recipients are:

 

-         26.3% were currently pursuing a Master’s degree or higher

-         98% reported that SSU had adequately prepared them for graduate studies

-         93% reported that they were satisfied that SSU had adequately prepared them for their current job

-         97% were satisfied with their level of access to full-time faculty

-         98% were satisfied with the quality of education they had received at the University

-         91.6% were satisfied that SSU had adequately prepared them to use information technology in multiple applications

-         89.4% were satisfied that SSU had adequately taught them to be aware of important local, regional, national, and international issues

-         88% would choose to attend SSU “if they were to do it over”

-         83% of those who were employed were employed full-time, and 65% of those were employed in Maryland

-         20% of those employed full-time were employed as teachers

-         48% of those employed full-time were earning $30,000 or more annually and,

-         85% of those working full-time were working in a field directly related to their major.

 

Similar data are provided at the program level, although sample size and respondent rate influence the results.  These data frame one portion of the dialogue regarding measures of student outcomes and how the curriculum, climate, and services might be enhanced.  Additionally, since the survey contains a number of academic and skills related questions, the responses provide an indication of the University’s success in advancing student abilities.

 

¨       Cooperative Institutional Research Program (CIRP) Survey

 

In September 1999, the Salisbury State University (SSU) Orientation Team administered the 1999 Cooperative Institutional Research Program (CIRP) survey to a sample of first-time, full-time freshmen.  These surveys were processed by the staff of the CIRP at the Higher Education Research Institute (HERI) at the University of California, Los Angeles, and returned to Salisbury State University in January 2000.  Data files of the results were forwarded to the University in May 2000.

In fall 2000, the review and analyses of CIRP data were conducted by the University.  As in 1997 (the previous year in which SSU participated), the 1999 CIRP survey was intended to assess the attitudes and lifestyles of incoming freshmen and to determine if those qualities are changing over time.  Four primary goals were of interest to the University:

 

1.       To compare SSU freshmen to the national norms and to institutionally comparable cohorts.

2.       To provide faculty and staff with student attribute profiles for use in strategic planning.

3.       To obtain current and accurate data about freshmen alcohol consumption prior to their college experience to use as baseline data for future comparisons.

4.       To determine the perspectives of SSU freshmen on certain University initiatives.

 

The final report was published in December 2000 and sent to the President’s Advisory Team, the Division of Student Affairs, and the advisors of each school.  The report was also published internally via an intranet link thereby rendering it accessible to the entire faculty and staff.  Some noteworthy highlights of the Final Report on the Responses of SSU Freshmen to the 1999 CIRP Student Questionnaire are:

 

-         12.6% of the entering freshmen were first generation college students

-         22.8% regarded the opportunity to engage in undergraduate research as very important

-         77% had attended a high school where community service was required for graduation.  The same percentage of the national sample did not attend a high school where community service was required for graduation.

-         80% considered raising a family essential or very important

 

¨       National Survey of Student Engagement (NSSE)

 

During the 1999-2000 academic year, the University participated in the first ever National Survey of Student Engagement (NSSE).  The NSSE provided an alternative survey to the traditional CIRP and SSI-type surveys, focusing specifically on the “student engagement” levels of a random sample of approximately 700 freshman and senior students from SSU.  Preliminary data were received in August 2000 while comparative data were received in October 2000.  By agreement, all results were embargoed until the official public release date by the coordinating agency in November 2000. 

The survey was designed to provide indicators or benchmarks of the level of academic challenge, active and collaborative learning, student interactions with faculty members, enriching educational experiences, and supportive campus environment.  Nationally, 276 four-year institutions and 63,000 students participated in the survey.  The data received by the University provided SSU data, peer data on comparable Master’s level institutions, and national data benchmarks.  The data were further stratified by class, while analyses were provided for each and every question. 

Although the results of the survey are still under review, a preliminary analysis reveals some significant insight into the opinions of SSU students versus comparable peers, and their perceived level of academic engagement.  Some noteworthy highlights of Salisbury State University’s National Survey of Student Engagement are:

 

-         Seniors rated the quality of their relationships with faculty as better than the peer average and the national average.  The difference was statistically significant against both samples.

-         Seniors rated the quality of their relationships with administrative personnel and offices as better than the peer average and the national average.  The difference was statistically significant against both samples.

-         Seniors rated their satisfaction with the entire educational experience as better than the peer average and the national average.  The difference was statistically significant against the peer group.

-         Freshmen said that the institutional emphasis encouraged contact among students from different economic, social, and racial or ethnic backgrounds at a level that was lower than the ratings of the peer and national samples.  The difference was statistically significant against the national sample.

-         Although the differences are not statistically significant, both freshmen and seniors believe that the level of academic challenge at SSU is not as great as that communicated by both the peer and national samples.

 

As with all data, the University approaches the interpretation of these results objectively and cautiously.  The NSSE does not consider or compensate for academic ability and achievement, which may strongly impact the opinions regarding academic challenge.  Thus, the results and the implications upon the curriculum are incomplete and will become a matter of discussion for University faculty.

 

¨       Then and Now Course Assessment

 

The Henson School of Science and Technology is piloting a course assessment model, which they call the “Then and Now Course Assessment.”  This model is based on a presentation entitled “Using the Post-then Method to Assess Learner Change” by Karl Umble, et al. at the 2000 AAHE Conference on Assessment.  In this model, the students, at the end of the semester, are asked to evaluate their level of understanding at the beginning (Then) and at the end of the semester (Now) relative to the desired student learning outcomes for the course.  The pilot study involves one faculty member from each of the seven departments in the Henson School and is being used in full range of courses, from freshman survey courses to senior majors courses.  A meeting was held on January 23, 2001, to evaluate the Fall 2000 semester assessment.  The results of the meeting were:

1)   All members found the course assessment a positive experience and wished to continue the assessment project for the Spring semester.  

2)   There was interest from others to join the pilot study.  It was agreed that two-to-three faculty per department could participate, but that the group should not become too large since most of the data processing is now done manually.

3)   The first results were displayed as a bar graph, the bars extending from the mean starting level to the mean ending level.  The graph had a relative scale based on the outcome with the highest outcome average.  The group wanted to examine other ways of displaying the result.  Alternative result displays will be examined during the Spring semester.

 

¨       General Education Proposal

 

By the end of the 2000 calendar year, after three and a half years of work, extensive research, and consultation with the Salisbury State University faculty, the General Education Task Force (GETF) submitted to the Provost two models suggesting curricular frameworks for the revision of general education.  Both models are assessment driven and are grounded in principles of general education as well as student learning goals that were previously endorsed by the Faculty Senate.  The GETF submitted a full report on these models, including faculty feedback on each, as well as proposals on an implementation process and stewardship of general education on campus.  The report was forwarded to the Provost’s Office with a request that the Provost conduct a cost analysis of both models, including a comparison to the costs of the current general education curriculum.  Once that analysis is complete, the report of the General Education Task Force, accompanied by the cost analysis, will be sent to the Faculty Senate.  At that point, the Faculty Senate will develop and implement a formal deliberative process that will result in a formal recommendation to the Provost regarding what revision should be made in the general education program. the General Education Task Force submitted its report. 

 

Part 2: Programmatic Assessment

¨       Accredited and Certified Programs

 

Several of Salisbury State University’s academic programs have earned specialized accreditation.  In many professional programs, specialized accreditation is essential to institutional graduates who must attest to completing studies in a program that is certified to meet the criteria and standards of a granting accrediting agency.  These agencies require periodic program assessment to measure, validate, or certify quality in higher education.  They also establish rigid criteria and standards for program accreditation and validate the success of the program through multiple measures, including student learning outcomes.  In many instances, the final hurdle that students must surmount is in the form of a standardized exam for licensure or certification that assesses content knowledge and skills. The following programs, along with the accrediting agency and review dates, are individually accredited or certified at Salisbury State University:

Academic Program

Accrediting

 Agency*

Accreditation

Review

  Last             Next

Accounting

AACSB

  1994             2004

Athletic Training

CAAHEP

  1998             2003

Business Administration

AACSB

  1994             2004

Chemistry

ACS-CPT

  1997             2002

Economics

AACSB

  1994             2004

Elementary Education

NCATE

  1999             2004

Environmental Health

NEHSPAC

  2000             2005

Management

AACSB

  1994             2004

Medical Technology

NAACLS

  1999             2006

Nursing

NLNAC

  1996             2004

Physical Education

NCATE

  1999             2004

Respiratory Therapy

CAAHEP

  1998             2005

Social Work

CSWE 

  1997             2005

 

The following table lists the licensure/certification passing rates of SSU students by program.

Licensure Exam

Passing Rate

 

FY 1997

FY 1998

FY 1999

FY 2000

Medical Technology

100%

100%

80%

100%

Nursing

97%

87%

87%

91%

Respiratory Therapy

100%

100%

100%

100%

Teaching

96%

99%

96%

96%

 

¨       Teaching and Learning Network

 

The Teaching and Learning Network (TLN) at Salisbury State University was initiated in 1999 to support and promote the advancement of excellence in teaching and learning. The TLN focuses on efforts to foster effective pedagogical practice and the use of technology to achieve excellence in the teaching/learning process, both on the SSU campus and in diverse educational settings at all levels of K-16 and in graduate education. In addressing its mission, the TLN promotes informed and reflective, research-based best practice in teaching and learning. The goal of the TLN is to foster a climate where the scholarship and practice of effective teaching is highly valued as a professional commitment of SSU’s faculty. The TLN also seeks to address the needs of, and interaction among, each of three constituencies that benefit from the commitment to excellence in teaching at SSU: the Higher Education community, K-12 Teachers on Maryland’s Eastern Shore, and the employing Business Community in Maryland. Since its formation, the TLN has been instrumental in initiating the following:

 

·         funding and organizing a faculty mentor program;

·         providing facilities for state-of-the-art design in the form of a development studio and resource center;

·         supporting writing across the curriculum by jointly sponsoring faculty workshops and conferences;

·         supporting innovative curriculum and program development and piloting an interactive Website;

·         serving as a forum for evaluating teaching strategies and determining ways in which outcome assessment can be quantitatively and/or qualitatively evaluated;

·         sponsoring and conducting workshops and seminars on pedagogical issues; and

·         supporting research and writing about teaching and learning as a professional activity through minigrants and providing assistance in publishing results of research.

 

As demonstrated, the University can attest to significant activities in university-wide outcomes and program assessment.  There are a number of academic programs that through specialized accreditation continually validate the success of student learning.  Additionally, there is an active Office of Institutional Assessment, Research, & Accountability that annually conducts, analyzes, and publishes institutional data on a variety of performance measures utilizing a variety of instrumentation.  Further, there are a number of institutional plans, including strategic and facilities plans that facilitate prioritization and utilize information to “close the feedback loop,” to assess progress, and to assist in decision-making.  By using multiple measures, Salisbury State University is able to draw inferences and conclusions from results that communicate similar information.