Loading...
 

Assessment and Institutional Effectiveness

The material for this Assessment wiki has been adapted from the 2018 Mohawk Valley Community College (MVCC) Self-Study document that was submitted for reaccreditation by the Middle States Commission on Higher Education (MSCHE), 3624 Market Street, Philadelphia, PA 19104 (Telephone: 267-284-5000, website: www.msche.org).



Assessment at MVCC

There are various ways that MVCC works to extend the culture of assessment throughout the college community. An Institutional Effectiveness and Assessment (IE&A) Plan serves as the guiding document to detail expectations of the assessment activities conducted at MVCC. The IE&A Plan is modified annually to reflect changes in assessment priorities and is composed of specific objectives and work steps addressing the various institutional effectiveness and assessment processes. The IE&A Plan work steps have been linked to the Middle State Standards.

The Institutional Effectiveness Council (IEC), a sub-council of the Strategic Planning Council, oversees the assessment processes. The IEC is co-chaired by the Director of Institutional Research and Analysis and the Assessment Liaison. IEC membership includes representation from all divisions of the College. Progress on the work steps of the IE&A Plan is monitored by the IEC, and annual updates are provided to the Strategic Planning Council.

The MVCC Institutional Effectiveness website provides information about the IEC, and is a point of reference for assessment information on procedures and results of assessment activities. Included on the site is the statement of MVCC’s Principles of Assessment, an Assessment Handbook, and a Survey and Data resource page. The MVCC Assessment Statement, on this website, as well as in the College Catalog, provides expectations of participation with assessment. An online Blackboard Basic Assessment Course has been developed which could be of benefit for assisting members of Academic Affairs in understanding the principles and processes of assessing academic student learning.

The assessment work is overseen by the the Director of Institutional Research and Analysis (DIRA) and the Assessment Liaison. The Office of Institutional Research and Analysis (OIRA), in addition to overseeing the institutional research functions, supports assessment and institutional effectiveness by coordinating all survey efforts, monitoring research proposals, and collecting, analyzing, and reporting assessment results.

Budgets for supporting assessment and institutional effectiveness activities are managed through the Office of the Vice President of Learning and Academic Affairs and through the Office of Institutional Research and Analysis. In addition, the College Foundation funds an annual Eye of the Hawk Award.

Procedures for IEC and Assessment work

Procedures for IEC and IE and A Plan work

Learning Outcomes

The first step in providing effective assessment of student learning is the identification of learning outcomes. Faculty and staff work with the Assessment Liaison and DIRA to develop measurable learning outcomes. Blooms Taxonomy is utilized to reflect the appropriate level of learning expectation.

MVCC demonstrates an integrated approach, which includes student learning outcomes at the course, program (academic, general education, and diversity and global view), and institutional (College-Wide Competencies) levels. The program-level and institutional-level learning outcomes are consistent with the College Mission. Curriculum maps show the relationships between courses and the program goals for each of the academic programs. For the academic program goals and supporting outcomes, at least one goal is required to link to each of the College-Wide Competency statements to ensure that opportunities within the program will provide for students to meet each institutional-level learning outcomes.

Procedures for Learning Outcomes Work

Procedures for Learning Outcomes Work

Course-level Learning Outcomes

Course-level learning outcomes for credit and developmental courses are developed by full-time faculty members who teach the courses. When new courses are developed, or major changes are made to course content, the Assessment Liaison reviews the student learning outcomes before final approval is made by the Vice President for Learning and Academic Affairs. Each course has a course outline that includes a core set of student learning outcomes; additional learning outcomes may be added by individual instructors. Regardless of who is teaching the course, or the location or mode of delivery, the expectation is that the instructor will provide opportunities for the students to achieve the core learning outcomes. Course learning outcomes are distributed on faculty syllabi are displayed on the course search page of the College website. Learning outcomes for non-credit courses are developed by the instructors hired through the Center for Corporate and Community Education (CCED) and are distributed to students in the course.

Assessment of Course-Level Learning Outcomes

At the course level, the assessment of student learning outcomes is under the purview of the faculty members teaching the course and is reinforced within the MVCC Faculty Bill of Rights. Online courses are assessed similarly to on-campus courses, as appropriate to the online learning environment.

A variety of direct measures to assess student learning are used, including but not limited to objective tests, skills testing, laboratory reports, oral presentations, and assignments and research papers that are graded by rubrics. Indirect measures of assessing student learning are also employed, including Classroom Assessment Techniques and the Student Class Survey, which is administered toward the end of each course.

Procedures for Course-level Assessment work

Procedures for Course-Level Assessment Work

Program-Level Learning Outcomes

Academic

For each academic program, goals and supporting outcomes, some of which are learning outcomes, have been developed by full-time discipline faculty who teach within the program. Those learning outcomes provide the expectations of learning for students completing the program. Program goals and outcomes are displayed on the website for each degree and certificate. Discipline faculty regularly review and revise program-level learning outcomes based on business and industry requirements, transfer opportunities, SUNY and NYSED requirements, national standards, and program advisory board recommendations.

Assessment of Academic Program-Level Learning Outcomes

The assessment of academic program learning outcomes includes a process by which discipline faculty complete a Program Outcomes Assessment Report (POAR) template for each program to monitor formative and summative assessment of program goals and outcomes. The POAR template displays the program goals and outcomes, as well as the assessment measures and benchmarks for determining how well students are meeting them. A column also shows how the program goals link to the institutional-level learning goals (College-Wide Competencies). After assessments are conducted, discipline faculty comments on what action may be taken to address the results. All POAR templates are located in the IEC folder located on the MVCC shared M:drive.

A five-year Academic Program Review (APR) process is conducted for programs to examine key components of how the program supports the College Mission and expectations from national standards, and how assessment results are used for improvement of the program. Components of the review are outlined in the Academic Program Review Tool, which displays congruence with the Middle States Standards. A strengths and weaknesses section is completed as part of the review, including suggested actions to address the identified challenges. Discipline faculty who teach in the program complete the Academic Program Reviews and are encouraged to have an external reviewer examine the APR document, meet with students, and discuss ideas for program improvement.

The Assessment Liaison, the Vice President for Learning and Academic Affairs, and the appropriate Assistant Vice President and department Associate Dean review completed APRs. Results of the reviews are shared with appropriate groups (discipline faculty, program Advisory Boards, et al.), examined to make recommendations for purchasing of equipment/materials, and to initiate changes in the curriculum, faculty numbers, and course sequencing. Planning and budgeting allocations incorporate the resources to address the suggestions, as feasible considering budgeting constraints.

Procedures for Academic Program Assessment work

Procedures for Academic Program Assessment work

General Education

The learning outcomes for the General Education (GE) program are displayed in the College Catalog and on the Institutional Effectiveness website, along with additional expectations for successful completion of the GE program. These learning outcomes are integrated with the expectations of the MSCHE general education essential skills and the learning outcomes set forth by the SUNY General Education initiative. A rigorous process is in place to approve courses for General Education status and to provide the opportunity for students to meet the appropriate General Education learning outcomes.

The College Catalog contains a complete list of courses satisfying General Education requirements and a Quick Reference Guide is maintained to help faculty and students understand expectations of the program. The General Education aims and course list provide clear congruence between the SUNY and MSCHE requirements.

Oversight of the MVCC General Education program is the responsibility of the College Senate General Education Committee. The actions and recommendations of the Committee are forwarded to the Vice President for Learning and Academic Affairs and, where approval is needed, to SUNY. An annual report is submitted to the Senate and, in specific cases, plans are submitted for Senate approval.

Assessment of General Education

Assessment activities for the General Education learning outcomes are documented in the MVCC General Education Assessment plan, and revolve on a four-year cycle. Each fall, the appropriate Associate Deans are notified about the classes that have been selected for assessment that academic year. Most assessments for the GE outcomes focus on course-embedded activities. Faculty report the General Education assessment results to the Assessment Liaison for aggregate reporting.

The SUNY competency of Information Management is assessed using the online Information Management/DGV/CT Assessment administered at the time of graduation. This assessment instrument was developed by Library faculty and the General Education Committee. When applying for graduation on the website, prospective graduates are requested to complete the online assessment designed to measure their ability to perform basic computer operations, use basic research techniques, and locate, evaluate, and synthesize information from a variety of sources. Results are tabulated by the Office of Institutional Research and Analysis, and data are sorted by academic program and reviewed during the five-year APR Process.

Critical thinking is assessed in two ways. All students in AA, AS, and AAS programs are required to take EN102, which provides the opportunity for students to complete assignments that meet the SUNY critical thinking learning outcomes. The assignments are then scored according to the SUNY Critical Thinking Rubric. In addition to the in-course assessment, a component that focuses on a broader definition of critical thinking is part of the online Information Management/DGV/CT Assessment. This portion includes a passage, followed by five questions pertaining to the written narrative.

Procedures for General Education Assessment work

Procedures for General Education Assessment work

Diversity and Global View (DGV)

The MVCC Diversity and Global View program provides students with an opportunity to expand their knowledge and understanding of the diverse populations with which they interact. The DGV plan identifies learning outcomes and graduation requirements for the program. The DGV program is monitored by the Diversity and Global View committee, a subcommittee of the College Senate.

Assessment of DGV

Assessment of the College Diversity and Global View learning outcomes is accomplished through two processes. The Information Management/DGV/CT Assessment previously mentioned has five items dealing with the DGV learning outcomes. In addition, as a graduation requirement, students complete DGV Tutorials through Blackboard, and each tutorial includes a quiz that must be completed with an 80% success rate.

Procedures DGV Assessment work

Procedures for DGV Assessment work

Institutional-Level Learning Outcomes


College-Wide Competencies

The MVCC College-Wide Competencies serve as the institutional level-learning goals and are overseen by the General Education Committee. Although there is significant overlap between the Competencies and the General Education learning outcomes, per New York State Education law, AOS and Certificate programs are not expected to include courses in the liberal arts and sciences. Therefore, the College-Wide Competencies designate the knowledge and skills that are expected of all program and certificate graduates, at a level appropriate to the coursework involved.

MVCC College-Wide Competencies 2 and 5 address the learning outcomes associated with information literacy skills. A crosswalk between the learning outcomes for SUNY and MVCC GE Information Management, the MVCC Competencies, and the MSCHE Information Literacy Skills has been developed to provide assurance that MVCC is addressing the SUNY and MSCHE identified learning skills for this area.

Assessment of College-Wide Competencies

As part of the process to identify academic program goals and supporting outcomes, discipline faculty must make sure that at least one program goal links to each of the College-Wide Competencies. As faculty assess student learning at the program level, attainment of the institutional-level learning outcomes is also assessed. The College-wide Competencies dealing with information literacy are assessed on the online Information Management/DGV/CT instrument.

Procedures for College-wide Competencies Assessment work

Procedures for College-wide Competencies Assessment work

College Unit Goals and Outcomes

The areas of Student Affairs, Administrative Services, and those reporting to the President have developed operational goals and supporting outcomes of the programs/services/activities of each department. Units within Student Affairs have begun to incorporate student-learning outcomes as part of the goals for their departments.

Assessment of College Units

College units under the areas of Student Affairs, Administrative Services, and those reporting to the President have developed Program Outcomes Assessment Report (POAR) templates, as appropriate to the activities and programs offered through the unit. These templates delineate the operational goals and supporting outcomes of the programs/services/activities of the departments. In addition, they provide for systematic assessment, review of data, and action plans for improvement for the operational programs and services offered within the departments. Assessments are completed for the listed goal outcomes, usually on an annual basis. Data are reviewed to determine whether the programs/services are effective or if changes need to be made in the goals/outcomes, to inform resource allocation, and to discuss ways to improve the operations of the department. All Unit POAR templates are located in the IEC folder located on the MVCC shared M:drive.


Procedures for College Unit Assessment work

Procedures for College Unit Assessment work

Institutional Effectiveness Assessment


In addition to the assessment of programs/services/activities reported on the POAR templates, the Office of Institutional Research and Analysis is responsible for monitoring surveys and database repositories used to assess institutional effectiveness. Surveys or measures may be added or removed as necessary from the following list to maintain currency in the measure of important institutional goals or initiatives.

Integrated Postsecondary Educational Data System (IPEDS): As a participant in federal student aid programs, MVCC reports data on enrollments, program completions, graduation rates, student outcomes, faculty and staff, finances, institutional prices, and student financial aid. MVCC data are reported on the MVCC Institutional Effectiveness web page.
Achieving the Dream (ATD): MVCC ATD data continue to provide three years of information on key performance indicators.
SUNY Student Opinion Survey (SUNY SOS): The SOS survey results provide feedback from students on a range of items and are available on the password-protected section of the MVCC Institutional Effectiveness web page.
National Community College Benchmark Project (NCCBP): The National Community College Benchmark Project (NCCBP) displays core indicators of institutional effectiveness allowing peer comparisons. MVCC results are found on the password-protected section of the MVCC Institutional Effectiveness web page.
Community College Survey of Student Engagement (CCSSE): The CCSSE surveys students on College programs and services. MVCC results are available on the CCSSE website.
Voluntary Framework of Accountability (VFA): The VFA is a national accountability system for community colleges that uses a number of Metrics dealing with the areas of Developmental Education Progress Measures, Two-year Progress Measures, Six-year Outcomes Measures, Career and Technical Education Measures, Non-Credit Workforce Courses, and Adult Basic Education/GED.
Student Achievement Measure (SAM): SAM, required by SUNY, provides a way to report student progress and graduation by reporting on the percentage of students who have graduated, are still enrolled, have transferred to a subsequent institution, or whose enrollment or completion status is unknown. Results are available on the SAM website.
MVCC Commencement Survey: This survey is administered to students at the time of their graduation and asks for information about their experiences at the College. Results are found on the password-protected section of the MVCC Institutional Effectiveness web page.
MVCC Graduate Survey: This survey is administered shortly after graduates have left the College and asks for information pertaining to their future plans. Results are found on the password-protected section of the MVCC Institutional Effectiveness web page.
MVCC Annual Institutional Update for MSCHE (AIU): The MVCC AIU provides longitudinal data on a number of College areas (finances, enrollment, etc.) and student information. The update also includes progress on recommendations made as a result of the MSCHE Self-Study reaccreditation process. Results are found on the password-protected section of the MVCC Institutional Effectiveness web page.

Procedures for Institutional Effectiveness Assessment work

Procedures for Institutional Effectiveness Assessment work

Communication and Use of Assessment Results

Assessment results obtained from the Program Outcomes Assessment Report (POAR) templates are reviewed by department members and the appropriate supervisor. Documented results are archived on the M:drive in the IEC folder. Data gathered from surveys are reported on the MVCC Institutional Effectiveness web page. Results, along with feedback from the IEC, are presented to the President’s Cabinet by the Director of Institutional Research and Analysis. The President’s Cabinet reviews the analysis of the data and then uses the information to inform resource allocation and planning. Changes instituted through the examination of assessment data and longitudinal results are documented by each program/department.

Procedures for Communicating and Documenting Use of Assessment Results

Procedures for Communicating and Documenting Use of Assessment Results