Below are questions about assessment that have been asked by CHFA faculty. Click on each question to go to the answer. Answers have been developed by members of the CHFA SOA Committee. Do you have a FAQ about assessment? Share your question with the SOA Committee by e-mailing martha.reineke@uni.edu.
We already track student learning. It’s called grading. Why do we have to do something more?
Doesn’t mandated assessment violate the academic freedom of professors?
What is the most important reason to do assessment?
What is the role of accreditation in assessment?
What is assessment?
Assessment is a practice for evaluating whether students in our programs are learning. We engage in assessment when we consider in systematic ways whether and to what degree our teaching methods are effective. Assessment enables us to ask, “Are our students learning what we want them to learn? What can we do to improve student learning?”
Faculty members already ask questions that aim to assess student learning:
· How strong are our students' writing skills?
· Because students in our major can select from numerous electives, few students who graduate from our program have shared a common learning experience. Does this matter?
· Faculty members teaching the Senior seminar are noticing gaps in students’ learning. What can we do to assist students in filing these gaps prior to the Senior seminar?
· Is the sequencing of courses in our major effective? Would changing the sequence enhance student learning?
Finding answers to such questions is one of the most important roles for assessment.
When we reflect systematically on such questions, we engage in program assessment. Tom Angelo’s definition of program assessment is widely cited for its explanatory clarity and completeness:
Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education.
~Tom Angelo, “Reassessing (and Defining) Assessment.” The AAHE Bulletin, 48 (2), November 1995, pp. 7-9.
Martha Reineke
Professor of Religion
Faculty members often say, “We already do assessment; we grade our students.” Grades are a measure of student learning. However, grades do not offer particularly helpful data for program outcomes assessment. Course grades offer a score that sums up a student’s performance across a host of learning outcomes. Student grades across the major result in a major gpa. But course grades and gpas don’t tell us which course or major outcomes students have mastered. Because grades don’t tell us about student learning in the aggregate on particular learning outcomes, they provide little information on the overall success of a program in assisting students in meeting learning outcomes. Barbara Walvoord helpfully sums up the different emphases and strengths of grading and assessment:
Grading is a way to assess learning.
- The audience is students, potential employers, or a graduate admissions committee.
- Its focus is on individual achievement.
- It results in grades and letters of recommendation.
Graded activities also can be used for program-level assessment IF:
- Data about learning are diagnostic
NOT diagnostic: Students earned an average grade of C+ in this program.
Diagnostic: “On their capstone projects, students were strong in identification of the problem but weak in considering alternative solutions.”
- Data are aggregated to yield information about strengths/weaknesses in a group of students. Aggregate data are anonymous (student and faculty names are removed).
- Data are used to suggest courses of action to improve learning.
In an outcomes assessment process, faculty can draw on classroom work that also is being graded, using it for embedded assessment by aggregating the data and analyzing it with rubrics specific to the assessment process. This data can be combined in an assessment plan with other types of data (standardized tests, student surveys, senior recitals and exhibitions, conference presentations, portfolios).
Sources: Barbara Walvood, presentation at the Higher Learning Commission 2005 Meeting, April 2, 2005, Chicago, IL; http://www.umass.edu/oapa/oapa/publications/online_handbooks/program_based.pdf
Doesn’t mandated assessment violate the academic freedom of professors?
Responding to a “mandate” from an external source does not necessarily violate academic freedom. It depends on the faculty’s role in deciding how to respond. Outcomes assessment is being strongly promoted by external stakeholders such as accrediting agencies, the Iowa legislature, and the Board of Regents. Even parents and students have increasing interest in accountability in higher education. These stakeholders ask us, “What do you want students to learn and how do you know if they are learning it? Or, “What are you doing to enhance student learning in your major?” If faculty members make the decisions regarding how to answer these questions, then academic freedom is not violated. Establishing the content and setting the standards for academic programs have long been a faculty prerogative and faculty have typically shared this information with others. We publish our major requirements in brochures and on the web. We prepare syllabi for our students. We engage in the processes of program review and curriculum revision. By participating in assessment, faculty are extending this collegial work. We reflect systematically each year on successes and challenges we have encountered with our students. We gather information from our students and use it to make decisions about curriculum in order to enhance our students’ learning. The keys are that the faculty have an opportunity to participate in open discussion of assessment, the faculty are involved in measuring how well students are performing, and the faculty decide what curricular or program changes are indicated in order to improve students’ knowledge of their major. The assessment process is consistent with academic freedom as long as the process is in the hands of the faculty, and faculty set the standards for what our graduates should know, value, and be able to accomplish.
Lauren Nelson
Associate Professor, Communicative Disorders
Yes, the initial SOA business was too top-down and too often the messengers were domineering and condescending. It is important to recognize that this new approach to SOA at UNI is premised on the idea of student learning. What our students learn should be a question of interest to all of us. Additionally, this new approach is directly tied to improving programs in that the information gathered about student learning should be (will be) applied to program modification. Most important is departmental and faculty control and ownership of this process. The process is intended to achieve improvement. It is not a process done for its own sake or to mollify upper administration.
Richard Colburn
Professor of Art
This question really is two-fold. It asks whether standardized tests are appropriate and if faculty really need to be involved in assessment.
Are standardized tests appropriate? We should use standardized tests in the assessment of our majors IF faculty in our departments concur that the information we gain from these tests is valuable. If we draw on the results of the tests to review our majors, revise our courses, develop new ones, and otherwise use insights we have garnered from the test scores to enhance student learning, tests are helpful measures of student learning. But if we don’t think that students’ learning can be measured in that way, we will need to use other assessment mechanisms that reflect the complexity and/or distinctiveness of our goals for students. For example, if we want to assess students’ thinking skills, we may decide that activities other than a standardized test more richly catch our students in the act of critical reflection. Other media (e.g., embedded assessment that draws on classroom work such as essays, research projects) may also reflect our learning values for students better than does a standardized test.
Do faculty members need to be involved in assessment or can someone else do it for us? Faculty already set the standards of what a graduate of a program should know, be able to do, and value. Faculty already measure how well students are performing. Faculty already decide what curricular or program changes are needed in order to improve students’ knowledge of their major. Faculty members are typically reluctant to cede these responsibilities to others. Assessment is really an extension of our existing faculty responsibilities. It is simply a systematic process for gathering, analyzing, and interpreting our teaching efforts and for using the resulting information to document, explain, and improve students’ learning. At UNI, this systematic process is integral to academic program review.
Furthermore, when we engage in this process, we ready ourselves for external calls for accountability. At present, debate is ensuing nationwide at state and federal levels about mandated testing for students in higher education. If faculty members are already found to be engaging in assessment of our own choosing and demonstrating to our publics that we already are holding ourselves accountable for student learning as we prepare for and engage in regular program review, we may be less vulnerable to high-stakes, mandated testing on the model of No Child Left Behind.
April Chatham-Carpenter
Associate Professor, Communication Studies
Students who embrace the principle of ineffability in their learning believe they are born writers or artists or musicians. Indeed, we sometimes hear students say, “I am not good at writing,” as if this is a natural given. If we don’t concur with this way of thinking, we need to be able to articulate for our students what count as successful writing outcomes, as optimum critical thinking outcomes, or as outstanding performances in the arts, and we need to establish markers for achieving those outcomes. Rather than dismiss assessment, if the measures that are offered to us by assessment experts don’t fit, we need to use our disciplinary expertise to develop strategies for documenting, describing, and assessing student learning that do fit. We certainly do this when we are engaged in the peer review of each other’s work – whether we are jurying an art exhibition or reviewing a book manuscript. We can do this for our students too.
The point of SOA is not to “bump off” some courses or protect other courses within the department but to promote discussions that enhance the quality of students’ learning. These discussions can result in changes in what we teach and how we teach. However, these discussions, in themselves, do not place at particular risk any sub-category of courses (e.g., advanced courses and seminars). We need to remain open to discussion in order to explore with care how our courses contribute to student learning and changes we might make in our courses that would enhance the capacity of our courses to do so.
When faculty make the case for how our courses contribute to student learning, as part of outcomes discussions, faculty may experience challenges answering that question for multiple courses in the major, and not only for their advanced courses. For example, when departments choose key learning outcomes and look in the curriculum to identify courses in which students are given opportunities to achieve that outcome, it is not uncommon for faculty members to discover that students are not exposed to a particular outcome in any course. Faculty members may choose to drop that outcome from their list. Or, one or more faculty members may volunteer to incorporate opportunities for students to work toward that outcome into a course or courses that they typically teach.
In respect to advanced courses, some that may appear initially not to contribute to outcomes for a major may be discovered to contribute when they are considered as a “cluster” within the major that collectively serve to further specified, advanced outcomes. For example, some critical thinking rubrics identify advanced skills of which students are capable only at the end of their college careers. When faculty members align the curriculum with outcomes, they may make an effort to look at the place of such high-level outcomes in the advanced courses of a major.
What is the most important reason to do assessment?
The main reasons to do outcomes assessment are to help faculty to 1) discover the extent to which students are learning and are able to use or adapt what they have learned in useful ways after they have completed a program of study and 2) better understand whether what students have learned is relevant and is meeting program goals identified by program faculty, such as content and skills. The focus of outcomes assessment is not on the individual student but on students in a program collectively. If students in the aggregate are not learning/retaining, it may be necessary to adjust the delivery of the content. If the content is not relevant or appropriate, curricular adjustments are probably necessary. Outcomes assessment (rather than standard grading evaluation) is very likely the only process that will give a true picture of student learning and program content relevance.
Alan Schmitz
Professor of Music
What is the role of accreditation in assessment?
In Fall, 2005 the Higher Learning Commission announced a set of new questions for universities that will guide reviewers from the North Central Association when they come to our campus the next time. The questions that the NCA will ask when their representatives visit our campus (and on which our campus self-study will report) are:
· How are your stated student learning outcomes appropriate to your mission, program, and degrees?
· What evidence do you have that students are achieving your stated learning outcomes?
· In what ways do you analyze and use evidence of student learning?
· How do you ensure shared responsibility for assessment of student learning?
· How do you evaluate and improve the effectiveness of your efforts to assess and improve student learning?
The HLC would like universities to use these questions as conversation starters at multiple levels: including program majors and the LAC core. In CHFA, we will be using these questions each year in our departmental conversations about assessment. We also will respond to these questions in our annual SOA reports to the Dean. These reports will establish documentation for the NCA that departments in CHFA have engaged regularly in conversations about assessment and in assessment practices referenced by these conversations.