«Site: Intensive English Program American University of Kuwait Type: Programmatic Dates of Visit: February 21-24, 2009 Site Reviewers: Sarah ...»
Step 2: Identify the most important outcomes of the department or program Learning outcomes are the knowledge, skills, values, and attitudes that students gain from a learning
experience. They address the following questions:
What should students know and be able to do when they have finished their particular program at AUK?
What knowledge, skills, or attitudes distinguish graduates from your program from other students?
How do these outcomes tie in with the university’s mission and educational goals?
Answering these questions produce statements of learning outcomes or learning goals (the two phrases are used interchangeably). The list does not need to include all learning outcomes, only the most important; more than two and less than eight is ideal.
Learning outcomes need to be specific, clear, and measurable and ideally include knowledge that students acquire, skills that students demonstrate, and attitudes that students develop. Well-defined outcomes are often stated as: “Students will …” or “Upon graduation, students will…”
In addition, when developing outcomes:
• Focus on the ends, not the means -- what students will do after completing the course or program, what the desired “end state” should be.
• Use an “action” verb to describe in an observable way what students should be able to do.
• Try not to be too broad or too specific.
Finally, share outcomes with students and staff. Students learn more effectively when they are given clear goals to help them focus on what’s most important, understand how individual assignments or courses fit with the goals of the department, and how this course or program will help prepare them for life or careers after graduation. Program outcomes should be listed on the program’s website, and course outcomes should be listed on course syllabi.
Sample Departmental Outcomes Example 1: Business Administration (Bowling Green State University) (Student Achievement Assessment Committee (SAAC) 2007)
Graduates will be able to:
1. Demonstrate problem-solving, critical-thinking, oral and written communications, and team and leadership skills
2. Apply business tools and concepts in domestic and global contexts
3. Integrate foundational and functional business areas in making decisions
4. Show commitment to ethical values and behavior, continuous learning, and professional growth
5. Show understanding and appreciation for cultural, racial, and gender differences Example 2: Computer Science (Bowling Green State University) ("Department and Program Learning Outcomes” 2007)
Graduates will be able to:
1. Program in a higher-level language
2. Work effectively with a client and members of a software development team to analyze, specify, design, implement, test, and document software that meets the client's needs
3. Acquire new computer-related skills independently as technologies evolve Review Team Report Responses 2009 Page 37
4. Communicate technical concepts to non-technical persons, both orally and in writing
5. Develop a plan to integrate hardware and software into a particular environment
6. Conduct themselves in an ethical and professional manner Example 3: Biology (AUC)
The graduates of the Biology Department will be able to:
1. Think critically, identify biological issues and formulate solutions to biological problems.
2. Use computers and information technology effectively to address biological problems.
3. Function effectively in a teamwork environment.
4. Apply knowledge in basic mathematics, general chemistry, calculus bases physics and statistics to solving biological problems.
5. Use their knowledge and comprehension of basic biological principles, concepts, and theories.
6. Evaluate and synthesize information and ideas from a variety of sources and formats.
7. Competently collect, analyze, organize, evaluate, and present scientific data.
8. Understand, analyze, and evaluate original research literature in support of current research projects.
9. Compete effectively for entry level employment and/or placement in graduate or professional training facilities.
Step 3: Ensure that students have adequate opportunities to achieve these outcomes A program’s curriculum needs to ensure that all students in the program have the opportunity to achieve these goals before they graduate. Program planners need to ask, “In what courses or experiences do students learn these skills or acquire this knowledge?” A matrix can be a useful tool to map outcomes with the curriculum and learning experiences to ensure that all students are presented with adequate learning opportunities.
Step 4: Define how you will assess progress towards these outcomes Assessments don’t have to be complicated and, when used well, can be a powerful tool for improvement, providing better information for planning, budgeting, changes in curriculum, new programs, staffing, and student support. Student learning assessment data helps us understand what our students are learning, where they might be having difficulty, and how we can change the way we teach and how we can shape our curriculum to help them learn better. Assessment is not an evaluation of individual students, faculty or courses.
Start by taking an inventory of the kinds of tools your department or program is already using.
Many departments and programs are already accessing student learning outcomes. These assessments might take the form of capstone courses, theses, papers, individual or group projects, performances, documentaries, presentations, student portfolios, alumni or employer surveys, student opinion surveys, focus groups, standardized tests, entry or exit tests or surveys, reports from internship supervisors, or other measures.
Listed below are direct and indirect measures of student learning. Effective assessment plans must include a mix of direct and indirect methods of assessment.
Direct methods of evaluating student learning provide tangible evidence that a student has acquired a skill, demonstrates a quality, understands a concept, or holds a value tied to a specific outcome. They answer the question, “What did students learn as a result of this (assignment/project/exam…)?” and “How well did they learn?” Direct methods generally result in student “products” like term papers or performances.
Direct Methods of Assessing Student Learning:
• Capstone courses
• Review of senior projects by external evaluators (using scoring guidelines – see appendix 3)
• Licensure or certification exams Review Team Report Responses 2009 Page 38
• Places in the curriculum where multiple faculty members examine student work, e.g. theses, video documentaries, art projects, research projects, etc. Scoring guidelines should be used
• Portfolios and e-portfolios, with material showing progression throughout major
• Entry and exit exams
• Homework assignments, examinations and quizzes, term papers and case studies
• Evaluations of student performance in internships, research projects, field work, or service learning.
• Classroom Assessment Techniques (CATs)
• Standardized tests
• Videotape of oral presentations or performances Indirect methods provide more intangible evidence, demonstrating characteristics associated with learning but only implying that learning has occurred. When a student answers a question correctly, there is direct evidence that he or she has learned. When a student says that he or she has an excellent understanding of the topic, there is indirect evidence. While both methods of assessing learning are valuable, indirect evidence is more meaningful when it is tied to direct evidence.
Indirect Methods of Assessing Student Learning:
• Retention and graduation statistics
• Job placement or graduate school acceptance
• Career development over time
• Student perception surveys
• Course evaluations, with questions added regarding learning
• Alumni surveys or focus groups
• Employer surveys or focus groups
• Student activities
• Teaching strategies that promote learning
• Course grades not based on scoring guidelines or not linked to clear learning goals.
• Number of student hours spent on homework
• Number of student hours spent on service learning
• Number of student hours spent on cultural or intellectual activities related to learning outcomes
• Entry and exit student surveys At the course level, course learning outcomes should be listed on the syllabi, and the course should be structured so that there are multiple opportunities for students to achieve the course outcomes.
Aren’t Course Grades Enough? Assessment tries to link student performance to specific learning outcomes.
Grades can be an excellent assessment tool, if the performance being graded is linked to a specific outcome.
Traditional course grades tend to provide a summary measure of students’ performance across many outcomes, which doesn’t provide the kind of specific feedback necessary to link student performance to improvement.
They can also include factors like attendance, participation, and test-taking skills. Course grades can provide insight, however, into a student’s understanding of the course content and can serve as an indirect method of assessment.
What about Course Evaluations? Course evaluations are not a direct measure of student learning because they focus more on student perceptions of the quality of teaching than on learning outcomes.
Some universities have modified their course evaluations to include questions that address student perceptions of learning as well. These kinds of questions would ask students how well they thought they achieved the learning goals of the course. An example of a revised course evaluation that does both is available at http://www.idea.ksu.edu/StudentRatings/ index.html.
Step 5: Develop the assessment plan Review Team Report Responses 2009 Page 39 Once the mission, learning outcomes and assessment methodologies have been developed, the assessment plan must be completed. See Appendix 6 for a template for an assessment plan at the program level. Program assessment coordinators should use this template to develop their plans and reports or create a text document that provides the same information in a similar format, e.g. assessment measures and benchmarks should be listed for each outcome, along with results and action plans for each outcome.
This template can also be helpful for faculty planning assessment at the course level. Remember, not all outcomes need to be assessed – only those that are the most important. More than two and less than eight is generally a manageable number. In addition, not all outcomes must be assessed each year. Departments and programs can schedule assessment of outcomes over several years, if needed.
Before starting your plan, consider the following:
1. Are your learning outcomes well-stated? Are they measurable? Do they focus on outcomes rather than the process? Are they tied to AUK’s institutional learning outcomes?
2. Are all of your outcomes being taught? Are they taught in a sensible sequence?
3. Are different sections of the same course sharing the same outcomes? While course content and teaching methods can differ, it often helps to ensure that all sections of the same course share the same learning goals.
4. When and how often will assessment information be collected and shared? With whom will it be shared?
5. How will you use the information? How will it be used to inform the department’s decision making? How will it affect course content and sequencing, testing, availability of labs and library resources, faculty-student interaction, course staffing, class size, student advising, and more?
Step 6: Carry out the assessment Once the plan is developed and submitted, the assessment process needs to be implemented. Remember, for program assessment, the goal is to assess program-level outcomes, not to evaluate individual students or faculty members. The assessment coordinator, or chair of an assessment committee, will manage the program’s assessment process and will create a detailed timeline for the assessment cycle. The timeline might include dates for when work will be collected, when results will be tabulated and analyzed across the program, and when faculty will meet to discuss the results of the process and recommend changes.
Items to consider include which courses and learning experiences are better suited for assessment, timelines and schedules, whether all students should be assessed or only a sample, and how to best protect the confidentiality of the students being assessed.
Step 7: Collect, analyze, communicate, and report on your findings After assessment information is collected, the results need to be analyzed and communicated in useful ways to the faculty, who can consider changes to teaching methods, the curriculum, resource availability and scheduling, course content, and other factors.
At the end of the year, faculty members should complete an assessment report, similar in format to the plan, stating each course’s learning outcomes, assessment tools used, results of the assessment, and how the results were used to make changes to help students and improve learning. A template for the report is included in the appendix.
The program’s assessment coordinator should collect and tabulate results across the program and/or department and report that information back to the department or program faculty. The program’s assessment coordinator should share the department/program’s overall report with the Director and Coordinators, which will provide timely feedback and comments. Departments and programs are encouraged to share their results with all stakeholders.
Review Team Report Responses 2009 Page 40 Assessment results should be used in preparation of departmental budgets and changes to the long-range plans.
The results should also be used to review and adjust the department’s assessment plans, to improve student learning.