ASSESSMENT AND TESTING: MEASURING UP TO EXPECTATIONS

A Digest from the

ERIC Clearinghouse for Commuity Colleges.

by Christine Cress

Contents
External Pressures and Challenges
Frameworks for Assessment
Techniques and Methodology
Diversity Considerations
Garnering Institutional Support for Assessment
Conclusion
References

Historically, community colleges have not embraced research as a primary mission and function. Many institutions utilize assessment measures to ensure accurate student placement into courses and to self-monitor realization of educational goals. However, with increasing pressure from external and internal constituencies (accreditation bodies, legislators, taxpayers, parents) community colleges have been called upon to "prove" their efficiency and effectiveness. In essence, assessment has become synonymous with accountability (McMillan, 1994).

This digest will examine external pressures on and challenges to assessment practices, frameworks and techniques for assessment, considerations in working with diverse populations, and the importance of garnering institutional support for assessment.

EXTERNAL PRESSURES AND CHALLENGES

Spurred by the Student Right to Know Act, state coordinating and governing boards have reacted to pressure from legislatures, business and the public to report on the outcomes of full-time students. Inquiries include: how many students graduate from the community college with an associate degree and in what time frame; how many students transfer to four year institutions, how many graduates with associate degrees or vocational certificates get jobs; and how much does it cost to educate students? (McMillan, 1994).

Community college environments include transient student populations, wide ranges of student ability, large numbers of adjunct faculty, and disparate academic goals among students (Mittler & Bers, 1994a). Therefore, cross-state or cross-national comparisons of graduation rate or semester to semester retention may not take into account the unique mission an individual college plays in its local community (Seybert, 1994). The fundamental purpose of assessment should be improvement of campus instructional and support programs thereby increasing the prospects of individual student success. Otherwise, "At what point does a college spend more resources responding to exterior demands than improving or even practicing its teaching and learning role?" (McMillan, 1994, p.8).

FRAMEWORKS FOR ASSESSMENT

Whether the assessment of outcomes is internally or externally motivated, the importance of determining who will assess the information and how it relates to student learning and instruction is critical.

Mittler & Bers (1994a) suggest that institutions consider the following questions in the who, what, when and how of assessment:

1) Who should be assessed? If a student attended more than one institution, which college, even theoretically, could be expected to have most affected the student's achievements?

2) What should be assessed? Competency in basic skills such as writing and math, or overarching achievements such as good citizenship and critical thinking?

3) What is the timeframe? During what time period are assessments rendered moot because of time elapsed between the educational experience and the assessment activity?

4) How often can institutions reasonably ask students to participate in assessment?

5) Is collaboration between institutions possible in order to eliminate redundancy?

TECHNIQUES AND METHODLOGY

In selecting appropriate assessment methods, investigators may want to refer to research literature and examine models at other institutions. There are a wide range of options including competency-based models, self-reports and third-party reports. The relative advantages and disadvantages of each should be carefully evaluated. Prus and Johnson (1994) suggest pilot-testing techniques in an educated trial and error method before proceeding to full-scale implementation.

A challenge that institutions often face is identifying the type and amount of assessment information that exists outside the college's office of institutional assessment. By engaging in an "assessment audit" (a compilation of what sorts of information about students is gathered at the institution) duplicative or potentially complementary assessments existing in isolation can be compiled and an understanding gained of how information is collected, analyzed, stored, and used (Mittler & Bers, 1994b). This exercise may also be valuable in establishing baselines of information about students. By knowing the initial characteristics of students, institutions are better able to assess the outcomes.

The usefulness of techniques may vary depending on the issues being evaluated. Qualitative forms of assessment should be incorporated as a way to complement and sometimes challenge interpretations of quantitative data. Methods include focus groups, in-depth interviews, participant observations and case studies. Although the information can be difficult to generalize to populations, the goal is to listen and watch for factors that influence outcomes. This technique directly involves community college members in the process instead of relying solely on the "truthfulness" of numbers. It allows individuals to share in their own words how they perceive their environment and what areas they consider effective or ineffective (Mittler & Bers, 1994b).

Oakton Community College (OCC) in Des Plaines, Illinois utilizes exit interviews and alumni surveys as a form of assessment for programs and services. They recently asked their students, "Where do you go to find help?" Like most colleges they spend significant amounts of time and money producing catalogs, brochures and other literature in the hope of answering students' questions and informing them about what they need to know to succeed. Their students indicated that they got college information not from the printed materials, but rather from friends, faculty, counselors and librarians. Basically, they receive most of their information from people. As a result, OCC placed a renewed emphasis on communicating with students by having academic advisers roam registration lines, increasing the availability of faculty advisers, and having identifiable staff available to answer student questions during the start of each quarter (Mittler & Bers, 1994b).

DIVERSITY CONSIDERATIONS

Community college student populations are characterized by nontraditional attendance and matriculation patterns, part-time and returning students, and those attending for upgrading of technical or vocational skills. Students with learning disabilities and students whose native language is not English are often highly represented (Seybert, 1994). Assessment activities need to take into consideration the needs of culturally, ethnically, religiously, and linguistically diverse students. When conducting campus wide assessments researchers must look for variations in outcomes on the basis of students' different educational backgrounds as well as differences in ethnic or cultural backgrounds. Not all students learn in the same manner. Not all students enter college with the same levels of preparation. Kerlin and Britz (1994) recommend that campuses seek information that informs them about how students differ in areas such as retention, graduation rates, and transfer to four year institutions.

North Seattle Community College conducted a campus-wide multicultural climate study and systemwide evaluation on the performance of its diverse students. The assessment not only focused on measurements of difference, but also on the changing environments which influence outcomes. Data and anecdotal information were presented to the faculty, resulting in changes made at the course, department and institutional level. Specifically, the division of social sciences reviewed all curricula and hired a new faculty member specializing in multicultural issues. Training seminars and workshops on multiculturalism were also offered to assist with the redesign and integration of curriculum.

Done effectively, multicultural assessments can heighten the awareness of and sensitivity to cultural diversity. But because it elevates issues of race and ethnicity, this type of assessment can also serve to heighten tensions. Before engaging in assessments, researchers might ask themselves, what is the campus wide commitment to diversity? Are there pockets of resistance? What resources is the institution willing to devote to assessment of diverse students and what is the willingness of its members to try new techniques or programs? (Kerlin & Britz, 1994).

GARNERING INSTITUTIONAL SUPPORT FOR ASSESSMENT

Various approaches, such as providing pizza or handing out baseball caps, have been used to encourage students' participation in assessment activities. If students as well as faculty and staff are to participate in and take assessment seriously, extrinsic rewards are rarely enough. Assessment becomes an integral part of the institution when this focus is clearly stated in the mission of the college and emphasized as a part of insuring student success (Duvall, 1994). Barriers to assessment processes often are faculty resistance, finding ways to motivate students to put forth their best efforts, and lack of resources (Seybert, 1994). Students, faculty and staff can take pride in their institution if they know that excellence is planned and quality is measured (Duvall, 1994).

CONCLUSION

Assessment activities engaged in by institutions have increased steadily to meet the needs and demands of constituencies. The best student outcomes assessment processes and methodologies are of little value unless the results are used to improve the curriculum and teaching process. Rather than searching for a single indicator to demonstrate success, institutions can foster climates that value the use of many different benchmarks as evidence of institutional effectiveness, thereby, assuring the public and themselves that students are being well-served by higher education.

REFERENCES

This digest is drawn from "Assessment and Testing: Myths and Realities," New Directions for Community Colleges, Number 88, edited by Trudy H. Bers and Mary L. Mittler; published in Winter, 1994. The cited articles include: "Assessment from the State Perspective," by Virginia K. McMillan; "Assessment from a National Perspective: Where are We, Really?" by Jeffrey A. Seybert; "Assessment and Transfer: Unexamined Complexities," by Mary L. Mittler and Trudy H. Bers (a); "Obtaining Student Cooperation for Assessment," by Betty Duvall; "Assessment and Diversity: Outcome and Climate Measurements," by Scott P. Kerlin and Patricia B. Britz; "Qualitative Assessment: An Institutional Reality Check," by Mary L. Mittler and Trudy H. Bers (b); "A Critical Review of Student Assessment Options," by Joseph Prus and Reid Johnson.

Return to the Clearinghouse's Home Page.

updated

Citations preceded by an asterisk (*) refer to journal articles or other material unavailable from EDRS. Journal articles may be acquired through regular library channels, or purchased from article clearinghouses such as CARL Uncover at (800) 787-7979. Other material may be available either directly through the publishing agency, or from out-of-print vendors. ERIC documents (references with "ED" numbers) may be read on microfiche at approximately 900 libraries worldwide. To identify your local ERIC Resource Collection, connect to: http://www.ed.gov/BASISDB/EROD/eric/SF. Most documents may be ordered on microfiche or in paper copy from the ERIC Document Reproduction Service (EDRS) at (800) 443-ERIC. For more information about our products and services, please contact the ERIC Clearinghouse for Community Colleges by telephone at (800) 832-8256, or via the Internet at ericcc@ucla.edu.

Please help us keep the ERIC database current and comprehensive by submitting your written works to the Clearinghouse for review for inclusion in the ERIC system. Send two copies of each manuscript to the attention of our Acquisitions Coordinator.

ERIC Clearinghouse for Community Colleges
University of California, Los Angeles
3051 Moore Hall
P.O. Box 951521
Los Angeles, CA 90095

Updated 10/10/01
Maintained by ericccus@gseis.ucla.edu