Quality Enhancement Plan
D. Assessment: Strategy, Protocols, and Instruments
Cognitive Learning Objectives
By the completion of any QEP course, undergraduate students will be expected to achieve the three primary goals defined below:
Goal #1: Students will have acquired rigorous, discipline-specific inquiry skills. They will be able to:
- Define a research or design problem.
- Design an inquiry method that takes the researcher from problem to solution.
- Demonstrate basic knowledge of and facility with a repertoire of appropriate research or design techniques and instruments.
- Identify, access, and evaluate the validity of appropriate evidence and scholarship in their field of inquiry.
- Work as self-directed researchers.
Goal #2: Students will be able to apply theories to, or construct models for, solving real-world problems. Specifically, they will be able to:
- Identify relevant theories, principles, laws, or models for application to specific problems.
- Recognize how a specific problem relates to broader forces and/or larger questions.
- Synthesize information from multiple disciplines for application to specified problems independent of the academic setting.
- Identify and distinguish more practical or realistic solutions from other less feasible alternatives.
- Critically analyze existing theory and/or models in light of their research/design experience.
Goal #3: Students will acquire enhanced ability to interact with, and present their work effectively to, audiences beyond the academic community. They will be able to:
- Collaborate with non-academic partners to define and study a problem.
- Identify appropriate audiences and methods of communication for formal presentations and informal interactions.
- Demonstrate effective presentation skills and understanding of appropriate forms of dissemination for different audiences.
- Solicit and respond appropriately to feedback from audiences within and beyond the academic community.
Experiential Learning Objectives
Undergraduates' participation in the Civic Inquiry and Civic Experience Programs will also contribute to the achievement of the following objectives:
Goal #4: Upon graduation, undergraduates will consider a vital connection to urban Houston to be a distinctive feature of their Rice education. Students will:
- Develop a lasting sense of public responsibility and an enhanced commitment to diverse forms of civic engagement.
- Explore and better appreciate the diverse artistic and cultural offerings of urban Houston.
Goal #5: Undergraduates will better understand the roles that larger communities play in their education and life after graduation. Students will be able to:
- Describe how their understanding of complex social problems was enriched by their community experiences.
- Recognize the qualities that prepare them for citizenship in multicultural, urban communities.
- Articulate how community experiences clarified their career and educational goals.
- Continue their civic involvement after graduation through service to their home communities.
- Value and respect non-academic skills and talents.
Community and Institutional Objectives
While principally concerned with creating meaningful learning opportunities for our undergraduates, Rice University's decision to develop a QEP centered on community-based inquiry is also mindful of an institutional commitment to serve and learn from the Houston community. As President Leebron said at his inauguration, "Rice and Houston can achieve great things in the years ahead, but neither can do it without the other. We are joined together, great research university and great city, in a symbiotic relationship." 38 So, while we will look to community partners to provide learning opportunities for our students, we will look also to respond to the needs of our neighbors and partners and to teach our students the values of service and civic engagement.
Goal #6: The QEP will develop a culture of civic engagement across the Rice academic community. The program will:
- Foster discussion of community issues in Rice classrooms and residential colleges.
- Support and increase participation in undergraduate volunteer organizations that assist with the problems addressed by civic organizations in Houston.
- Increase the number of undergraduate alumni who choose to live and work in Houston.
Goal #7: The QEP will leverage Rice University's intellectual capital for the benefit of our city, our local economy, and our quality of life. The program will:
- Foster and develop enduring relationships between the university and community organizations and between individual faculty members and community organizations.
- Encourage Rice undergraduates to take the lead in working with fellow Houstonians to identify and implement solutions to the social ills that afflict our community.
Based on these seven goals, the success of the QEP will be measured in five ways:
- The extent to which enhancements in student learning have been demonstrated.
- The extent to which QEP programs foster an enhanced culture of civic engagement on campus.
- The increase in the number of undergraduates participating in civic engagement programs and community-based research and design.
- The expressed satisfaction by community partners with the contributions made to partners' organizations and goals by QEP programs.
- The lasting benefits, in terms of skills, knowledge and attitudes, from participation in community-based research and design as reported by alumni of the QEP.
Assessment Instruments: The above processes identified five types of assessment efforts (see tables 3a-c):
- Cognitive Learning Objectives will be assessed via an instrument to be developed at Rice-the Measurement of Student Learning Indicators (MSLI). The MSLI will be used to measure student cognitive and experiential outcomes. MSLI will be administered to all students in QEP Courses and to a Comparative Control Cohort (CCC) of students for each graduating class (2007-2011).
- Experiential Learning Objectives will be assessed primarily by administration of the National Survey of Student Engagement (NSSE). NSSE offers an extremely well-tested and reliable method of assessing student engagement, which research has shown is an important predictor of student learning and personal development. 39 More than 850 different four-year colleges and universities have administered NSSE at least once since 2000. NSSE will be administered in year two (2007-08) and year four (2009-2010) of the QEP to measure supplemental outcomes, such as student engagement and campus climate issues. 40
- Rice will conduct assessment surveys of alumni, matching a comparison group of students who have not participated in QEP research projects to a group of QEP participants. These surveys will allow Rice to assess how alumni value community-based research and design experiences in comparison to other out-of classroom educational experiences, and whether alumni with undergraduate experience in community-based research and design "perceive themselves to have attained higher levels of important general cognitive and personal abilities and skills than comparable alumni" with no community-based research and design experience. 41
- Assessment instruments will be developed to allow community partners to provide feedback on the contributions made by faculty and staff to their programs. These instruments will be administered at the conclusion of each semester.
- Data will be collected on student participation in co-curricular civic engagement activities and programs, such as those sponsored by the Community Involvement Center, Passport to Houston, and Office of Student Activities. It is our expectation that the number of undergraduates participating in community-based research and design projects, and community-service programs will increase, as measured by NSSE, in comparison to 2001 baseline data.
Collection, Dissemination, Evaluation, and Use of Assessment Data:
- Annual: The administration of the MSLI will be coordinated at the end of each semester by the Center and disseminated to the Faculty Advisory Board for evaluation. The Faculty Advisory Board will meet in February and submit recommendations for academic program adjustments to the Dean of Undergraduates no later than March 10.
- Annual: Collection of data on student participation in co-curricular programs will be coordinated by the Community Involvement Center (CIC) in April.
- Annual: The Dean of Undergraduates will convene the QEP Implementation Team (Associate Dean of Undergraduates, Executive Director of the Center for Civic Engagement, Director of the CIC, etc.) to evaluate data and determine program adjustments.
- Annual: Beginning in year three of the QEP (2008-2009), Rice will administer a survey to all alumni who have participated in QEP programs. Each QEP alumnus will be "matched as closely as possible with two alumni who shared the same academic major, year of graduation, and cumulative grade point average" but did not participate in a QEP research and design project. We also anticipate a sub-cohort, comparison group of students who did not participate in QEP projects but who participated in campus-based research and design projects. 42
- Annual: Student participation data will be collected by the Center.
- Years Two and Four of the QEP: NSSE will be administered.
- 2009-2014: Annual administration of the Career Services Center's Post-Graduate Survey will be conducted.
Dissemination and Use
The following groups will receive the results of the QEP assessment:
- The faculty teaching the QEP courses will receive the results of the MSLI and the evaluations of student projects from the community partners for students enrolled in their classes. Faculty will use these results for the improvement of existing, and the design of new, QEP courses.
- The Faculty Advisory Board will receive the results of the MSLI reported in aggregate and the evaluations from the community partners for all QEP projects. The Board will meet annually to review the effectiveness of the overall program design and will make recommendations to the Director of the Center for Civic Engagement about program changes and enhancements.
- The Director of the Center for Civic Engagement will receive the results of the NSSE and reports on student participation in co-curricular civic engagement activities and programs, such as those sponsored by the Community Involvement Center, Passport to Houston, and Office of Student Activities. The Director and the Dean of Undergraduates will review the coordination of co-curricular activities and the efforts to increase student participation in these activities, and the Dean will implement program changes and enhancements.
- The Director of the Center will receive the results of all of the assessment activities and the reports from the Faculty Advisory Board and the Campus Community Engagement Committee. This information will be used to evaluate issues affecting the relationship of the QEP and the general undergraduate curriculum.
- The Director of the Center will provide an annual report to the Dean of Undergraduates, summarizing the effectiveness of the courses and co-curricular activities in satisfying the QEP learning objectives and making recommendations for program enhancements.
- "The Inauguration Address of David W. Leebron," David W. Leebron, October 2, 2004, available at http://www.professor.rice.edu/professor/Inauguration_Speech_2004.asp .
- George D. Kuh, "What We're Learning from NSSE: Benchmarks for Effective Educational Practices," Change (March/April 2003); R. Carini, G. Kuh, and S. Klein, "Student Engagement and Student Learning: Testing the Linkages" (2004), available at http://nsse.iub.edu/index.cfm .
- National Survey of Student Engagement Viewpoint, November 2004. John Hayek, "National Survey of Student Engagement: Student Engagement in Learning," Presentation at SACS Institute on Quality Enhancement and Accreditation, Orlando, FL, July 25, 2005.
- The alumni assessment will include the Post-Graduate Survey, used by the Rice Office of Career Services, and an instrument based on a model used successfully by the University of Delaware, which demonstrated that "alumni judgment of undergraduate experience can be an accurate measure of undergraduate achievement." K. Bauer and J. Bennett, "Alumni Perceptions Used to Assess Undergraduate Research Experience," Journal of Higher Education 74, no. 2 (2003): 213-214.
- Bauer and Bennett (2003): 216.