CACREP Self Study Program Evaluation
During the 2000 calendar year, as part of its CACREP Self-Study effort, the IU-IUPUI Counseling and Counselor Education Program conducted an evaluation on the Bloomington campus. This effort replicated in many ways the Counseling and Counselor Education Program Evaluation that was implemented on the IUPUI campus during the 1998-1999 academic year.
Four groups of stakeholders were surveyed: recent graduates, current students, site-based practicum and internship supervisors, and employers of our recent graduates. Each group was asked to respond to a different albeit related survey form. With respect to content, the survey forms were designed to address six questions that relate to the program generally and to the curriculum in particular: a) the amount and quality of preparation in counseling skill areas; b) the structure and delivery of the curriculum; c) program advising and support; d) the quality and efficiency of the practicum and internship experiences; e) the infrastructure within which the program operates; and f) program modifications needed to better meet the needs of various stakeholders as well as requirements of Indiana state licensing and other accrediting bodies.
Survey forms varied in length with the recent graduate and current student forms being the longest and the most
comprehensive. Respondents were asked to indicate on six point response scales their levels of satisfaction (6: very satisfied - 1: very dissatisfied) with the counseling/counselor education program in the areas being evaluated. All four survey forms also included an open invitation “to provide impressions on the major strengths and weaknesses of the Counseling and Counselor Education Program on the Bloomington campus.”
Survey return rates were mixed with stakeholder samples ranging between 17 (employers of recent graduates) to 44 (recent graduates). With that limitation in mind, we are limiting the present discussion to a reporting of findings generated from the Likert type items that appear in the four different survey forms. Despite the small return rates, some trends in the data appear to be developing and warrant discussion here. These trends are noted for the separate respondent groups and then presented again in a summary statement. The discussion concludes with an enumeration of recommendations for improvement.
Return to Top of the Document
Findings
Survey of Recent Graduates
Surveys were returned by 44 recent graduates of the Counseling and Counselor Education Program. With respect to mean satisfaction values for the amount and quality of preparation in counseling skill areas, recent graduates reported greatest satisfaction with the preparation they had received in their counseling practicum and internship field experiences, individual counseling, ethics, and group counseling. Least satisfaction was associated with amount (i.e., frequency of offering) appraisal/assessment, marriage and family counseling, and career counseling.
With respect to the structure and delivery of the curriculum, 64% of graduates reported being either very satisfied or generally satisfied, while fewer than 6% reported being very or generally dissatisfied. Graduates also reported high satisfaction for the program’s descriptive materials and relative low satisfaction for the counseling students association. Over all, 53% of the respondents to the recent graduates’ survey reported that they were very or generally satisfied with program advising and support services while 13% indicated they were very or generally dissatisfied.
Evaluations for the items categorized under the practicum and internship program component were mixed. With respect to over-all satisfaction with sites, over-all satisfaction with practicum program, and over-all satisfaction with internship program, 61% of the recent graduate sample reported that they were very satisfied or generally satisfied over-all while fewer than 11% indicated they were very or generally dissatisfied. Dissatisfaction is associated primarily with placement procedures, availability of information on prospective sites and assistance in obtaining a site.
In summary, program graduates generally provide quite positive evaluations regarding the amount and quality of preparation in the counseling skill areas surveyed. The curriculum is seen as superior to what colleagues in their own work setting report they experienced in their graduate programs. Graduates are especially complimentary regarding the curriculum in individual counseling, group counseling, and the infusion of ethics education throughout the curriculum. However, the graduates’ evaluations point to room for improvement in several areas. Current students are somewhat less positive over-all. These concerns are separately discussed in a later section of this report.
Return to Top of the Document
Survey of Current Students
Surveys were distributed to 40 current counseling students; 34 surveys were returned. In general, current students (1999-20000 cohort) were less positive over-all in their evaluations as compared to recent graduates. From their perspective, the strongest areas of the curriculum, both in terms of amount and quality, are the program’s field experiences, i.e., counseling practicum and counseling internship, ethics training, individual counseling, and group counseling. Least positive evaluations were given to marriage and family counseling (amount), research skills, and career counseling.
With respect to the structure and delivery of the program’s curriculum, the two areas receiving the lowest ratings (availability each semester of required courses and courses on specific topics) both relate to the availability of course offerings, a problem brought on by recently enforced requirements associated with Indiana state licensure legislation and the program’s effort to attain compliance with CACREP accreditation standards for three separate specializations, school, community, and clinical mental health. The program advising and support services component of the survey elicited their most negative evaluations.
Current student evaluations of the practicum and internship program component were mixed. Over-all satisfaction with practicum sites, quality of supervision by site supervisors, and over-all satisfaction with the internship program received the most positive evaluations. The least positive evaluations were associated with assistance in obtaining a site, availability of information on prospective sites, on-site monitoring by faculty, and over-all satisfaction with practicum placement procedures.
For the most part the student group that completed our survey has now graduated. At the time they completed the survey the program was in the throws of change following an extended period of stability and bliss. In many respects, change was imposed from the outside in the form of pending legislation relating to Indiana counseling licensure and growing interest in CACREP accreditation. Members of this particular cohort (1999-2001) and the two that preceded it (1997-1999 and 1998-2000) were the most seriously impacted by the transformations that were underway. The negative impact of these transformations is reflected most directly in the contrasting evaluations by the student sample and by the remaining three stakeholder groups who are more glowing in their evaluations. It now looks like the current cohort (i.e., cohort 2001-2003) is strategically situated to be the first cohort to realize the positive gains and benefits that derive from the program transformations that have been under way over the past several years.
Before moving on we need to acknowledge the considerable contributions made by the current student survey group. The fact remains, some of the problems and areas of need would not have surfaced if we had limited ourselves to the survey data generated from recent graduates, practicum and internship on-site supervisors, and employers of recent graduates. Most importantly to note is the fact that current student data supports all renewal decisions we as a faculty have made as a result of the program’s self-study.
Return to Top of the Document
Survey of Practicum and Internship Site Supervisors
A total of 19 surveys were returned by on-site practicum and internship supervisors. Despite the small sample size, some trends in the data warrant mention. On-site supervisors appear to be quite satisfied with respect to the availability and usefulness of the printed materials they receive regarding the program’s expectations and the procedures to be followed in the practicum and internship experiences. Practicum and internship site supervisors as a group also give quite favorable ratings with respect to student preparedness to begin the field experience component of counselor training. Most favorable preparedness ratings were given in the areas of professional ethics, individual counseling, group counseling, and research skills. Preparedness in the area of career counseling was rated least favorably.
In summary, on-site supervisors of our practicum students and interns are generally positive in their evaluations. They consider our preparation program to be highly effective in developing in our students the counseling skills that are required to be successful in the field experience component of the program. The over-all mean rating given by site supervisors was the second highest among the four stakeholder groups surveyed. Only the employers of recent graduates group generated a higher over-all mean assessment of preparedness and satisfaction.
The negative feedback centers around the lack of on-going communication between supervising faculty and on-site supervisors. They point specifically to the need for faculty supervisors to have a stronger presence at placement site. Looking at these data and the graduate survey data reported earlier, it is clear that graduates and on-sight supervisors share this perception and propose similar recommendations for improvement.
Return to Top of the Document
Survey of Employers of Recent Graduates
,p>Employers of recent graduates as a group generated the most positive over-all evaluation. Over ninety-one percent (91%) of the employer responses were either a 5 (generally satisfied) or a 6 (very satisfied). Because the evaluations were so positive, differentiation among items is quite restricted. Mean evaluations within the professionalism domain ranged from a perfect 6.00 (ethical in their work) to a low of 5.65 (versatility). In between we have mean ratings of 5.76 (they are reliable), 5.71 (they are professional), and 5.70 “getting along with co-workers”. With respect to the four counseling skill areas, mean evaluations ranged between 5.73 (group counseling) to 5.13 (career counseling) with individual counseling and assessment and diagnosis situated in between. The final item, an over-all rating by employers, generated a mean of 5.30 with ninety-three percent (93%) of employers of recent graduates giving ratings of very competent.
Return to Top of the Document
Recommendations for Improvement
Amount And Quality of Preparation in Counseling Skill Areas
Major modifications are needed in the career development curriculum with more emphasis given to practical applications of theories of career counseling and career development in K-12 schools and agency settings. We have addressed this deficiency through the addition this year of a senior faculty member with a national reputation in the area of career development and career counseling.
Additional preparation in school counseling methods and issues is needed. These efforts should focus on helping students better understand and function more effectively in the K-12 “school culture” starting with their school practicum placement. This need has become most apparent on the Bloomington campus because increasing numbers of our school track students matriculate into our program directly out of undergraduate programs other than teacher education. In many cases their singular experience with K-12 environments is limited to their own experience as students in those contexts. This deficiency has been addressed by requiring that all students specializing in school counseling enroll in G562 School Counseling: Intervention, Consultation, and Program Development and G542 Organization and Development of School Counseling Programs.
Survey data also point to the need for expanded pre-service preparation opportunities in the area of marriage and family counseling. This applies particularly to students whose internship placements and future employment require that they work with couples and families. This will be a continuing challenge because the marriage and family courses currently fall in the
elective category and students’ programs of study are already over-prescribed.
Return to Top of the Document
Structure and Delivery of the Curriculum
With respect to structure and delivery of the curriculum, graduates and current students are equally positive in their evaluations. Program faculty are regarded as competent, committed, and caring. They, along with adjunct faculty, are seen as excellent instructors, supervisors, and role models. Here too, however, there is room for improvement. One strong recommendation that surfaces has to do with making all required courses available at least two times a year.
Return to Top of the Document
Program Advising and Support
Program advising receives mixed evaluations from graduates of the program and current students. There is a need for faculty to make more time available to work closely with students’ on their individual needs, especially in the areas such as career guidance and job placement. There also is a need for greater accuracy and consistency in the information the program disseminates through its printed and electronic information. This latter deficiency has been addressed through widespread revisions in both the printed and electronic information the program makes available to its constituents. The revised Student Handbook is a good example of an effort that was guided by feedback derived from this survey. These improvements have freed up considerable faculty advising time for working more with advisees on an individual needs basis.
Return to Top of the Document
Quality and Efficiency of Practicum and Internship Experiences
Evaluations by program graduates and current students clearly indicate that they see practicum and internship experiences as the crown jewel in the curriculum. Placement sites and on-site supervision are perceived to be of high quality and this translates into successful job searches and a readiness to perform with professionalism and accountability right from the outset of their transition to full time employment. However, the evaluations also point to several areas needing improvement. Most importantly, the placement process needs to operate more smoothly. Also, there needs to be greater involvement by campus faculty in the on-site activities of practicum students and interns. We have addressed the former need through revisions in our placement procedures, including the creation of a graduate student assistantship that provides logistical support for the faculty member who coordinates the entire placement operation. The second need is being addressed through more careful written articulation of the program’s expectations and requirements with respect to campus-field experience communications and personal contact with our community partners. Included among these expectations is regular contact between faculty and site supervisors and through more regular site visits, telephone conferencing, and email contacts.
Return to Top of the Document
Facilities and Physical Properties of Infrastructure
The physical properties of the infrastructure within which the program operates are perceived by our graduates to be among the best in the country. They single out the high tech features that are everywhere present in the building, including the Department’s clinical training clinics, the Center for Human Growth and the Institute for Child Study. Although these positives are attributed to both campuses, the Bloomington campus stands out due to the fact that it is one of the “most wired” campuses in the country.
Return to Top of the Document
Acknowledgements
Appreciation is extended members of the various stakeholder groups who completed and returned surveys. You obviously took the task seriously. In keeping with CACREP requirements, we will be conducting similar surveys on a regular basis and we hope that stakeholder groups surveyed in the future will be equally generous in their forthrightness and commitment to assist. Again, we thank each of you.
Return to Top of the Document