Contents

Abstract

Program evaluations are routinely conducted in most educational institutions. This article reports the use of focus groups in the evaluation of a program of school library education at the University of Alberta, Edmonton, Canada. The program evaluation began with a self-study conducted by the program coordinator. In order to gain information from the groups served by the program, focus group interviews were organized by a ministry of education consultant. From the analysis of the focus group interview data, six themes emerged: taking a leadership role, developing the school library program, managing the program, coping with technology, sequencing of learning, and modelling and mentoring. The themes were found to be generally consistent with the content and pedagogy of the current program. The focus group methodology, in addition to providing valuable evaluation data, was found to be an effective means of communicating with practitioners in the field about the program.

1. Introduction

Courses in school library education have been offered at the University of Alberta in Edmonton, Canada for over 40 years. Through the years, the program has been periodically revised in response to philosophical and technical changes in the field of school librarianship. This article reports one aspect of an extensive and in-depth program evaluation planned and carried out by the coordinator for the diploma program in school library education working with the library consultant from the provincial ministry of education. Data for the program evaluation were collected using a variety of strategies. The evaluation process began with a self-study. Course outlines, reading lists, assignments and examinations were analyzed to identify program content and requirements. Then the elements of the program were assessed against the competencies identified in the national standards. Student course evaluation data from the previous five years, which included both numeric ratings and written comments, were reviewed for areas of student concern. Finally, focus group interviews were used to get the opinions of a representative sample of the groups served by the program. Previous to the program evaluation reported in this article, educators in schools and school districts in the geographic area served by the program had not been routinely consulted as part of program revision.

2. Literature Review

2.1 Evaluation of School Library Education Programs

Focus groups may have been used elsewhere in the evaluation of programs of school library education but no reference to this was found in the literature. Some of the other methods used to evaluate programs of school library education reported in the literature included matching program elements against lists of criteria or competencies, eliciting graduates' perceptions of the program, and measuring the performance of graduates.

Criterion-based methods involve measuring program elements against a list of standards defined by expert groups. A number of professional associations such as IFLA (the International Federation of Library Associations and Institutions), CSLA (the Canadian School Library Association), and AASL/NCATE (the American Association of School Librarians and the National Council for the Accreditation of Teacher Education) have developed statements of competencies for teacher-librarians and curriculum guidelines for programs of school library education. [1].

Surveying graduates to determine their satisfaction with the program is another evaluation method frequently used. For example, program evaluation studies by Taylor, Dabiri, Azarmza, and Scheeren [2] included collection of data through questionnaires mailed to program graduates. In studies by Ball, Cepek, and Hough [3] school media specialists were surveyed to determine the extent to which their professional education prepared them to manage school library programs.

The performance of graduates is an outcome-based method for program evaluation. Correll [4] reported an evaluation of programs in which the immediate supervisors of recent graduates were interviewed by telephone. Supervisors were asked to rate graduates in five competency areas.

Evaluations often involve collecting data from a variety of stakeholder groups, using several data-gathering methods. Case et al [5] reported an evaluation of six programs that used data gathered from instructors, from students enrolled in the programs, from graduates, and from supervisors of graduates. A checklist and a structured interview questionnaire were used to gather the data.

Although no reference to the use of focus groups in evaluation of school library education was found in the literature, focus groups are increasingly being used in other areas of library and information work. They are being used by libraries seeking to develop or improve services and facilities, by vendors wanting to develop or improve information products, and by researchers investigating information use.

The following discussion is provided for readers who may be unfamiliar with focus groups and their uses. Focus groups are used to collect qualitative data. The purpose of a focus group is to develop an understanding of the participants' experiences or views about an activity, a program, or a product or service. The purpose of a focus group is not to reach consensus, provide recommendations, or make decisions. Typically a focus group is composed of 8 to 10 people who do not know each other. The participants are selected because they have certain characteristics or experiences in common. Participants are asked to share ideas and perceptions about a particular area of interest in a relaxed, comfortable atmosphere. The discussion is carefully planned around a few open-ended questions.

The focus group interviewer or moderator must be skilled at setting up a non-threatening environment which promotes self-disclosures. He or she should not be in a position of power over the group. The moderator plays an unobtrusive role, encouraging comments, both positive and negative, and being careful not to make judgmental comments. At the opening the moderator ensures that participants understand that they are expected to express opinions and feelings and that alternative explanations are expected. Participants are assured that there are no right answers, only alternative points of view. The moderator promotes interaction and assures that the discussion remains on topic. The interview follows pre-determined questions arranged in a logical order [6].

In collecting data needed for program evaluation, focus groups have a number of advantages over surveys or individual interviews. Focus groups are a rich source of information because participants' comments build on the insights of other participants. A wider range of insight is provided because a comment by one person often triggers a chain of responses. Focus groups provide a quick and cost effective method of gaining a large amount of information.

3. Context of the Program Evaluated

University of Alberta, located in the provincial capital, Edmonton, is the largest of four universities in the province of Alberta and the only one providing a program of school library education. The school library courses are administratively located within the Department of Elementary Education, but these courses are available to all post baccalaureate students within the Faculty of Education. A specialization in school librarianship can be taken through a 24-credit diploma program, culminating in Diploma of Elementary or Secondary Education in School Libraries or a 48-credit master's program, leading to either a Master of Education or a Master of Library and Information Studies. The largest number of students completing school library studies do so through a diploma program.

The diploma program is designed to provide basic training in school librarianship and to prepare teacher-librarians to work at the school level. The diploma program typically consists of five core courses in school librarianship and three complementary courses chosen from children's literature, curriculum studies, or media and technology studies. Courses chosen depend upon an individual's background and needs. The five core courses currently offered are School Librarianship, School Library Programs, Organization of School Library Materials, School Library Information Materials, and School Library Collection Development.

The program was revised in the early 1980's to meet requirements of the Canadian School Library Association's Qualifications for School Librarians and Recommended Curriculum for Education for School Librarianship. After publication of Alberta standards for school libraries in 1985, the program was adjusted to reflect the philosophy of Focus on Learning: An Integrated Program Model for Alberta School Libraries [7].

The core courses in school library education are taught by one full-time faculty member and a number of part-time instructors. Most of the part-time instructors are currently employed as teacher-librarians or as supervisors or consultants in school districts. Core courses are offered during Fall, Winter, and Summer terms. Courses are also offered in Calgary, a city about 350 kilometers south of Edmonton, during Fall and Winter terms.

Students who take courses in school librarianship at the University of Alberta are generally experienced teachers or practicing teacher-librarians. The majority of these students from urban areas. To be employed in Alberta schools, teacher-librarians must have a current teaching certificate; formal library education is recommended by the ministry of education but not required.

4. Use of Focus Groups in the Program Evaluation

4.1 Purposes

Focus groups were held at four locations in the province of Alberta to provide for the geographically diverse areas served by the program. The groups were held in Edmonton, where the University of Alberta is located; in Fort McMurray, approximately 500 km to the north; in Calgary, 350 km to the south; and in Jasper, 450 km to the west of Edmonton. In order to minimize cost, groups were brought together in connection with a conference, an evaluation, and other regular meeting opportunities.

One of the evaluators, acting as moderator, began each focus group session by explaining the purpose of the group and how the information provided would be used. The interview questions led from consideration of the participants' own training and experience to their recommendations for training of new teacher-librarians. At the end of each focus group, participants were provided with information about the current program and given opportunities to ask questions and make additional comments. Each focus group interview lasted approximately 90 minutes. Discussions were recorded on audio tape and later reviewed by the two evaluators. The key concepts were categorized around six themes.

5. Themes

One of the most prominent themes raised by participants centered around the nature of the role of the teacher-librarian in the school. Developing a philosophy and plan for the school library program and articulating these to the principal and teachers requires leadership and collaborative skills. One participant, a district supervisor, stated:

[There should be a] real emphasis on leadership. What is leadership? People are often quite nervous about taking a leadership role and I say, 'You don't take it, you've already got it!

An understanding of the dynamics of the change process was cited as a valuable asset to the job. Many recognized that implementing a program of cooperative planning and teaching can involve changes in the culture of the school. A graduate of the program said:

The people issues are the ones that are so incredibly important in this role. The course that gave me a focus was the business of working with change. Our backgrounds did not equip us in any way to cope with, deal with, talk about, and understand the process of change. . . When you look at the research on how change takes place, you realize that the teacher-librarian is so much at the place in the school and the place in the learning process where change takes place. That person can have an impact on the change that takes place in the school, more so than most anyone.

The task of advocating for school library programs involves working with the adults in the school in a new relationship, akin to administration, but without the power of position.

The other thing . . . is we are not good at advocacy issues. We are not good at letting people know what this program is really about, without them thinking it's a vested interest. And so we need to have some strategies . . . to help us market the program, without making it sound like we are emphasizing it at the expense of something else.

Marketing [is] not just promotion and not just advocacy. Marketing within the whole program is essential and it has to start right at the beginning in the introduction to teacher-librarianship.

The self-confidence and personal knowledge required to supervise support staff, negotiate with administrators, and collaborate with teachers should be nurtured through the program.

I think it's important to have some kind of personnel management course, because, besides the administration, we are the only ones who are asked to manage secretaries and volunteers.

You need to understand where teachers are coming from. [You] can't negotiate if you don't have an appreciation for their point of view. You don't have to agree with their point of view.

The teacher-librarian must be a negotiator.

Collaborative skills are really helpful.

A person needs to have a lot of self-knowledge, because so much of the job involves working with other adults. . . something that would cause people to reflect back on the kinds of people they are and the needs they have in working with other adults would give them a real start.

5.2 Developing the School Library Program

As well as interpersonal communication skills, cooperative program planning and teaching requires curriculum knowledge and pedagogical expertise. Participants pointed out that teacher-librarians need to understand the philosophy and general content of each curriculum program, even if they are not specialists in the subject. Knowledge about the research process, learning styles, thinking skills, and strategies of cooperative learning were all cited as important skills.

The teacher-librarian has to be, first of all, an educator.

First of all they [teacher-librarians] need to know the ultimate goal and that what they are really there to do is to help students to learn.

The school library program is not just for students going on to university. The school library program is a program for everyone as they graduate to life.

Getting them [the teachers] to understand how kids learn and how they research is how 90% of my time is spent.

The philosophy of how learning takes place has been very valuable to me.

Even though they believed in the centrality of the research process in today's school library programs, most participants also expressed the need to be knowledgeable about children's and young adult's literature.

Because of the emphasis on integrated learning with literature, I feel that I need to have some background [in children's and young adult literature].

5.3 Managing the Program

Administrative tasks such as planning, budgeting, collection development, and evaluating the program were mentioned as important. In school jurisdictions where library clerks are assigned to handle clerical tasks, the in-depth knowledge of technical processes such as cataloguing and circulating materials have been de-emphasized. However, participants expressed the need to have at least basic awareness of a wide variety of technical tasks. They pointed out that the ability to question current procedures and implement time and cost saving alternatives is dependent upon understanding the reasons for standard library procedures. In addition to supervision of the technical tasks, many teacher-librarians are expected to provide training in management procedures when new library clerks are hired.

I think we would all benefit from looking at what the management tasks are and saying, 'Here are a variety of ways of doing them.' . . having some options . . .

I've just lost an assistant who was there for ten years and I . . . don't know a lot of the mundane, routine things of managing a library. It's a technical disaster!

Cataloguing was one area that I felt apprehensive about.

One of the things that I worried about most was how to spend that money responsibility. I didn't know what to buy. I didn't know what a reviewing aid was or which ones were reliable. Having that budget and spending it stupidly really bothered me.

5.4 Coping with Technology

New technologies were mentioned in all but one of the focus groups as a current concern in the field that must be addressed in continuing education as well as the school library education program. Participants were especially concerned that teacher-librarians be able to evaluate and implement library automation systems and new formats such as CD-ROM and videodiscs. The most recent graduates of the diploma program were the only ones to express confidence about their ability to use and evaluate the new technologies.

[Technology] is moving faster than any of our budgets will ever keep up to so you just have to step somewhere and decide what is reasonable.

Has technology become the new . . . fear for every teacher, not just some people in specific designations? The greatest fear, I hear, is the fear of technology and it's a real barrier and it's difficult.

The other side is the people who believe that technology will solve everything and so they want you to automate the library by next month and then everything will be okay.

We are perceived, rightly or wrongly, as being the information experts in our schools and also with that comes the expectation that we know how to hook up a CD-ROM and that we know some of the technical background of how to make the thing work. We are the only people in many schools that have some expertise in these areas.

The majority of the concerns related to the technical aspects of technology. However, some participants also emphasized the need for teacher-librarians to be aware of and responsive to the instructional challenges and opportunities involved. A few participants suggested that the program should address issues related to media literacy or the skills associated with evaluating the content and impact of non-print media. The focus should be, according to some, especially district supervisors and consultants and teacher-librarians working in automated environments, on helping teachers and student understand the impact of technology to maximize learning.

The technology is interesting, but I'm really concerned about the "random access to information." . . The kids are pulling out bits of information and I'm wondering if there are a lot of implications for their ability to skim and scan and synthesize and really integrate the information and understand how important or how irrelevant that bit of information is to the totality of what they are doing.

5.5 Sequencing of Learning

The participants commented on the differing learning needs of teacher-librarians through the various stages of their careers. Before entry to the field, teacher-librarians were concerned that they be able to cope with the technical and technological aspects of the job. Some were also concerned with the enormity of collection development tasks. Once into their jobs, issues related to involving teachers in the program and to balancing technical and instructional aspects began to be more important. Experienced teacher-librarians were more aware of the need to be updated frequently in relation to new technologies and new trends in educational theory and practice.

Life-long learning about ourselves requires constant renewal. Professional development is an on-going, continual thing whether you get credit for it or not.

The new teacher-librarian has to have a much broader concept of media; not just computer media, but all media and the blend of formats . . . using the learner as the focal point for learning . . . Not just for the delivery of instruction, but for learner access. In the near future . . . you will look at the whole building as being a learning centre with students having the ability to identify the location of particular items, no matter what the format.

Question: What does the beginning teacher-librarian really need to know?
Answer: The phone number of an old teacher-librarian!

The critical importance of learning from models was especially emphasized by those participants who had not worked with teacher-librarians during their classroom teaching careers. Students returning to teaching without current placement in a school found the lack of models difficult. Two students had solved the problem by volunteering in school libraries.

I've learned an immense amount of material. I've been volunteering in a school so I can feel my way back into the system.

Because I'm just new, I've been volunteering in two school libraries one day a week, just to try to learn what is happening in libraries. That has been very helpful to me. I've had the opportunity to see some of these things in action.

Those coming to school library education with the experience of cooperative planning and teaching, were able to draw on that experience in their coursework and felt more confident in their ability to be effective teacher-librarians.

Our former teacher-librarian and I had worked really closely together . . . so I was able to model what I did on what I had seen her do in a really strong program.

My background was in cooperative teaching with an excellent teacher-librarian.

Some participants reported that they had entered the profession through the mentorship of another teacher-librarian. Mentorship or other means of socializing novices into the profession was identified by many participants as a valuable part of school library education as well as an essential support for new teacher-librarians in the field.

The participants generally agreed that an effective program would include many opportunities to learn the role of teacher-librarian through simulations, role-plays, videotapes and other non-lecture approaches. They expected that class activities and assignments would give them the opportunity to develop and refine the practical skills of teacher-librarianship, from planning with teachers, to budgeting, to giving inservices.

One of the projects that we had to do in our cooperative planning course was to plan a unit with a teacher. I found this rather difficult, but I learned an awful lot!

6. Implications

6.1. Findings in Relation to Present Program

Many of the themes raised by the participants are covered in the present program. Taking a leadership role and developing the program are addressed throughout the diploma courses but specifically in two courses: School Librarianship, an overview introduction to the field, and School Library Programs, an in-depth examination of the content and implementation of the Alberta school library program model. The only area of weakness identified by the focus group participants was media literacy education. Participants suggested that there needed to be more emphasis on the evaluation and understanding of nonprint media and its impact on the user.

Managing the program is addressed in three of the core courses, School Librarianship, School Library Collection Development, and Organization of School Library Materials course. Coping with technology was judged to be the aspect least well addressed by the Diploma program. Although an aspect of each of the core courses, focus group participants emphasized that the demands of the new technology required more in-depth and concentrated learning experiences. This was seen to be a high priority need by most groups, from students in the program to district consultants and supervisors.

Sequencing of the program was unevenly achieved because of the variety of routes by which students accessed the program and because of limited frequency of course offerings. The recommended sequence of core courses begins with School Librarianship, continues with Information Materials, Collection Development, and Organization of Materials, and ends with School Library Programs. When students enter the program without background or expertise in computers, children's or young adult literature, or media studies, they are advised to build complementary courses in these areas into their program early in the sequence. Students can complete the Diploma program in the recommended sequence only if they begin in the Fall (September) term, or in odd-numbered years, in the Summer (July) term.

The differing backgrounds of students when they enter the Diploma program creates challenges for instructors and students. Students who began without the experience of working with or working as a teacher-librarian had more difficulty in the early parts of their coursework and continued to feel at a disadvantage throughout the Diploma program. These students in particular felt more need for mentorship opportunities.

Participants valued the modelling of the teacher-librarian role throughout the courses. They also commended the instructors for their efforts to provide assignments and other learning activities that developed and refined the practical skills of teacher-librarianship.

In addition to providing information useful for program evaluation, some other benefits of the focus group methodology were identified by the evaluators. First, the participants valued the opportunity to express their opinions. They appreciated having their voices recognized. One participant, a student currently in the program, stated:

First of all I want to say that I've never been asked my opinion on something like this and I think this is very nice indeed. This is a very welcome opportunity! Although students regularly complete course evaluations, the focus group experience gave them the opportunity to look at the program as a whole, as well as broader issues in school library education.

After each focus group session was completed, the program was reviewed briefly and participants were given written course outlines. There was time for questions and comments. For some of the participants, the program had changed significantly since they had completed their training. Some participants had credentials other than a University of Alberta Diploma. The discussion raised awareness of the program and of the efforts of the university in providing school library education.

7. Models of Professional Education: Reflections

In Alberta, continuing education is not a requirement for the continuing certification of teachers. Continuing professional education is available through membership in associations and through other individual endeavors. The participants in the focus groups emphasized the need for additional avenues for continuing professional education. Does the university have a role and a responsibility to provide professional development opportunities for teacher-librarians? To what extent does the university program prepare students to continue their learning throughout their professional lives?

8. Conclusion

The focus group methodology appears to have been an effective way to increase the quantity and quality of information needed for program evaluation. Although the focus group approach did not provide a clear direction for future program changes, it did affirm the current direction of the program and it did widen the scope of the inquiry into the program. It also appears to have an important benefit in terms of building awareness of the current school library education program within the professional community, especially for the many individuals who have limited contact with the university after completing their formal professional education [8].

References

Acknowledgements