“Evaluate programs and services on measurable criteria”
Introduction
In any work environment that provides services and programs, it’s important to be able to determine how productive those services and programs are to the people using them. The library and information science field is no different. Determination of effectiveness of services and programs can be accomplished in several ways, but usually involves measurable criteria such as the number of users who accessed a particular web page on the library’s site, or a count of participating children in an after-school homework hour. Ours is a data-driven profession.
Programs are scheduled events such as computer classes, knitting groups, concerts and story hours. Services are actions like having a reference librarian answer questions, or having a computer lab instructor assist. Both programs and services are free to patrons of the library. They are not free for the library to provide. Costs of programs and services need to be monitored and compared to how much they are being used by patrons. For example, a public library continues to provide a phone reference librarian as a service to patrons, as they’ve been doing for the last thirty years. However, determining if this service is still being used by patrons enough to offset the cost is an important thing to consider.
Evaluation criteria for programs can include: tracking the number of participants, recording the amount of money spent, and giving out questionnaires to determine effectiveness. For example, a public library hosts a small concert headlining a local violinist. The librarian running the concert records the amount of money spent, and does a head count of attendees. This information can help determine if the library should host another similar concert.
Established criteria in service evaluation will differ from organization to organization. Generally, this criteria will include things like the longevity, popularity, cost, and other associated aspects. Often data from services are recorded for budgetary tracking in addition to evaluation. The length of how long a service has been offered in a organization is only one aspect of evaluating the effectiveness of a service. Of course longevity is not a reason by itself to continue a program or service. In my earlier example the phone reference librarian service had been a part of the public library for quite sometime. Let’s suppose that there had been a severe drop in usage rates of that service over the past 3 years. This information is discovered because the phone reference librarians keep track of how many phone call queries they receive per hour and per day. The head of the reference department can take that data and see that perhaps the phone reference librarian service is antiquated and unnecessary now. This means that the public library’s meager budget will be able to focus on improving existing services and programs.
Taken in conjunction, all of these aspects create a clearer picture of how effective a particular service is. Another example, a craft hour at a small rural library has been offered for many years, but seems to be waning severely in popularity. However, it costs very little to offer. Considering these criteria together, a library administrator may decide to keep this craft hour and start aggressively advertising the program in order to attract new patrons.
Careful evaluation can improve programs or services, such as in the example above. But it’s also important to underscore that this process is best done recursively. A service or program can be evaluated for a schedule change, then slightly altered in length of time or frequency. Performance data collected before and after the change can then be evaluated, and further fine tuning of the schedule and comparing of the data will yield optimal results. In this way can a library’s services and programs be kept up to date, relevant, and flexible, with little investment beyond careful record keeping and honest data analysis. Such creative evaluation can inspire examples for other programs. Perhaps the small rural library may want to copy the craft hour’s successful advertisement and apply this tactic to their own branch’s services.
Evidence
My first piece of evidence is a post from week 13 in LIBR 247. This discussion post asked me to discuss the various methodologies and data gathering tools used to evaluate thesauri. Thesauri interfaces can be evaluated from a user point of view, which involves examining the user’s interaction with the thesaurus and how satisfied she is with the retrieved results and overall process. Formative evaluation of the thesaurus is done by an evaluator during the development or during an improvement phase of the thesaurus development. Other criteria used in evaluation are comparing the structure of the thesauri to national and international standards, connectedness ration, accessibility, subject coverage, and the cost-benefit of using it.
My second piece of evidence is a critical note from LIBR 200. This critical note examines a study evaluating an academic library’s services and other statistics. The study emailed 85 randomly selected international students a questionnaire which had 61 responses. The questionnaire contained 6 items on personal information and seven items on information needs, library resources, services usage frequency, information viewing formats, most commonly used databases, and search engines. There was also an open-ended question on what additional resources and services the participants needed. This is an example of both program and service evaluation.
My third piece of evidence is a computer class hand out on Pinterest I created while volunteering at a local public library. In my volunteer position, I was tasked with updating and creating a desired computer class curriculum. This was in response to patron requests for new computer class services and programs at this branch. My class handout was created using a very rough template, yet includes step-by-step instructions on how to join and use the Pinterest social media service. It includes screenshots, explanations of the different areas and some aspects of the website, and suggests activities for the classroom.
Conclusion
Programs and services exist in LIS environments to provide extra benefits to library users and patrons beyond the collection itself. Accurately determining the performance, price, popularity and other aspects of programs and services allows them to be critically considered for improvement or reinvention. It is therefore essential to the supporters of these programs and services that they be evaluated with measurable criteria. Changes resulting from these evaluations improves the LIS work environment for the library users as well as LIS professionals. Continued success of any system requires evaluation using criteria, alteration, and then re-evaluation.