Following Mr. Penniman's introduction of the panelists, John Bertot made his presentation.
"In terms of measuring the value of library services, there are a few assumptions that aid researchers: We know what to measure; we know how to measure it; and we know why we are measuring it.
"In knowing what to measure, one of our key assumptions is that we have identifiable elements and variables. In some ways, looking at electronic transactions is the easier side of things. The software is improving in terms of measuring the network traffic, that is: 1) How many documents are downloaded? 2) What is the overall bandwidth?; and 3) What is the amount of networking trafficking in all of these areas? The truth is that behind all of these questions there are some key elements missing in terms of actual value. This is partly because we do not yet have a new definition of value.
"I would argue that 'value' is something that the user determines. The follow-up to that is: 1) What, in fact, did they do with those documents? 2) How did they help? and 3) How did it make a difference? Those are the more qualitative aspects of data collection activity that are not very easy to obtain.
"In determining what to measure we need agreement on the 'unit of analysis.' One of the things that we found when looking at the Maryland Sailor Network is that defining the network, itself, becomes quite difficult. It is not just a 'network.' You have to determine what part of that network you are looking at: 1) Are you looking at the physical structure? 2) Are you looking at the backbone? 3) Are you looking at an individual PC? 4) What level of the network are you going to look at? and 5) What types of things are you going to measure within those areas?
"There are a whole range of issues within the network services, on-line data bases, and on-line data base services, that you may provide: 1) Who is managing the network? and 2) At what level?. Within each of these areas there is a different notion of value. In some sense, value is a moving target , and, it is getting harder to pin down. You have to define exactly what is 'value' within each of those areas.
"I want to reiterate the point that value is at the user level. The basic problem is to determine how to get the user level data on a national basis and have it standardized across all of the libraries that you measure.
"There is really an amazing level of complexity in the Sailor Network. Maryland provides some baseline Internet service, and every public library system in the State of Maryland is linked to Internet, at some level. However, individual library systems have added on to that in providing additional Internet-based services; some are even providing e-mail account to users. What Internet-level service are you going to measure? How are you going to count that nationally? As a result, you have multiple services.
"One terminal provides multiple services. Questions to be answered are: 1) Which services are you trying to track? and 2) How do you divide up the costs of the terminals for each of the services they provide? In our 1996 study we found that 30 percent of public libraries could not figure out how much of their operating expenditures went to 'information technology' alone. What percentage of the technology budget went to Internet-based services? As a result, you have to estimate percentages and overall costs.
"You need to know how frequently you want to collect data. You can do it monthly or annually. You have to standardize the collection. You have to have agreement on the data elements and the units of analysis. Everyone has to agree—the collection agency as well as those actually collecting the data in the field.
"One debate now is 'urban versus rural libraries.' Basically, we determined that anything under 25,000 population is rural. This was a quick way of looking at the urban/rural distinction, and, as it turned out, there was much controversy.
"In trying to determine costs, for example, cost per terminal for the service you are trying to provide, I do not think there is one good way right now. When we conducted our cost-model study, we tried to determine 'cost elements'. There are, of course, hardware costs, software costs, planning costs, renovation costs, building costs, telecommunications costs, and so forth.
"One mistake made in our 1996 study was that when we asked how many libraries were connected to the Internet, we were talking about all of the different costs elements. What we really should have asked was: 'When did you get your connection?' Because, within that framework, there is the difference between 'ongoing costs' versus 'initial costs.' If you already have an OPAC system, and you have 100 different terminals hanging off that system that have been in use for the last five years, and you are providing text-based services to the Internet, there are really no additional costs for the hardware side of things or for providing Internet services. In some cases, it is only turning on a switch. Those are the kinds of things that we need to look at!
"In terms of methodologies, I have learned that determining quality is not necessarily a quantitative process. 'Yes' and 'no' questions are very broad, and you do not reach the level that you need in terms of value questions. When we ask, "Are you connected to the Internet in any way? and you say , 'Yes', there are about 200 questions I can think to ask underneath that.
"Finally, you have the local library at the one level that resides within a library system, within a local government system, within a state government system, and within the federal government system. Why are you collecting this data? For internal purposes? For informing the state? For informing your local government? For a national debate on libraries? In many ways, this is the first step to be addressed in terms of data collecting.
"On the bad news situation—our setting is a complex new frontier. We are not really sure which components we should measure. The patron community is very spread out, and it is no longer necessary to walk into a library to get services. The good news is, to some extent, we are able to track services more accurately through automatic data collection types of software that we can hang off of the servers and terminals.
"NCLIS took the lead by going ahead and doing the first national survey of public libraries and their use on the Internet. There really is a need for an on-going survey of this type. We can argue about what it is to collect on that survey, but, at some point, we have to get into institutionalized data collection activity. It is a moving target. It may change rapidly, and it must be flexible. But, at the same time, it must allow for ongoing data collection so we can do more longitudinal-types of collection to study the changes over the past few years. I think the FSCS network is probably the most logical choice to carry out that task.
"In a pitch for my career with the research communities, I think skills in the electronic services environment are needed, and that falls on the purview of the research community to provide those skills.
"My thanks for being given the opportunity to talk to this group this morning. I also want to thank John Bertot for his paper, which provided a wonderful set-up for my own presentation.
"Before beginning my substantive comments, I want to make four introductory comments:
"First, my presentation this morning is taken from a soon-to-be published paper entitled, A Framework for Evaluating Public Investment in Urban Libraries, Bottom Line, 9:4 (Summer 1996). This paper is exactly what its title suggests —a framework. We still have to develop measurements. This morning's talk is derived primarily from this Framework paper.
"Second, the rationale for this research appears under my authorship as Something More than Soundbites: Communicating Value to Library Constituencies, Bottom Line, 9:3 (Spring 1966). A couple of my comments will be drawn from this paper.
"Third, as soon as the 'Framework' publication appears in print, the entire piece will be electronically mounted on the MCB Press homepage and the Urban Libraries Council homepage, and the full text can be downloaded from either location.
"Fourth, the project on which I am working has two co-authors: Donald Elliott, Professor of Economics, and Christopher Dussold, a Lecturer in Economics, both in the Department of Economics at Southern Illinois University at Edwardsville. I publicly state my gratitude for their willingness to collaborate with me. I would not have attempted a library benefits valuation project without the contributions of able economic co-researchers.
"Let me tell you why I became interested in our current research project.
"I think public libraries need better tools to convey their value to their constituents than the 'warm and fuzzies' they mostly use now. Some constituents—among whom are public officials and corporate leaders—want a financial or dollar-value sense of the payback from public investment in libraries. Our research project is designed to create a tool which can be used to assess the benefits of public expenditure on libraries. I believe the result will be to increase the credibility of my public library and public libraries, generally.
"I have seen the successes which cultural institutions can have in utilizing such value-discerning tools. Through a succession of economic-impact studies, St. Louis area museums have been able to convey their positive effect on their regional economy. Their statements have helped establish economic credibility. Libraries should have the same kind of tools.
"Given that positive experience, why have Professor Elliott and I chosen to use cost-benefit analysis rather than economic-impact analysis to ascertain library benefits? For those who know the two methodologies, the reason is basic: Economic impact analysis does not work well in describing the economic effects of public libraries. Public institutions yield high-economic impact only if they attract investment or expenditure from outside the region. Nationally known museums, for example, attract out-of-town visitors who spend at area retail and hospitality institutions. Such out-of-region 'income' yields high-economic impact. Because public libraries are primarily 'locale-serving', and they do not attract 'income' from outside the region; their operation usually does not produce a high-economic impact.
"Cost benefit analysis does a better job of describing the economic effects of public library operation. It already has been used to measure the benefits of similar kinds of locale—serving institutions, colleges, schools, swimming pools, and reading programs. These methodologies are established, and those studying public libraries can use this literature to hurry along the determination of library benefits.
"Cost benefit calculations also match nicely in the way taxpayers think: If you pay more taxes, you should get more benefits! In the focus groups that the St. Louis Public Library (SLPL) has been conducting for several years, participants have been almost eager to assign benefit values to library services. And, cost-benefit analysis is flexible: It can be adapted to focus on specific library functions or individual branches, or it can be aggregated to focus on the costs and benefits of the public library as a whole.
"Finally, cost-benefit analysis matches well with a prior body of research information on public libraries. In the cost-benefit formulation which Professor Elliott and I have established, the elements we will be analyzing are: 1) resources; 2) capability or quality; 3) utilization; and (4) beneficial effects or value. This cost-benefit analysis fits nicely in the work that library professionals have been doing for years to determine their costs and to measure their utilization, even though the latter measure has seldom been in economic terms. Librarians also have strong non-quantitative impressions regarding both capability and the value of their services.
"Can those qualitative impressions of professionals and the public be transformed into measurable quantitative benefits? "The answer is yes, but the work will be difficult. Our measurement techniques will be the same ones that John Bertot talked about. We know it is possible to measure the private benefits which people derive from using the library. If a swimming pool developer can calculate that individuals obtain 'X' value from using the facility, then a public library should be able to get users to assign 'Y' value to the hundreds of fiction books they check out and read each year.
"Measuring public benefits is harder but not impossible. If juveniles become more knowledgeable and literate because of their library use, they are less likely to be unemployed and more likely to earn and to vote. These are public benefits. People are willing to set values on such services if they are given appropriate information and choices.
"This value setting will be an extension of work already underway in St. Louis. A good deal of the work done to determine the costs of its services has been conducted by the SLPL staff. Library staff have also accumulated significant detail on who uses which services. In addition, the library staff and consultants have experience in getting focus groups of users to talk about the values of library use.
"Within the context of this whole research experience, user focus groups will be asked questions about economic value. The results of those focus groups can be used in appropriately designed surveys by which other users—and non-users—can be asked questions about the economic value they are willing to set on library services.
"There will be other research questions as well, such as: 1) What estimating system should be used to accumulate value? 2) What about discount rates? and 3) What formulae should be used to establish indirect external benefits after having completed the measurement of direct benefits?
"As you can see, our project's study efforts will start very simply. We are ready to do pilot measurements. We expect quick results, but we will need to replicate studies in St. Louis and elsewhere. Our studies will be built on established social science and economic literature, and on SLPL's previous research experiences. We will reinvent no wheels that already run smoothly!
"Professor Elliott and I both are methodological conservatives. We expect our first outcomes to be cruder than we would like. We expect that methodological refinements and replications of value testing in St. Louis and elsewhere will provide more accurate pictures of the benefits to be derived both from the aggregate of library services and from specified individual services.
"With a grant from the Public Library Association, the next phase of our research will involve initial cost-benefit valuations of two services: 1) Library basic literacy services for youth, and 2) the library's business services. The reason for our interest in both services is derived from SLPL's operating situation.
"A single statistic illustrates our interest in the impact of basic literacy. This past year, the St. Louis Public Schools graduated 1,050 seniors. SLPL user statistics, meanwhile, note the existence of 973 17-year olds who are regular library users. Quite obviously, all of us connected with SLPL Youth Services wonder if there is a correlation between the success of those seniors who graduated high school and continued library card holding, There are either equally fascinating library-use/children's-success relationships that seem worth exploring.
"The Library's interest in business services flows from an equally intriguing relationship. Sixty-six percent of all library income is derived from property taxes associated with businesses, not from residences. SLPL provides many business services in return, especially to small businesses but to large corporations as well. Exploring the relationship between library use and business success will be just as intriguing as that between library use and school success.
"What are the policy implications of the proposed research? "
First, and most importantly, we hope to demonstrate the social return on investment in public libraries. Such a calculation can have wide implications and can make arguments for continuing and/or improved public and philanthropic funding.
"Second, we believe that cost-benefit analysis may be useful in defining what can be called 'the edge of service'. In the 21st century, it seems unlikely that public libraries will be able to do everything for everybody. Instead, policy makers may have to begin to self-consciously decide which services provide the most benefits and which need to be dropped because their benefits do not match the income. If resource constraints increase, how will library leaders decide 'the edges of their service'? Will it be simply arbitrary? Or, will some economic formulation be part of the decision-making equation?
"I do not want you to think that I would ever use a cost-benefit formulation as the sole reason for increasing or cutting a service. Public library administrators have to decide service levels based on many factors, including economics. To put the matter simply, I am not one of those professionals who is searching for the ultimate set of statistics that will take most of the risk out of policy making. On the other hand, not to utilize any set of statistical or econometric measures which help inform policy is like using only half a brain to make a service decision. Statistics and tools of economic measurement should not drive policy making, but they can help establish a powerful framework for policy.
"Cost-benefit analysis is not going to be a magic bullet, nor are we going to produce elegant public-library cost-benefit formulations at our first attempt. Time and refinement will be required to obtain that which will be meaningful from a policy standpoint. By its very nature, social science moves forward in incremental steps. Professor Elliott and I are beginning to take one-in-a-series of steps in the public library field.
"Is this exercise worthwhile? My answer is similar to a previous reply by Don King. In 1982, Professor Herbert White, whom I respect enormously for his conceptual and methodological contributions to library and information science, wrote, 'I see no real reason for cost-benefit determination because the premise that libraries are good is still widely accepted.'
"Does anybody want to stand by that comment today when attempting to get things accomplished in Washington, DC?
"Times have changed, and cost-benefit analysis has a place in library and information science."
"This has been an extraordinarily stimulating conference. The talks we have heard so far, have hit very important issues. I plan to go a little more deeply into one that I might prefer to avoid. I will describe my talk as 'value in libraries' and it is based upon research that I did in collaboration with Tefko Saracevic and a number of other people at Rutgers University, with support by the Council on Library Resources. This 300-page report is available in paper for $35.00 or over the Internet for free. ftp://soils.rutgers.edu/pub/APLab/cost.value.study.
"I have come to believe that the Internet is, in some sense, the 'big public library' for the entire Nation. The study I will talk about was motivated by the belief that we have to find ways to value libraries today which are going to persist and make sense across this substantial transformation from paper-based, visit-based, to electronic-based, outreach-based. I think that the one element to remain unchanged during that transformation is the impact on users.
"My own career in studying libraries has gradually worked across the chain from input through processing in libraries, to outputs in libraries, and, now, to the impact on users. In a sense, when we study processes, we ask how the library is doing what it does? When we study outputs, we ask how much is the library doing what it does?
"I think of the measurements at various stages in this as either being measurements of quality or quantity, or sometimes, both. When you are dealing with the inputs to a library in terms of personnel, there are quantity measures that may not show up—such as amount of training. With processes, it becomes very natural to talk about the quality of the process, such as success rates, turnover, and so forth. When you look at the outputs, it becomes very natural to talk about the quantity. When looking at the impact of uses, you deal with both quantity and quality. There is some qualitative phenomenon that occurs when the user is affected by what happens at the library.
"Our particular study was conducted at five major research libraries, looking at about 20 different services within the libraries. We conducted more than 500 interviews with users. Our goal was to try to use uniform methodology across these very diverse services, ranging from the entire art collection at one library to a specialized electronic record service at another. One of the results of the survey was a very large manual, a kind of do-it-yourself manual.
"Metrology, applied to measuring libraries, asks the question, 'How good is the library?' Taxonomy, which we do not talk about very often, asks, 'How is the library good?' We found that we are not really breaking totally new ground. The oldest reference we found was Aristotle, the ancient Greek philosopher, who pointed out that the value of a thing is of two kinds—its value in use and its value in exchange. For libraries and information services of all kinds, value and use are proper foci.
"We also made a brief diversion. We went into a philosophical study of value called, 'Axiology.' We found a list of some 20 kinds of value, of which I will summarize only two: 1) intrinsic value, and 2) instrumental value. We say that something has intrinsic value when it is just good for its own sake. Instrumental value is the value that something has when it helps you to do something else that you have chosen to do.
"It is also useful to distinguish between 'information' and 'information services'. In thinking about information, there is a rather-well developed economic theory of the value of information which says, 'The value of the information is the difference between how well you do when you have it, and how well you do if you do not have it.' We cannot, at least yet, apply that measure to libraries. In thinking about information services, we recognize that people go through three different important steps to get to the information. The first we call the 'acquisition' of the information. The second we call 'cognition'. Something has to happen inside your own head. If you think about it, you are somehow transformed by the information that you have obtained. And, then in some cases there is an 'application'—but, not always. If what you get from the libraries are detective stories, you read them. There is some cognitive process, but you are done. You do not actually go out and solve any crimes yourself.
"Before we could talk about those processes, we had to look at libraries to learn as much as possible about the impact of these information services. We took an approach which in social sciences is called 'grounded theory.' What that means is: 'Look at things before you make up your mind.' We did that with the 500+ interviews. We interviewed people by asking them two-part questions: 'What kind of score would you give to the library on your last visit?' This did not tell us very much. We asked, 'Why did you give that score?' This told us quite a lot. Then, we asked the people: 'What was the reason or purpose of the project that brought you to the library in the first place?' Finally, we asked: 'What was the impact on what you got from the library for the project or research?'
"It turns out that when you ask people these kinds of things, they do not respond to the particular question that you asked. Rather, they tell you the whole story. We taped and transcribed the interviews. Many hours were spent reading the transcripts, and in shaping the interviews into groups. We took those results and shaped them even more to see if they could make sense. What seemed to emerge is that you can make a very broad, high-level grouping of the responses that people give to you in terms of 'reasons for use', 'interactions', and 'results'.
"The details of both our methodology and the resulting taxonomy will appear in two papers in JASIS, sometime in 1996 or 1997.
"We do not claim that this taxonomy will describe all users of every library. We are sure that for many library settings, other terms, concepts, and descriptions will arise. But, we are very comfortable in saying that this methodology can be applied, and it can be used to find out how the library is good to the users. Transparencies show just a typical interaction might be, 'I searched the CD-ROM.' The typical kind of result might be, 'How I wrote my paper.'
"All in all, we feel that we have a new beginning. The question, 'How is the library good?' should be preceded by the question, 'How good is the library?' because it defines the scales and dimensions along which we can actually communicate with the libraries and their users. It may turn out that money is just not a very good thing to question them about. In fact, as one side question, we asked: 'Have you ever paid for the information services?' Most of the users replied that they never had.
"To bring this down to the focus of this particular Forum and propose an answer to the question, 'What should be asked of libraries when they are surveyed?', I think that we ought to begin to ask, 'On what terms do patrons value your services?'
"Personally, I am extremely interested in the impacts of technology, and I think the corresponding questions are:
"Thank you for inviting me here today.
Having looked at the value of information, starting back in 1968, and as a result of numerous studies, we began to study the value of libraries. Over the years we have built a framework that is useful for examining the economics of libraries of all types. Very briefly, I would like to go over this framework.
"This framework has two dimensions. One dimension deals with the level of service being looked at. In other words, you can apply the framework to looking at resources, such as staff and equipment. Or, you can look at it as specific services or from the perspective of the entire library.
"The second perspective deals with gathering information that could be useful from a number of different perspectives. Certainly, one has to look at it from the perspective of the users and from a higher level, from the organization's standpoint, whether that be the community served by public library or aggregated up to industry and society.
"There are basically five measuring factors one might study. One has to do with the input to the services—the application of resources necessary for library services. (Resources being such things as staff, space, equipment, supplies, and so forth.) Then you can measure those in terms of the amount of resources involved, or you can put a common unit on those resources. And, you can put a cost to space, equipment, supplies, and the like. Those are all measures of input one should consider as well as the attributes that will have an impact.
"We like to think of outputs in terms of the quantities of output of the services and also the attributes of the outputs you are charging for the services, such as the price, quality, timeliness, availability, and accessibility. In a sense, some people consider those attributes as being the value of the services.
"The first measuring tool has to do with the usage of the services. The amount of use and non-use can be measured by looking at the factors that affect use and non-use. There are alternative ways of gathering that information, and people will choose whether they use the library or an alternative choice. And, one should recognize that alternative choices do exist.
"A second measuring tool has to do with the purpose of use for public libraries.
"A third measuring unit has to do with getting some assessment of the importance and satisfaction with specific attributes of those services.
"Awareness is a big issue. I am always flabbergasted by the lack of awareness of the user and potential user of the services being provided.
"Then we go to the outcomes. In looking at the outcomes in 'special' libraries, we find that using the information provided by the library results in saving time, improving productivity, improving quality of work, improving timeliness of work, achieving organization, and so forth. We have a different set of outcomes that we use in 'public' libraries.
"The fifth measure is the number of people served in the community. The outcome measure is designed to calculate cost per-capita.
"One problem has been that the measures, by themselves, do not have much use. It is really the relationship of the measure that begins to tell you something about the usefulness and value of libraries. So, we developed a series of 'derive makers' which relate the input to the outputs. We call that 'performance' because that is something you can control within the libraries. Obviously, unit-costs are one measure of performance. Another measure is the productivity of staff, for example, the output divided by the number of hours necessary to provide the services. It is also useful to look at performance in light of the attribute because you find that you provide better services in terms of quality, timeliness, availability, and/or accessibility. That has an impact on the cost, and you can look at the unit-cost in light of the attribute. So, we always try to do this as well.
"Then we relate the output, particularly in regard to the attribute, and the extent to which those favorable attributes have an affect on demand —that is, the extent to which the library services are used. We want to think 'usage' in terms of the use of the information, not necessarily the use of the service.
"Then you can begin to relate the outputs, particularly the attributes of the output and the extended usage. You always see a high correlation between the attributes and the extent to which the library services are used. If you begin to do this, then you begin to say, 'If I get better equipment and staff it will cost me more, but I know that we are going to get higher attributes and, therefore, more use out of the services'. That finding has some bearing on justifying the budget and resources.
"We also measure the cost against the extent to which the services and the information is used. Then, we look at the impact relating to the outcome of the services. You can then look at the value in terms of the consequences of having used that information.
"In 'special' libraries, in particular, we are able to quantify the fact that the more people use information, the greater their productivity.
"The final kind of measure we have found useful is cost-benefit. When I first started looking at value, I really looked in terms of measuring the costs against the outcome of the services. The way I look at cost-benefit now is to look at the library, or any of its services, against some alternative to that library or service. If the comparison is favorable, I look at that as being a benefit. If the comparison is unfavorable, I consider that to be a cost. This is a very, very useful way of measuring.
"Oftentimes, we find that people are willing to indicate that if they did not have the needed service available in their library, they would go to an alternative source to get it and are willing to make a rough estimate for the cost of that alternative. We have been able to do this in' special' and 'public' library environments. Thank you."
"I am currently working on two projects which relate to the measurement of the benefits and costs of library services. The four speakers before me—John Bartow, Donald King, Glen Holt, and Paul Kantor—provided good examples of the value of cost-benefit analysis in library and information science research. As an economist, I feel strongly that economic analysis is an important tool in measuring the value of library services.
"In cost-benefit analysis, costs are relatively easy to calculate. Costs typically involve summing library expenditures on a particular project or in a cost center. A percentage of overhead costs may be added to account for managerial time spent on the project or service. Costs may also involve measuring the opportunity cost to patrons for using a particular library service.
"Calculating benefits is more difficult, particularly in a library setting. Economists measure benefit using the demand for a good or service. If a service is provided, customers or patrons will be willing to pay a given amount for that good or service depending on the benefit they anticipate deriving from it. For example, one way to measure the benefit of health and safety classes offered by the American Red Cross is to measure the demand or willingness to pay for these classes. Given data on price, number of students, and number of classes, an economist can measure the benefit of these classes by estimating student demand. This information can then be used to determine what price the American Red Cross may want to charge students in order to maximize the benefit while collecting sufficient revenue to cover the costs to provide these classes.
"Measuring the benefit from goods or services provided by a library is more difficult. Typically, calculating the benefit from a service is not meant to determine a price to charge for the service in order to cover costs; instead, it is meant to illustrate that there is some measurable value to providing the service. Measuring the value of a library service may be necessary to convince a government agency director, a library funder, or donor that the library should provide this worthwhile project. However, without information on the willingness to pay a price charged library patrons, it is a difficult task to measure the benefits derived from library services. Data on how much patrons use the library and the opportunity cost of patrons' time can be used as a proxy for the willingness to pay or value of library services.
"I have completed two research projects that provide examples of cost-benefit analysis within a library setting. The first project, 'The Economics of Access Versus Ownership in Academic Libraries,' was funded by the Council on Library Resources. This project was in collaboration with the State University of New York Libraries at Albany, Binghamton, Buffalo, and Stony Brook. The complete results of this project can be found in The Economics of Access versus Ownership, Haworth Press (1996). This project calculated the cost-efficiency of providing patron access to scholarly journal articles at an academic library via purchasing a journal subscription versus providing access via interlibrary loan.
"Previous research on the economics of access versus ownership calculated the costs of a journal subscription, shelf space, and the cost of staff time to catalogue, bind, and re-shelf the journal. The costs of supplies and labor to provided access via interlibrary loan has also been calculated by other researchers. However, previous research failed to include the cost and benefits to patrons of access versus ownership. Providing access to journal articles by interlibrary loan implies a patron must wait a few hours or a few weeks to access the information. This project calculated the 'opportunity cost' to patrons of having to wait for access to a journal article provided by interlibrary loan.
"Patrons were surveyed about their willingness to pay for immediate access to a journal article. Their response to this survey was used to calculate the opportunity costs of their time spent waiting for delivery.
"Surveying patrons about their willingness to pay for a library service is difficult. Patrons cannot be asked how much they are willing to pay for a service since their responses may be an inaccurate estimate of what they are truly willing to pay. Patrons may respond with a low-willingness to pay if they believe their response may be used to implement fees for the service. Likewise, patrons may respond with a high-willingness to pay if they believe their response may be used to determine the benefit received from a service and influence whether the service will continue to be offered. In this project, a carefully-constructed survey was essential to accurately measure patrons' opportunity cost of time waiting for access to journal articles.
"On average, patrons were willing to pay $2.55 to have immediate access to a journal article. This number, along with numbers on the financial cost of providing interlibrary loan and journal subscriptions, can be used to measure the economic cost of providing access to journal articles by interlibrary loan versus a journal subscription. This information, along with data on the use and subscription price of journal subscriptions, can be used by academic libraries to determine the most cost-efficient method of access to scholarly information.
"Data on pricing and use collected at the University Libraries which participated in the study was used to make recommendations to these libraries about which subscriptions to retain and/or cancel. For many high-price and low-use journal subscriptions it was more economically efficient to provide access by interlibrary loan than journal subscriptions. Many of the techniques of cost-benefit analysis used in this study can be found in The Economics of Information: A Guide to Cost-Benefit Analysis, Libraries Unlimited, (1996).
"A second example of economic analysis applied to libraries is research on entrepreneurship in public libraries and measuring the impact of new streams of revenues on more traditional local tax support. This project is in collaboration with the American Library Association and the Mandel Center for Nonprofit Organizations at Case Western Reserve University. Funding for this project is provided by the Kellogg Foundation.
"This research examines non-tax sources of revenue for public libraries, including donations, bequests, sales, and fines, and its effect on financial support from local tax revenues. It is a well-known result in the economics literature that an increase in revenues from government sources for nonprofit organizations may result in a decrease in charitable contributions. This research examines the opposite effect—the effect of a change in revenues from other sources on government revenues—for public libraries. More importantly, this research examines the strategies public library directors can employ to insure that revenues gained from donations, sales, and bequests do not adversely effect revenues from local government sources.
"Data from public libraries in Ohio, Minnesota, and New York were used to determine the amount of support these libraries receive from tax and various non-tax sources. Final results of this project are not yet available, however, preliminary results suggest that there exists a positive relationship between some non-tax sources of revenue and local tax revenues. This effect holds constant other variables including population, type of library, and the value of local taxable property.
"The second stage of this project is to visit the public libraries which are, and those which are not, successful at raising revenues from entrepreneurial sources. These site visits will determine the differences in library management and supporting groups, such as friends of libraries and library foundations, between libraries which are successful in raising revenues from entrepreneurial sources and those which are not.
"This research and the other research described in this session illustrates the usefulness of economic analysis applied to topics in library and information science. However, valid, empirical research requires accurate data."
Panel Discussion
Joan Challinor asked, "Bruce, how did the subscribers arrive at the $2.55 value?" Bruce Kingma responded, "We surveyed all of the interlibrary loan patrons, and, as I said before, it is extremely important how to ask the right question. Actually, the survey was completed a couple of years earlier by some researchers who asked, basically, 'How much would you pay for this?' I think that is the wrong way to ask that question. What we asked them, in a couple of different ways, is, 'If you had to wait for this article from interlibrary loan or if we implemented a priority service that would provide it within an hour, how much would you be willing to pay?" As though, 'Here is a carrot for you.' We came pretty close to getting an estimate of the value they placed on having immediate access. Recognize, too, that the $2.55 is an average value."
Joan Challinor: "What do you do about the people who say that if they did not get it from the library, they would not get it at all, and, they would not really care all that much? In other words, the value is the existence of the library, and there is no actual value." Bruce Kingma: "It is not the value of the information. It is the value of immediate access versus waiting two weeks. We are not saying you are not going to get it. The $2.55 is not the value of the information, it is the value of the opportunity-cost of waiting two weeks." Don King responded: "What Bruce is measuring is really the value of the attribute of the timeliness of response. Another way to do that is to look at all of the attributes of the interlibrary loan and document delivery in terms of speed of response, the quality of the photocopying, and so forth, and do a conjoint measurement in order to determine the relative value of each of those different attributes."
David Penniman asked, "Can you give a layman definition of conjoint? It is a marketing research tool that looks at different attributes, speed of delivery, quality, and the like, and using a statistical technique where you get people to make choices about the different levels of those attributes. You force people to say, "I would prefer a price over the quality or speed of delivery." Paul Kantor replied, "There is a paper in the library literature, College and Research Libraries by Greg Crawford, on trying to apply the conjoint analysis that Don King brought into the field."
Sam Memberg asked, "Do you have a cost-benefit analysis, which is not negative, regarding staff access to the Internet?" Glen Holt replied, "There is a study of making books available electronically at Columbia University. We asked the people: 1) With regard to such and such a book, will you access it in paper or on-line? 2) Which way is most productive? and 3) Which enables you to do better quality work? We are just beginning to get a toe-hold. We are not ready to ask, 'How much better?' yet. We are waiting to see if it is better at all."
Don King responded, "We have to think of any cost-benefit analysis in terms of the information that is derived from the service. The Internet is only as good as the information it provides. We must constantly think of it in those terms. It is a tool." John Bertot then added, "In the public library setting, access to the Internet is relatively new. We really are at the anecdotal stage. I am not as optimistic in terms of being able to assess cost-benefit analysis in the electronic environment at this point. An academic setting might be different. As an assistant professor, I know what journals I like to use and what those subscriptions costs. But, in a public library setting, I have yet to run across a patron who could immediately say, 'Time Magazine costs X amount, and this is what it is worth for me to have it in two days from now.' I think the electronic environment will dramatically change our expectations of what is 'immediacy' and what is 'timeliness.'"
Glen Holt suggested, "Before you decide whether or not the staff is getting value out of it, maybe libraries ought to invest in staff computing? I know you are not being negative, but we have to be an exception to the rule. We invest in staff computing before we invest in the Internet. Forty percent of all of the computers in our system (we have 415 in our system) are physically for staff use. When you invest at that level and start conceptualizing the value of Internet, using it as a way of communicating reference questions, I am suggesting that I would be real careful about measuring apples and oranges. I think the Internet system that we are creating for staff support is a different system than the Internet. I would think of the Internet access as part of that system only in the sense of reference help, e-mail communications, and so forth."
Sam Memberg asked, "Are you using it specifically for Internet?"
David Penniman stated, "Glen, I think the audience would be interested in some background on how you employ the GIS system to help you make some strategic decisions." Glen Holt replied, "We established a tract and sub-tract information system for our community. We aligned this against activity of the library user so we can match library use against tract data and show that geographically. This gives us pockets of use. We have what we call a full-patron equivalent, an invisible human being that runs around from place to place. What I am suggesting is that the GIS system we employ is an enormous policy tool. It is also an economic-development tool for people in the community, so it serves two principle functions. I am not being defensive about GIS. Fifteen percent of our total use comes out of a virtual library. We deliver actively to 100 elderly sites; we deliver actively to 200 day-care sites."
Walt Terrie asked, "Most of the public libraries in the country need to do the kinds of things that you are talking about but are probably not able to for many reasons. How can the work you do be generalized in order to assist a library that cannot afford to do this to make a case? For instance, the one thing they do frequently know is inputs; that may be all they know." Don King responded, "Most of these studies are not public yet. There are enormous studies, and they are all going to go away because they are all done individually. One of the recommendations that Sam Memberg and Joe Shubert came up with is to identify this information and pull it all together. I would like to do that, if I can."
Paul Kantor then responded, "Since Don King has locked in his vaults more data on this then the rest of us will ever see, he knows better than the rest of us that it would be hopeless to try to claim that because my library spends 'X' dollars and so did another library across the country, therefore, we are having just as much impact on our community. We find enormous variations in the relation between any input and output. I do not think there is going to be a magic bullet which says, 'If all you are measuring right now is the turnstile count at the door, that can be translated into a full picture of your impact to a community.' It just ain't so. The question is, 'Can these things that we do at preposterously high costs, on an individual study basis, be turned into something that you can do in the privacy of your home?' That means really thinking about a certain kind of teaching; not the kind of teaching done in the classroom, but practical teaching transferring certain amount of skills and know-how. One technique, of course, is to write a 'how to do it manual.' I earned a relatively good living for a number of years as a consultant by having written 'how to do it' manuals, and then I got paid to go out and read it to people because the manual did not really explain how to do it. If we are going to do this, we have to develop techniques that are learner-friendly and make use of newer technologies. For example, I cannot think of a better way to teach focus-group techniques then to have a short video showing how to do Discussion Groups badly.
"Again, this is a project for the public good and so the generation of these materials would have to be financed on speculation because these materials by themselves will not produce the answers or information about any particular library."
Peter Young asked, "I am curious if any of the panelists would like to take on the task of measuring the value of library services within the macro of looking at 'information services and publishing' in general, especially with interaction with electronic networking?" In response, Don King stated, "Within the field of science, I have tried to do that. I received a small grant from the Special Libraries Association to look into electronic publishing and how it will affect the communication process. I do not know anyone that is trying to do that within the public library environment."
Paul Kantor added, "It is very important to understand that the public library function is changing. Much of the value that public libraries provide to their community is going to be in terms of making that 'Internet-thing' of greater value to the local community. The library will be maintaining local resources for the local community because no one else cares to and providing, through various kinds of finding aids, the kind of guidance that, in the past, was basically provided by buying the right books and not the wrong books. If libraries do not identify this as the 21st century definition of what it means to be a library, then people are going to start stepping over our bodies. As Don King said, 'What they really want is the information'."
Helene Yurth asked, "Paul Kantor mentioned that we learned from President Reagan that power of anecdotes is proven. I think we also learned from him that to repeat an idea over and over again at every opportunity it can become to be an accepted idea. It seems that, as a profession, we have recognized that we need to have more time for measurement and evaluation. But, at the same time, we seem to be adopting an apologetic stance about not stopping that. If we cannot prove our value, we cannot expect anyone to value us. I wonder if that is a problem? Can we not be improving our analysis at the same time that we maintain the position that libraries are important and valuable, just because they are?"
Glen Holt replied, "We should do everything possible to demonstrate that libraries are winners. The language of winning is a language that Ronald Reagan held a 'teflon-coated patent' on for years and years. He taught people how to communicate. I think that is one important step. My reason for getting into cost-benefit analysis was a very personal one. I had some very conservative people whom I was using to help raise income that is not publicly supported because we are trying to increase that part of the revenue that the public cannot, or will not, provide for us. And, those people are not influenced by 'warm and fuzzies'.
"I used to do a lot of work with Fritz Machlup, and he argues very persuasively that you cannot measure value. The problem is that in 'real life' you have to begin to pull together information that is persuasive for the people who fund and use your services. You have to have some indicators that will help in the funding process."
David Penniman added, "Obviously, you do not want to discount the value of the anecdotal information. Don King and José-Marie Griffith, and I am sure others, have developed some techniques for systematically collecting anecdotal information. There are techniques for turning it into quantitative data."
Helene Yurth then responded, "You do not see it as an exclusive thing, but it does seem that we adopting this stance and that we are apologetic about ourselves."
Don King added, "In the special library environment, we observed that they were collecting anecdotal information. The problem is that those instances are fairly rare. But, in a statistical way, if you use critical incidence method you can get people to indicate how often those things do occur, and, sometimes, they will provide an estimate of value. About one percent of the usages of libraries result in some kind of savings, from $10.00 to $50,000. You can look at it statistically."
The panel discussion ended at 10:30 a.m.
Three underlying themes in the group discussions were detected. These themes are further defined by the questions listed below:
Theme 1: Understanding Measurement
Many of the discussions pointed out the urgent need for detailed data at the local level.