Schedule of Changes

Overview

In order to create PLDF3, a series of programs was run for each state's data and in them, the newkey was initally created from the last year each library reported, then sorted by newkey and year. The data were subsequently printed to a file and this file was examined for anomalies. In effect, a library's FSCSKEY in the last year of the library's reporting determines that library's newkey—with resolution of collisions discussed by methods described below. Next these programs were edited to make changes in the newkey as a result of that examination. The programs were run recusively and the files examined until all anomalies appeared to be resolve. In some case this process was repeated a number of times. When the printout looked correct, it was initally thought to send it to the State Data Coordinators for examination. This idea was not particularly effective because the lists were often long and daunting, and these days SDCs have more than enough to do. There were no replies to these emails although subsequently I found questions about the odd case here or there generally got a reply. As a general principle, after many year-to-year changes in the FSCSKEY, the lack of systematic assignment of keys has been addressed and anomalies each year are increasingly rare. And, as an additional general principle, the newkey assigned to each library is the FSCSKEY of the last year of the data for that library. Given that few changes in FSCSKEYs assigned to libraries have been changed, there have been few changes in newkeys recently. But, there are a few each year and they cause small adjustments here and there in the number of libraries reporting by state in earlier years. If you had a copy of the table below, you would find evidence of this effect. Such changes for FY 2005 are noted below.

In the case of the states listed below with asterisks, there are no changes in the FSCSKEY from year to year so the newkey is the same as the FSCSKEY. So far, in these states writing the program to create the state's PLDF3 file has generally been straightforward because there have only been a few cases of ambiguity and one case where untangling of the early years of the FSCSKEY was quite difficult. However, the very complexity of the task with nearly 165,000 cases has meant that each year—even this year—changes in the FSCSKEY for individual libraries missed in earlier years have been found and adjustments made in the newkey. The code that is copied into each of the various state "schedules" copied below has been changed in realively few cases for the FY 2005 data.

The most important variable is newkey

As we saw on the index page, the FSCSKEY is usually stable after about 1996 or 1997 so in those cases, the stable FSCSKEY is the newkey for that library. If there is ambiguity in the FSCSKEYs, the FSCSKEY from last year of the reported data—usually 2005— is that library's newkey and it is included in the List of Libraries. Otherwise, by definition, the FSCSKEY and newkey should be the same for the year listed here. In the List of Libraries the last year also supplies the ZIP, state, CITY, and LIBNAME. Hence, of the many forms of LIBNAME, CITY, and so forth, one sees, this latest one will be what appears in the List. If a LIBNAME changes when the 2006 data come out, then it will be changed in that updated List. One characteristic of the CITY and LIBNAME now is the use of upper case. I have been told it is a postal regulation that makes it easier for machines to read. It sure makes it hard for people to read them which might explain why so much of what shows up in my mailbox isn't mine.

The purpose of newkey is to be a unique number for any one library through all years of the data. The newkeys for libraries that die are not reused. When two libraries merge, they become a new library and get a different newkey. If they divorce, they get their old newkeys back but there will be years missing for that library during the years it was merged. In some cases, libraries died and the FSCSKEY was reassigned by NCES and using the last year of reporting resulted in two different libraries having the same newkey. These kinds of collisions have been resolved by creating a separate newkey for the library that closed or where collusions occur for any other reason starting with XX9975 and assigned sequentially with 'XX' standing for the two character state code.

span is a variable that indicates a characteristic of the data for each library. It has three possible values: A indicates the library reports data for all years of the data. Given that states started reporting in different years, A refers to reporting all possible years for that state so the years covered is a function of the first year reporting to FSCS. S libraries report some years. E is a special case of S where the library reports at least the end years of the span of years but are missing some years in the middle. A libraries would be useful in measuring behavior of all libraries through the years while A and E libraries would be useful in measuring changes from the first to last years of a span of years. There are 7,765 A libraries, 1,774 S libraries and 354 E libraries. Minnesota did not report data in 2001, as a result, all Minnesota libraries are coded with a span of S.

first year is the first year this library reports data to FSCS.

last year is the last year this library reports data. Last year is most often 2004 for the current dataset.

The changes documented in the Schedules are primarily to newkey.

It is a temptation to correct obvious errors that are noted in the printouts and not just the FSCSKEYs. Inspection of the files find clear typographic errors in the demographic variables used in the construction of PLDF3 and, too, there are not-so-obvious errors that the SDCs will be aware of. In addition, there are changes in formats used for the names of systems and libraries. Why not “correct” them? The temptation to correct all these errors must be resisted. The Hippocratic Oath of data is: First Do No Harm. There is no telling what mischief might result from an attempt to correct what is a digital text in an archival sense—these datasets constitute a text worthy of preservation. Creating a superset with new variables as with newkey allows the old variables to be preserved and the new “corrected” (we hope) variables to exist at the same time. Anyone using this edition may then make such changes as he or she fancies.

Those looking at the Schedules will note that there are cases where changes for CITY for 2002 were commented out. There are two examples where the SDCs agreed that the CITY was wrong in 2002 and it was obvious by inspection. Iowa had three examples, one: LAPORTE CITY was corrected to LA PORTE CITY. The changes were made so that the List of Libraries, which is drawn from the last years of the data, will be as accurate as possible to anyone using it and not ambiguous when checking the data file itself. As I was going over the data in 2003, I decided changing the CITY was not a good idea so I didn't do subsequently but left the code in as a reminder of the folly of such changes. So, these changes have been reversed so that the errors in those files are, again, as they were in the orginal NCES data.

The schedules are pasted from the SAS code for each state and are in ASCII for manipulation. The programs themselves are available to anyone who wants to see them.

There are a total number of 164,872 observations in PLDF3 through the FY 2005 data. More information on them can be found in the revision history of PLDF3.

There are currently no changes for California, the District of Columbia, Georgia, Guam, Hawaii, Louisiana, the Marianas, Maryland, Mississippi, Nevada, Ohio, or the Virgin Islands.

The changes in each file are in order by the FSCSKEY except for Colorado, Oregon, and Washington which are in order by the CITY. The FSCSKEY is often in order by CITY at least in the early years. In retrospect, I should not have rearranged the data by CITY but left them in the order they were in the data so that my coding could be more easily followed for anyone checking for errors.

How to Read the Schedules of Changes

The programs creating newkey from FSCSKEY first create a dummy newkey from the FSCSKEY for 2003 and then changes are made in the newkey in the program. The programs are written in SAS and the code changes are what appear in the Schedule of Changes. They are pasted from the programs without editorial changes. They should be readable by people who do not know SAS. Here is an example:

/* OTIS              */
if year = 1987 and FSCSKEY = 'CO0071' then newkey = 'CO0091';
if year = 1988 and FSCSKEY = 'CO0102' then newkey = 'CO0091';

These are the changes for Otis which were used as an example from the List of Libraries. The “/*” and “*/” indicate comments and are used here as an easy means of finding lines of code related to different libraries. These comments, typically, have the CITY. The first line indicates that SAS is to look for any case where the year is 1987 and the FSCSKEY is CO0071. In those cases (and this is only one where those two conditions hold), change the newkey to CO0091 from the dummy CO0071 that was used to seed this variable. The last year of Otis was 1989, you will recall. And there was no other library that was using that FSCSKEY in a later year so it is used as the newkey. In 1988, Otis had the FSCSKEY of CO0102 and this, too, was changed to CO0091. The semicolon is a SAS end of command.

The Schedules of Changes are available as text files by state. Most of the state Schedules are in order by FSCSKEYs and years changed in the code—often in pairs, that is, in order by the sort of FSCSKEY, then year and often there were pairs of changes in different FSCSKEYs. This method gives the changes no apparent order but they are in order by the data. In a handful of cases, I sorted the libraries by CITY, thinking this would make things easier to find. I regret having done that because rather than making changes easier to find, it has made them harder to find when you do have the data. The most likely use of these files is for people to check my work and sorting by CITY cannot make this process easier.

There is a spreadsheet (OpenOffice, xls) giving a fuller list of the number of reporting libraries by state and year than what is given below.

FY 2005 Changes

There are only a few cases of changes in the FSCSKEYs in the FY 2005 data that affect the newkey. Most were where I had missed changes in annual FSCSKEYs in individual libraries. That is, one library was in more than one FSCSKEY over the years and I had not corrected the newkeys. There was one case of an apparent anomaly in the way one library was assigned an FSCSKEY. Given that the software that assigns FSCSKEYs has improved so much over the years, this case is profoundly odd. It is impossible but, nonetheless, it occurred. That case occurred with data from Oregon. See Oregon for the table that lists data for the Wagner Community Library. Otherwise the few changes are noted in the code included in the various state schedules with a comment indicating the changes were made for the FY 2005 data. See the page below for Illinois, Oregon, and Pennsylvania for cases where anomalies in assignment of FSCSKEYs required adjustments in the newkey and the underlying code required to produce PLDF3. Alabama had a minor spelling error in a library name corrected in FY 2005.

Links

* indicates states with no changes.
** Minnesota did not report in 2001.

December 27, 2007