Early this month the INDI project team made the INDI Sandbox available so that archivists and others interested in the distributed workflow system could kick the tires and get a feel for how the Integrated Digital Special Collections works. Late last week team members completed the INDI Walkthrough that will guide first time users through the various features available in INDI. This document is available at http://www.lib.byu.edu/indi/indi-walkthrough/ and is meant to be used in conjunction with the INDI Sandbox. It features step by step instructions on how to use INDI Desktop, MyProjects, and MyContacts, as well as detailed instructions for creating appraisal and accession projects. We hope that the walkthrough will make using the INDI Sandbox an easier experience.

In other INDI news, the project team is wrapping up the documentation for INDI Phase I, and once the final bug is fixed and the source code has been appropriately packaged INDI Phase I will be made available to the archival community. Phase I is comprised of INDI Desktop, MyProjects and MyContacts as well as the Apprasial module and the Acquisition and Accession module. Please watch the Downloads page for links to the source code for these items.

With the completion of Phase I the project team is turning its attention to Phase II. Phase II will focus on the creation of the Arrangement and Description module of INDI. This module will gather descriptive data about archival collections and then make that data available for export in a variety of output formats (EAD, EAC, MARCXML, MODS, Dublin Core, etc.). The project team is currently creating the requirements statements that will serve as the basis for programming this module of INDI. The team is also comparing the various standards that will be available for output and working to create a comprehensive data model. This data model is seen as the key to enabling INDI to output the specified formats as well as INDI’s future extensibility.

We would like to invite any archivist or manuscripts curator interested in the automation of archival workflow processes to visit the INDI (Integrated Digital Special Collections) web site (http://www.lib.byu.edu/indi/) and to become more familiar with the INDI project. Recent INDI developments include the completion of the INDI core tools (MyProjects, MyContacts, INDI Desktop, and the Administrative Interface) as well as the Appraisal and Accession Modules. Planning is nearly complete for the Arrangement and Description module and the development of this key module will begin shortly.

A new feature of particular interest on the INDI website is the INDI Sandbox. If you would like to become more familiar with INDI functionality enter the Sandbox and experiment with the core tools and the various modules. Collection information for four hypothetical collections has been entered. To experiment with INDI, enter or input keyword information for any one of the following collections: 1. Westwood Architectural firm records, 2. Murphy family papers, 3. George Washington legends, and 4. Department of Social Work records. Within the INDI Sandbox, you can also create your own collections as well; all data for created collections is deleted at the end of the day.

There are several ways to create an INDI project; as you become familiar with INDI, experiment with the system and discover the various ways to create new projects. For first time users, however, we recommend that you first create a project by clicking on the MyProjects icon (a clock face and pencil superimposed over a graph) at the bottom of the screen. Next, locate the File menu item located in the upper left hand corner of the screen. Click on the File menu and a drop-down menu will appear. Follow the drop-down menu to New Project and then to Accession and from there just follow the instructions to create a new project.

As you become more familiar with INDI functionality, please share your impressions with us. We welcome your input and your suggestions.

As we begin wrapping up the first phase of INDI, we put increased emphasis on documenting the application as it exists. While we had hoped to complete our documentation in time for the release of this phase of the project, documentation has also provided us with a chance to step back and orient our new programmer on the development of the system.

This work has included both user and technical documentation, including user guides, walkthroughs, internal help content, and design documentation. However, the most useful part of this effort has been the development of a revised software requirements specification. Our previous version of a requirements document, while adequate at the time, had been surpassed to such a point that it no longer provided a guide for development. At some points, INDI’s programming has been very organic, and under-documented features were added to the application. Returning to documentation has provided us a chance to recapture and redirect the course of development.

While we hope that this process will not significantly delay the release of phase 1, we believe the project overall will be improved due to this work.

The INDI project has been a very thought provoking endeavor for INDI team members.  It has forced us to examine the ways that archival principles are applied in the L. Tom Perry Special Collections and to ask ourselves whether or not we are meeting our intended goals.  We have carefully analyzed the ways in which we apply the concepts of appraisal, acquisition, arrangement, description, and open access to the varied collections in our stewardships.  We have noted the interrelationships between each of these concepts and struggled to decide on the best way to automate them.  This process has been ongoing for the last 18 months.  It was recently revitalized with the hire of a new lead programmer, Curtis Thacker, for the INDI project.  The project team has spent the last couple of months working with Curtis to help him gain a better understanding of the archival needs that have driven the creation of the system and we are now working to better document the decisions that we have made and how they need to be implemented in the INDI system.  It has been exciting to conceptualize new and improved ways of accomplishing our primary tasks as archivists.  It has also re-emphasized the importance of being standards compliant.  This has become particularly evident as we have been working on the data model and flowcharting the processes of arrangement and description in preparation for beginning work on the Arrangement and Description Module in early 2008.  In building the data model we have consulted closely Describing Archives: A Content Standard (DACS) and looked at best practice guidelines concerning the use of Encoded Archival Description (EAD).  This close consultation led us to two conclusions:  1) if we were going to build compliance with DACS into INDI then perhaps our staff should know how to use it and 2) if we planned to leverage Encoded Archival Description in our consortial environment, then we needed to have our own best practice guidelines.  Recently Cory Nimer and Gordon Daines completed work on the Church Consortium of Libraries and Archives (CCLA) EAD Best Practice Guidelines.  These new guidelines are being used to help finalize the arrangement data model and they are being used to build a new database delivery system for our finding aids.  Gordon and Cory have also recently completed a series of trainings on how to use DACS in the L. Tom Perry Special Collections and the project team is working to integrated DACS concepts into the INDI help system.  All archivists should be forced to revisit the way that they apply archival principles and to think seriously about why we do what we do–we might be surprised by what we find.

You’ll notice that both of the standards that we are using as the basis of our arrangement data model are descriptive standards.  It begs the question–Where are our arrangement standards?  Archivists need to seriously examine arrangement to see if it can be standardized and to develop a more solid footing for arrangement theory.  We also need to think carefully about the relationship between arrangement and description.  Should our descriptions reflect a physical arrangement or should they reflect an intellectual arrangement?  What does original order mean?  Whose original order?  Original order at what point in time?  Just a few questions to ponder.

The last several months have been pretty hectic for the INDI Development team. Our lead programmer, Jason Read, took another position and left the Harold B. Lee Library in early October. Curtis Thacker was hired as our new programmer shortly before Jason Read left and spent about two weeks working with Jason in an effort to gain a better understanding of the underlying architecture of the INDI system. To help jump start the learning process for Curtis, the INDI team developed a new module for him to build. The new module deals with the processes inherent in the appraisal of archival material. The change in programmers has wreaked havoc on our development schedule and we will be updating it shortly. We plan to release the INDI core tools (MyProjects, MyContacts, INDI Desktop, Administrative Interface) as well as the Accession and Appraisal Modules in early 2008. Another project that Curtis is working on is the development of an INDI sandbox that will be publicly available for interested individuals to examine INDI first hand. We hope to release the sandbox soon. The INDI project team is also working to update the INDI screencast to include information about the Appraisal module.

This past summer, Cory Nimer, Blake Miller, and I were tasked with the responsibility to develop the INDI Appraisal module. Appraisal, as suggested in the Society of American Archivists Glossary for Archivists, is the process of identifying records that have long-term historical or retention value. One of the most important appraisal tools available to the archivist or manuscripts curator is the collection development policy. The well-defined collection development policy identifies the types or kinds of historical materials acquired by archival institutions.

Traditionally, archivists have maintained paper-based case files to document or otherwise record appraisal activity. In terms of the web-based nature of the INDI Project, however, it was important that we also maintain or otherwise preserve a permanent electronic record documenting all appraisal transactions.

To accomplish this task we created a flow chart incorporating all major elements (from our perspective at least) of the appraisal process beginning with the identification of a potential collection or a potential collection donor—what we called in INDI parlance “lead identified.” Following steps, as outlined in our flow chart, and as developed by INDI software programmer, Curtis Thacker, include:

1. “Collection description” information, including a description of the contents of a specific collection.
2. “Contact” information for the donor or vendor, including an address, phone number, e-mail address, etc.
3. “Fieldwork reports”, including correspondence with donors and vendors; and
4. An “appraisal” statement indicating whether the collection in question was acquired and the reasons for its acquisition; as well as the “disposition” of materials not selected, and the reasons why the materials in question were not acquired.

The long-term importance of the case file, including the permanent retention of appraisal information, cannot be overstated. Appraisal records not only document the provenance of the collection they become part of our institutional histories. They are frequently consulted by archivists and manuscripts curators and they inform our understanding of the collections in our care. It is our expectation that INDI will further facilitate the creation and maintenance of accurate case file and appraisal information.

In building the MyContacts tool in INDI, one of the primary goals of the design team was the ability to use the data it contains for authority control. Although there has been a long (and ongoing) debate over the importance of archival authority records, it appears from the recent formation of the Encoded Archival Context (EAC) Working Group that the archival community has decided to formalize the creation and sharing of these records. Yet as various writers have noted, archives face some difficulties in building EAC authority files, especially in attracting library participation in EAC databases.

For libraries, the primary concern of traditional authority control is headings management. By using a documented, authorized form of a person/corporate name in descriptive records, library systems are able to collocate materials and provide improved access to their materials. Many academic libraries are associated with the Name Authority Cooperative Program (NACO), and contribute authority records to the Library of Congress Name Authority File (LCNAF). Here at Brigham Young University we are also a NACO participant.

For many librarians, there is little to be gained from contributing to an archival authority file in addition to library authority file. Archives and special collections are oftentimes housed in the library in academic institutions, and contribute to the LCNAF as part of their MARC cataloging. In these circumstances, creating a second authority file would require unnecessary database management problems as they tried to keep both up-to-date. Additionally, some librarians do not see an advantage in recording the contextual information possible in EAC to authority records.

The recent publication of the Functional Requirements of Authority Data (FRAD) draft suggests that archival concerns about recording context are beginning to be addressed by the library community. While it is noted in the draft that EAC was not evaluated in the preparation of the document, the writers did consult the International Standard Archival Authority Record for Corporate Bodies, Persons and Families (ISAAR(CPF)), developed by the International Council on Archives and embodied in Describing Archives: a Content Standard.

In the FRAD model, names and persons are defined as separate entities. Names include various attributes, such as type, scope of usage, dates of usage, language, script, and transliteration scheme. Name records can then be associated with person records (or corporate bodies/families records), which contain information about the person, including dates, titles, gender, associated locations, languages, occupations, and biographical histories. Person records (or corporate bodies/families records) can also be associated with other entities to describe parent/child, hierarchical, or other relationships.

The resulting FRAD framework seems reminiscent of the EAC model, except that it separates the authorized form of the name (EAC identity element) as a separate record. This is also somewhat similar to the agency history records created as part of the RLG Government Records Project, which used LCNAF authorized forms for headings while keeping administrative history information in the bibliographic record. While FRAD is still in draft, it will be interesting to see what effect its conceptual model might have in the development of Encoded Archival Context.

In INDI we have already begun implementing the FRAD model by separating headings management (name record data) from contextual information (person/corporate name/family record data). The decision to do so was made primarily due to pragmatic considerations, but we believe will have long-term benefits for our program and should be considered by the wider archival community as we move toward the development of archival authority files. In INDI, the MyContacts record acts as the FRAD contextual record, containing information about the entity and associations with other entity records and with archival collections. For headings management, though, INDI relies on the LCNAF through record linking.

To accomplish this, MARC 21 Authorities records are currently synched with the Oracle database used by the library ILS system, and are searchable within INDI. Once the correct record is selected it is linked to the MyContacts record on the record number. While this setup requires close integration between INDI with the library ILS, it reduces the need for headings management to a single authority file. It is hoped that as the Library of Congress develops an Search/Retrieval via URL (SRU) interface for LCNAF similar to that currently used in their catalog that INDI will be able to do this searching and linking directly with the national authority file itself.

Then for EAC output, INDI will harvest the information from the linked LC authority record, using it in the identity section of the record. The MyContacts data in turn will form the contextual information for the description and relationship sections of the record. In future releases of EAC, perhaps this type of linking relationship could be integrated in the standard to prevent the duplication of record content between library and archival authority files.

We hope that by approaching authority control in this way we will be able to both honor our commitments to the NACO program and library community locally, while participating in the emerging archival authority control community. Attending presentations on EAC at the SAA conference in Chicago, it seems that many members of the archival community are ready to write off libraries as partners in archival authority file development, with the intent of building completely independent systems. Yet it seems foolhardy to ignore the infrastructure and detailed authority work already done by librarians. By using linked authority systems we may be able to leverage that system, integrating heading management and contextual data as recommended in the FRAD model.

INDI team members have been busy over the summer working on the data model and workflow for the Arrangement and Description Module as well as testing the Accession module.

The Accession Module of INDI was released in a limited fashion to several students and curators in the L. Tom Perry Special Collections in July 2007. These students and curators used INDI to accession new collections and provided feedback on INDI’s usability. They recommended that contextual help be implemented as soon as practical and the project team began drafting the text for this help to be added to the Accession module. The Accession module of INDI was released to all of the manuscript curators and their students in early August and use of the system has been consistent and very few problems have been reported.

Team members carefully analyzed Describing Archives: A Content Standard, the EAD Schema, Dublin Core and several other national standards in the construction of the data model that will underly the Arrangement and Description Module. They also carefully analyzed the manuscripts Workflow in the L. Tom Perry Special Collections and created a workflow diagram to aid in the creation of the forms that will gather the arrangement and description data. Team members are working to understand the arrangement and description practices of the Wilson Folklore Archives in order to meet the unique descriptive needs of folklore materials through the INDI system.

This week the project team has been creating a webcast to highlight the completed portions of INDI. It can be viewed on the Downloads page of this site under Screencasts.

As you may have noticed, this site is under development. We’ll let you know when it’s polished and ready to be unveiled. :)

During our recent meeting, it was decided that we would work to redevelop our Web site to act as a clearing house for information about the INDI Project.

Over the past few months we have received a number of inquiries about the project from other institutions. These include archivists interested in how the system compares to other archival management applications (such as Archon or Archivists Toolkit), programmers interested in the technological underpinnings of the system, and archival administrators interested in the benefits of workflow management in archival processing. In the past we have created various documents to meet the needs of these groups, but we believe that it would be more effective to develop a single site that might meet the needs of these various audiences.

By developing this site, we also hope to generate more interest in the archival community in our developing product. Over the past year there have been a number of other products made available to archivists to help manage their work. While these offer workable solutions for some institutions, we believe that INDI’s focus on workflow management and collaboration sets it apart from its competitors and makes an important contribution to the archival community.

We hope that readers will be able to track our progress on the project, as we move toward our release date.