NDSA:Wednesday, June 8, 2011

We had a rather lively discussion of the three documents members worked up to describe some of the themes, trends, and tensions that come out of the responses to the storage implementation questionnaire which several members responded to. As a next step, Karen will bring these documents together into a single document that she will share as a Google doc for members to literally highlight key points and add any additional comments. We can workshop the document together in this shared environment. We will also workshop this document at the meeting next month.

Call Participants

• Karen Cariani
• Dan Dodge
• Trevor Owens
• Gene Mopsik
• Gene Hurr
• Cory Snavely
• Micah Altman
• Dean Farrell
• John Unsworth
• Robert Cartolano
• Andrea Goethals
• Cal Lee
• Elizabeth Joffrion

What is it we have and what can we say?

We started by discussing what exactly it is that we have here and what we could say based on what we have. Importantly, these responses do not provide any kind of numerical information about a representative sample of members’ approaches to large scale storage systems. That would require a more targeted and fully formed survey. With that said, the open ended questions have brought back some very interesting stories from different partners working on different problems and there seem to be some clear common trials, and some clear common values within this very diverse set of respondents from the membership which are suggesting some similar approaches and perspectives across different institutions.

There seemed to be general consensus that putting the three drafted documents together to try and further tease out these themes, values, and approaches was valuable.

Audience: Who do we think would want to hear what we have here?

We then discussed what audience this document might be of interest to. Several members suggested that this kind of document, when fully formed, could be valuable to other organizations looking for guidance about what our collective experience suggests are some of the key things to consider when trying to plan for implementing and maintaining these systems. A second audience was identified as storage providers, in this case the presumed value of the document would be to clearly explain how the values of digital stewardship organizations explain our decisions about which technologies to buy, use, and when to migrate to and from them. Lastly, it can serve as a valuable activity for ourselves to explore and share the challenges we are each facing around storage and attempt to articulate some of the principles that are guiding our individual approaches.

Formats for the final product from this

First and foremost, the document we are working on can be refined and revised to the point where it could stand as its own report. One idea was to highlight common key points where development is needed and identify key issues and problems that need to be addressed.

There was also some discussion about developing suggested requirements that should be considered when choosing or developing a digital preservation storage system. There would not be a single list but several, dependent upon the scope and needs. We would try to suggest as much coverage as possible, but will need to clearly state the scope of each. Some initial suggestions: some of the requirements came out in the questions – simple, self managed, fixity check – what are the core things we need to document for a digital repository.

Some others: Ingest – got to be able to ingest quickly Need to have public users access quicker See thumbnail and uncompressed file that may take a little longer

Bring out themes that different responses indicate, group them together in what seems logical – access and latency. Common themes – need to develop additional data integrity practices above storage system as a software layer, check sum validation – to do this well, need these things. We agree about it how to implement? David Rosenthal at Stanford could step in and say this is the best way to do check sum validation. Let’s all solve together.

Others: Trade offs around simple file systems. Easy to move around. Un-rigorous access to data. But all went that route. Keep it simple principle. Encryptions – we could distill themes from it all – technology choice for preservation (risk averse technology trend)

Develop a decision tree for people… Along with this, Micah Altmen volunteered that once the draft document is put together he would be willing to work up an accompanying decision tree that tries to distill what the needs for a system suggest for the kinds of decisions and technologies we are using.

Workshop Session on Large Scale Storage for Meeting

The group briefly discussed what we want to do at the meeting. There is general consensus that we will plan to use the session to further discuss and vet the document we are working on. To this extent, we could present a short deck of slides that describes some of the values, tensions, and challenges that come out of this work and then use the rest of the time to break into groups and further solicit stories and feedback from additional participants. We could use this as an opportunity to identify much more targeted radio button like questions for a short quantitative survey that we could then put out to the broader membership, or we could use this as an opportunity to break into groups and workshop the text we are drafting.

Next steps: Action items

• Boil 3 docs to one - Karen will do that.

• Put that up as shared google doc for everyone to comment on, edit.

• Identify from q 10-13 common themes commonality of approaches that organizations have taken. Highlight common themes.

• From common themes, design a less open ended survey to back up data from others in NDSA.

• Discuss at meeting n Washington and engage others in NDSA.