==Gap Analysis & Use Cases==
This is a workspace for distributed digital preservation groups to self-describe their orientation to the various Archive Interoperability models described in Section 6 of the OAIS Reference Model, as well as to OAIS in general (terminology, concepts, functional areas, roles & responsibilities, etc.). It's primary purpose is to identify important elements that could be elaborated upon to help describe a framework for effectively applying the OAIS Reference Model and derived auditing metrics such as TRAC to these groups and environments.

This gap analysis and set of use cases can then be drawn upon to articulate the proposed white paper, and lay the groundwork for finalizing a more complete and accurate [[NDSA:DDP_OAIS_Frameworks | Framework for Applying OAIS to Distributed Digital Preservation]].

===Gap Analysis===
A list of distributed digital preservation groups, projects, service providers or practitioners describing their environment's orientation to Section 6 of the OAIS Reference Model: Archive Interoperability. See [[NDSA:Archive_Interoperability | here]] for Section 6 OAIS descriptions of Independent, Cooperating, Federated and Archives with Shared Infrastructures.

* [[NDSA:MetaArchive_Gap_Analysis | MetaArchive Cooperative]]

===Use Cases===
A list of distributed digital preservation groups, projects, service providers or practitioners describing individual use case examples of applying the Reference Model and comparing/contrasting its concepts to their environment.

* [[NDSA:MetaArchive_Use_Cases | MetaArchive Cooperative]]

[[NDSA:DDP_OAIS_Frameworks | << Back to Main Page]]

NDSA:MetaArchive Use Cases

2011-04-07T13:18:57Z

Matt.schultz:

=MetaArchive Use Cases=
The following use case is intended to spotlight a couple of examples of applying the Reference Model and comparing/contrasting its concepts to the MetaArchive Cooperative.

==Open Pull vs. Push on Ingest==

===Use Case===
Preparing content for ingest in the MetaArchive stands somewhat in distinction from other digital archives where the final preparation of content for preservation is almost entirely in the hands of Archival Management (an archive and its staff). In such cases, Producers (content providers) typically supply half-formed Submission Information Packages (SIPs), and the Archival Management stages it prior to preservation to round out the SIP as an Archival Information Package (AIP) conforming to the repository’s requirements.

Because of our participatory and decentralized model, and our use of the LOCKSS peer-to-peer software, the roles and responsibilities of ingest and formation of SIPs (and even AIPs) rest primarily with the Producer (member institution), who has to very closely align the organization of data with defining the on-going and open ingest mechanism for that data. This in turn largely determines the successful formation of an AIP. Again, this is a little different than most archives where the final preparation of content, moving it from a SIP to an AIP, is often entirely in the hands of Archival Management (which is an archive and its internal staff).

This may be characteristic (but not at all intrinsic) of other repositories that rely heavily upon on-going and open web crawl mechanisms for ingest – making use of a “pull” vs. “push” metaphor.

We would invite further use cases.

===Comments===
The nature of SIPs and AIPs and their formation is not really at issue. It is perhaps more so an issue of clarifying or modeling definitions of Producers and Archival Management. The Reference Model is often keen to point out that one Archive may be a Producer or Consumer of another Archive. In such arrangements the functional elements of Archival Management remain encapsulated within an Archive. For participatory and distributed models such as MetaArchive it might serve to affirm that various responsibilities of Archival Management (completing an AIP) may be accomplished in some cases by those entities also serving otherwise as Producers. [[NDSA:User:Matt.schultz|Matt.schultz]]

===Questions===
#What additional safeguards should an Archive implement to ensure proper AIP formation without subsuming Producer participation & responsibility? [[NDSA:User:Matt.schultz|Matt.schultz]]

==Dark Archiving, Full Preservation & Designated Communities==

===Use Case===
Because the MetaArchive is engaging in dark archiving (i.e., not providing on-going end-user access) and doing so in a cooperative fashion, it has, to date, largely shifted responsibility for end-user (Access) concerns closer to the Producer (who also occupies the role of the Consumer – through their extended scholarly communities).

This has meant that issues related to ensuring “Full Preservation” (such as vigorously monitoring representation information, requiring normalization, carrying out format migration, readability via access interfaces, etc.) get considered in a very deliberative fashion – balancing preservation benefits versus impacts of cost and effort, and properly allocating responsibilities.

Through on-going communication with and by our members and through routine reporting, monitoring, and recovery testing, the MetaArchive members gauge their needs in tandem and experiment with issues related to long-term renderability of contributed content as resources and interest permit. The member buy-in ensures graded approaches and accountability around these issues over time.

Clearly this arrangement blends together many characteristics of Consumers, Producers, and Archival Management and makes for a less delineated notion of an Archive preserving on behalf of a Designated Community (more typically defined as an external requesting archive and/or end-user community of AIPs and DIPs; i.e., Consumers).

===Comments===
The Reference Model acknowledges that Archives may occasionally restrict access in a dark archive fashion – dark archiving itself is not at issue. However its participatory and distributed implementation in the MetaArchive means that the Consumer for all intents and purposes is also the Producer, and as mentioned earlier in some ways the Archival Management.

There is an intersection of both concern and desirability for ease of decision-making in regards to prioritizing the long-term renderability of contributed content. Each member Consumer/Producer, to date, has agreed to manage Full Preservation on an individual basis as opposed to positioning it as a universal requirement for Archival Management. The Archive places no technical barriers to formats, format migration, or the management of representation information nor does it impose it.

We think this a completely valid and acceptable approach given the decentralized accountability.

===Questions===
#Should the Reference Model better account for such participatory models, overlapping roles, and responsibilities through statements of exception when assigning the activities of Producers, Consumers for things like Content Information & Preservation Description Information on behalf of content for Designated Communities? [[NDSA:User:Matt.schultz|Matt.schultz]]

#What would best practice recommendations be for member institutions and an archive in such a participatory model to effectively communicate to their Designated Communities their rationale for a measured and differentiated approach to servicing "Full Preservation"? [[NDSA:User:Matt.schultz|Matt.schultz]]

[[NDSA:DDP_OAIS_Gap-Uses | << Back to Other Gap Analyses & Use Cases]]

NDSA:DDP OAIS SoP

2011-03-04T19:41:16Z

Matt.schultz: Created page with '==DDP/OAIS Framework Statement of Purpose== The Educopia Institute and the National Digital Stewardship Alliance seek to convene a working group with the broader digital preserva…'

==DDP/OAIS Framework Statement of Purpose==
The Educopia Institute and the National Digital Stewardship Alliance seek to convene a working group with the broader digital preservation community to develop an approach for implementing the OAIS Reference Model and the derived Trustworthy Repository Auditing & Certification metrics (TRAC) in a distributed digital preservation (DDP) environment.

To foster innovation, interoperability, and mutual understanding, the digital preservation field needs a conceptual framework that acknowledges the contributions of OAIS and the various metrics derived from that model. Such a framework would articulate a common set of terms, and describe an extended implementation of the OAIS functional areas for a variety of alliances, systems, and strategies that comprise distributed preservation activities.

Distributed approaches are distinct from centralized repositories, and they often use technological and organizational alignments to preserve digital content. OAIS is frequently interpreted in terms of centralized repositories. As the field of digital preservation continues to evolve, institutions need expanded vocabularies and models to help them understand OAIS in both centralized and decentralized repository environments.

Describing a Framework for Applying the OAIS Reference Model to Distributed Digital Preservation would accomplish the following aims:

#Describe the roles and responsibilities at institutions which are working to replicate and preserve digital information in an organizationally distributed environment;
#Provide a common vocabulary and set of recommendations for trustworthy interoperable systems development; and
#Inform administrative and technical practices between groups that seek to collaboratively preserve digital information using distributed mechanisms.

This work will synthesize the findings of a variety of current DDP initiatives and provide a solid foundation upon which new preservation activities may build and innovate. It will offer a viable and formalized extension of the standard OAIS-derived central preservation repository system for institutions seeking to replicate their collections and distribute them organizationally for preservation purposes.

==Activities Proposed==
We propose to Describe a Framework for Applying the OAIS Reference Model to Distributed Digital Preservation through the following steps:

'''Research & Outreach'''. In 2011 the Distributed Framework Coordinator (Matt Schultz), in conjunction with various NDSA Working Group participants and contributors from the broader digital preservation community, will perform a gap analysis between OAIS Section 6: Archive Interoperability and various known DDP uses cases to produce a two-to-five page white paper. This white paper will be drafted in wiki format for collaborative purposes and cover the following items:

:#Problem Statement
:#Background
:#Current Trends & Practices
:#Current Challenges
:#Recommendations

In parallel with the drafting of this white paper the Distributed Framework Coordinator in partnership with various core contributors will begin collaboratively identifying upcoming digital preservation conferences, events, and relevant practitioners/groups to engage with.

Core contributors to this collaborative work will then seek opportunities at one or more of these events to form recommendations and garner feedback on the white paper findings. These events will be key opportunities to solidify Collaborative Teams to begin drafting a final framework for submission to the Consultative Committee for Space Data Systems (CCSDS) in a form similar to the OAIS supplemental Producer-Archive Interface Methodology Abstract Standard (PAIMAS). See here: [public.ccsds.org/publications/archive/651x0m1.pdf public.ccsds.org/publications/archive/651x0m1.pdf].

'''Production'''. In 2011-2012, the Distributed Framework Coordinator in partnership with various Collaborative Teams will help lead the effort to draft a semi-final Framework. This work will be accomplished through a combination of event-hosted meetings and virtual collaboration (video/teleconferences, wikis, listservs, Google Docs, etc.). The Collaborative Teams will aim to synthesize the Framework and make it available for public comment, likely no later than September 2012. At the close of the public comment period, the Distributed Framework Coordinator and the Collaborative Teams will finalize the framework for submission to the Consultative Committee for Space Data Systems (CCSDS) for consideration as a supplemental abstract standard.

'''Dissemination'''. In 2012, upon completion of the Framework and throughout its phase of consideration by the CCSDS, the document will be broadly promoted through targeted announcements on listservs, blogs, and newsletters. Its authors will produce at least one peer-reviewed article on the process of creating the Framework. The team will host a meeting on Distributed Digital Preservation that will highlight existing efforts and encourage the development of new distributed digital preservation efforts.

[[NDSA: | << Back to Main Page]]

NDSA:DDP OAIS Frameworks

2011-03-04T19:36:10Z

Matt.schultz: Created page with '==Action Team Scope & Purpose== The Educopia Institute and the National Digital Stewardship Alliance seek to convene a working group with the broader digital preservation communi…'

==Action Team Scope & Purpose==
The Educopia Institute and the National Digital Stewardship Alliance seek to convene a working group with the broader digital preservation community to develop an approach for implementing the OAIS Reference Model and the derived Trustworthy Repository Auditing & Certification metrics (TRAC) in a distributed digital preservation (DDP) environment.

Read the full [[NDSA:DDP_OAIS_SoP | Statement of Purpose]].

==Actvities Proposed==
We propose to Describe a Framework for Applying the OAIS Reference Model to Distributed Digital Preservation through the following steps:

===White Paper & Recruitment===
In 2011 the Distributed Framework Coordinator (Matt Schultz), in conjunction with various NDSA Working Group participants and contributors from the broader digital preservation community, will perform a gap analysis between [http://public.ccsds.org/sites/cwe/rids/Lists/CCSDS%206500P11/Attachments/650x0p11.pdf OAIS Section 6: Archive Interoperability] and various known DDP uses cases to produce a two-to-five page white paper.

In parallel with the drafting of this white paper the Distributed Framework Coordinator in partnership with various core contributors will begin collaboratively identifying upcoming digital preservation conferences, events, and relevant practitioners/groups to engage with.

Core contributors to this collaborative work will then seek opportunities at one or more of these events to form recommendations and garner feedback on the white paper findings. These events will be key opportunities to solidify Collaborative Teams to begin drafting a final framework for submission to the Consultative Committee for Space Data Systems (CCSDS) in a form similar to the OAIS supplemental Producer-Archive Interface Methodology Abstract Standard (PAIMAS). See here: [http://public.ccsds.org/publications/archive/651x0m1.pdf Producer-Archive Interface Methodology Abstract Standard (PAIMAS)].

The white paper will cover the following items:

====Problem Statement====

====Background====

====Current Trends & Practices====

====Current Challenges====

====Recommendations====

===Production===
In 2011-2012, the Distributed Framework Coordinator in partnership with various Collaborative Teams will help lead the effort to draft a semi-final Framework. This work will be accomplished through a combination of event-hosted meetings and virtual collaboration (video/teleconferences, wikis, listservs, Google Docs, etc.). The Collaborative Teams will aim to synthesize the Framework and make it available for public comment, likely no later than September 2012. At the close of the public comment period, the Distributed Framework Coordinator and the Collaborative Teams will finalize the framework for submission to the Consultative Committee for Space Data Systems (CCSDS) for consideration as a supplemental abstract standard.

===Dissemination===
In 2012, upon completion of the Framework and throughout its phase of consideration by the CCSDS, the document will be broadly promoted through targeted announcements on listservs, blogs, and newsletters. Its authors will produce at least one peer-reviewed article on the process of creating the Framework. The team will host a meeting on Distributed Digital Preservation that will highlight existing efforts and encourage the development of new distributed digital preservation efforts.

==Gap Analysis & Use Cases==
See [[NDSA:DDP_OAIS_Gap-Uses | here]] for a wiki page devoted to specific distributed digital preservation uses cases and identifying gaps between them and OAIS (specifically [http://public.ccsds.org/sites/cwe/rids/Lists/CCSDS%206500P11/Attachments/650x0p11.pdf OAIS Section 6: Archive Interoperability]) related to bridging of terminology & concepts, application of functional areas, and/or roles & responsibilities, etc.

NDSA:Standards and Best Practices Working Group

2011-03-04T17:55:26Z

Matt.schultz:

== Standards and Practices Working Group ==

[[NDSA:Standards and Practices Working Group Charter ]] (December 10, 2010)

== Current Members ==
A list of current members is posted here: [[NDSA:Standards and Practices Working Group Members]].

== Statement of Purpose ==

The Standards and Practices Working Group will work to facilitate a community-wide understanding of the role and benefit of standards in digital preservation and how to use them effectively to ensure durable and usable collections. The Group will also develop, recommend, promote, and disseminate information about effective methods for selecting, organizing, describing, managing, preserving and serving digital content, in collaboration with other individuals and organizations where appropriate.

== Current Scope of Work ==

'''THE CONTENT OF THIS SECTION IS TENTATIVE PENDING OUR JANUARY 12 WORKING GROUP PHONE MEETING'''

Working group members may initiate and engage in new work at any time by forming Action Teams focused on specific projects or tasks. In addition, the Working Group will conduct the following project.

===Survey and document the digital preservation standards landscape===

====Objectives:====
* Identify and describe existing digital preservation standards and best practices
* Identify opportunities for collaboration with non-NDSA individuals and organizations who are currently working on digital preservation standards and best practices
* Identify gaps in digital preservation standards and best practices coverage that could be addressed by this working group in future activities

====Scope:====
* Compilation of existing digital preservation standards and best practices, including but not limited to:
** Authoring individuals and organizations
** Status of standard or best practice
** Description of standard or best practice
** Characterization of the standard or best practice (need to define categories)
** Examples of where and how they are used
** Related documents and Web sites
* Analysis and synthesis
* Identify gaps, areas of potential collaboration, etc.
* Publication of survey as an online resource

====Deliverables:====
* Online resource (perhaps a wiki?) that can be continually updated by the NDSA Standards and Practices group and accessed by the general public
* Report

====Schedule====
* Early Spring 2011: Project planning
* Late Spring 2011: Rough draft of report
* Summer 2011: “Live” online resource
* Summer 2011: Report on work at the NDSA annual conference

====Logistics & methodology====
* Progress will be reported via the regular working group phone meetings.
* Discussions will happen on listserv and via emails and phone calls as needed
* NDSA Standards and Practices working group wiki space will be used to collate resources and work on the deliverables
* A Google Doc Spreadsheet may work better to collate resources because of its support for collaborative editing and column sorting.

====Participants====
* Lead: Small group of Working Group members?
* Responsibilities: Manage the project (keep it on-track, provide status updates), Design contribution process (e.g. who can contribute, editorial review process?, how to publicize contribution process), Design information framework (e.g. the elements to record about each standard, controlled vocabulary values), Perform analysis on collated standards and best practices, Write summary report
* Participants: Entire Working Group
* Responsibilities: Identify and describe relevant standards and best practices

== Work Spaces for Standards Survey==

[[NDSA:Survey Template]]

[[NDSA:Categories and Action Teams]] (sign up for an action team here)

== Meeting Schedules, Minutes and Agendas==

* [[NDSA:February 7, 2011 Standards Working Group Notes and Agenda]]
* [[NDSA:January 12, 2011 Standards Working Group Notes and Agenda]]
* [[NDSA:December 6, 2010 Standards Working Group Notes]]
* [[NDSA:November 17, 2010 Standards Working Group Notes]]

== NDSA Standards Working Group Listserv Archives==

The archives of the Standards Working Group Listserv can be found here (login will be required): http://list.digitalpreservation.gov/SCRIPTS/WA-DIGITAL.EXE?A0=NDSA-STANDARDS&X=25F57E4CACD543490D&Y

== NDSA Standards Working Group Google Group Space==

http://groups.google.com/group/ndsa-standards-working-group?hl=en

== Related Action Teams==

[[NDSA:DDP_OAIS_Frameworks | Describing a Framework for Applying OAIS to Distributed Digital Preservation]]

NDSA:Categories and Action Teams

2011-02-06T16:48:35Z

Matt.schultz:

Following is a short list of categories for the Standards Survey project. Feel free to edit these as you see fit and add more of your own. The whole idea here is that we work on what we're interested in so that we leverage our individual expertise in this project. Please put your name and email address under the category or categories you're interested in working on.

We already have the following action team:

'''Survey Template Editing'''

Kate Murray

Andrea Goethals

Shane Beers

== '''Standards Survey Categories''' ==

'''Audiovisual File Formats'''

Courtney Michael - courtney_michael@wgbh.org

Kate Murray - kate.murray1@nara.gov

'''Preservation Metadata'''

'''Preservation Models & Evaluation Metrics'''

Matt Schultz - matt.schultz@metaarchive.org

'''Preservation Infrastructure'''

Kris Carpenter - kcarpenter@archive.org

'''Preservation Practices'''

Andrea Goethals - andrea_goethals@harvard.edu

'''''Add Your Own!'''''

[http://www.loc.gov/extranet/wiki/osi/ndiip/ndsa/index.php?title=Standards_and_Best_Practices_Working_Group NDSA Standards Group Home]