https://wiki.diglib.org/api.php?action=feedcontributions&feedformat=atom&user=Csnavely 2026-05-10T18:55:12Z User contributions MediaWiki 1.44.0 https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2522 2011-06-06T20:52:06Z

<p>Csnavely: /* HathiTrust Response to Implementations of Large Scale Storage Architectures */</p> <hr /> <div>==HathiTrust Response to Implementations of Large Scale Storage Architectures==<br /> <br /> # What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> #* HathiTrust's mission is (jointly) multi-institutional long-term preservation and access of digitized library materials.<br /> # What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?<br /> #* HathiTrust does not use cloud storage providers due to potential legal issues with copyrighted content. At least within the scope of digital preservation of library materials, we would consider ourselves to be a cloud storage provider, or similar to one. The large scale storage system we use is from Isilon which was chosen for its ability to scale in capacity and performance while keeping maintenance requirements constant and low. All well-known storage vendors (about 15) were considered.<br /> # Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)<br /> #* Currently, print volumes comprises the bulk of preserved content. Pilot projects for continuous tone images and audio are well underway.<br /> # How big is your collection? (In terms of number of objects and storage space required)<br /> #* As of May 2011, HathiTrust is preserving 8.7 million print volumes which consume approximately 400TB of storage.<br /> # What are your performance requirements? Further, why are these your particular requirements?<br /> #* Fixity checking (checksum validation) requires all data to be read over an approximate 90-day periodicity, which, at the current repository size, translates to 54MB/s continuous read activity over 90 days (or higher over fewer days). This has, so far, been easily attainable through parallelization. The most recent peak in the ingest rate was ~500,000 volumes per month, which translates to an average of ~7MB/s write activity, also easily attainable.<br /> # What storage media have you elected to use? (Disk, Tape, etc) Further, why did you choose these particular media?<br /> #* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) providing 6 months of previous-version retention (backups). Disk was chosen for primary storage because the archive is light, with materials continuously accessible, and because repository activities such as full-text search indexing and fixity checking require frequent low-latency access to content. Tape was chosen for backups because, as is the conventional wisdom, previous-version retention inflates storage requirements (in our case, roughly 25%) but does not demand low-latency access, and so tape is the most cost-effective medium.<br /> # What do you think are the key advantages of the system you use?<br /> #* Being NAS-based, the storage environment is a simple filesystem which offers easy integration with applications and easy movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.<br /> # What do you think are the key problems or disadvantages your system presents?<br /> #* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.<br /> # What important principles informed your decision about the particular tool or service you chose to use?<br /> #* The need to use an in-house storage system was heavily influenced by the sensitive nature of copyrighted materials that are the subject of an ongoing lawsuit.<br /> # How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations?<br /> #* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another whole migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency; the equipment lifetime is 3-4 years.<br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> #* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.<br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> #* We have developed fixity checking and other repository auditing tools as a check on our use of the storage system, ensuring that our processes have stored data correctly. This check doubles as bit rot detection in addition to that performed by the system. We have developed a small diagnostic script to isolate problems in the data synchronization from one site to another if the storage system reports an error.<br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?<br /> #* There are no issues or shortcomings of the system with respect to TRAC requirements.</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2521 2011-06-06T19:35:57Z

<p>Csnavely: /* HathiTrust Response to Implementations of Large Scale Storage Architectures */</p> <hr /> <div>==HathiTrust Response to Implementations of Large Scale Storage Architectures==<br /> <br /> # What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> #* HathiTrust's mission is (jointly) multi-institutional long-term preservation and access of digitized library materials.<br /> # What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?<br /> #* HathiTrust does not use cloud storage providers due to potential legal issues with copyrighted content. At least within the scope of digital preservation of library materials, we would consider ourselves to be a cloud storage provider, or similar to one. The large scale storage system we use is from Isilon which was chosen for its ability to scale in capacity and performance while keeping maintenance requirements constant and low. All well-known storage vendors (about 15) were considered.<br /> # Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)<br /> #* Currently, print volumes comprises the bulk of preserved content. Pilot projects for continuous tone images and audio are well underway.<br /> # How big is your collection? (In terms of number of objects and storage space required)<br /> #* As of May 2011, HathiTrust is preserving 8.7 million print volumes which consume approximately 400TB of storage.<br /> # What are your performance requirements? Further, why are these your particular requirements?<br /> #* Fixity checking (checksum validation) requires all data to be read over an approximate 90-day periodicity, which, at the current repository size, translates to 54MB/s continuous read activity over 90 days (or higher over fewer days). This has, so far, been easily attainable through parallelization. The most recent peak in the ingest rate was ~500,000 volumes per month, which translates to an average of ~7MB/s write activity, also easily attainable.<br /> # What storage media have you elected to use? (Disk, Tape, etc) Further, why did you choose these particular media?<br /> #* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) providing 6 months of previous-version retention (backups). Disk was chosen for primary storage because the archive is light, with materials continuously accessible, and because repository activities such as full-text search indexing and fixity checking require frequent low-latency access to content. Tape was chosen for backups because, as is the conventional wisdom, previous-version retention inflates storage requirements (in our case, roughly 25%) but does not demand low-latency access, and so tape is the most cost-effective medium.<br /> # What do you think are the key advantages of the system you use?<br /> #* Being NAS-based, the storage environment is a simple filesystem which offers easy integration with applications and easy movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.<br /> # What do you think are the key problems or disadvantages your system presents?<br /> #* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.<br /> # What important principles informed your decision about the particular tool or service you chose to use?<br /> #* The need to use an in-house storage system was heavily influenced by the sensitive nature of copyrighted materials that are the subject of an ongoing lawsuit.<br /> # How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations?<br /> #* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another whole migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency; the equipment lifetime is 3-4 years.<br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> #* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.<br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> #* We have developed fixity checking and other repository auditing tools as a check on our use of the storage system, ensuring that our processes have storage data correctly. This check doubles as bit rot detection in addition to that performed by the system. We have developed a small diagnostic script to isolate problems in the data synchronization from one site to another if the storage system reports an error.<br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?<br /> #* There are no issues or shortcomings of the system with respect to TRAC requirements.</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2520 2011-05-16T16:38:27Z

<p>Csnavely: /* HathiTrust Response to Implementations of Large Scale Storage Architectures */</p> <hr /> <div>==HathiTrust Response to Implementations of Large Scale Storage Architectures==<br /> <br /> # What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> #* HathiTrust's mission is (jointly) multi-institutional long-term preservation and access of digitized library materials.<br /> # What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?<br /> #* HathiTrust does not use cloud storage providers due to potential legal issues with copyrighted content. At least within the scope of digital preservation of library materials, we would consider ourselves to be a cloud storage provider, or similar to one. The large scale storage system we use is from Isilon which was chosen for its ability to scale in capacity and performance while keeping maintenance requirements constant and low. All well-known storage vendors (about 15) were considered.<br /> # Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)<br /> #* Currently, print volumes comprises the bulk of preserved content. Pilot projects for continuous tone images and audio are well underway.<br /> # How big is your collection? (In terms of number of objects and storage space required)<br /> #* As of May 2011, HathiTrust is preserving 8.7 million print volumes which consume approximately 400TB of storage.<br /> # What are your performance requirements? Further, why are these your particular requirements?<br /> #* Fixity checking (checksum validation) requires all data to be read over an approximate 90-day periodicity, which, at the current repository size, translates to 54MB/s continuous read activity over 90 days (or higher over fewer days). This has, so far, been easily attainable through parallelization. The most recent peak in the ingest rate was ~500,000 volumes per month, which translates to an average of ~7MB/s write activity, also easily attainable.<br /> # What storage media have you elected to use? (Disk, Tape, etc) Further, why did you choose these particular media?<br /> #* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) providing 6 months of previous-version retention (backups). Disk was chosen for primary storage because the archive is light, with materials continuously accessible, and because repository activities such as full-text search indexing and fixity checking require frequent low-latency access to content. Tape was chosen for backups because, as is the conventional wisdom, previous-version retention inflates storage requirements (in our case, roughly 25%) but does not demand low-latency access, and so tape is the most cost-effective medium.<br /> # What do you think are the key advantages of the system you use?<br /> #* Being NAS-based, the storage environment is a simple filesystem which offers easy integration with applications and easy movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.<br /> # What do you think are the key problems or disadvantages your system presents?<br /> #* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.<br /> # What important principles informed your decision about the particular tool or service you chose to use?<br /> #* The need to use an in-house storage system was heavily influenced by the sensitive nature of copyrighted materials that are the subject of an ongoing lawsuit.<br /> # How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations?<br /> #* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency.<br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> #* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.<br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> #* We have developed fixity checking and other repository auditing tools as a check on our use of the storage system, ensuring that our processes have storage data correctly. This check doubles as bit rot detection in addition to that performed by the system. We have developed a small diagnostic script to isolate problems in the data synchronization from one site to another if the storage system reports an error.<br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?<br /> #* There are no issues or shortcomings of the system with respect to TRAC requirements.</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2519 2011-05-16T16:35:06Z

<p>Csnavely: /* HathiTrust Response to Implementations of Large Scale Storage Architectures */</p> <hr /> <div>==HathiTrust Response to Implementations of Large Scale Storage Architectures==<br /> <br /> # What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> #* HathiTrust's mission is (jointly) multi-institutional long-term preservation and access of digitized library materials.<br /> # What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?<br /> #* HathiTrust does not use cloud storage providers due to potential legal issues with copyrighted content. At least within the scope of digital preservation of library materials, we would consider ourselves to be a cloud storage provider, or similar to one. The large scale storage system we use is from Isilon which was chosen for its ability to scale in capacity and performance while keeping maintenance requirements constant and low. All well-known storage vendors (about 15) were considered.<br /> # Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)<br /> #* Currently, print volumes comprises the bulk of preserved content. Pilot projects for continuous tone images and audio are well underway.<br /> # How big is your collection? (In terms of number of objects and storage space required)<br /> #* As of May 2011, HathiTrust is preserving 8.7 million print volumes which consume approximately 400TB of storage.<br /> # What are your performance requirements? Further, why are these your particular requirements?<br /> #* Fixity checking (checksum validation) requires all data to be read over an approximate 90-day periodicity, which, at the current repository size, translates to 54MB/s continuous read activity over 90 days (or higher over fewer days). This has, so far, been easily attainable through parallelization. The most recent peak in the ingest rate was ~500,000 volumes per month, which translates to an average of ~7MB/s write activity, also easily attainable.<br /> # What storage media have you elected to use? (Disk, Tape, etc) Further, why did you choose these particular media?<br /> #* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) providing 6 months of previous-version retention (backups). Disk was chosen for primary storage because the archive is light, with materials continuously accessible, and because repository activities such as full-text search indexing and fixity checking require frequent low-latency access to content. Tape was chosen for backups because, as is the conventional wisdom, previous-version retention inflates storage requirements (in our case, roughly 25%) but does not demand low-latency access, and so tape is the most cost-effective medium.<br /> # What do you think are the key advantages of the system you use?<br /> #* Being NAS-based, the storage environment is a simple filesystem which offers easy integration and movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.<br /> # What do you think are the key problems or disadvantages your system presents?<br /> #* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.<br /> # What important principles informed your decision about the particular tool or service you chose to use?<br /> #* <br /> # How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations?<br /> #* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency.<br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> #* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.<br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> #* We have developed fixity checking and other repository auditing tools as a check on our use of the storage system, ensuring that our processes have storage data correctly. This check doubles as bit rot detection in addition to that performed by the system. We have developed a small diagnostic script to isolate problems in the data synchronization from one site to another if the storage system reports an error.<br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?<br /> #* There are no issues or shortcomings of the system with respect to TRAC requirements.</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2518 2011-05-16T16:16:12Z

<p>Csnavely: </p> <hr /> <div>==HathiTrust Response to Implementations of Large Scale Storage Architectures==<br /> <br /> # What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> #* HathiTrust's mission is (jointly) multi-institutional long-term preservation and access of digitized library materials.<br /> # What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?<br /> #* HathiTrust does not use a cloud storage provider. At least within the scope of digital preservation of library materials, we would consider ourselves to be similar to a cloud storage provider. The large scale storage system we use is from Isilon.<br /> # Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)<br /> #* Currently, print volumes comprises the bulk of preserved content. Pilot projects for continuous tone images and audio are well underway.<br /> # How big is your collection? (In terms of number of objects and storage space required)<br /> #* As of May 2011, HathiTrust is preserving 8.7 million print volumes which consume approximately 400TB of storage.<br /> # What are your performance requirements?<br /> #* Checksum validation requires all data to be read over an approximate 90-day periodicity, which, at the current repository size, translates to 54MB/s read activity which is attainable through parallelization. The most recent peak in the ingest rate was ~500,000 volumes per month which translates to an average of ~7MB/s writing. <br /> # What storage media have you elected to use? (Disk, Tape, etc)<br /> #* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) for backup and 6 months of previous-version retention.<br /> # What do you think the key advantages of the system you use?<br /> #* Being NAS-based, the storage environment is a simple filesystem which offers easy integration and movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.<br /> # What do you think are the key problems or disadvantages your system present?<br /> #* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.<br /> # What important principles informed your decision about the particular tool or service you chose to use?<br /> #* The storage solution was chosen for its ability to scale in capacity and performance while keeping maintenance requirements constant and low.<br /> # How frequently do you migrate from one system to another?<br /> #* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required.<br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> #* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.<br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?<br /> #* There are no issues or shortcomings with respect to TRAC requirements.</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2517 2011-05-16T15:34:34Z

<p>Csnavely: /* HathiTrust Response to Implementations of Large Scale Storage Architectures */</p> <hr /> <div>==HathiTrust Response to Implementations of Large Scale Storage Architectures==<br /> <br /> # What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> # What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice? <br /> # Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.) <br /> # How big is your collection? (In terms of number of objects and storage space required)<br /> # What are your performance requirements?<br /> # What storage media have you elected to use? (Disk, Tape, etc)<br /> # What do you think the key advantages of the system you use?<br /> # What do you think are the key problems or disadvantages your system present?<br /> # What important principles informed your decision about the particular tool or service you chose to use? <br /> # How frequently do you migrate from one system to another?<br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2516 2011-05-16T15:34:16Z

<p>Csnavely: Created page with &#039;==HathiTrust Response to Implementations of Large Scale Storage Architectures== # What is the particular preservation goal or challenge you need to accomplish? (for example, re-…&#039;</p> <hr /> <div>==HathiTrust Response to Implementations of Large Scale Storage Architectures==<br /> <br /> # What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> <br /> # What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice? <br /> <br /> # Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.) <br /> <br /> # How big is your collection? (In terms of number of objects and storage space required)<br /> <br /> # What are your performance requirements?<br /> <br /> # What storage media have you elected to use? (Disk, Tape, etc)<br /> <br /> # What do you think the key advantages of the system you use?<br /> <br /> # What do you think are the key problems or disadvantages your system present?<br /> <br /> # What important principles informed your decision about the particular tool or service you chose to use? <br /> <br /> # How frequently do you migrate from one system to another?<br /> <br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> <br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> <br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:Cloud_Presentations&diff=2033 2011-05-16T15:31:51Z

<p>Csnavely: </p> <hr /> <div>In each case we would want to identify who would present, who will contact them. Then when they will present. <br /> <br /> From there we can include specific questions we would like them to respond to. <br /> <br /> ==Presentation Schedule and Slides==<br /> # Feb 1, Tues, 1:00 EST call with iRods Reagan Moore ([[NDSA:Media:NIAID.ppt|presentation]])<br /> # Feb 14, Monday, 11:00 EST call with Duracloud ([[NDSA:Media:DuracloudNDSA.ppt|presentation]])<br /> # Feb 17, Thurs, 11:00 EST call with MetaArchive/GDDP Katherine Skinner, Matt Schultz and Martin Halbert MetaArchive NDSA ([[NDSA:Media:MetaArchive NDSA Infrastructure.ppt|presentation]])<br /> <br /> ==People/Projects to Contact==<br /> *Chronopolis (Mike Smorul will contact)<br /> *Open questions from the Educopia Guide to Distributed Digital Preservation <br /> *Commercial providers? (Who specifically would we want here? Please add them.)<br /> **Azure (Leslie to contact)<br /> **Amazon (Who will contact?)<br /> <br /> ==General Questions for Cloud Service Presenters==<br /> Here we are working on a set of general questions for presenters to develop talks around. <br /> <br /> # What sort of use cases is your system designed to support? What doesn't this support?<br /> # What preservation standards would your system support? <br /> # What resources are required to support a solution implemented in your environment? <br /> # What infrastructure do you rely on?<br /> # How can your system impact digital preservation activities?<br /> # If we put data in your system today what systems and processes are in place so that we can get it back 10 years from now? (Take for granted a sophisticated audience that knows about multiple copies etc.)<br /> # What types of materials does your system handle? (documents, audio files, video file, stills, data sets, etc) And give examples of those types in practice<br /> <br /> ===Responses to questions===<br /> ====[[NDSA:iRODS]] direct responses====<br /> <br /> Other general notes:<br /> <br /> * [Snavely] The need for each storage target to support a specific set of operations, and consistently with other storage targets, seems like a risk that comes along with the elegant abstraction that iRODS provides. Clear specifications help mitigate this risk.<br /> <br /> ====[[NDSA:DuraCloud]] direct responses====<br /> Other general notes:<br /> <br /> * [Snavely] Treatment of cloud provider is generally as a black box, without a strong sense of actual reliability of underlying storage systems. Cloud providers tend to promise checksum validation of contents, but recourse if validation fails was unknown (right?). Additional checksum validation has been augmented on top of cloud storage service by Duracloud.<br /> <br /> ====[[NDSA:MetaArchive/GDDP]] direct responses====<br /> Other general notes:<br /> <br /> * [Snavely] Built on LOCKSS, so data integrity assurances are provided by robust networked software model augmented to commodity hardware and storage. Federated nature provides integrity assurance but also a lack of central control in that the accidental loss of multiple caches is unlikely but e.g. scheduled maintenance or upgrades could coincidentally collide.<br /> <br /> ====Chronopolis====<br /> # ...<br /> ====MicroSoft Azure====<br /> # ...<br /> ====Amazon S3/EC2====<br /> # ...<br /> <br /> ==Questions for Member Institution Implementations of Large Scale Storage Architectures==<br /> #What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> #What large scale storage or cloud technologies are you using to meet that challenge? Further, why did you choose these particular technologies?<br /> #Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.) <br /> #How big is your collection? (In terms of number of objects and storage space required)<br /> #What are your performance requirements? Further, why are these your particular requirements?<br /> #What storage media have you elected to use? (Disk, Tape, etc) Further, why did you choose these particular media?<br /> #What do you think the key advantages of the system you use?<br /> #What do you think are the key problems or disadvantages your system present?<br /> #What important principles informed your decision about the particular tool or service you chose to use? <br /> #How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations? <br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?<br /> <br /> ===Responses to questions===<br /> <br /> ====[[NDSA:Florida Center for Library Automation]]====<br /> <br /> ====[[NDSA:HathiTrust]]====<br /> <br /> ====[[NDSA:National Library of Medicine Responses]]====<br /> <br /> ====[[NDSA:Penn State]]====<br /> <br /> ====[[NDSA:WGBH Responses]]====<br /> <br /> ====[[NDSA:Your Institution Here]]====<br /> <br /> ==General Concerns==<br /> # confidential data<br /> # encrypted data<br /> # auditing<br /> # preservation risks<br /> # legal compliance<br /> # ...<br /> <br /> ==Solution Models and Environments==<br /> {| border=&quot;1&quot;<br /> !Name<br /> !Offered as Service<br /> !Deployed Locally<br /> !Opensource<br /> !Authentication Scheme<br /> !Ingest Mechanism<br /> !Export Mechanism<br /> !Integrity/Validation Mechanism<br /> !Replication Mechanism<br /> !Administration Model (Federated, etc.)<br /> !Tiering Support<br /> |-<br /> |iRODS<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |DuraCloud<br /> |yes<br /> |yes<br /> |yes (Apache2)<br /> |Basic Auth<br /> |1:web-ui, 2:client-side utility, 3:REST-API<br /> |1:web-ui, 2:client-side utility, 3:REST-API<br /> |Checksum verified on ingest. On-demand checksum verification service.<br /> |Built-in support for cross-cloud replication.<br /> |<br /> |<br /> |-<br /> |MetaArchive/GDDP<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Chronopolis<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Microsoft Azure<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Amazon S3/EC2<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |}</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:Cloud_Presentations&diff=2004 2011-03-18T21:23:39Z

<p>Csnavely: /* DuraCloud */</p> <hr /> <div>In each case we would want to identify who would present, who will contact them. Then when they will present. <br /> <br /> From there we can include specific questions we would like them to respond to. <br /> <br /> ==Presentation Schedule==<br /> Once we start scheduling presenters we will keep a list of the talks here.<br /> # Feb 1, Tues, 1:00 EST call with iRods Reagan Moore ([[NDSA:Media:NIAID.ppt|presentation]])<br /> # Feb 14, Monday, 11:00 EST call with Duracloud<br /> # Feb 17, Thurs, 11:00 EST call with MetaArchive/GDDP Katherine Skinner, Matt Schultz and Martin Halbert<br /> <br /> ==People/Projects to Contact==<br /> *DuraCloud/Duraspace (Leslie to contact)<br /> *Chronopolis (Mike Smorul will contact)<br /> *Open questions from the Educopia Guide to Distributed Digital Preservation http://www.metaarchive.org/GDDP (Martin will contact)<br /> *Irods: Reagan Moore, 2/1/2011 see slides: NIAID.ppt <br /> *Commercial providers? (Who specifically would we want here? Please add them.)<br /> **Azure (Leslie to contact)<br /> **Amazon (Who will contact?)<br /> <br /> ==General Guiding Questions for Presenters==<br /> Here we are working on a set of general questions for presenters to develop talks around. <br /> <br /> # What sort of use cases is your system designed to support? What doesn't this support?<br /> # What preservation strategies would your system support? <br /> # What preservation standards would your system support? <br /> # What resources are required to support a solution implemented in your environment <br /> # What infrastructure do you rely on?<br /> # How can the cloud environment impact digital preservation activities?<br /> # If we put data in your system today what systems and processes are in place so that we can get it back 50 years from now? (Take for granted a sophisticated audience that knows about multiple copies etc.)<br /> <br /> ===Responses to questions===<br /> ====iRODS====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] The need for each storage target to support a specific set of operations, and consistently with other storage targets, seems like a risk that comes along with the elegant abstraction that iRODS provides. Clear specifications help mitigate this risk.<br /> <br /> ====DuraCloud====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] Treatment of cloud provider is generally as a black box, without a strong sense of actual reliability of underlying storage systems. Cloud providers tend to promise checksum validation of contents, but recourse if validation fails was unknown (right?). Additional checksum validation has been augmented on top of cloud storage service by Duracloud.<br /> <br /> ====MetaArchive/GDDP====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] Built on LOCKSS, so data integrity assurances are provided by robust networked software model augmented to commodity hardware and storage. Federated nature provides integrity assurance but also a lack of central control in that the accidental loss of multiple caches is unlikely but e.g. scheduled maintenance or upgrades could coincidentally collide.<br /> <br /> ====Chronopolis====<br /> # ...<br /> ====MicroSoft Azure====<br /> # ...<br /> ====Amazon S3/EC2====<br /> # ...<br /> <br /> <br /> ==General Concerns==<br /> # confidential data<br /> # encrypted data<br /> # auditing<br /> # preservation risks<br /> # legal compliance<br /> # ...<br /> <br /> ==Solution Models and Environments==<br /> {| border=&quot;1&quot;<br /> !Name<br /> !Offered as Service<br /> !Deployed Locally<br /> !Opensource<br /> !Authentication Scheme<br /> !Ingest Mechanism<br /> !Export Mechanism<br /> !Integrity/Validation Mechanism<br /> !Replication Mechanism<br /> !Administration Model (Federated, etc.)<br /> !Tiering Support<br /> |-<br /> |iRODS<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |DuraCloud<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |MetaArchive/GDDP<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Chronopolis<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Microsoft Azure<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Amazon S3/EC2<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |}</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:Cloud_Presentations&diff=2003 2011-03-18T21:22:01Z

<p>Csnavely: /* MetaArchive/GDDP */</p> <hr /> <div>In each case we would want to identify who would present, who will contact them. Then when they will present. <br /> <br /> From there we can include specific questions we would like them to respond to. <br /> <br /> ==Presentation Schedule==<br /> Once we start scheduling presenters we will keep a list of the talks here.<br /> # Feb 1, Tues, 1:00 EST call with iRods Reagan Moore ([[NDSA:Media:NIAID.ppt|presentation]])<br /> # Feb 14, Monday, 11:00 EST call with Duracloud<br /> # Feb 17, Thurs, 11:00 EST call with MetaArchive/GDDP Katherine Skinner, Matt Schultz and Martin Halbert<br /> <br /> ==People/Projects to Contact==<br /> *DuraCloud/Duraspace (Leslie to contact)<br /> *Chronopolis (Mike Smorul will contact)<br /> *Open questions from the Educopia Guide to Distributed Digital Preservation http://www.metaarchive.org/GDDP (Martin will contact)<br /> *Irods: Reagan Moore, 2/1/2011 see slides: NIAID.ppt <br /> *Commercial providers? (Who specifically would we want here? Please add them.)<br /> **Azure (Leslie to contact)<br /> **Amazon (Who will contact?)<br /> <br /> ==General Guiding Questions for Presenters==<br /> Here we are working on a set of general questions for presenters to develop talks around. <br /> <br /> # What sort of use cases is your system designed to support? What doesn't this support?<br /> # What preservation strategies would your system support? <br /> # What preservation standards would your system support? <br /> # What resources are required to support a solution implemented in your environment <br /> # What infrastructure do you rely on?<br /> # How can the cloud environment impact digital preservation activities?<br /> # If we put data in your system today what systems and processes are in place so that we can get it back 50 years from now? (Take for granted a sophisticated audience that knows about multiple copies etc.)<br /> <br /> ===Responses to questions===<br /> ====iRODS====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] The need for each storage target to support a specific set of operations, and consistently with other storage targets, seems like a risk that comes along with the elegant abstraction that iRODS provides. Clear specifications help mitigate this risk.<br /> <br /> ====DuraCloud====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] Treatment of cloud provider is generally as a black box, without a strong sense of actual reliability of underlying storage systems. Cloud providers tend to promise checksum validation of contents, but recourse if validation fails was unknown (right?). Additional checksum validation has been augmented on top of service by Duracloud.<br /> <br /> ====MetaArchive/GDDP====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] Built on LOCKSS, so data integrity assurances are provided by robust networked software model augmented to commodity hardware and storage. Federated nature provides integrity assurance but also a lack of central control in that the accidental loss of multiple caches is unlikely but e.g. scheduled maintenance or upgrades could coincidentally collide.<br /> <br /> ====Chronopolis====<br /> # ...<br /> ====MicroSoft Azure====<br /> # ...<br /> ====Amazon S3/EC2====<br /> # ...<br /> <br /> <br /> ==General Concerns==<br /> # confidential data<br /> # encrypted data<br /> # auditing<br /> # preservation risks<br /> # legal compliance<br /> # ...<br /> <br /> ==Solution Models and Environments==<br /> {| border=&quot;1&quot;<br /> !Name<br /> !Offered as Service<br /> !Deployed Locally<br /> !Opensource<br /> !Authentication Scheme<br /> !Ingest Mechanism<br /> !Export Mechanism<br /> !Integrity/Validation Mechanism<br /> !Replication Mechanism<br /> !Administration Model (Federated, etc.)<br /> !Tiering Support<br /> |-<br /> |iRODS<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |DuraCloud<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |MetaArchive/GDDP<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Chronopolis<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Microsoft Azure<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Amazon S3/EC2<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |}</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:Cloud_Presentations&diff=2002 2011-03-18T21:14:15Z

<p>Csnavely: /* DuraCloud */</p> <hr /> <div>In each case we would want to identify who would present, who will contact them. Then when they will present. <br /> <br /> From there we can include specific questions we would like them to respond to. <br /> <br /> ==Presentation Schedule==<br /> Once we start scheduling presenters we will keep a list of the talks here.<br /> # Feb 1, Tues, 1:00 EST call with iRods Reagan Moore ([[NDSA:Media:NIAID.ppt|presentation]])<br /> # Feb 14, Monday, 11:00 EST call with Duracloud<br /> # Feb 17, Thurs, 11:00 EST call with MetaArchive/GDDP Katherine Skinner, Matt Schultz and Martin Halbert<br /> <br /> ==People/Projects to Contact==<br /> *DuraCloud/Duraspace (Leslie to contact)<br /> *Chronopolis (Mike Smorul will contact)<br /> *Open questions from the Educopia Guide to Distributed Digital Preservation http://www.metaarchive.org/GDDP (Martin will contact)<br /> *Irods: Reagan Moore, 2/1/2011 see slides: NIAID.ppt <br /> *Commercial providers? (Who specifically would we want here? Please add them.)<br /> **Azure (Leslie to contact)<br /> **Amazon (Who will contact?)<br /> <br /> ==General Guiding Questions for Presenters==<br /> Here we are working on a set of general questions for presenters to develop talks around. <br /> <br /> # What sort of use cases is your system designed to support? What doesn't this support?<br /> # What preservation strategies would your system support? <br /> # What preservation standards would your system support? <br /> # What resources are required to support a solution implemented in your environment <br /> # What infrastructure do you rely on?<br /> # How can the cloud environment impact digital preservation activities?<br /> # If we put data in your system today what systems and processes are in place so that we can get it back 50 years from now? (Take for granted a sophisticated audience that knows about multiple copies etc.)<br /> <br /> ===Responses to questions===<br /> ====iRODS====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] The need for each storage target to support a specific set of operations, and consistently with other storage targets, seems like a risk that comes along with the elegant abstraction that iRODS provides. Clear specifications help mitigate this risk.<br /> <br /> ====DuraCloud====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] Treatment of cloud provider is generally as a black box, without a strong sense of actual reliability of underlying storage systems. Cloud providers tend to promise checksum validation of contents, but recourse if validation fails was unknown (right?). Additional checksum validation has been augmented on top of service by Duracloud.<br /> <br /> ====MetaArchive/GDDP====<br /> # ...<br /> ====Chronopolis====<br /> # ...<br /> ====MicroSoft Azure====<br /> # ...<br /> ====Amazon S3/EC2====<br /> # ...<br /> <br /> <br /> ==General Concerns==<br /> # confidential data<br /> # encrypted data<br /> # auditing<br /> # preservation risks<br /> # legal compliance<br /> # ...<br /> <br /> ==Solution Models and Environments==<br /> {| border=&quot;1&quot;<br /> !Name<br /> !Offered as Service<br /> !Deployed Locally<br /> !Opensource<br /> !Authentication Scheme<br /> !Ingest Mechanism<br /> !Export Mechanism<br /> !Integrity/Validation Mechanism<br /> !Replication Mechanism<br /> !Administration Model (Federated, etc.)<br /> !Tiering Support<br /> |-<br /> |iRODS<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |DuraCloud<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |MetaArchive/GDDP<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Chronopolis<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Microsoft Azure<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Amazon S3/EC2<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |}</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:Cloud_Presentations&diff=2001 2011-03-18T21:05:23Z

<p>Csnavely: /* iRODS */</p> <hr /> <div>In each case we would want to identify who would present, who will contact them. Then when they will present. <br /> <br /> From there we can include specific questions we would like them to respond to. <br /> <br /> ==Presentation Schedule==<br /> Once we start scheduling presenters we will keep a list of the talks here.<br /> # Feb 1, Tues, 1:00 EST call with iRods Reagan Moore ([[NDSA:Media:NIAID.ppt|presentation]])<br /> # Feb 14, Monday, 11:00 EST call with Duracloud<br /> # Feb 17, Thurs, 11:00 EST call with MetaArchive/GDDP Katherine Skinner, Matt Schultz and Martin Halbert<br /> <br /> ==People/Projects to Contact==<br /> *DuraCloud/Duraspace (Leslie to contact)<br /> *Chronopolis (Mike Smorul will contact)<br /> *Open questions from the Educopia Guide to Distributed Digital Preservation http://www.metaarchive.org/GDDP (Martin will contact)<br /> *Irods: Reagan Moore, 2/1/2011 see slides: NIAID.ppt <br /> *Commercial providers? (Who specifically would we want here? Please add them.)<br /> **Azure (Leslie to contact)<br /> **Amazon (Who will contact?)<br /> <br /> ==General Guiding Questions for Presenters==<br /> Here we are working on a set of general questions for presenters to develop talks around. <br /> <br /> # What sort of use cases is your system designed to support? What doesn't this support?<br /> # What preservation strategies would your system support? <br /> # What preservation standards would your system support? <br /> # What resources are required to support a solution implemented in your environment <br /> # What infrastructure do you rely on?<br /> # How can the cloud environment impact digital preservation activities?<br /> # If we put data in your system today what systems and processes are in place so that we can get it back 50 years from now? (Take for granted a sophisticated audience that knows about multiple copies etc.)<br /> <br /> ===Responses to questions===<br /> ====iRODS====<br /> # ...<br /> <br /> Other general notes:<br /> <br /> * [Snavely] The need for each storage target to support a specific set of operations, and consistently with other storage targets, seems like a risk that comes along with the elegant abstraction that iRODS provides. Clear specifications help mitigate this risk.<br /> <br /> ====DuraCloud====<br /> # ...<br /> ====MetaArchive/GDDP====<br /> # ...<br /> ====Chronopolis====<br /> # ...<br /> ====MicroSoft Azure====<br /> # ...<br /> ====Amazon S3/EC2====<br /> # ...<br /> <br /> <br /> ==General Concerns==<br /> # confidential data<br /> # encrypted data<br /> # auditing<br /> # preservation risks<br /> # legal compliance<br /> # ...<br /> <br /> ==Solution Models and Environments==<br /> {| border=&quot;1&quot;<br /> !Name<br /> !Offered as Service<br /> !Deployed Locally<br /> !Opensource<br /> !Authentication Scheme<br /> !Ingest Mechanism<br /> !Export Mechanism<br /> !Integrity/Validation Mechanism<br /> !Replication Mechanism<br /> !Administration Model (Federated, etc.)<br /> !Tiering Support<br /> |-<br /> |iRODS<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |DuraCloud<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |MetaArchive/GDDP<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Chronopolis<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Microsoft Azure<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Amazon S3/EC2<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |}</div>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:Cloud_Presentations&diff=1996 2011-02-07T18:47:58Z

<p>Csnavely: /* Solution Models and Environments */</p> <hr /> <div>In each case we would want to identify who would present, who will contact them. Then when they will present. <br /> <br /> From there we can include specific questions we would like them to respond to. <br /> <br /> ==Presentation Schedule==<br /> Once we start scheduling presenters we will keep a list of the talks here.<br /> # Feb 1, Tues, 1:00 EST call with iRods Reagan Moore ([[NDSA:Media:NIAID.ppt|presentation]])<br /> # Feb 14, Monday, 11:00 EST call with Duracloud<br /> # Feb 17, Thurs, 11:00 EST call with MetaArchive/GDDP Katherine Skinner, Matt Schultz and Martin Halbert<br /> <br /> ==People/Projects to Contact==<br /> *DuraCloud/Duraspace (Leslie to contact)<br /> *Chronopolis (Mike Smorul will contact)<br /> *Open questions from the Educopia Guide to Distributed Digital Preservation http://www.metaarchive.org/GDDP (Martin will contact)<br /> *Irods: Reagan Moore, 2/1/2011 see slides: NIAID.ppt <br /> *Commercial providers? (Who specifically would we want here? Please add them.)<br /> **Azure (Leslie to contact)<br /> **Amazon (Who will contact?)<br /> <br /> ==General Guiding Questions for Presenters==<br /> Here we are working on a set of general questions for presenters to develop talks around. <br /> <br /> # What sort of use cases is your system designed to support? What doesn't this support?<br /> # What preservation strategies would your system support? <br /> # What preservation standards would your system support? <br /> # What resources are required to support a solution implemented in your environment <br /> # What infrastructure do you rely on?<br /> # How can the cloud environment impact digital preservation activities?<br /> # If we put data in your system today what systems and processes are in place so that we can get it back 50 years from now? (Take for granted a sophisticated audience that knows about multiple copies etc.)<br /> <br /> ===Responses to questions===<br /> ====iRODS====<br /> # ...<br /> ====DuraCloud====<br /> # ...<br /> ====MetaArchive/GDDP====<br /> # ...<br /> ====Chronopolis====<br /> # ...<br /> ====MicroSoft Azure====<br /> # ...<br /> ====Amazon S3/EC2====<br /> # ...<br /> <br /> <br /> ==General Concerns==<br /> # confidential data<br /> # encrypted data<br /> # auditing<br /> # preservation risks<br /> # legal compliance<br /> # ...<br /> <br /> ==Solution Models and Environments==<br /> {| border=&quot;1&quot;<br /> !Name<br /> !Offered as Service<br /> !Deployed Locally<br /> !Opensource<br /> !Authentication Scheme<br /> !Ingest Mechanism<br /> !Export Mechanism<br /> |-<br /> |iRODS<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |DuraCloud<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |MetaArchive/GDDP<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Chronopolis<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Microsoft Azure<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |Amazon S3/EC2<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |<br /> |-<br /> |}</div>

Csnavely