https://wiki.diglib.org/index.php?action=history&feed=atom&title=NDSA%3AHathiTrust 2026-05-10T18:21:30Z Revision history for this page on the wiki MediaWiki 1.44.0 https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2523&oldid=prev 2016-02-11T19:18:28Z

<p>7 revisions imported: Migrate NDSA content from Library of Congress</p> <table style="background-color: #fff; color: #202122;" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 15:18, 11 February 2016</td> </tr><tr><td colspan="4" class="diff-notice" lang="en"><div class="mw-diff-empty">(No difference)</div> </td></tr> <!-- diff cache key wiki_production-dlfwiki_:diff:1.41:old-2522:rev-2523 --> </table>

Dlfadm https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2522&oldid=prev 2011-06-06T20:52:06Z

<p><span class="autocomment">HathiTrust Response to Implementations of Large Scale Storage Architectures</span></p> <table style="background-color: #fff; color: #202122;" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 16:52, 6 June 2011</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l24">Line 24:</td> <td colspan="2" class="diff-lineno">Line 24:</td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* We have developed fixity checking and other repository auditing tools as a check on our use of the storage system, ensuring that our processes have <del style="font-weight: bold; text-decoration: none;">storage </del>data correctly. This check doubles as bit rot detection in addition to that performed by the system. We have developed a small diagnostic script to isolate problems in the data synchronization from one site to another if the storage system reports an error.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* We have developed fixity checking and other repository auditing tools as a check on our use of the storage system, ensuring that our processes have <ins style="font-weight: bold; text-decoration: none;">stored </ins>data correctly. This check doubles as bit rot detection in addition to that performed by the system. We have developed a small diagnostic script to isolate problems in the data synchronization from one site to another if the storage system reports an error.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* There are no issues or shortcomings of the system with respect to TRAC requirements.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* There are no issues or shortcomings of the system with respect to TRAC requirements.</div></td></tr> <!-- diff cache key wiki_production-dlfwiki_:diff:1.41:old-2521:rev-2522:php=table --> </table>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2521&oldid=prev 2011-06-06T19:35:57Z

<p><span class="autocomment">HathiTrust Response to Implementations of Large Scale Storage Architectures</span></p> <table style="background-color: #fff; color: #202122;" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 15:35, 6 June 2011</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l20">Line 20:</td> <td colspan="2" class="diff-lineno">Line 20:</td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The need to use an in-house storage system was heavily influenced by the sensitive nature of copyrighted materials that are the subject of an ongoing lawsuit.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The need to use an in-house storage system was heavily influenced by the sensitive nature of copyrighted materials that are the subject of an ongoing lawsuit.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another <ins style="font-weight: bold; text-decoration: none;">whole </ins>migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency<ins style="font-weight: bold; text-decoration: none;">; the equipment lifetime is 3-4 years</ins>.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.</div></td></tr> <!-- diff cache key wiki_production-dlfwiki_:diff:1.41:old-2520:rev-2521:php=table --> </table>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2520&oldid=prev 2011-05-16T16:38:27Z

<p><span class="autocomment">HathiTrust Response to Implementations of Large Scale Storage Architectures</span></p> <table style="background-color: #fff; color: #202122;" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 12:38, 16 May 2011</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l14">Line 14:</td> <td colspan="2" class="diff-lineno">Line 14:</td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) providing 6 months of previous-version retention (backups). Disk was chosen for primary storage because the archive is light, with materials continuously accessible, and because repository activities such as full-text search indexing and fixity checking require frequent low-latency access to content. Tape was chosen for backups because, as is the conventional wisdom, previous-version retention inflates storage requirements (in our case, roughly 25%) but does not demand low-latency access, and so tape is the most cost-effective medium.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) providing 6 months of previous-version retention (backups). Disk was chosen for primary storage because the archive is light, with materials continuously accessible, and because repository activities such as full-text search indexing and fixity checking require frequent low-latency access to content. Tape was chosen for backups because, as is the conventional wisdom, previous-version retention inflates storage requirements (in our case, roughly 25%) but does not demand low-latency access, and so tape is the most cost-effective medium.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key advantages of the system you use?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key advantages of the system you use?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* Being NAS-based, the storage environment is a simple filesystem which offers easy integration and movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* Being NAS-based, the storage environment is a simple filesystem which offers easy integration <ins style="font-weight: bold; text-decoration: none;">with applications </ins>and <ins style="font-weight: bold; text-decoration: none;">easy </ins>movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key problems or disadvantages your system presents?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key problems or disadvantages your system presents?</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What important principles informed your decision about the particular tool or service you chose to use?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What important principles informed your decision about the particular tool or service you chose to use?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#*  </div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* <ins style="font-weight: bold; text-decoration: none;">The need to use an in-house storage system was heavily influenced by the sensitive nature of copyrighted materials that are the subject of an ongoing lawsuit.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another? Further, what is it that prompts you to make these migrations?</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency.</div></td></tr> <!-- diff cache key wiki_production-dlfwiki_:diff:1.41:old-2519:rev-2520:php=table --> </table>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2519&oldid=prev 2011-05-16T16:35:06Z

<p><span class="autocomment">HathiTrust Response to Implementations of Large Scale Storage Architectures</span></p> <table style="background-color: #fff; color: #202122;" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 12:35, 16 May 2011</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l4">Line 4:</td> <td colspan="2" class="diff-lineno">Line 4:</td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust&#039;s mission is (jointly) multi-institutional long-term preservation and access of digitized library materials.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust&#039;s mission is (jointly) multi-institutional long-term preservation and access of digitized library materials.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust does not use <del style="font-weight: bold; text-decoration: none;">a </del>cloud storage <del style="font-weight: bold; text-decoration: none;">provider</del>. At least within the scope of digital preservation of library materials, we would consider ourselves to be <del style="font-weight: bold; text-decoration: none;">similar to </del>a cloud storage provider. The large scale storage system we use is from Isilon.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust does not use cloud storage <ins style="font-weight: bold; text-decoration: none;">providers due to potential legal issues with copyrighted content</ins>. At least within the scope of digital preservation of library materials, we would consider ourselves to be a cloud storage provider<ins style="font-weight: bold; text-decoration: none;">, or similar to one</ins>. The large scale storage system we use is from Isilon <ins style="font-weight: bold; text-decoration: none;">which was chosen for its ability to scale in capacity and performance while keeping maintenance requirements constant and low. All well-known storage vendors (about 15) were considered</ins>.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* Currently, print volumes comprises the bulk of preserved content. Pilot projects for continuous tone images and audio are well underway.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* Currently, print volumes comprises the bulk of preserved content. Pilot projects for continuous tone images and audio are well underway.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How big is your collection? (In terms of number of objects and storage space required)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How big is your collection? (In terms of number of objects and storage space required)</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* As of May 2011, HathiTrust is preserving 8.7 million print volumes which consume approximately 400TB of storage.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* As of May 2011, HathiTrust is preserving 8.7 million print volumes which consume approximately 400TB of storage.</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># What are your performance requirements?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># What are your performance <ins style="font-weight: bold; text-decoration: none;">requirements? Further, why are these your particular </ins>requirements?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* <del style="font-weight: bold; text-decoration: none;">Checksum </del>validation requires all data to be read over an approximate 90-day periodicity, which, at the current repository size, translates to 54MB/s read activity <del style="font-weight: bold; text-decoration: none;">which is </del>attainable through parallelization. The most recent peak in the ingest rate was ~500,000 volumes per month which translates to an average of ~7MB/s <del style="font-weight: bold; text-decoration: none;">writing</del>.  </div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* <ins style="font-weight: bold; text-decoration: none;">Fixity checking (checksum </ins>validation<ins style="font-weight: bold; text-decoration: none;">) </ins>requires all data to be read over an approximate 90-day periodicity, which, at the current repository size, translates to 54MB/s <ins style="font-weight: bold; text-decoration: none;">continuous </ins>read activity <ins style="font-weight: bold; text-decoration: none;">over 90 days (or higher over fewer days). This has, so far, been easily </ins>attainable through parallelization. The most recent peak in the ingest rate was ~500,000 volumes per month<ins style="font-weight: bold; text-decoration: none;">, </ins>which translates to an average of ~7MB/s <ins style="font-weight: bold; text-decoration: none;">write activity, also easily attainable</ins>.</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># What storage media have you elected to use? (Disk, Tape, etc)</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># What storage media have you elected to use? (Disk, Tape, etc) <ins style="font-weight: bold; text-decoration: none;">Further, why did you choose these particular media?</ins></div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) <del style="font-weight: bold; text-decoration: none;">for backup and </del>6 months of previous-version retention.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) <ins style="font-weight: bold; text-decoration: none;">providing </ins>6 months of previous-version retention <ins style="font-weight: bold; text-decoration: none;">(backups). Disk was chosen for primary storage because the archive is light, with materials continuously accessible, and because repository activities such as full-text search indexing and fixity checking require frequent low-latency access to content. Tape was chosen for backups because, as is the conventional wisdom, previous-version retention inflates storage requirements (in our case, roughly 25%) but does not demand low-latency access, and so tape is the most cost-effective medium</ins>.</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># What do you think the key advantages of the system you use?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># What do you think <ins style="font-weight: bold; text-decoration: none;">are </ins>the key advantages of the system you use?</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* Being NAS-based, the storage environment is a simple filesystem which offers easy integration and movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* Being NAS-based, the storage environment is a simple filesystem which offers easy integration and movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key problems or disadvantages your system <del style="font-weight: bold; text-decoration: none;">present</del>?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key problems or disadvantages your system <ins style="font-weight: bold; text-decoration: none;">presents</ins>?</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What important principles informed your decision about the particular tool or service you chose to use?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What important principles informed your decision about the particular tool or service you chose to use?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* <del style="font-weight: bold; text-decoration: none;">The storage solution was chosen for its ability to scale in capacity and performance while keeping maintenance requirements constant and low.</del></div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#*  </div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another?</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another<ins style="font-weight: bold; text-decoration: none;">? Further, what is it that prompts you to make these migrations</ins>?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required<ins style="font-weight: bold; text-decoration: none;">. The initial move was simply the initial hardware acquisition, and the hardware replacement is an annual cycle to maintain hardware currency</ins>.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>#* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* We have developed fixity checking and other repository auditing tools as a check on our use of the storage system, ensuring that our processes have storage data correctly. This check doubles as bit rot detection in addition to that performed by the system. We have developed a small diagnostic script to isolate problems in the data synchronization from one site to another if the storage system reports an error.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>#* There are no issues or shortcomings with respect to TRAC requirements.</div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>#* There are no issues or shortcomings <ins style="font-weight: bold; text-decoration: none;">of the system </ins>with respect to TRAC requirements.</div></td></tr> <!-- diff cache key wiki_production-dlfwiki_:diff:1.41:old-2518:rev-2519:php=table --> </table>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2518&oldid=prev 2011-05-16T16:16:12Z

<p></p> <table style="background-color: #fff; color: #202122;" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 12:16, 16 May 2011</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l2">Line 2:</td> <td colspan="2" class="diff-lineno">Line 2:</td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?  </div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* HathiTrust's mission is (jointly) multi-institutional long-term preservation and access of digitized library materials.</ins></div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)  </div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* HathiTrust does not use a cloud storage provider. At least within the scope of digital preservation of library materials, we would consider ourselves to be similar to a cloud storage provider. The large scale storage system we use is from Isilon.</ins></div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* Currently, print volumes comprises the bulk of preserved content. Pilot projects for continuous tone images and audio are well underway.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How big is your collection? (In terms of number of objects and storage space required)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How big is your collection? (In terms of number of objects and storage space required)</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* As of May 2011, HathiTrust is preserving 8.7 million print volumes which consume approximately 400TB of storage.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What are your performance requirements?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What are your performance requirements?</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* Checksum validation requires all data to be read over an approximate 90-day periodicity, which, at the current repository size, translates to 54MB/s read activity which is attainable through parallelization. The most recent peak in the ingest rate was ~500,000 volumes per month which translates to an average of ~7MB/s writing. </ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What storage media have you elected to use? (Disk, Tape, etc)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What storage media have you elected to use? (Disk, Tape, etc)</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* HathiTrust uses two instances of disk in different cities (several hundred miles apart) for primary storage and two instances of tape in one city (several miles apart) for backup and 6 months of previous-version retention.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think the key advantages of the system you use?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think the key advantages of the system you use?</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* Being NAS-based, the storage environment is a simple filesystem which offers easy integration and movement between vendor platforms. The Isilon system has a number of advanced data integrity features that are well-suited to digital preservation including internal checksums and inline data correction to protect against bit rot, misdirected/torn writes, and similar data storage risks.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key problems or disadvantages your system present?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key problems or disadvantages your system present?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div># What important principles informed your decision about the particular tool or service you chose to use?  </div></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* The simplicity of filesystems can be seen as a risk for the same reason it is beneficial; without imposed structure, filesystems can accommodate informal and unrigorous access to data. When using a filesystem, the price paid for simplicity is that greater attention is required to permissions and formalized processes for data management.</ins></div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div># What important principles informed your decision about the particular tool or service you chose to use?</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* The storage solution was chosen for its ability to scale in capacity and performance while keeping maintenance requirements constant and low.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another?</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* HathiTrust migrated to Isilon storage in 2007 from temporary DAS and has not since needed to do another migration, although the first cycle of hardware replacement (100TB at both locations) was completed using the built-in capability for removing and adding storage nodes; there was no direct handling of data required.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* The Isilon system enables us to have greater parity redundancy than conventional storage systems: we use N+3, as opposed to the conventional RAID 5 (N+1) or slightly novel RAID 6 (N+2). The latest release of Isilon system software computes checksums on write and validates checksums on read, correcting data as needed, and the cluster architecture scales the compute capacity required to do this work, as opposed to conventional head/tray design. The system provides features for built-in data migration for hardware replacement that eliminate the need for manual data handling or even downtime during hardware replacement.</ins></div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)</div></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div></td></tr> <tr><td colspan="2" class="diff-side-deleted"></td><td class="diff-marker" data-marker="+"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">#* There are no issues or shortcomings with respect to TRAC requirements.</ins></div></td></tr> <!-- diff cache key wiki_production-dlfwiki_:diff:1.41:old-2517:rev-2518:php=table --> </table>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2517&oldid=prev 2011-05-16T15:34:34Z

<p><span class="autocomment">HathiTrust Response to Implementations of Large Scale Storage Architectures</span></p> <table style="background-color: #fff; color: #202122;" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">← Older revision</td> <td colspan="2" style="background-color: #fff; color: #202122; text-align: center;">Revision as of 11:34, 16 May 2011</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l2">Line 2:</td> <td colspan="2" class="diff-lineno">Line 2:</td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><br></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?  </div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice?  </div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)  </div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.)  </div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How big is your collection? (In terms of number of objects and storage space required)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How big is your collection? (In terms of number of objects and storage space required)</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What are your performance requirements?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What are your performance requirements?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What storage media have you elected to use? (Disk, Tape, etc)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What storage media have you elected to use? (Disk, Tape, etc)</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think the key advantages of the system you use?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think the key advantages of the system you use?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key problems or disadvantages your system present?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What do you think are the key problems or disadvantages your system present?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What important principles informed your decision about the particular tool or service you chose to use?  </div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What important principles informed your decision about the particular tool or service you chose to use?  </div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># How frequently do you migrate from one system to another?</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)</div></td></tr> <tr><td class="diff-marker" data-marker="−"></td><td style="color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2" class="diff-side-added"></td></tr> <tr><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div></td><td class="diff-marker"></td><td style="background-color: #f8f9fa; color: #202122; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div># Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div></td></tr> <!-- diff cache key wiki_production-dlfwiki_:diff:1.41:old-2516:rev-2517:php=table --> </table>

Csnavely https://wiki.diglib.org/index.php?title=NDSA:HathiTrust&diff=2516&oldid=prev 2011-05-16T15:34:16Z

<p>Created page with &#039;==HathiTrust Response to Implementations of Large Scale Storage Architectures== # What is the particular preservation goal or challenge you need to accomplish? (for example, re-…&#039;</p> <p><b>New page</b></p><div>==HathiTrust Response to Implementations of Large Scale Storage Architectures==<br /> <br /> # What is the particular preservation goal or challenge you need to accomplish? (for example, re-use, public access, internal access, legal mandate, etc.)<br /> <br /> # What large scale storage or cloud technologies are you using to meet that challenge? Further, which service providers or tools did you consider and how did you make your choice? <br /> <br /> # Specifically, what kind of materials are you preserving (text, data sets, images, moving images, web pages, etc.) <br /> <br /> # How big is your collection? (In terms of number of objects and storage space required)<br /> <br /> # What are your performance requirements?<br /> <br /> # What storage media have you elected to use? (Disk, Tape, etc)<br /> <br /> # What do you think the key advantages of the system you use?<br /> <br /> # What do you think are the key problems or disadvantages your system present?<br /> <br /> # What important principles informed your decision about the particular tool or service you chose to use? <br /> <br /> # How frequently do you migrate from one system to another?<br /> <br /> # What characteristics of the storage system(s) you use do you feel are particularly well-suited to long-term digital preservation? (High levels of redundancy/resiliency, internal checksumming capabilities, automated tape refresh, etc)<br /> <br /> # What functionality or processes have you developed to augment your storage systems in order to meet preservation goals? (Periodic checksum validation, limited human access or novel use of permissions schemes)<br /> <br /> # Are there tough requirements for digital preservation, e.g. TRAC certification, that you wish were more readily handled by your storage system?</div>

Csnavely