NDSA:September 15, 2014 Standards and Practices Working Group Agenda and Notes: Difference between revisions
(Created page with '== New member introduction == == Announcements & Project Updates == *Video survey update *Email exploration meeting == Today's discussion topic: Optical Media == Topics and S…') |
m (5 revisions imported: Migrate NDSA content from Library of Congress) |
||
(4 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
'''Meeting minutes - NDSA Standards and Practices, September 15, 2014 | |||
*Minutes: Lauren Sorensen, [mailto:lsor@loc.gov email me] with any changes/updates''' | |||
== Announcements & Project Updates == | |||
Kate Murray: Ranking stumbling blocks for video preservation, put a call out to NDSA members, meeting on 26th of Sept. Contact Kate at kmur@loc.gov to join for meeting details | |||
Andrea Goethals: next call S&P call will be on October 20th - continue discussion on optical media 1-2pm. | |||
August 25th, a closed meeting happened about preserving and accessioning email records; attendees included Stanford, Harvard, NARA. Interest in opening this up to any NDSA members working with this material, aim to host listservs and list of toolsets. Coming soon: Signal blog post from Chris Prom about this meeting. | |||
Different aspects of optical discs; still interest in our group. Welcome anyone working in this area to contribute to discussions. | |||
== Today's discussion topic: Optical Media == | == Today's discussion topic: Optical Media == | ||
Topics and Speakers: | Topics and Speakers: | ||
<div style="column-count:2;-moz-column-count:2;-webkit-column-count:2"> | <div style="column-count:2;-moz-column-count:2;-webkit-column-count:2"> | ||
*Longevity of Optical Media Discs, Michele Youket, Library of Congress http://www.theatlantic.com/technology/archive/2014/05/the-library-of-congress-wants-to-destroy-your-old-cds-for-science/370804/ | *Longevity of Optical Media Discs, '''Michele Youket, Library of Congress''' http://www.theatlantic.com/technology/archive/2014/05/the-library-of-congress-wants-to-destroy-your-old-cds-for-science/370804/ | ||
*Introduction to Optical Media Preservation, Alex Duryee, AV Preserve http://www.avpreserve.com/wp-content/uploads/2014/04/OpticalMediaPreservation.pdf | *Introduction to Optical Media Preservation, '''Alex Duryee, AV Preserve''' http://www.avpreserve.com/wp-content/uploads/2014/04/OpticalMediaPreservation.pdf | ||
*Preserving Write-Once DVDs: Producing Disc Images, Extracting Content, and Addressing Flaws and Errors, Morgan Morel and George Blood, George Blood Audio Video Film | *Preserving Write-Once DVDs: Producing Disc Images, Extracting Content, and Addressing Flaws and Errors, '''Morgan Morel and George Blood, George Blood Audio Video Film''' http://www.digitizationguidelines.gov/audio-visual/documents/Preserve_DVDs_BloodReport_20140901.pdf | ||
*Data Extraction from Rewritable CDs and DVDs, John Passmore, WNYC | *Data Extraction from Rewritable CDs and DVDs, '''John Passmore, WNYC''' http://blogs.loc.gov/digitalpreservation/2014/02/getting-public-radios-legacy-off-ageing-rewritable-cds-an-interview-with-wnycs-john-passmore/?loclr=blogsig | ||
*Brief Introduction to M-Disc and Cost Comparisons for Data Storage, John Spencer, BMS/Chace | *Brief Introduction to M-Disc and Cost Comparisons for Data Storage, '''John Spencer, BMS/Chace''' | ||
</div> | </div> | ||
---- | |||
Speakers today: '''Michelle Youket (LOC), Alex Duryee (AVPS), Morgan Morel (George Blood Audio & Video).''' | |||
*'''Michelle Youket (LOC)''' | |||
**Goal is to develop strategies for preservation of optical media | |||
**Natural and accelerated aging | |||
**1996 - pilot study - 125 random optical discs were selected from LOC collections | |||
**Monitor effects of storage and use | |||
**Later expanded to 1200 discs and wide range of dates of manufacture | |||
**Follow 1SO 18921 standard | |||
**Adhesive labeled disc after 1000 hours aging error rates higher | |||
**Measured effects of laser engraving | |||
**CD-R more stable than DVD - larger size of data pitch for CD, more data to be corrupted in a smaller area with DVD. | |||
**Dye and reflective composition - significant factor in degradation - gold more stable than silver. | |||
**Cyanine, meant as long strategy dye. | |||
**Phthalocyanine - at times, hard to differentiate upon inspection. | |||
**LOC created error checker software as an assessment tool. | |||
**JVC archival disc system error checker disc drive | |||
**Disc-to-disc migration | |||
**Burn tests | |||
**Report fields: media / brand / dye / jitter | |||
*'''Alex Duryee (AudioVisual Preservation Solutions)''' [https://twitter.com/archivetype @archivetype] | |||
**Article, meant to be intro to optical disc preservation: http://www.avpreserve.com/wp-content/uploads/2014/04/OpticalMediaPreservation.pdf | |||
**Research being done by colleagues - studying preservation of physical media but little information exists about the process of extraction and logical structure of what is on a disc. | |||
**What is on this disc? How to view and assess? There is no go-to process for examining. | |||
**Precedent is law enforcement - however, few criminals use CDs and optical discs so often not supported by forensics tools. | |||
**92% migration failure rate for data extraction. | |||
**Researching discs as carriers of data. | |||
**Two major types of discs - audio and data | |||
**Audio: one of first uses of consumer optical media - designed to replace 8 track | |||
**Instead of filesystem paradigm these discs featured a single stream of modulated data running uninterrupted throughout the disc, with byte level metadata, such as track names. | |||
**Audio CDs more akin to tapes and vinyl because of this uninterrupted stream - only metadata differentiates “pieces” on the CD. | |||
**The human ear is bad at detecting small errors - CD audio standard is 44.1khz - 16 bit depth. | |||
**Even the best consumer/ professional hardware has a 98-99% accuracy in a given read - not good for preservation, as don’t know if you’re getting what you need - is it capturing important metadata? | |||
**CD-ROM/ data CDs - ISO 9660 - Similar to audio discs except broken to sessions and tracks | |||
**See it in a file browser and directories - convenient for archivists, what you see is what you get. | |||
**Data cds can contain multiple filesystems - older, early mid-90s HFS file systems were common. | |||
**Can contain all 3 filesystems and operating system used to extract data will default to one it can read. So: sometimes the OS can’t see the filesystem. | |||
**UDF filesystem started being used, manufacturers getting together to consistently use this filesystem. | |||
**ISO Buster - tool for this filesystem breakdown. | |||
**Other projects | |||
***[http://howtheygotgame.stanford.edu Cabrinety Collection] | |||
***WNYC - Audio disc project | |||
*'''Morgan Morel (George Blood Audio & Video)''' [https://twitter.com/av_morgan @av_morgan] | |||
**500 once written DVDs - American Folklife Collection Veterans History Project. | |||
**Staff at vendor asked to create findings report; outlining troubles and tools used. | |||
**Formats for medium term storage white paper; ISO disc image recommended. | |||
**ISO disc image can be mounted filesystem as if it were inserted into drive | |||
**Extract production master files | |||
**VideoTS folder and AudioTS folder | |||
***[Going through folders as they appear:] VideoTS is all video content - IFO is info about playback and navigation | |||
***VOB - audio and video data _0 menus - limit to 1GB content - need to connect to make more than one to make it playable | |||
***All folders listed needed to be intact | |||
**CLI: *ended up using most for project on-site at George Blood. | |||
***hdiutil* | |||
***dd | |||
***ddrescue* | |||
**GUI: | |||
***Mediagrabber | |||
***DVD Decrypter | |||
***Used hdutil and ddrescure - powerful for automated systems run from server | |||
***make workable iso images in case of damage etc | |||
**Tools for extraction: *ended up using most for project on-site at George Blood. | |||
***GUI: compressor, handbrake, mpegstreamclip, streamz | |||
***CLI: FFmpeg* | |||
**Out of 500 DVDs, 49 were problematic | |||
**9 had physical issues - ok after polishing | |||
**Cloning errors for rest - ddresue could still make working ISO images | |||
**31 were DVD-ROM instead of DVD-video. | |||
**http://www.digitizationguidelines.gov/audio-visual/documents/IntrmMastVidFormatRecs_20111001.pdf | |||
*'''John Passmore (WNYC)''' [https://twitter.com/WNYCarchives @WNYCarchives] | |||
**30,000 CD-DA - CD-Rs with encoded audio | |||
**Created in early 2000s by WNYC staff | |||
**Considered preservation masters for that era | |||
**Group of CDs were made using same machines, same brands of discs, so in a sense a control group | |||
**CDs function as material for the reference library for a long time - Librarians noticed that pulling CDs some were not playing, ripping properly, determined a need to get data off of the CDs ASAP, purchased equipment and made a RIP station. | |||
**Ripped about 8k CDs, dumping wav files at digital asset management station, cataloging. | |||
**First wanted to test some to learn about increase in errors and condition of collection. | |||
**Plextor drives used. | |||
**Large spectrum of discs regardless of year they were from and different levels of accessibility. | |||
**2400 Mitsui silver CD-Rs | |||
**20% of silver cds measuring previously detectable errors | |||
**None passed IASA’s specs for errors - technicians instead looking for what CD is getting kicked out and why. | |||
**Not gradual increase based on date - totally random variables | |||
**Conclusion: CD-DAs don’t last long and are unpredictable, accelerating aging only part of issues with optical media - hardware to make data, care and handling etc. | |||
**More open source solutions would be great to use with hardware that exists | |||
**Contact John Passmore at WNYC: jpassmore@nypublicradio.org | |||
---- | |||
Kate: Running out of time, John Spencer will present on MDisc at next month's meeting. | |||
*'''Discussion''' | |||
**Spencer: Plextor: only been a year that current iteration of the company has been making their own drives and have dropped support for some of their software tools. | |||
***Observation that understand nature of companies - rebranding burners. | |||
**Passmore: Optidrive (?), other software | |||
**Duryee: Secondary market now for equipment - price spike in true quality plextor drives - going for a lot used before bought by another company (current iteration of brand). | |||
**Blood: we’re always talking about a system - drive from trusted manufacturer, CD that isn’t playing, sometimes a cheap player can help resolve playability issue. | |||
**Youket: differences in disc composition - many different dyes with different patents - that’s why focus was on developing and using error checkers. | |||
**Blood: Interest in how to optimize burn speeds to get low error rates. Batch variation: Mitsui sold 4 times in 5 years. |
Latest revision as of 15:20, 11 February 2016
Meeting minutes - NDSA Standards and Practices, September 15, 2014
- Minutes: Lauren Sorensen, email me with any changes/updates
Announcements & Project Updates
Kate Murray: Ranking stumbling blocks for video preservation, put a call out to NDSA members, meeting on 26th of Sept. Contact Kate at kmur@loc.gov to join for meeting details
Andrea Goethals: next call S&P call will be on October 20th - continue discussion on optical media 1-2pm.
August 25th, a closed meeting happened about preserving and accessioning email records; attendees included Stanford, Harvard, NARA. Interest in opening this up to any NDSA members working with this material, aim to host listservs and list of toolsets. Coming soon: Signal blog post from Chris Prom about this meeting.
Different aspects of optical discs; still interest in our group. Welcome anyone working in this area to contribute to discussions.
Today's discussion topic: Optical Media
Topics and Speakers:
- Longevity of Optical Media Discs, Michele Youket, Library of Congress http://www.theatlantic.com/technology/archive/2014/05/the-library-of-congress-wants-to-destroy-your-old-cds-for-science/370804/
- Introduction to Optical Media Preservation, Alex Duryee, AV Preserve http://www.avpreserve.com/wp-content/uploads/2014/04/OpticalMediaPreservation.pdf
- Preserving Write-Once DVDs: Producing Disc Images, Extracting Content, and Addressing Flaws and Errors, Morgan Morel and George Blood, George Blood Audio Video Film http://www.digitizationguidelines.gov/audio-visual/documents/Preserve_DVDs_BloodReport_20140901.pdf
- Data Extraction from Rewritable CDs and DVDs, John Passmore, WNYC http://blogs.loc.gov/digitalpreservation/2014/02/getting-public-radios-legacy-off-ageing-rewritable-cds-an-interview-with-wnycs-john-passmore/?loclr=blogsig
- Brief Introduction to M-Disc and Cost Comparisons for Data Storage, John Spencer, BMS/Chace
Speakers today: Michelle Youket (LOC), Alex Duryee (AVPS), Morgan Morel (George Blood Audio & Video).
- Michelle Youket (LOC)
- Goal is to develop strategies for preservation of optical media
- Natural and accelerated aging
- 1996 - pilot study - 125 random optical discs were selected from LOC collections
- Monitor effects of storage and use
- Later expanded to 1200 discs and wide range of dates of manufacture
- Follow 1SO 18921 standard
- Adhesive labeled disc after 1000 hours aging error rates higher
- Measured effects of laser engraving
- CD-R more stable than DVD - larger size of data pitch for CD, more data to be corrupted in a smaller area with DVD.
- Dye and reflective composition - significant factor in degradation - gold more stable than silver.
- Cyanine, meant as long strategy dye.
- Phthalocyanine - at times, hard to differentiate upon inspection.
- LOC created error checker software as an assessment tool.
- JVC archival disc system error checker disc drive
- Disc-to-disc migration
- Burn tests
- Report fields: media / brand / dye / jitter
- Alex Duryee (AudioVisual Preservation Solutions) @archivetype
- Article, meant to be intro to optical disc preservation: http://www.avpreserve.com/wp-content/uploads/2014/04/OpticalMediaPreservation.pdf
- Research being done by colleagues - studying preservation of physical media but little information exists about the process of extraction and logical structure of what is on a disc.
- What is on this disc? How to view and assess? There is no go-to process for examining.
- Precedent is law enforcement - however, few criminals use CDs and optical discs so often not supported by forensics tools.
- 92% migration failure rate for data extraction.
- Researching discs as carriers of data.
- Two major types of discs - audio and data
- Audio: one of first uses of consumer optical media - designed to replace 8 track
- Instead of filesystem paradigm these discs featured a single stream of modulated data running uninterrupted throughout the disc, with byte level metadata, such as track names.
- Audio CDs more akin to tapes and vinyl because of this uninterrupted stream - only metadata differentiates “pieces” on the CD.
- The human ear is bad at detecting small errors - CD audio standard is 44.1khz - 16 bit depth.
- Even the best consumer/ professional hardware has a 98-99% accuracy in a given read - not good for preservation, as don’t know if you’re getting what you need - is it capturing important metadata?
- CD-ROM/ data CDs - ISO 9660 - Similar to audio discs except broken to sessions and tracks
- See it in a file browser and directories - convenient for archivists, what you see is what you get.
- Data cds can contain multiple filesystems - older, early mid-90s HFS file systems were common.
- Can contain all 3 filesystems and operating system used to extract data will default to one it can read. So: sometimes the OS can’t see the filesystem.
- UDF filesystem started being used, manufacturers getting together to consistently use this filesystem.
- ISO Buster - tool for this filesystem breakdown.
- Other projects
- Cabrinety Collection
- WNYC - Audio disc project
- Morgan Morel (George Blood Audio & Video) @av_morgan
- 500 once written DVDs - American Folklife Collection Veterans History Project.
- Staff at vendor asked to create findings report; outlining troubles and tools used.
- Formats for medium term storage white paper; ISO disc image recommended.
- ISO disc image can be mounted filesystem as if it were inserted into drive
- Extract production master files
- VideoTS folder and AudioTS folder
- [Going through folders as they appear:] VideoTS is all video content - IFO is info about playback and navigation
- VOB - audio and video data _0 menus - limit to 1GB content - need to connect to make more than one to make it playable
- All folders listed needed to be intact
- CLI: *ended up using most for project on-site at George Blood.
- hdiutil*
- dd
- ddrescue*
- GUI:
- Mediagrabber
- DVD Decrypter
- Used hdutil and ddrescure - powerful for automated systems run from server
- make workable iso images in case of damage etc
- Tools for extraction: *ended up using most for project on-site at George Blood.
- GUI: compressor, handbrake, mpegstreamclip, streamz
- CLI: FFmpeg*
- Out of 500 DVDs, 49 were problematic
- 9 had physical issues - ok after polishing
- Cloning errors for rest - ddresue could still make working ISO images
- 31 were DVD-ROM instead of DVD-video.
- http://www.digitizationguidelines.gov/audio-visual/documents/IntrmMastVidFormatRecs_20111001.pdf
- John Passmore (WNYC) @WNYCarchives
- 30,000 CD-DA - CD-Rs with encoded audio
- Created in early 2000s by WNYC staff
- Considered preservation masters for that era
- Group of CDs were made using same machines, same brands of discs, so in a sense a control group
- CDs function as material for the reference library for a long time - Librarians noticed that pulling CDs some were not playing, ripping properly, determined a need to get data off of the CDs ASAP, purchased equipment and made a RIP station.
- Ripped about 8k CDs, dumping wav files at digital asset management station, cataloging.
- First wanted to test some to learn about increase in errors and condition of collection.
- Plextor drives used.
- Large spectrum of discs regardless of year they were from and different levels of accessibility.
- 2400 Mitsui silver CD-Rs
- 20% of silver cds measuring previously detectable errors
- None passed IASA’s specs for errors - technicians instead looking for what CD is getting kicked out and why.
- Not gradual increase based on date - totally random variables
- Conclusion: CD-DAs don’t last long and are unpredictable, accelerating aging only part of issues with optical media - hardware to make data, care and handling etc.
- More open source solutions would be great to use with hardware that exists
- Contact John Passmore at WNYC: jpassmore@nypublicradio.org
Kate: Running out of time, John Spencer will present on MDisc at next month's meeting.
- Discussion
- Spencer: Plextor: only been a year that current iteration of the company has been making their own drives and have dropped support for some of their software tools.
- Observation that understand nature of companies - rebranding burners.
- Passmore: Optidrive (?), other software
- Duryee: Secondary market now for equipment - price spike in true quality plextor drives - going for a lot used before bought by another company (current iteration of brand).
- Blood: we’re always talking about a system - drive from trusted manufacturer, CD that isn’t playing, sometimes a cheap player can help resolve playability issue.
- Youket: differences in disc composition - many different dyes with different patents - that’s why focus was on developing and using error checkers.
- Blood: Interest in how to optimize burn speeds to get low error rates. Batch variation: Mitsui sold 4 times in 5 years.
- Spencer: Plextor: only been a year that current iteration of the company has been making their own drives and have dropped support for some of their software tools.