jump to navigation

NARA to Declassify 400 Million Pages of Documents in Three Years 2011/12/06

Posted by nydawg in Archives, Digital Archives, Digital Preservation, Electronic Records, Information Technology (IT), Media, Records Management.
Tags: , , , , , ,
add a comment

For a very long time, I have been trying to ask anyone who knows (from my colleagues to the AOTUS himself), why are we even attempting to preserve 250 million emails created during the Bush Administration.  As I’ve mentioned before, that works out to nearly one email every second for eight years!  (And remember, part of that time included Bush’s annual month-long vacations.)  So this story really seemed to give a bit of context in ways that the National Archives (NARA) deals with processing large collections of backlog materials.  “All of these pages had been piling up here, literally,” said Sheryl J. Shenberger, a former CIA official who is the head of the National Declassification Center (NDC) at the National Archives. “We had to develop a Costco attitude: We had 400 million pages . . . and we have three years to do them in.”

If you read Saturday’s article in the Washington Post, you’ll learn that “All of the backlogged documents date back 25 years or more, and most are Cold War-era files from the departments of Defense, State and Justice, among other agencies. The CIA manages the declassification of its own files.”  and that ““The current backlog is so huge that Americans are being denied the ability to hold government officials accountable for their actions,” [AOTUS David] Ferriero said. “By streamlining the declassification process, the NDC will usher in a new day in the world of access.”

If NARA is really trying to declassify, process, catalog, describe, preserve and make these pages available, I hope they’re planning on hiring some more archivists!  The problem is that when institutions are dealing with mass quantities of materials, the (quantitative) metrics we use, may actually hurt us in the future.  In the archival world, the prevailing wisdom seems to be MPLP (More Product, Less Process), but I would argue that archivists need to have qualitative metrics as well, if only to ensure that they are reducing redundancies and older, non-needed versions.  This gets to the crux of the distinction between best practices for records managers and best practices for digital asset managers (or digital archivists).  Ideally, a knowledgeable professional will collect and appraise these materials, and describe it in a way, so that a future plan can be created to ensure that these assets (or records) can be migrated forward into new formats accessible on emerging (or not-yet invented) media players and readers.

Ultimately, this leads to the most serious problem facing archivists: the metadata schemas that are most popular (DublinCore, IPTC, DACS, EAD, etc.) are not specific enough to help archivists plan for the future.  Until our metadata schemas can be updated to ensure that content, context, function, structure, brand, storage media and file formats can be specifically and granularly identified and notated, we will continue paddling frantically against the digital deluge with no workable strategy or plan, or awareness of potential problems (e.g. vendor lock-in, non-backwards compatible formats, etc.)  Sadly, in the face of huge quantities of materials (emails and pages), NARA will probably embrace MPLP, and ultimately hinder and hurt future access to the most important specific files, pages, emails, etc., because they will refuse to hire more professionals to do this work, and will (probably) rely on computer scientists and defense contractors to whitewash the problems and sell more software.

Advertisements

Comparing Documentation Strategy of Civil War and First Gulf War 2011/11/21

Posted by nydawg in Archives, Best Practices, Digital Archives, Digital Preservation, Media, Records Management.
Tags: , , , ,
add a comment

I’ve said it before, and I’ll say it again (paraphrasing someone
else): “We are at risk of knowing less about the events leading up to
the First Gulf War than events leading up to the Civil War, because
all of the records and documents from the Civil War were conserved and
preserved, whereas all the records from the First Gulf War were
created on Wang Wordprocessors and never migrated forward and now lost
forever.”

Case in point: Lincoln at Gettysburg; photo by Matthew Brady
http://blogs.archives.gov/prologue/?p=2564

Or 1991 Gulf War speech by Sec of Def Cheney:
http://en.wikipedia.org/wiki/File:Cheney_Gulf_War_news_conference.jpg
http://upload.wikimedia.org/wikipedia/commons/thumb/5/52/Powell,_Schw…

or http://www.pbs.org/mediashift/2007/08/the-tangled-state-of-archived-n…

Adobe Abandons Mobile Flash Video (Over Steve Jobs’ Dead Body) 2011/11/10

Posted by nydawg in Archives, Digital Preservation, Information Technology (IT), Intellectual Property, Media.
Tags: , , , , , , ,
add a comment

Wired Magazine ran an interesting news story that many have been expecting!  “On Wednesday morning, Adobe delivered the eulogy for its multi-media Flash platform for mobile, stating the company would no longer invest resources in porting its once-indispensable cross-browser technology to smartphones and tablets.  It’s a startling admission of failure from a company that vehemently defended Flash and its mobile strategy in the face of Apple’s refusal to allow it on the iPhone and iPad. Adobe even took on Steve Jobs in a war of words over Flash’s viability as a mobile platform, all in the public domain.  But the writing was on the wall for Flash years ago, and Adobe knew it. With no Flash announcements to be heard at its Adobe Max conference earlier this year and with the company slowly beefing up its toolkit of Flash alternatives, Wednesday’s move is in step with Adobe’s broader strategy of migrating its loyal Flash developer base to a new era, one where mobile platforms reign supreme.”

It’s interesting to watch how these advancements will change our archiving strategies as older formats are retired and/or unsupported.  Everyone knows that the H.264 codec is more energy-efficient, but is the quality also better, and is it worth those license fees?!  So  just for fun, you might want to check out Steve Jobs’ “Thoughts on Flash” from April 2010:   “I wanted to jot down some of our thoughts on Adobe’s Flash products so that customers and critics may better understand why we do not allow Flash on iPhones, iPods and iPads. Adobe has characterized our decision as being primarily business driven – they say we want to protect our App Store – but in reality it is based on technology issues. Adobe claims that we are a closed system, and that Flash is open, but in fact the opposite is true. Let me explain.”

dk
###

Carnegie and Other Library Philanthropy Misrepresented by NYTimes 2011/11/09

Posted by nydawg in Archives.
Tags: , , , , ,
add a comment

Leave it to the NYTimes to misrepresent Andrew Carnegie’s library philanthropy in hopes to suggest that one former Microsoft employee is even more generous. This article seems to be implying that any collection of “Dr. Seuss of Cambodia” books or unsold books of “Keeping Up with the Kardashians” would also qualify as a library! In my opinion, the brilliance of Carnegie’s Library philanthropy was that it required municipalities to make a commitment and buy in, and to provide land, continued funding and books, but we all know (don’t we?) that the best libraries actually hire professional librarians.

“ONE of the legendary triumphs of philanthropy was Andrew Carnegie’s construction of more than 2,500 libraries around the world. It’s renowned as a stimulus to learning that can never be matched — except that, numerically, it has already been surpassed several times over by an American man you’ve probably never heard of.. . . “He faced one challenge after another, not only in opening libraries but also in filling them with books that kids would want to
read. ”

http://www.nytimes.com/2011/11/06/opinion/sunday/kristof-his-libraries-12000-so-far-change-lives.html

NARA’s Erratic ERA Offers No Content-Searching 2011/10/29

Posted by nydawg in Archives, Digital Archives, Electronic Records, Records Management.
Tags: , , ,
add a comment

Many of us have been watching the unruly boondoggle of NARA’s ERA over
the years, but this story seems a bit overdue. . . . In a nutshell,
“Searching text impossible on NARA’s e-Records Archive”.
I hope soon they’ll take on the task of separating the wheat from the
chaff of those 250million Bush emails. (nearly one [out-of-office?]
email every second for 8 years)

“People trying to search the text of documents through the National
Archives and Records Administration’s $430 million Electronic Records
Archive are going to be disappointed, according to the agency’s
inspector general.  Under the currently deployed system, users can
search only by metadata. That typically includes tags for information
such as name of the original publication, date of publication, agency
that originated the document, and a small number of keywords. Users
who hope to locate a document by a word or phrase that isn’t part of
the metadata will be unable to. . . .

The public’s ability to use the ERA is likely to be hampered because
of the lack of a full text-based search capability, which would be
similar to what is available on Google.com or other commercial search
engines, NARA Inspector General Paul Brachfeld said in an interview on
Oct. 26.  Lack of full text search “is one of the profound problems
with the ERA at this point,” Brachfeld said. “Metadata alone does not
tell the story of what is in the documents.””

http://fcw.com/articles/2011/10/26/nara-electronic-archive-has-fundamental-flaw-in-search–it-says.aspx

#OccupyWallStreet Call for Information Managers, Librarians & Archivists 2011/10/05

Posted by nydawg in Archives, Education, Media.
Tags: , , , , ,
add a comment

I don’t know if anybody else has had an opportunity to walk around the
Liberty Plaza #OccupyWallSt #ows festival, but it is truly
incredible . . . Smart people are living there & there’s plenty of
free food donated throughout the day from around the world, and most
importantly, a lending library!  It’s true, i saw it with my own
eyes.  They also have drum circles, Helen Caldicott (last night) and
today, at 4:30 they’re co-hosting a big union rally!

“As the OccupyWallStreet protest movement has held firm and spread
since its inception September 17, the northeast corner of Zuccotti
Park (renamed Liberty Plaza by the protesters) in lower Manhattan has
become the home for the budding revolution’s People’s Library.  The
library already has a website, which proclaims that “information is
liberation,” and this morning, October 5, a “call for librarians” went
out.

“We need help building our catalog and writing our history. Our
readers are enthusiastic and some of them need help finding the right
book,” the post reads. “The right book for the right reader is
fundamental to successful librarianship, so we need public services
folks to come out and conduct reference interviews with people and
help them find ‘their’ book.”

http://www.libraryjournal.com/lj/home/892288-264/as_a_revolution_takes_root.html.csp
check out the library, and maybe you’ll find those American Archivists
back issues i dumped!
http://peopleslibrary.wordpress.com/

WikiLeaks’ Cablegate Links State Dept. Bureau of Diplomatic Security to Madness 2011/09/28

Posted by nydawg in Archives, Digital Archives, Digital Preservation, Electronic Records, Information Technology (IT), Media, Privacy & Security, Records Management, WikiLeaks.
Tags: , , , ,
add a comment

For the last year or so, I’ve been fascinated by the whole WikiLeaks Cablegate story.  As I posted previously, there are a number of factors that contribute to this story which make it particularly interesting for people concerned with records  management and best practices for accessing and sharing information.   In my opinion, Private first class Bradley Manning is a fall guy (lipsynching to Lady Gaga), but problems revealed serious systemic malfunctions.  So I was very interested to read this article by Andy Kroll: “The Only State Dept. Employee Who May Be Fired Over WikiLeaks“.

Peter Van Buren is no insurgent. Quite the opposite: For 23 years he’s worked as a foreign service officer at the State Department, and a damn good one from the looks of it. He speaks Japanese, Mandarin Chinese, and Korean; served his country from Seoul to Sydney, Tokyo to Baghdad; and has won multiple awards for his disaster relief work. So why was Van Buren treated like a terror suspect by his own employer? For linking to a single leaked cable dumped online by WikiLeaks earlier this month.”

Well, this led me to read a TomDispatch.com posting by Van Buren himself which offers a clear-headed look at the madness!  For one thing, Van Buren got into a heap of trouble and was “under investigation for allegedly disclosing classified information” for LINKING to a WikiLeaks document which was already on the Web!  As he put it: “two DS agents stated that the inclusion of that link amounted to disclosing classified material. In other words, a link to a document posted by who-knows-who on a public website available at this moment to anyone in the world was the legal equivalent of me stealing a Top Secret report, hiding it under my coat, and passing it to a Chinese spy in a dark alley.”

Van Buren goes on to analyze the situation by stating: “Let’s think through this disclosure of classified info thing, even if State won’t. Every website on the Internet includes links to other websites. It’s how the web works. If you include a link to say, a CNN article about Libya, you are not “disclosing” that information — it’s already there. You’re just saying: “Have a look at this.”  It’s like pointing out a newspaper article of interest to a guy next to you on the bus.  (Careful, though, if it’s an article from the New York Times or the Washington Post.  It might quote stuff from Wikileaks and then you could be endangering national security.)”

And, for me, the cherry on the top, and something I’ve been trying to state for most of the last year (including at the Archivists Round Table of Metropolitan New York meeting in January 2011), is the fact that “No one will ever be fired at State because of WikiLeaks — except, at some point, possibly me. Instead, State joined in the Federal mugging of Army Private Bradley Manning, the person alleged to have copied the cables onto a Lady Gaga CD while sitting in the Iraqi desert. That all those cables were available electronically to everyone from the Secretary of State to a lowly Army private was the result of a clumsy post-9/11 decision at the highest levels of the State Department to quickly make up for information-sharing shortcomings. Trying to please an angry Bush White House, State went from sharing almost nothing to sharing almost everything overnight. They flung their whole library onto the government’s classified intranet, SIPRnet, making it available to hundreds of thousands of Federal employees worldwide. . . . . State did not restrict access. If you were in, you could see it all. There was no safeguard to ask why someone in the Army in Iraq in 2010 needed to see reporting from 1980s Iceland. . . . . Most for-pay porn sites limit the amount of data that can be downloaded. Not State. Once those cables were available on SIPRnet, no alarms or restrictions were implemented so that low-level users couldn’t just download terabytes of classified data. If any activity logs were kept, it does not look like anyone checked them.

In other words, by pointing the finger of blame at a few (two) bad apples (Pfc Manning and Foreign Services Officer/ Author Van Buren), “… gets rid of a “troublemaker,” and the Bureau of Diplomatic Security people can claim that they are “doing something” about the WikiLeaks drip that continues even while they fiddle.”  Yet, the State Department and the Department of Defense still refuse to acknowledge the systemic problems of trying to provide UNRESTRICTED and UNTRACEABLE ACCESS to ALL CABLES to all LEVELS of employees from the highest administrative levels at State and Defense  to the lowliest of the low  (Private first class on probation or a contractor, like Aaron Barr, working in White Hat or Black Hat Ops.)  Okay, according to Homeland Security Today, there’s 3 million people (not just Americans, btw) with “secret” clearance and “only” half a million with access to SIPRNet!

This still strikes me as an example of the US acting like ostriches and burying its head so we will not have to acknowledge the serious problems that are all around us.  Mark my words: the system is still broken, and even though certain changes have been instituted (thumb drive bans), we have a much more serious and systemic problem which few dare to acknowledge.  What’s the solution?  Better appraisal and better records management!

No one will ever be fired at State because of WikiLeaks — except, at some point, possibly me. Instead, State joined in the Federal mugging of Army Private Bradley Manning, the person alleged to have copied the cables onto a Lady Gaga CD while sitting in the Iraqi desert. That all those cables were available electronically to everyone from the Secretary of State to a lowly Army private was the result of a clumsy post-9/11 decision at the highest levels of the State Department to quickly make up for information-sharing shortcomings. Trying to please an angry Bush White House, State went from sharing almost nothing to sharing almost everything overnight. They flung their whole library onto the government’s classified intranet, SIPRnet, making it available to hundreds of thousands of Federal employees worldwide.

CLIR: Future Generations Will Know More About the Civil War than the Gulf War 2011/09/22

Posted by nydawg in Archives, Best Practices, Digital Archives, Education, Electronic Records, Information Technology (IT), Records Management.
Tags: , , , , , , , , , , , ,
add a comment

When I was in Queens College Graduate Library School six years ago, I took Professor Santon’s excellent course in Records Management which led me to understand that every institution has to manage its records and its assets and Intellectual Property.   The vital role the archive and records center play for every day use and long-term functions was made clear by the fact that records have a life cycle, basically creation – – use – – destruction or disposition.   The course was excellent, despite the fact that the main text books we used were from the early 1990s (and included a 3 1/4″ floppy that ran on Windows 3.1).

While doing an assignment, I found a more recent article which really led me to a revelation: electronic records will cause a lot of problems!  The one part that stuck out most and I still remember to this day was in a 2002 article “Record-breaking Dilemma” in Government Technology.  “The Council on Library and Information Resources, a nonprofit group that supports ways to keep information accessible, predicts that future generations will know more about the Civil War than the Gulf War. Why? Because the software that enables us to read the electronic records concerning the events of 1991 have already become obsolete. Just ask the folks who bought document-imaging systems from Wang the year that Saddam Hussein invaded Kuwait. Not only is Wang no longer in business, but locating a copy of the proprietary software, as well as any hardware, used to run the first generation of imaging systems is about as easy as finding a typewriter repairman. ” (emphasis added)

Obviously that article impacted my thinking about the Digital Dark Ages greatly, and it got me to wondering what will best practices be for managing born-digital assets or electronic records for increasingly long periods of time on storage media that is guaranteed for decreasing periods of time.  Or  “”We’re constantly asking ourselves, ‘How do we retain and access electronic records that must be stored permanently?'” she said. ”  Well, this gets to the crux of the issue, especially when records managers and archivists aren’t invited into the conversations with IT.  So when we are using more and more hard drives (or larger servers even in the cloud), “Hard-drive Makers Weaken Warranties“.  In a nutshell : “Three of the major hard-drive makers will cut down the length of warranties on some of their drives, starting Oct. 1, to streamline costs in the low-margin desktop disk storage business.”

So if we’re storing more data on storage media that is not for long-term preservation, then records and archival management must be an ongoing relay race, with appropriate ongoing funding and support, as more and more materials are copied or moved from one storage medium to another, periodically, every 3-5 years (or maybe that will soon be  1-3 years?).   Benign neglect is no longer a sound records management strategy.

That’s the technological challenge.  But there’s more!  I’ve gone on and on and on before about NARA’s ERA program and how one top priority is to ingest 250 million emails from the Bush Administration.  (I’ve done the math, it works out to nearly one email every second of the eight years.)  So we know that NARA is interested in preserving electronic records.  But a couple years ago I read this scary Fred Kaplan piece, “PowerPoint to the People: The urgent need to fix federalarchiving policies” in which he learned that “Finally—and this is simply stunning—the National Archives’ technology branch is so antiquated that it cannot process some of the most common software programs. Specifically, the study states, the archives “is still unable to accept Microsoft Word documents and PowerPoint slides.””

Uhhhhh, wait!  Well, at least that was written in 2009, so we can hope they have gotten their act together, but if you think about it too much, you might wonder if EVERYTHING NEEDED TO ARCHIVE IS ON MICROSOFT’S PROPRIETARY FORMATS?  Or you might just be inspired to ask if anyone really uses Powerpoint in the military.  Well, as Kaplan points out “This is a huge lapse. Nearly all internal briefings in the Pentagon these days are presented as PowerPoint slides. Officials told me three years ago that if an officer wanted to make a case for a war plan or a weapons program or just about anything, he or she had better make the case in PowerPoint—or forget about getting it approved.”  Or this piece from the NYTimes “We Have Met the Enemy and He Is Powerpoint” in which “Commanders say that behind all the PowerPoint jokes are serious concerns that the program stifles discussion, critical thinking and thoughtful decision-making. Not least, it ties up junior officers — referred to as PowerPoint Rangers — in the daily preparation of slides, be it for a Joint Staff meeting in Washington or for a platoon leader’s pre-mission combat briefing in a remote pocket of Afghanistan.”

We Have Met the Enemy, and He Is PowerPoint

Keep Bit Rot at Bay: Change is Afoot as LoC’s DPOE Trains the Trainers 2011/09/20

Posted by nydawg in Archives, Best Practices, Digital Archives, Digital Archiving, Digital Preservation, Information Technology (IT), Media.
Tags: , , , , ,
add a comment

This was forwarded to me by a nydawg member who subscribes to the UK’s Digital Preservation listserv.  I don’t know if  it’s been posted publicly in the US, but I guess this first one is by invitation-only.  I would LOVE to hear what they are teaching and how they are doing it, so I hope someday to attend as well.

Library of Congress To Launch New Corps of Digital Preservation Trainers

The Digital Preservation Outreach and Education program at the Library of Congress will hold its first national train-the-trainer workshop on September 20-23, 2011, in Washington, DC.

The DPOE Baseline Workshop will produce a corps of trainers who are equipped to teach others, in their home regions across the U.S., the basic principles and practices of preserving digital materials.  Examples of such materials include websites; emails; digital photos, music, and videos; and official records.

The 24 students in the workshop (first in a projected series) are professionals from a variety of backgrounds who were selected from a nationwide applicant pool to  represent their home regions, and who have at least some familiarity with community-based training and with digital preservation. They will be instructed by the following subject matter experts:

*   Nancy McGovern, Inter-university Consortium for Political and Social  Research, University of Michigan
*   Robin Dale, LYRASIS
*   Mary Molinaro, University of Kentucky Libraries
*   Katherine Skinner, Educopia Institute and MetaArchive Cooperative
*   Michael Thuman,  Tessella
*   Helen Tibbo, School of Information and Library Science, University of  North Carolina at Chapel Hill, and Society of American Archivists.

The curriculum has been developed by the DPOE staff and expert volunteer advisors and informed by DPOE-conducted research–including a nationwide needs-assessment survey and a review of curricula in existing training programs. An outcome of the September workshop will be for each participant to, in turn, hold at least one basic-level digital-preservation workshop in his or her home U.S. region by mid-2012.

The intent of the workshop is to share high-quality training in digital preservation, based upon a standardized set of core
principles, across the nation.  In time, the goal is to make the training available and affordable to virtually any interested
organization or individual.

The Library’s September 2011 workshop is invitation-only, but informational and media inquiries are welcome to George Coulbourne, DPOE Program Director, at gcou@loc.gov.

The Library created DPOE  in 2010.  Its mission is to foster national outreach and education to encourage individuals and organizations to actively preserve their digital content, building on a collaborative network of instructors, contributors and institutional partners. The DPOE website is www.loc.gov/dpoe
http://digitalpreservation.gov/education/.  Check out the curriculum and course offerings here.

 

dk
###

U.S. Marshals’ Gross Mismanagement Undervalues Complex Assets 2011/09/20

Posted by nydawg in Archives, Electronic Records, Records Management.
Tags: , , , , ,
add a comment

People often wonder, “Why does anyone need a good, honest, ethical records manager anyway?”  or “Why don’t universities offer better programs in records management?” and “why do people responsible for hiring records managers not understand what records managers should do?” or maybe “what does the USMS Ethics Officer do?”

I wish I knew the answers, but here’s an interesting article revealing how shoddy records management can be described as Gross Mismanagement! “The unit of the United States Marshals Service that manages complex assets seized in criminal cases, including that of Bernard L. Madoff, kept such shoddy records that it could not say who bought assets or how much was paid for them in 8 of the 55 sales it handled from 2005 to 2010, according to an audit released on Tuesday. . . . . The report said that the auditors found significant problems in how the service managed complex assets and that those problems “increased the risk that the government could mismanage the administration and disposition of forfeited assets.” The team disposed of $136 million in seized and forfeited assets from January 2005 to August 2010. “This audit identified numerous deficiencies in the procedures the complex asset team implemented to track, safeguard, value and dispose of complicated and valuable assets,” the report said. The report added that the team’s overseer, the Asset Forfeiture Division, did not “vigorously oversee” the team. Read all about it in “Auditors Find Chaos in U.S. Marshal’s Asset Sale Record-Keeping“!

So where are all those jobs anyway? From the audit: “Additionally, we found that the limited staff and resources of the Complex Asset Team were disproportionate to its responsibilities.  From 2005 to 2009, the number of staff varied between two and four individuals. . . . While the 14 forfeiture financial specialist contractors had extensive experience relevant to forfeiture, their primary assignment during Briskman’s tenure was to assist USMS district offices and not the Complex Asset Team.” . . . “Between 2005 and 2010, the small staff of the Complex Asset Team disposed of over $136 million in assets, yet it operated in an environment lacking the procedures to guide its actions and decisions pertaining to seized and forfeited assets.” . . . Uh, hold on, so four employees were responsible for keeping records on $136 million in assets?! $34 million per employee?! I wonder what their daily salaries were!

“Further, we identified inaccuracies in some monthly reports Briskman compiled. The summaries Briskman provided for many assets stated, “No change from previous report.” Yet, when we compared these entries to previous entries for such assets, we noted that some of the entries contained unexplained changes from the previous monthly status report. For example, the listed taxable profit amount for one asset varied from $9.5 million to $12 million between different reports; however, the entries provided to summarize the changes for this particular asset in subsequent reports were “no change from previous report.”

and check out the recommendations including: “Recommendation 20: Ensure that managers know that they must thoroughly review financial disclosure forms and disclose any potential conflicts of interest to the USMS ethics office.”  Read the DoJ “AUDIT OF THE UNITED STATES MARSHALS SERVICE
COMPLEX ASSET TEAM MANAGEMENT AND OVERSIGHT” here.

dk