jump to navigation

CLIR: Future Generations Will Know More About the Civil War than the Gulf War 2011/09/22

Posted by nydawg in Archives, Best Practices, Digital Archives, Education, Electronic Records, Information Technology (IT), Records Management.
Tags: , , , , , , , , , , , ,
add a comment

When I was in Queens College Graduate Library School six years ago, I took Professor Santon’s excellent course in Records Management which led me to understand that every institution has to manage its records and its assets and Intellectual Property.   The vital role the archive and records center play for every day use and long-term functions was made clear by the fact that records have a life cycle, basically creation – – use – – destruction or disposition.   The course was excellent, despite the fact that the main text books we used were from the early 1990s (and included a 3 1/4″ floppy that ran on Windows 3.1).

While doing an assignment, I found a more recent article which really led me to a revelation: electronic records will cause a lot of problems!  The one part that stuck out most and I still remember to this day was in a 2002 article “Record-breaking Dilemma” in Government Technology.  “The Council on Library and Information Resources, a nonprofit group that supports ways to keep information accessible, predicts that future generations will know more about the Civil War than the Gulf War. Why? Because the software that enables us to read the electronic records concerning the events of 1991 have already become obsolete. Just ask the folks who bought document-imaging systems from Wang the year that Saddam Hussein invaded Kuwait. Not only is Wang no longer in business, but locating a copy of the proprietary software, as well as any hardware, used to run the first generation of imaging systems is about as easy as finding a typewriter repairman. ” (emphasis added)

Obviously that article impacted my thinking about the Digital Dark Ages greatly, and it got me to wondering what will best practices be for managing born-digital assets or electronic records for increasingly long periods of time on storage media that is guaranteed for decreasing periods of time.  Or  “”We’re constantly asking ourselves, ‘How do we retain and access electronic records that must be stored permanently?'” she said. ”  Well, this gets to the crux of the issue, especially when records managers and archivists aren’t invited into the conversations with IT.  So when we are using more and more hard drives (or larger servers even in the cloud), “Hard-drive Makers Weaken Warranties“.  In a nutshell : “Three of the major hard-drive makers will cut down the length of warranties on some of their drives, starting Oct. 1, to streamline costs in the low-margin desktop disk storage business.”

So if we’re storing more data on storage media that is not for long-term preservation, then records and archival management must be an ongoing relay race, with appropriate ongoing funding and support, as more and more materials are copied or moved from one storage medium to another, periodically, every 3-5 years (or maybe that will soon be  1-3 years?).   Benign neglect is no longer a sound records management strategy.

That’s the technological challenge.  But there’s more!  I’ve gone on and on and on before about NARA’s ERA program and how one top priority is to ingest 250 million emails from the Bush Administration.  (I’ve done the math, it works out to nearly one email every second of the eight years.)  So we know that NARA is interested in preserving electronic records.  But a couple years ago I read this scary Fred Kaplan piece, “PowerPoint to the People: The urgent need to fix federalarchiving policies” in which he learned that “Finally—and this is simply stunning—the National Archives’ technology branch is so antiquated that it cannot process some of the most common software programs. Specifically, the study states, the archives “is still unable to accept Microsoft Word documents and PowerPoint slides.””

Uhhhhh, wait!  Well, at least that was written in 2009, so we can hope they have gotten their act together, but if you think about it too much, you might wonder if EVERYTHING NEEDED TO ARCHIVE IS ON MICROSOFT’S PROPRIETARY FORMATS?  Or you might just be inspired to ask if anyone really uses Powerpoint in the military.  Well, as Kaplan points out “This is a huge lapse. Nearly all internal briefings in the Pentagon these days are presented as PowerPoint slides. Officials told me three years ago that if an officer wanted to make a case for a war plan or a weapons program or just about anything, he or she had better make the case in PowerPoint—or forget about getting it approved.”  Or this piece from the NYTimes “We Have Met the Enemy and He Is Powerpoint” in which “Commanders say that behind all the PowerPoint jokes are serious concerns that the program stifles discussion, critical thinking and thoughtful decision-making. Not least, it ties up junior officers — referred to as PowerPoint Rangers — in the daily preparation of slides, be it for a Joint Staff meeting in Washington or for a platoon leader’s pre-mission combat briefing in a remote pocket of Afghanistan.”

We Have Met the Enemy, and He Is PowerPoint

Advertisements

Keep Bit Rot at Bay: Change is Afoot as LoC’s DPOE Trains the Trainers 2011/09/20

Posted by nydawg in Archives, Best Practices, Digital Archives, Digital Archiving, Digital Preservation, Information Technology (IT), Media.
Tags: , , , , ,
add a comment

This was forwarded to me by a nydawg member who subscribes to the UK’s Digital Preservation listserv.  I don’t know if  it’s been posted publicly in the US, but I guess this first one is by invitation-only.  I would LOVE to hear what they are teaching and how they are doing it, so I hope someday to attend as well.

Library of Congress To Launch New Corps of Digital Preservation Trainers

The Digital Preservation Outreach and Education program at the Library of Congress will hold its first national train-the-trainer workshop on September 20-23, 2011, in Washington, DC.

The DPOE Baseline Workshop will produce a corps of trainers who are equipped to teach others, in their home regions across the U.S., the basic principles and practices of preserving digital materials.  Examples of such materials include websites; emails; digital photos, music, and videos; and official records.

The 24 students in the workshop (first in a projected series) are professionals from a variety of backgrounds who were selected from a nationwide applicant pool to  represent their home regions, and who have at least some familiarity with community-based training and with digital preservation. They will be instructed by the following subject matter experts:

*   Nancy McGovern, Inter-university Consortium for Political and Social  Research, University of Michigan
*   Robin Dale, LYRASIS
*   Mary Molinaro, University of Kentucky Libraries
*   Katherine Skinner, Educopia Institute and MetaArchive Cooperative
*   Michael Thuman,  Tessella
*   Helen Tibbo, School of Information and Library Science, University of  North Carolina at Chapel Hill, and Society of American Archivists.

The curriculum has been developed by the DPOE staff and expert volunteer advisors and informed by DPOE-conducted research–including a nationwide needs-assessment survey and a review of curricula in existing training programs. An outcome of the September workshop will be for each participant to, in turn, hold at least one basic-level digital-preservation workshop in his or her home U.S. region by mid-2012.

The intent of the workshop is to share high-quality training in digital preservation, based upon a standardized set of core
principles, across the nation.  In time, the goal is to make the training available and affordable to virtually any interested
organization or individual.

The Library’s September 2011 workshop is invitation-only, but informational and media inquiries are welcome to George Coulbourne, DPOE Program Director, at gcou@loc.gov.

The Library created DPOE  in 2010.  Its mission is to foster national outreach and education to encourage individuals and organizations to actively preserve their digital content, building on a collaborative network of instructors, contributors and institutional partners. The DPOE website is www.loc.gov/dpoe
http://digitalpreservation.gov/education/.  Check out the curriculum and course offerings here.

 

dk
###

Authors’ Guild Sues HathiTrust for Using Unauthorized Scans 2011/09/20

Posted by nydawg in Copyright, Digital Preservation, Intellectual Property, Media.
Tags: , , ,
add a comment

A few months ago, Judge Denny Chin put the kibosh on GoogleBooks’ attempt to digitize millions of library books and provide (mostly limited) access to the OCR.  Well, at that time, the good news was that ” HathiTrust, an organization set up to help them archive and distribute digital works” was still doing important work, but now HathiTrust is named as a defendant.  “The suit seeks to block two separate efforts. In the first, the universities have created a pooled digital archive of the contents of their libraries, maintained by the Hathitrust. No one contests that these works remain in copyright, or that the universities have rights to the nondigital forms of these works. What the authors object to is the fact that the digital works are derived from an unauthorized scan, and will be stored in a single archive that is no longer under the control of the university from which the scan was derived. The suit suggests that the security of this archive is also suspect, and may allow the mass release of copyrighted work.

“A separate issue in the suit is an orphaned works project started by the Hathitrust that focuses on some of the works within this archive. The group is attempting to identify out-of-copyright books, and those where the ownership of copyright cannot be established. If attempts to locate and contact any copyright holders fail, and the work is no longer commercially available, the Hathitrust will start providing digital copies to students without restrictions. This has not gone over well. The executive director of the Australian Society of Authors, Angelo Loukakis, stated, “This group of American universities has no authority to decide whether, when or how authors forfeit their copyright protection. These aren’t orphaned books, they’re abducted books.”  Read the Ars Technica article.

And if you’re still confused about the legal issues related to GoogleBooks’ recent problems with copyright infringement with an eye towards orphans, out-of-copyright and copyrighted materials, here’s an excellent multimedia presentation at Harvard by Lawrence Lessig in which he makes the argument  that tigers, as cubs, are extremely cute or to read why he thinks it is a “path to insanity,” check out TechCrunch or his longer “For the Love of Culture” essay in The New Republic.

U.S. Marshals’ Gross Mismanagement Undervalues Complex Assets 2011/09/20

Posted by nydawg in Archives, Electronic Records, Records Management.
Tags: , , , , ,
add a comment

People often wonder, “Why does anyone need a good, honest, ethical records manager anyway?”  or “Why don’t universities offer better programs in records management?” and “why do people responsible for hiring records managers not understand what records managers should do?” or maybe “what does the USMS Ethics Officer do?”

I wish I knew the answers, but here’s an interesting article revealing how shoddy records management can be described as Gross Mismanagement! “The unit of the United States Marshals Service that manages complex assets seized in criminal cases, including that of Bernard L. Madoff, kept such shoddy records that it could not say who bought assets or how much was paid for them in 8 of the 55 sales it handled from 2005 to 2010, according to an audit released on Tuesday. . . . . The report said that the auditors found significant problems in how the service managed complex assets and that those problems “increased the risk that the government could mismanage the administration and disposition of forfeited assets.” The team disposed of $136 million in seized and forfeited assets from January 2005 to August 2010. “This audit identified numerous deficiencies in the procedures the complex asset team implemented to track, safeguard, value and dispose of complicated and valuable assets,” the report said. The report added that the team’s overseer, the Asset Forfeiture Division, did not “vigorously oversee” the team. Read all about it in “Auditors Find Chaos in U.S. Marshal’s Asset Sale Record-Keeping“!

So where are all those jobs anyway? From the audit: “Additionally, we found that the limited staff and resources of the Complex Asset Team were disproportionate to its responsibilities.  From 2005 to 2009, the number of staff varied between two and four individuals. . . . While the 14 forfeiture financial specialist contractors had extensive experience relevant to forfeiture, their primary assignment during Briskman’s tenure was to assist USMS district offices and not the Complex Asset Team.” . . . “Between 2005 and 2010, the small staff of the Complex Asset Team disposed of over $136 million in assets, yet it operated in an environment lacking the procedures to guide its actions and decisions pertaining to seized and forfeited assets.” . . . Uh, hold on, so four employees were responsible for keeping records on $136 million in assets?! $34 million per employee?! I wonder what their daily salaries were!

“Further, we identified inaccuracies in some monthly reports Briskman compiled. The summaries Briskman provided for many assets stated, “No change from previous report.” Yet, when we compared these entries to previous entries for such assets, we noted that some of the entries contained unexplained changes from the previous monthly status report. For example, the listed taxable profit amount for one asset varied from $9.5 million to $12 million between different reports; however, the entries provided to summarize the changes for this particular asset in subsequent reports were “no change from previous report.”

and check out the recommendations including: “Recommendation 20: Ensure that managers know that they must thoroughly review financial disclosure forms and disclose any potential conflicts of interest to the USMS ethics office.”  Read the DoJ “AUDIT OF THE UNITED STATES MARSHALS SERVICE
COMPLEX ASSET TEAM MANAGEMENT AND OVERSIGHT” here.

dk

DMCA, DRM and The unFair Use Act 2011/09/09

Posted by nydawg in Archives, Copyright, Digital Preservation, Information Literacy, Intellectual Property, Media, Privacy & Security.
Tags: , , , , , , , ,
add a comment

A few weeks ago, I reserved a copy of a Clay Shirky e-Book to take with me to Chicago.  When the book became available from NYPL, I was excited and hoped it would be a DRM (Digital Rights Management)-free PDF copy so I could download it (click the emailed link) to my netbook and transfer it to my eReader Tablet.  (It was my first time, so I was clueless.)  Oh well.  Obviously you can’t do that. . . .  or maybe you can, on a Sony eReader and the York Library.

Yesterday an old friend on facebook asked about borrowing eComic Books with the intent of ultimately preserving on some portable medium.  So I was intrigued enough to do a little research on DRM and found this informative piece from the ASIS&T Bulletin website: “Digital rights management (DRM) is commonly defined as the set of technological protection measures (TPM) by which rights holders prevent the use of digital content they license in ways that could compromise the commercial value of their products.  Restrictions on such uses as downloading, printing, saving and emailing content are encoded directly in the products or the hardware needed to use them and are therefore in immediate effect.”

The whole article is worth reading, but this one part caught my eye: “The New York Public Library (NYPL), for instance, has been considering bringing its digitized collection of dance and performance videos closer to the public outside the NYPL system as long as it is possible to restrict access to this online content to library locations only. These examples show that DRM may actually provide opportunities to expand access to online materials in ways previously not possible.”  The essay continues by examining the DMCA and its relation to DRM, pointing out that “Since the DMCA was enacted in 1998, the Library of Congress has enforced exceptions three times – in 2000, 2003 and 2006 – and was scheduled to do so again in 2009. Of the six exceptions passed in 2006, one specifically allows film and media studies professors to circumvent TPM to make film clip compilations for coursework using DVD copies held by their institution’s film-studies library. A movement has been underway to expand this exception to include K-12 educators, all subject areas and all legally obtained copies.”

And to give you a sense of what is at stake, the author writes “In February 2007, the Fair Use Act was introduced in Congress, but never passed. It would have codified into law all six exceptions from 2006, which are currently rule-made and remain subject to periodic reviews. The Fair Use Act would have permitted the circumvention of TPM for, among other cases, (1) access to public domain works, (2) access to works of public interest for criticism, scholarship, reporting or research, (3) compilations of educational film clips and (4) preservation in libraries. The latter is of particular importance as the various media with historical content, including DVDs, begin to deteriorate. Smith argued that what frightens publishers about the Fair Use Act is that, if implemented, it would render ineffectual the anti-circumvention rules. Fair use would constitute an exception so broad that decisions regarding the right to circumvent would often be made after the actual circumvention. If a content owner objected, the user could take the matter to court, and only then would a judge decide whether fair use can justify that particular circumvention. The Fair Use Act would thus defeat the anti-circumvention rule’s self-help purpose.”

So in other words, the encryption that libraries are using is controlling access to their eBooks, and the anti-copy encryption that companies are using on their deteriorating DVDs are conspiring with the law to keep libraries from providing open access in the future to our resources.  I say instead of “Fair Use Act”, we need a “Fair Copy Act” so libraries will be free to begin their media refreshment, digital migration and whatever they need to do to make sure their media collections do not become time capsules, moments of time captured on obsolete media formats.

 

NARA, Why Is the Government Destroying Our History? 2011/09/07

Posted by nydawg in Archives, Electronic Records, Intellectual Property, Privacy & Security, Records Management.
Tags: , , , , , , , ,
add a comment

A colleague posted this sad (but true) story about the National Archives asking “Why Is the Government Destroying Our History?” and I noticed this set-up, “The U.S. National Archives and Records Administration (NARA) said it will destroy millions of federal court records and bankruptcy files from 1970 through 1995 but will hold those records the government deems “historically valuable.” . . . Ok, for those of you who think archivists and information purists are dirty curmudgeons who toil away amid dust balls to avoid socializing (on say, Facebook!), consider what is actually lost when these records are destroyed:

. . . Incrimination.  You are about to hire an executive. You call us to do a background check.  We find out he was charged with running a prostitution ring in the ’80s. Or, you are about to hire a new CFO.  You call us and during our research we find he has filed for personal bankruptcy protection three times in the last 15 years. ”

So, this is very troubling.  Offhand I don’t know what the retention schedules for court records and bankruptcy files are, but now it seems like the historians at NARA are convinced that they can describe these files as having “historic” value, but they won’t go near the “evidential” or “transactional” value.   Professional records managers are not making these decisions at NARA, because they would recognize the legal value.   So NARA, in its “infinite wisdom” will decide whether or not large parts of our shared legal history have “historical value”, at the same time that they believe that redundant digital junk (e.g. 250 million George W. Bush emails) merit long-term preservation, but court records related to criminal activity may not have value in the eyes of a .  They’re going to throw out the original, authentic records and create a black hole in our shared knowledge of our judicial system!

Anyone remember when George W. Bush signed Executive Order 132333 to limit access to President Reagan’s records?  Well, now imagine that NARA is doing the same thing with federal records.  So what does the Federal Records Act (FRA) have to say about court records, or how does NARA deal with court records and bankruptcy files?

Well, in Spring 2008,this power was held by the federal records centers (FRCs) of the National Archives and Records Administration (NARA).  In a promotional piece, “Ready Access NARA’s Federal Records Centers Offer Agencies Storage, Easy Use for 80 Billion Pages of Documents they were providing ready access.   “However, the majority of federal records—approximately 95 percent—are considered “temporary records.” Every temporary record has an official records retention schedule—that is, the amount of time it must legally be preserved for use before it is destroyed (usually by recycling). Retention schedules for temporary federal records vary widely, ranging from a few months to more than a century. For example, most agency information request correspondence is kept for less than a year. Individual tax returns are preserved for seven years. Corporate tax returns, while not considered “permanent,” must be retained for 75 years. And certain aircraft certification engineering files must be kept for 100 years.”

I’m not exactly sure what they are doing ,but I assume it’s something like saying that since the papers were digitized (scanned), the originals are no longer needed.  But for public records, NARA is steward.   “The public can also access federal court records held by FRCs. These records include files from U.S. bankruptcy courts, the U.S. court of appeals, and U.S. district court civil and criminal files. FRCs make court documents available for researchers such as reporters writing stories on high-profile cases, former bankruptcy court litigants applying for mortgages or other loans, companies conducting background checks on individuals, and legal professionals researching precedents.”

Okay, so it’s an interesting piece from NARA, but this part really stopped me in my tracks: “The federal records centers have ably served the federal government and the citizens of the United States for more than 50 years. As the needs of federal agencies change and grow, NARA’s FRCs are also changing and growing to ensure that they will continue to protect the information assets of the federal government.”

I hope I’m not the only person to cry foul on this!  It drives me crazy especially when you check the FRC website and see how heavily invested they are in having a social media presence (Twitter, Facebook).

 

Curating Google Doodle Highlights incl. Freddie Mercury’s Tribute 2011/09/06

Posted by nydawg in Curating, Digital Archives, Digital Archiving, Digital Preservation, Information Technology (IT), Intellectual Property, Media.
Tags: , , , , ,
add a comment

Hi everyone: Maybe this isn’t totally an archival or curatorial issue, but in some ways, these GoogleDoodles do what a good archive strives to do: provide easy access to available information and resources.  So pump up the volume, click on today’s GoogleDoodle, look for the cc [closed-captioning] button for lyrics to sing-along as you watch an animated music video tribute to the late great Queen singer Freddie Mercury.  http://www.google.com/

and check out Queen guitarist Brian May’s blog tribute here.

But if you want more of those awesome GoogleDoodles, don’t forget some of my favorites including: Alex Calder’s moving mobiles;  playable and recordable Les Paul guitar; John Lennon’s hand-drawn Imagine (animation); Martha’ Graham’s “Thought of You” dance; Mr. Men and Little Miss; Charlie Chaplin’s 122nd Birthday; and who can forget GoogleDoodle Dots, Jules Verne or the Google PacMan?

Those are some of my favorites, but I can probably think of a dozen more if i put my head to it. . . .If you’re interested in learning about the doodle history, check it out here.  And if i’m missing any good ones, please let me know!

WikiLeaks’ Cablegate and Systemic Problems 2011/09/06

Posted by nydawg in Best Practices, Digital Archives, Electronic Records, Information Technology (IT), Media, Privacy & Security, Records Management, WikiLeaks.
Tags: , , , , ,
1 comment so far

WikiLeaks Cablegate

Since late November of last year, the whole world has been watching as WikiLeaks got its hands on and slowly released thousands of classified cables created and distributed by the US over the last four decades.  As you may recall, the suspected leaker was Army Private First Class Pfc Bradley Manning who, undetected, was able to locate all the cables, copy them to his local system, burn them to CD-R (while allegedly lipsyncing Lady Gaga), and uploading an encrypted file to WikiLeaks.  (I’ve written previously , so I won’t get too detailed here.)

But last week, the story changed dramatically when The Guardian revealed that “A security breach has led to the WikiLeaks archive of 251,000 secret US diplomatic cables being made available online, without redaction to protect sources.  WikiLeaks has been releasing the cables over nine months by partnering with mainstream media organisations.  Selected cables have been published without sensitive information that could lead to the identification of informants or other at-risk individuals.”  To further confuse matters related to the origin of this newest leak, “A Twitter user has now published a link to the full, unredacted database of embassy cables. The user is believed to have found the information after acting on hints published in several media outlets and on the WikiLeaks Twitter feed, all of which cited a member of rival whistleblowing website OpenLeaks as the original source of the tipoffs.”  The Cablegate story, with all its twists and turns over the months, has left a big impression on me and, as an archivist and records manager, I think it is important to strip this story of all its emotionality and look at it calmly and rationally so that we can get to the bottom of this madness.

The first problem I have with the story, or more specifically, with the records management practices of the Defense Department is the scary fact that a low-level Private first class (Pfc) would have full access to the Army’s database.  This became a bit scarier when we learned that Pfc Manning used SIPRNet (Secret Internet Protocol Router Network) to gain full access to JWICS (Joint Worldwide Intelligence Communications System) as well as the [cilivian/non-military] diplomatic cables generated by the State Department.

So the first question I had to ask was: why does DoD have access to the State Department’s diplomatic cables, are they spying on the State Department?!  Well, maybe, but even if not, this staggering fact from a different Guardian article sent shivers down my spine:  “The US general accounting office identified 3,067,000 people cleared to “secret” and above in a 1993 study. Since then, the size of the security establishment has grown appreciably. Another GAO report in May 2009 said: “Following the terrorist attacks on September 11 2001 the nation’s defence and intelligence needs grew, prompting increased demand for personnel with security clearances.” A state department spokesman today refused to say exactly how many people had access to Siprnet.”

Other factors that scare the heck out of me related to “bad records management” and WikiLeaks Cablegate are the fact that there is a lack of CONTROL of these assets (they store everything online?!  Really?!); the DoD and State Department don’t use ENCRYPTION or cryptographic keys or protected distribution systems; the names of confidential sources were  not REDACTED in the embassy before uploading and sharing the cables with the world; their RETENTION SCHEDULES do not allow for some cables to be declassified and/or destroyed (so they keep everything online for decades and/or years); the majority of cables were UNCLASSIFIED suggesting that so many cables are created that they don’t even have enough staff to describe and CLASSIFY them in a better way?  The DoD didn’t have a method for setting ACCESS PRIVILEGES, or PERMISSIONS or AUTHORIZATION to ensure that a Pfc who is on probation would not be able to access (and copy and burn to portable media) all those cables undetected?!  There’s a question about password protection and authorization, but those problems could probably be covered with better ACCESS PRIVILEGES and PERMISSIONS.  Another question that leaves archivists confused is the idea that there seems to be limited version control.  In other words, it seems as if once a cable is completed, someone immediately uploads it, and then if the cable is updated and revised, a second cable will be created and uploaded.  This doesn’t seem to be a very smart way of trying to control the information when multiple copies may suggest differing viewpoints.

But perhaps the scariest part of the whole WikiLeaks’ Cablegate madness is simply that there was no TRACKING or TRACING mechanism so that the DoD could, through LOGS, trace data flows to show that one person (or one machine or one room in one building or whatever) had just downloaded a whole collection of CLASSIFIED materials!  [From the IT perspective, large flows of data may actually impact data flow speeds for other soldiers on the same network!]  And the fact that Pfc Manning was able to burn the data to CD-R suggests that when IT deployed the systems they forgot or neglected to DISABLE the burn function on a classified network!  (Okay, they’ve made some recent changes, but is it too late?!)

Many assume that Digital Forensics will provide a new way to authenticate data.  Well, if so, then why can’t they run a program on the cables and find out which system was used to burn the data and then trace the information back to the person who was using the machine at that time, as opposed to putting a soldier in jail, in solitary confinement, awaiting trial, convicted merely on a hearsay online chat he had with a known hacker?!  One other important consideration that also scares me: The military uses Outlook for their email correspondences, and Outlook creates multiple PST files.  As the National Journal puts it: “So how did Manning allegedly manage to get access to the diplomatic cables? They’re transmitted via e-mail in PDF form on a State Department network called ClassNet, but they’re stored in PST form on servers and are searchable. If Manning’s unit needed to know whether Iranian proxies had acquired some new weapon, the information might be contained within a diplomatic cable. All any analyst has to do is to download a PST file with the cables, unpack them, SNAP them up or down to a computer that is capable of interacting with a thumb drive or a burnable CD, and then erase the server logs that would have provided investigators with a road map of the analyst’s activities.”

Obviously the system was broken, informants’ security was compromised, our secrets are exposed, and the cat is out of the bag!  Yet even now, many are unwilling to listen to or heed the lessons we need to learn from this debacle.  Back in January, I attended a WikiLeaks panel discussion hosted by the Archivists Round Table of Metropolitan New York and was surprised to hear that most of these issues raised above were ignored.  I tried to ask a question regarding the systemic problems (don’t blame Manning), but even that was mostly ignored (or misunderstood) and not answered by everyone on the panel.

In my opinion, we have very serious problems related to best practices for records management.  If you look closely at DoD 5015.2, you can see that the problems are embedded in the language for software reqs, and nobody is looking at these problems in the ways that many archivists or records managers do (or should).  But honestly, the most insightful analysis and explanation were confessed by Manning himself: ““I would come in with music on a CD-RW labeled with something like ‘Lady Gaga,’ erase the music then write a compressed split file,” he was quoted in the logs as saying. “[I] listened and lip-synced to Lady Gaga’s ‘Telephone’ while exfiltrating possibly the largest data spillage in American history. Weak servers, weak logging, weak physical security, weak counter-intelligence, inattentive signal analysis … a perfect storm.

So maybe it is time for the military, the US National Archives, and all computer scientists and IT professionals to stop relying on computer processing and automated machine actions and start thinking of better ways to actually protect and control their classified and secret data.   Perhaps a good first move would be to hire more archivists and try to minimize the backlog quantity of Unclassified cables!  Or maybe it’s time to make sure that the embassies take responsibility for redacting the names of their sources before uploading the cables to a shared network?  And maybe it is time to consider a different model than the life cycle model which will account for the fact that often these cables will be used for different functions by different stakeholders through the course of its existence.

Digital New York: Still a Few Bugs in the System 2011/09/05

Posted by nydawg in Curating, Digital Archiving, Education, Electronic Records, Information Technology (IT), Media.
Tags: , , , ,
add a comment

Hurricane Irene (not to scale)

Many of you know that I missed all the excitement last week as Hurricane Irene bore down on the New York area.  I was in Chicago for the 75th Annual Meeting of the SAA (Society of American Archivists) and it got so bad that I received warning emails from my mother and my oldest brother.  [I assume they had received but not read my itinerary which clearly showed that I was heading to Minneapolis/St Paul after the meeting.]  So I figured I was in the clear until I realized sometime on Friday, “Whoops! I forgot to close my windows!”  So I guess I can say I was tangentially affected (by guilt caused) by Tropical Storm Irene. . . .

But as the story was developing, I was in touch with friends back East and learned that some who live in my neighborhood were advised to evacuate!  My ex-girlfriend evacuated our two (Brooklyn) cats to Manhattan, and sent me pictures!  Well, I live close enough to the East River to start to worry about my (second floor) apartment. .  With a little research, I learned that I could find the evacuation areas from nyc.gov.  But on Saturday, I didn’t have any luck accessing the PDF or whatever it was.

So this morning, I stopped for a cup of coffee in Champion, and happened to read an article that “The New York Times reported that the city’s official website, www.nyc.gov, was down on the morning of Friday, Aug. 26.  The news outlet suggested that the site was overwhelmed by people looking for information about the hurricane. As of 1:30 p.m. Pacific time, however, the site was back online.   The timing couldn’t have been worse. In what New York City Mayor Michael R. Bloomberg called a “first time” for the city, he ordered a mandatory evacuation of various coastal areas of the city’s five boroughs, covering roughly 250,000 people.”  So this is dysfunctional modern-day disaster planning.

From the TimesCity Learns Lessons From the Storm, Many of Them the Hard Way” we learn that “For example, the mayor’s office had predicted a surge in Web traffic on nyc.gov when it issued the evacuation order. But nobody expected five times the normal volume of traffic. By Friday afternoon, computer servers had become severely overloaded. The Web site sputtered and crashed for hours, when New Yorkers needed it most.  In the future, the city will try to modify the Web site so that it can be quickly stripped down to a few essential features  —  like an evacuation map, searchable by ZIP code —  that are in highest demand during an emergency.”

Hurricane Irene: NYC Evacuation Zones

I’m curious about what is the “normal volume” of traffic on that webpage?  But it seems to me that this is ultimately a problem wit making information accessible, but not thinking it through to the extent that an end-user (who may have to evacuate his/her house!) has to first click on the PDF, then download it, wait for it to finish downloading, launch it, and then search for the data needed. . . . .  The fact that this is not an integrated system where a person can easily plug his/her zip code into an online system to find out if his house is in an evacuation zone  suggests that the system is not very functional, best practices are not in use, and further, that perhaps the metrics used to show how vital Digital New York is, are the wrong metrics to use.

Why wouldn’t the IT staff at DoITTT consider creating mirror sites for downloading the PDFs?  So the first victim of Hurricane Irene was NYC.gov.  “In a tweet earlier this morning the city’s Chief Digital Officer apologized for the outage while giving specific links (which were also frequently down) to find the city’shurricane evacuation map (we’ve included it below for your convenience). And the city’s main Twitter feed just put out a similar tweet. Which means, damn, a LOT of people must be trying to access the city’s website. We’ve e-mailed to find out just how many users it takes to take down nyc.gov but have yet to hear back.”

Well, fortunately, they’ve probably learned some lessons from this hysteria, and it seems like no one suffered much damage in this area and, ironically (or fortunately) September is a good time to Get Prepared: “National Preparedness Month . .  . a nationwide campaign to promote emergency preparedness and encourage volunteerism.”  To learn more about NYC’s Digital Strategy and the Chief Digital Officer check here for the Road Map. (more…)

Arab Spring Diplomatics & Libyan Records Management 2011/09/05

Posted by nydawg in Archives, Best Practices, Digital Archives, Digital Preservation, Electronic Records, Information Technology (IT), Media, Records Management.
Tags: , , , , , , , , , , ,
add a comment

At 75th Annual Meeting of the SAA (Society of American Archivists) last week, I had the fortunate opportunity to attend many very interesting panels, speeches and discussions on archives, archival education, standards, electronic records, digital forensics, photography archives, digital media, and my mind is still reeling.   But when I heard this story on the news radio frequency, I needed to double-check.

As you all know, the Arab Springrevolutionary wave of demonstrations and protests in the Arab world. Since 18 December 2010 there have been revolutions in Tunisia and Egypt;
civil uprisings in BahrainSyria, &Yemen; major protests in AlgeriaIraqJordanMorocco, and
 

Omanand minor protests civil war in Libya resulting in the fall of the regime there; in

Kuwait, LebanonMauritaniaSaudi ArabiaSudan, and Western Sahara! Egypian President Hosni Mubarak resigned (or retired) and there’s a Civil War going on in Libya.   Meanwhile, with poor records management, documents were found in Libya’s External Security agency headquarters showing that the US was firmly on their side in the War on Terror:

“CIA moved to establish “a permanent presence” in Libya in 2004, according to a note from Stephen Kappes, at the time the No. 2 in the CIA’s clandestine service, to Libya’s then-intelligence chief, Moussa Koussa.  Secret documents unearthed by human rights activists indicate the CIA and MI6 had very close relations with Libya’s 2004 Gadhafi regime.

The memo began “Dear Musa,” and was signed by hand, “Steve.” Mr. Kappes was a critical player in the secret negotiations that led to Libyan leader Col. Moammar Gadhafi’s 2003 decision to give up his nuclear program. Through a spokeswoman, Mr. Kappes, who has retired from the agency, declined to comment.  A U.S. official said Libya had showed progress at the time. “Let’s keep in mind the context here: By 2004, the U.S. had successfully convinced the Libyan government to renounce its nuclear-weapons program and to help stop terrorists who were actively targeting Americans in the U.S. and abroad,” the official said.””

Shudder.

So I guess that means that if all of those documents from the CIA are secret, there would be no metric for tracing a record (at least on the US side).   In other words, every time a record is sent, copied or moved, a new version is created, but where is the original?  Depending on the operating system, the metadata may have a new Date Created.  How will anybody be able to find an authentic electronic record when it’s still stored on one person’s local system which is probably upgraded every few years?

There is a better way, a paradigm shift, and by looking at the Australian records continuum, “certainly provides a better view of reality than an approach that separates space and time”, we can find a better way so all [useless] data created is not aggregated.   With better and more appraisal, critical and analytical and technical and IP content, we can select and describe more completely the born digital assets and separate the wheat from the chaff, the needles and the haystacks, the molehills from the mountains, and (wait for it)  . . . see the forest for the trees.  By storing fewer assets and electronic records more carefully, we can actually guarantee better results.  Otherwise, we are simply pawns in the games of risk played (quite successfully) by IT Departments ensuring (but not insuring) the higher-ups that “we are archiving: we backup every week.” [For those who are wondering: when institutions “backup” they backup the assets one week, moves the tapes offsite and overwrite the assets the following week.  They don’t archive-to-tape for long-term preservation.]

Diplomatics may present a way for ethical archivists in to the world of IT, especially when it comes down to Digital Forensics.  But the point I’m ultimately trying to make, I think, is that electronic (or born digital) records management requires new skills, strategies, processes, standards, plans, goals and better practices than the status quo.  And this seems to be the big elephant in the room that nobody dares describe.