Do the New Rules Discourage IT Improvements?

Do the new Federal Rules of Civil Procedure (”FRCP”) reward the continued use of outdated computer systems where information is hard to find and retrieve?  Overall, no.  But surprisingly one of the new rules does reward inefficient IT, and it is an important rule at that:  Rule 26(b)(2)(B).  Although it is not the intended purpose of the rule, there is no getting around the fact that 26(b)(2)(B) is a disincentive to IT upgrades designed to improve accessibility to stored information. It does so by providing special protection from discovery to hard-to-access computer data, protection not afforded to any other type of digital evidence.

Fortunately for hardware and software vendors the impact of this one “pro-Luddite” rule is outweighed by the terms of the other new rules, primarily Rules 16(b) and 26(f).  They require a party in federal litigation to very quickly identify, preserve, search, and produce relevant computer data.  The new FRCP timelines established by these rules cannot be met by businesses unable to quickly access the high volumes of electronic records stored on their computer systems.  For that reason, the overall impact of FRCP, despite Rule 26(b)(2)(B), supports the upgrade and migration of IT systems to improve information accessibility.  A company in litigation today must be able to quickly and efficiently locate and retrieve information relevant to the dispute. It is foolish to continue to use outdated technology where that is often difficult, if not impossible.  In fact, that can often make the difference between winning and losing a lawsuit.  That is why many companies are now moving away from unsearchable backup tapes, to new searchable tape systems, archiving, live mirroring, or other solutions.  That is also why they are investing in software to more efficiently search and preserve information, especially emails.

Rule 26(b)(2)(B) FRCP goes against this trend and provides some small comfort to procrastinators who put off these needed upgrades.  This rule actually encourages inefficient IT because it places hard-to-find computer data off limits from discovery, absent a special showing of good cause not required for all other computer evidence.  In other words, if you store computer data so that it is hard to access, then, as a general rule, when you are sued, this information is, to a large extent, exempt from discovery. Unlike other relevant computer information that is stored efficiently, so that it can be reasonably accessed, if information is very difficult to access, then you do not have to access it at all. You do not even have to search it to see if it contains relevant evidence, and so, of course, you also do not have to produce it. You just have to disclose that it exists, and explain why it is hard to get to. Then it is up to the requesting party to try to persuade the court that special circumstances exist to justify the burden and expense of discovery.  This is a so-called “two-tiered” system of discovery, with “not reasonably accessible” data requiring a second-tier of good cause proof to be discoverable.

Here is the full text of Rule 26(b)(2)(B):

(B) A party need not provide discovery of electronically stored information from sources that the party identifies as not reasonably accessible because of undue burden or cost. On motion to compel discovery or for a protective order, the party from whom discovery is sought must show that the information is not reasonably accessible because of undue burden or cost. If that showing is made, the court may nonetheless order discovery from such sources if the requesting party shows good cause, considering the limitations of Rule 26(b)(2)(C). The court may specify conditions for the discovery. (emphasis added)

Information considered by the law to be “not reasonably accessible” and thus protected from ordinary discovery by this rule includes backup tapes, “legacy data,” and “double deleted” files.  By “legacy data”, I mean information stored on hardware or requiring software that is no longer used or easily available. By “double deleted” files, I mean files no longer listed in FAT, but still located on a hard drive or other storage device because they have not been written over yet by new information. (For more on what I mean by “double deleted” as opposed to single deleted, see the Blog entry of June 2, 2007.)

The “pro-Luddite” aspect of FRCP inherent in 26(b)(2)(B) seems to have escaped the notice of most IT commentators and vendors, especially those in the information storage and retrieval field.  They promote the idea that compliance with FRCP requires companies to upgrade their IT systems.  They affirm that FRCP now mandates businesses to make all of their computer data reasonably accessible, especially their email, which is the primary focus of discovery in most lawsuits.  To read all of the hype in this area, you would think that the new FRCP contains provisions requiring specific technology improvements, such as archiving of email. 

They do not, and in fact, as you can see by the terms of Rule 26(b)(2)(B),  this rule protects old, inefficient systems.   It serves to put information that is not reasonably accessible - that is, information stored in a manner that is burdensome or expensive to retrieve - outside the scope of normal discovery.  Therefore, in that way, Rule 26(b)(2)(B) rewards a company that maintains its data inefficiently by excluding that data from discovery.  The company is saved from the expense of preserving or searching for that kind of data.  To put it simplistically, you do not have to produce what you can’t easily find. So, why make anything easy to find? If you do, you just make it discoverable. Following this argument, the best strategy to avoid the high expense of e-discovery is to store your company’s business information in sources that are not reasonably accessible.  This is what I call a Luddite strategy of “deliberate inaccessibility”.  

Although I do not agree with this strategy for reasons I will explain below, I recognize that the argument does have some merit. I think this aspect of FRCP should be openly discussed, and not just swept under the rug.  In fact, this language in 26(b)(2)(B) is, in my view, and others, one of the most significant weaknesses in the new rules, and will cause problems for years to come. See Garrie, Armstrong & Burdett, Hiding the Inaccessible Truth: Amending the Federal Rules to Accommodate Electronic Discovery, 25 Rev. Litig. 115, 2006.  As Garrie, Armstrong & Burdett point out in another law review article on Sarbanes-Oxley and e-discovery, the Zubulake decisions have the same technology-chilling effect in so far as they place a special burden of cost-sharing on a party seeking discovery of inaccessible data, including legacy data and backup tapes.  

Zubulake thus creates a perverse disincentive that prevents companies from investing in more efficient data storage technologies, because parties with efficient storage systems are generally forced to produce more digital documents than parties using legacy storage systems. Although companies eventually may determine that the need for a newer storage system exceeds the risks posed by broad electronic discovery, litigants should not be forced to weigh potential adverse legal consequences against the benefits that could be realized by investing in appropriate systems for their business needs. (footnotes omitted)

Electronic Discovery and the Challenge Posed by the Sarbanes-Oxley Act, Garrie & Armstrong, Electronic Discovery and the Challenge Posed by the Sarbanes-Oxley Act, 2005 UCLA J.L. & Tech. 2, Para 13.

Other commentators acknowledge the problem, but then dismiss it too easily by stating that courts will see through any fraudulent schemes to hide evidence in inaccessible systems. This argument assumes fraudulent intent.  For authority these commentators usually cite the Rambus saga of cases.   Rambus, Inc. v. Infineon Tech. AG, 220 F.R.D. 264, 284 (E.D. Va. 2004); Rambus, Inc. v. Infineon Techs. AG, 155 F.Supp.2d 668, 680-83 (E.D.Va.2001); Rambus, Inc. v. Infineon Techs. AG, 222 F.R.D. 280, 286 (E.D.Va.2004); Samsung Electronics Co., Ltd. v. Rambus, Inc., 439 F.Supp.2d 524 (E.D.Va., July 18, 2006). These cases expose the deliberate actions by Rambus to change its record retention policies and physically destroy evidence in anticipation of bringing patent litigation. (See my Blog of March 3rd, 2007 for further discussion of this interesting case and the employee “shred days.”)  

This kind of facile response - ”don’t worry about 262(b)(2)(B) because the courts will pick up on any fraud” - begs the question.  Obviously if a company moves all of the data it does not want found into a legacy system before filing suit, instead of just destroying it as in Rambus, then this is a fraudulent scheme and will likely be exposed.  But this does not address the more realistic scenario of a company that simply continues the status quo of maintaining old systems with unreasonably accessible data.  It does not address the non-fraudulent scheme.  The strategy of “deliberate inaccessibility” is not necessarily fraudulent. In most cases it will more likely be driven by procrastination and short-sighted thinking.

Again, to reiterate, I do not think a company should follow this kind of strategy of inertia.  A company should not put off upgrades to their IT systems to make more information accessible, just because this might take the information out of the protection of Rule 26(b)(2)(B). The strategy of inaction is too risky. Businesses should improve their IT systems, especially email, and move away from old technologies that are hard to search.  Everyone, especially businesses involved in frequent litigation, should move towards making their information retrieval more efficient and reliable.  I think they should do so both for business purposes, and for litigation readiness purposes. But it is not a simple black and white matter of compliance with FRCP as some contend. The reality is, FRCP is a mixed bag, and companies should act now to improve information accessibility primarily for business and records management efficiency reasons, and only secondarily because of litigation.

Even from the litigation perspective alone, a full risk analysis counsels in favor of technology improvement, not status quo.  Attempts to rely on 26(b)(2)(B) to protect data from discovery by using obsolete storage, such as tapes and legacy systems, are too risky.  The risk is unacceptable because the exemption of inaccessible data from discovery is in turn subject to a second-tier ”good cause” exception.  The vague “good cause” exception allowing discovery is not defined by the rules.  Under this second-tier analysis a court can require the production of even inaccessible records, regardless of how much burden and expense this may cause.  This is an unacceptable risk because the expenses in a situation like that can be enormous.  Costs of  three million dollars in just five months are not that unusual.  See Kentucky Speedway, LLC v. NASCAR, 2006 U.S. Dist. LEXIS 92028 (E. D. Ky. Dec. 18, 2006). It is true that a court has discretion to shift some of these costs onto the requesting party, but you cannot count on that.

Good cause for discovery of inaccessible data can be shown in any number of ways, depending in large part on how the governing judge views the case and the discovery sought.  The Rules Committee Commentary specifies some of the factors they think a court should consider in finding “good cause”.  These are suggestions only, and are not binding on any court.  These Commentaries are quoted in full in the Rule 26 Page on the top of the Blog.  Here is my summary of the factors the Committee suggested be considered to decide whether there is good cause to allow discovery of ”not reasonably accessible” information:

1. The specificity of the discovery request.

2. The quantity of information available from other, more easily accessed sources.

3. The failure to produce relevant information that seems likely to have existed but is no longer available from more easily accessed sources.

4. The likelihood of finding relevant, responsive information that cannot be obtained from other, more easily accessed sources.

5. Predictions as to the importance and usefulness of the requested information.

6. The importance of the issues at stake in the litigation.

7. The parties’ resources.

It is a long list, and not exclusive, and everyone knows that it will be one of the most heavily litigated parts of the new FRCP for many years to come.

Since the good cause exception is so general and vague, not to mention hard to predict, it is foolish to keep information in the dark based on the hope that it will, for that reason, never be subject to discovery.   You never know when a court may require you to search in the dark at great expense. 

Also, and even more importantly in my opinion, that is no way to run a business or manage records.  Why keep information if you don’t want it, and can’t use it?  It might all come back to haunt you some day (just ask Morgan Stanley), and in the meantime it costs money to store. The best solution to an archaic computer system is to throw it out, and upgrade.  The best solution to archaic information is to throw it out, period.   If there is no valid business purpose or legal requirement to save records, then destroy them.  Don’t just save information because you have the ability to do so.  That is a natural pack-rat tendency of most IT departments, and the one that annoys lawyers the most.  Of course, don’t act rashly either.  Think long and hard, and be sure that you don’t need the records.  Then take the time to purge and destroy, and do it the right way.  Follow the guidelines of Rule 37, document everything, and so protect against any later spurious charges of spoliation.

District Court in LA Decides Computer RAM Memory Must Be Preserved and Produced

A Los Angeles District Court rocked the e-discovery world last month by holding, for the first time, that the contents of a computer’s Random Access Memory (”RAM”) memory are discoverable.  Columbia Pictures Industries v. Bunnell, Case No. CV 06-1093 (FMC(JCx) (Doc. No. 176). The Order dated May 29, 2007, by Magistrate Judge Jacqueline Chooljian was entered in a copyright infringement case against the owners of the popular media search-engine web, TorrentSpy. The order has been highly criticized by computer experts and e-discovery lawyers alike for greatly expanding the legal duty to preserve and produce electronic data to the most elusive and transitory of information, that held only in a computer’s RAM chips.

Many contend that this a rogue decision, and if not reversed on appeal could force companies and individuals to store and produce vast amounts of data that would otherwise exist only temporarily on their computer’s RAM hardware.  Ken Withers, director of The Sedona Conference, was quoted by CNET News as stating that he feared the judge’s decision may mean a “tremendous expansion” of the scope of discovery in civil litigation, creating yet another what he calls “weapon of mass discovery.” By this,he means a discovery burden that could be tremendously expensive to meet, and thus could be exploited by plaintiffs and defendants alike to force the settlement of a dispute to avoid that expense. 

Commentators all seem to agree that in most circumstances, if not all, it would be prohibitively burdensome for a person or company to try to preserve and produce the transitory contents of computer memory.  The burdensomeness would derive not only from the expense, but also from the disruptive actions required to comply. 

Moreover, many contend that an order requiring the production of RAM data necessarily requires a party to create computer records where none before existed.  This goes against a fundamental legal precept that a party is only required to produce documents and electronically stored information that  have already been created and stored.  A party to litigation is never required to generate and create new documents and information so as to satisfy the curiosity of the opposing party.  As Ken Withers stated in the CNET article:

There’s never been a requirement that (defendants) must create documents that they wouldn’t ordinarily maintain for the purpose of satisfying some (plaintiff’s) discovery requests. 

Judge Chooljian considered this argument, but rejected it, because in her view, the information already existed in RAM.  She concluded that the defendants were not required to create new data, they were just required to transfer it to permanent form and preserve it.

Withers and others think the order is ill conceived in part because it is based on a misunderstanding of technology and computer memory.   Computer memory RAM files are temporary files that are created by the computer for operational efficiency.  Computer memory like this is by nature information designed only for very temporary storage in the memory chips of a computer.  It is quickly overwritten and is always purged - literally disappears - when a computer is turned off.  (Unpowered RAM memory chips contain no information of any kind.)  The same CNET News article also quotes Dean McCarron, principle analyst at Mercury Research, as saying:

RAM is the working storage of a computer and designed to be impermanent.  Potentially your RAM is being modified up to several billions of times a second. The judge’s order simply reveals to me a lack of technical understanding.

In short, RAM memory is temporary and transitory, and leaves no trace or record.  Still, a computer system can be designed so that information temporarily stored in RAM is copied onto a hard drive or flash memory storage where it then becomes fixed.

Before this decision, most e-discovery attorneys I know thought that only information fixed on a storage device, such as a hard drive, flash memory or Read Only Memory (ROM) (which are permanent), CD, DVD, etc., would be considered “stored” and thus discoverable under the new Rules as “Electronically Stored Information”. See, e.g., Rule 34 and commentary in the Page 34 at the top of the Blog.  This decision broadens the meaning of “stored” to include the temporary holding of information in a volatile memory chip. 

The opinion concedes that computer RAM memory has never previously been adjudicated to be Electronically Stored Information (”ESI”) under new Rule 34.  For that reason, Judge Chooljian declined to impose sanctions on defendants for failing to preserve and produce the RAM ESI in the past, but she did order its preservation and production going forward.   Although the opinion concedes it is a case of first impression for construction of Rule 34, Judge Chooljian argues that her conclusion is compelled by the unique circumstances of this case and prior Ninth Circuit law, primarily MAI Systems Corp. v. Peak Computer, Inc., 991 F.2d 511, 518-519 (9th Cir. 1993).  The MAI Systems case is well known for its holding that the copying of software into RAM memory is equivalent to  “affixing” it in a tangible medium for purposes of copyright law, where ”fixed” is a term of art.  Judge Chooljian reasoned that if information in RAM was sufficiently fixed to constitute a copyright infringement, then it was sufficiently fixed for purposes of discoverability as electronically stored information.

In this case, the RAM memory at issue was the information stored for up to six hours, but no longer, on the webserver for the TorrentSpy web, as to what web-pages were downloaded, when, by whom, and other similar information.   That is called a webserver log.  For this particular kind of RAM data, it is not technically difficult to set up a webserver so that the log is transferred from RAM to a permanently stored file on the hard drive. Many webs are set up to do this.  These website owners want to keep track of who visits their site, which pages are the most popular, etc. 

But TorrentSpy did not set up their system that way.  They did not want to have any tracking information on their users, and in fact they put this privacy guaranty in their user agreement.  The Plaintiffs in this case, Columbia Pictures Industries, et al., were frustrated by the fact that the defendants did not know who its users were, and in fact did not want to know.  Plaintiffs claimed that TorrentSpy’s users were illegally downloading their copyrighted materials - their movies, music and television shows - based on information they obtained from TorrentSpy’s web. Plaintiffs claimed that TorrentSpy set up their computer systems with RAM anonymity on purpose in order to encourage their users to steal plaintiffs’ copyrighted materials without fear of being caught. 

Note that the plaintiffs do not claim that their copyrighted materials are stored on TorrentSpy’s web, and do not claim that TorrentSpy’s users downloaded the files from TorrentSpy. It appears to be undisputed that TorrentSpy “only” provides information as to where these materials are located, usually on the computers of individuals located around the world.  Plaintiffs sued the TorrentSpy web owners, and not the individuals who illegally downloaded the files or the individuals who supplied the files (for free), based on theories of vicarious copyright infringement, contributory infringement, and inducement.

Here, in my view, is where an old legal axiom comes in: “bad facts make bad law.”  The facts in this case are unusual.  The judge was convinced that the webserver logs were essential to the plaintiffs’ case.  This much is certain, and this is a highly unusual situation. Moreover, the tone of the opinion suggests that the judge thinks the plaintiffs are likely to prevail if they obtain these logs, that it will enable them to prove their infringement conspiracy theories.  The plaintiffs have successfully argued that defendants were hiding this crucial evidence in their RAM, and that this was an essential part of their conspiracy to steal their intellectual property. 

Further, the defendants were, according to Judge Chooljian, unable to demonstrate that it would be “unduly burdensome” for them to transfer all of the webserver logs on an ongoing basis from RAM to hard drive, and thereafter to preserve and produce this evidence.  The court was referring to the burden of proof placed on a party opposing discovery under new Rule 26(b)(2)(B). Under this rule, ESI is not discoverable if the party opposing discovery can show that it is “not reasonably accessible because of undue burden or cost.” Personally, I am surprised defendants could not prove undue burden or costs.  If they had, then under Rule 26(b)(2)(B) the burden of proof would have shifted to the plaintiffs to provide good cause that the “not reasonably accessible” information be produced anyway.  Apparently defendants failed because the court went entirely with the plaintiffs’ expert, and rejected defendants’ contrary testimony.  This certainly shows the importance of credible experts, especially in a case like this. 

Judge Chooljian appears to recognize that her decision would be controversial, and took some pains to note that it was not intended to serve as precedent for the routine discovery of computer memory in other cases.  Her position is explained in footnote 31, at page 31 of the 35-page opinion:

The court emphasizes that its ruling should not be read to require litigants in all cases to preserve and produce electronically stored information that is temporarily stored only in RAM. The court’s decision in this case to require the rentention and production of data which otherwise would be temporarily stored only in RAM, is based in significant part on the nature of this case, the key and potentially dispositive nature of the Server Log Data which would otherwise be unavailable, and defendants’ failure to provide what this court views as credible evidence of undue burden and cost.

I do appreciate this footnote. 

POSTSCRIPT: The Magistrate’s discovery ruling was appealed and later approved by the district court judge. See:  Order Denying Defendants’ Motion For Review dated Aug. 24, 2007

Should a Litigation Hold Include Backup Tapes?

I think Judge Scheindlin got it right in Zubulake IV when she held that a litigation hold should not, as a general rule, include backup tapes. Zubulake v. UBS Warburg LLC, 220 F.R.D. 212, 218 (S.D.N.Y. 2003):

The scope of a party’s preservation obligation can be described as follows: Once a party reasonably anticipates litigation, it must suspend its routine document retention/destruction policy and put in place a litigation hold to ensure the preservation of relevant documents.  As a general rule, that litigation hold does not apply to inaccessible backup tapes, for example, typically maintained solely for the purpose of disaster recovery, which may continue to be recycled on the schedule set forth in the company’s policy.  (emphasis added)

Judge Sebelius in Kansas recently agreed with Judge Scheindlin and held that a company is not required to suspend the normal recycling of  its backup tapes as part of a litigation hold.  Oxford House, Inc. v. City of Topeka, Kansas, 2007 WL 1246200 at *4 (D.Ka. April 27, 2007).  Such suspensions as part of a routine litigation hold can be very disruptive and expensive to any large organization with substantial ESI to backup. In most e-discovery cases, the tapes are never needed, and so the cost of routine preservation in every case is simply not justified. Further, disaster recovery type backup tapes are not reliable and decay over time. Even when the tapes are preserved, they are always difficult to recover, data is often lost, and you are never certain that the relevant evidence can be restored.

Admittedly in some rare cases, backup tapes may be the only place relevant ESI is still located by the time an organization receives notice of a suit.  In these circumstances, the imposition of a hold at that time would be the only way to preserve relevant evidence.  If an organization has notice of these unusual facts, then the general rule should probably not apply, and the hold notice should cover backup tapes.  But such circumstances are highly unusual.  Most of the time a defendant at the time of hold does not know whether the tapes will be needed.  In a few unusual cases, it is later discovered that the tapes were needed, and should have been preserved. These few exceptional cases do not justify the disruption caused in the vast majority of cases where the tapes are never needed.  

The general rule of exclusion of backup tapes from a hold is tested in the rare case; the case where it turns out the tapes should have been preserved. Oxford House is just such a case. Here a Topeka City Commissioner was found during discovery to have deleted certain relevant email in June of 2005.  The plaintiff claimed spoliation and sought sanctions.  The court rejected this argument because it found that litigation was not reasonably anticipated until August of 2005, two months after the email deletion.  The destruction of email before a duty to preserve arises cannot constitute spoliation.

That part is fairly cut and dried.  But the plaintiff also argued that the city should have preserved its email backup tapes.  It did not.  Instead, even after the August hold date, the city continued its normal backup tape recycling.  It continued to reuse the same tapes and thus erase the old emails and other information from the previous backups.  The city was on notice of pending litigation at the time it continued to recycle the tapes, and so from a time perspective at least, the duty to preserve had arisen.  But the city did not then know that the Commissioner had deleted relevant emails and the tapes might be the only place they could still be found.  

This situation raises the issue of the scope of the duty to preserve; did it extend to backup tapes?  Clearly the preservation duty applied to “live” ESI, to the easily accessible email on the city’s computers, but did it also run to the backup tapes?  Some would say yes, especially legal counsel representing a party with little electronic evidence at issue.  They disregard or attempt to minimize the cost and disruption caused by the routine imposition of holds on backup tape operations. The plaintiff, Oxford House, was of that ilk, and argued that the city had a duty to preserve backup tapes; but the court did not agree.

The backup tapes used by Topeka were disaster recovery type tapes.  They were not reasonably accessible, meaning they could be accessed, but only at great cost and expense.  In fact, the city estimated a cost of at least $100,000 to restore and review the tapes.  Judge Sebelius in Kansas, like Judge Scheindlin in Manhattan, was right to hold that these kinds of tapes do not have to be preserved.  A company should not have to stop its normal, good faith operation of recycling disaster recovery tapes every time litigation is threatened or materializes.

The City of Topeka first opposed plaintiff’s sanctions argument on the backup tapes the same way it opposed sanctions for deletion of the live email.  The city argued that the relevant emails had already been deleted from the backup tapes before August 2005, and so a duty to preserve never arose before deletion.  They made this claim because the backup tapes are supposed to be recycled every six weeks.  Since there was an approximate eight week gap between the estimated time of email deletion, and notice of pending litigation, the city argued that the emails in question were no longer on the backup tape in August, 2005.  In effect, they argued that the notice to preserve came two weeks too late, and so there was no need for the court to consider whether the scope of duty included backup tapes. 

Judge Sebelius recognized that it was a close question as to whether the emails would still have been on the backup tapes or not. Of course the city argued that its normal recycling procedures had been followed, and for that reason the emails would already have been written over in six weeks. But this is a slippery slope of an argument because it is hard to know if the exact timing of the recycling protocols were in fact followed, and the timing here was close.

If the facts alleged by the city had been proven, which is difficult to do, then once again there would have been no spoliation based on timing.  The erasure of the backup tapes, just like the deletion of the emails, would have occurred before a duty to preserve arose.  Then the court would not have had to decide whether the duty to preserve extended to backup tapes.  But the city’s argument was on shaky grounds, and apparently not well confirmed by affidavits or depositions.  In any event, the judge chose to reject it, and face the issue of the scope of preservation head on. Still, in other circumstances with a more fully developed evidentiary record, this argument alone might succeed. 

The court denied plaintiff’s motion for sanctions for backup tape spoliation based on the assumption that the deleted emails were still on the backup tapes in August at the time the duty arose.  In the words of Magistrate Judge Gary Sebelius:

In the court’s view, even if such back up tapes were conclusively shown to possess the deleted e-mail communications, “as a general rule, a party need not preserve all backup tapes even when it reasonably anticipates litigation.” Zubulake, 220 F.R.D. at 217. When parties put a litigation hold policy on destruction of documents in response to pending litigation, “that litigation hold does not apply to inaccessible back-up tapes (e.g., those typically maintained solely for the purpose of disaster recovery), which may continue to be recycled on the schedule set forth in the company’s policy.” Id.  The record in this case indicates that the back-up tapes are used for disaster recovery purposes.

The decision goes on to consider the related issue of whether the city should be forced to search its current backup tapes now, almost two years after the emails in question had been deleted.  Plaintiff argued that there was a chance the emails were still on the tapes in spite of their reuse. Plaintiff wanted to put the City of Topeka to the expense of undertaking the restoration and search of backup tapes on the chance that the emails in question had never been written over.  Plaintiff argued that the emails could possibly still be on the tapes because they might have been stored at the very end of the tapes, and no subsequent backups went that far.  This argument was based more on hope than reason, and was rejected by the Court:

It is certain that the cost of retrieval of data would be high. However, this factor is not dispositive–consideration must also be given to the potential efficacy of a technique seeking discoverable information. The court finds such efficacy is minimal at best in the instant case. The City of Topeka continually rewrites new data over the prior date on its back-up tapes.  Therefore, unless the latter back-up tape did not write as close to the end as the previous back up tape, this information has likely already been written over. As the likelihood of retrieving these electronic communications is low and the cost high, this court further finds that the unanswered portion of Plaintiffs’ Interrogatory No. 4 is unduly costly. The court thereby denies plaintiffs’ motion to compel further responses to Plaintiffs’ Interrogatory No. 4.

Aside from the odd procedural basis of this dispute arising from an objection to an interrogatory, the opinion itself is notable for never mentioning new Rule 37(f), which states:

Absent exceptional circumstances, a court may not impose sanctions under these rules on a party for failing to provide electronically stored information lost as a result of the routine, good faith operation of an electronic information system.

The full commentary for Rule 37 (f) is contained in the Blog page above “37″. 

You would expect a full analysis of the issues in this case to include the new ”safe harbor” rule.   The “exceptional circumstances” exclusion to the safe harbor is intended to apply to some situations where a duty to preserve has arisen.  In this case, the plaintiff would argue that the duty arose by notice of the pending litigation. There are other circumstances where the duty can arise, and the safe harbor be lost, such as by statute or regulation requiring emails or other ESI to be retained for certain periods. 

Perhaps the Court did not go there because it did not have evidence that the Commissioner’s email deletion was a “routine, good faith operation.”  In any event, the Court chose instead to rely upon the well established common law elements of spoliation.  It did not go into the related, but separate, and as yet largely untested provisions of the new rule.  Still, in most circumstances you would want to include the Rule 37(f) arguments in this kind of spoliation dispute.

Metadata Mistake by Top Spy Agency

Metadata, the hidden data contained in computer files, makes many people nervous, but especially lawyers.  They are uncomfortable with anything hidden, or for that matter anything poorly understood, and metadata qualifies as both.  Adding to this discomfort are the horror stories, well known in the profession, of other counsel accidentally producing documents to opposing counsel that contain embarrassing metadata.  For instance, a Word document containing secret comments they added, then hid, and then forgot to delete or “scrub” before production. 

This can be very serious for attorneys because of the high ethical duty they are under to maintain the secrecy of the confidential information disclosed to them by their clients.  For this reason, attorneys, more than most, tend to worry about inadvertently disclosing these secrets by metadata.  The concern is magnified for litigation attorneys who have opposing counsel watching their every move for mistakes. That is one of the reasons for the advent of “clawback” agreements and new Rule 26(b)(5)(B). If a privileged communication in a computer file is accidentally disclosed, say because it was contained in metadata that they did not see, they want to get it back, and prevent a waiver.

For these reasons it is especially troubling to lawyers to hear that even the best professionals at secret keeping, our country’s top spies, mess up on metadata.  If anyone is serious about secrecy, it is spies. Their very lives may be at stake. So if espionage professionals accidentally reveal state secrets because of metadata, then it could happen to anyone.  

Metadata is by definition out of sight.  And what is out of sight is out of mind, and so easily forgotten.  The latest metadata mistake story  proves this point in classic “spy versus spy” fashion. 

The Office of the Director of National Intelligence (”DNI”) is the highest intelligence agency in the United States.  The DNI oversees all federal intelligence agencies, including the CIA.  The size of the total US intelligence budget that DNI oversees is one of the government’s most closely guarded secrets.

On May 14, 2007, a Senior Procurement Executive of DNI gave an unclassified presentation to a group of outside contractors in Colorado entitled “Procuring the Future.” Her PowerPoint included a slide with two graphics depicting the trend of award dollars to contractors from 1995 to 2006.  Because these figures are highly classified, a scale of the total number of award dollars was omitted from the Y-axis of the bar chart.  The government contractors were only shown the graphical bars, and were left to guess what the actual expenditures were that the bars depicted.  A copy of this unclassified slide is shown below.

Apparently the DNI employee had used this same PowerPoint slide before to make classified presentations to persons with top secret clearance.  I presume this because the slide contains metadata revealing the actual amount of the award dollars. It would have been simple to reconfigure the graphic in PowerPoint to display the actual numbers to an audience with clearance.

The metadata revealing the classified information is not visible when the unclassified version of the PowerPoint is shown.  But when you open the native file using PowerPoint software, it is easy to make this embedded information visible.  All you have to do is double click on the bar chart shown above and a spreadsheet is revealed. It is that easy.  

The spreadsheet embedded as metadata in the graphic image controls the appearance of the bar graph. If you change the numbers in the spreadsheet, the columns change. Further, if you change the configuration of the graphic, the numbers will not be revealed at all.  This is exactly what they did in the unclassified version of the PowerPoint shown in Colorado.  In this way the secret numbers in the graphic were not displayed when the PowerPoint was shown.

So far, so good. The secrecy of the nation’s total intelligence budget was maintained at the conference. The big mistake was made after the conference when someone in the agency posted the PowerPoint file on the Defense Intelligence Agency website in native format.  There anyone could (and did) download the PowerPoint file onto their own computers. (Spies are like litigation counsel, they have opposing spies that watch their every move and look for mistakes.) Once the file was downloaded and opened in PowerPoint, the embedded numbers underlying the expenditure graphic were quickly uncovered.  One of America’s top secrets was lost due to careless handling of metadata.

The key mistake was to have posted the PowerPoint in its original native format. The file should have been flattened first and posted as an image file.  Then the metadata would have been stripped, and the secret would have been safe. Alternatively, the graphic in slide eleven should have had the secret spreadsheet removed.  But the top spies forgot to do that.  Out of sight, out of mind.  Metadata like that is a time bomb waiting to happen.

Do not look for the PowerPoint on the Defense Intelligence Agency website.  It was removed as soon as the press reported the error, and reported the once-secret intelligence budget.   Still, I had no difficulty finding another copy of it based on later press reports and a quick search on Google.  Information in today’s Internet Age is like a cat; once it is out of the bag, you can never get it back in.  This is the stuff of nightmares of spies and lawyers alike. 

Keyword Searches v. Concept Searches

An opinion this month by Judge Facciola distinguishes between keyword searching and concept searching.  Disability Rights Council of Greater Wash. v. Wash. Metro. Area Transit Auth., 2007 WL 1585452 (D.D.C. June 1, 2007).  The plaintiff had proposed simple keyword searching of email by people’s names, but Judge Facciola suggested the parties instead consider concept searching. This is the first opinion to recognize the distinction between the two types of searches according to Jason R. Baron, Director of Litigation of the National Archives and Records Administration.  He wrote to me earlier today to bring this to my attention.  Jason should know, as he is an expert and strong proponent of concept searching.  Indeed, Judge Faccio cites to his article in the opinion. Here is the operative language from Disability Rights Council at *9:

I bring to the parties’ attention recent scholarship that argues that concept searching, as opposed to keyword searching, is more efficient and more likely to produce the most comprehensive results. See George L. Paul & Jason R. Baron, Information Inflation: Can the Legal System Adapt? 13 Rich. J.L. & Tech. 10 (2007).

My blog of April 9, 2007, reviews this article in depth, and I mentioned it again yesterday when discussing the need for cooperation between counsel in my blog on Intellectual Foundations. Concept searching is just one of many cutting edge ideas discussed in Paul & Baron’s article; one that I have not discussed before.  It pertains to promising new software technology that may allow for far better searching than simple keyword matching.  But before I go into that,  a little more about the interesting Disability Rights Council case itself. 

The defendant Transit Authority configured their Groupwise email system so that all emails were automatically deleted after only 60 days.  The only exception was when a user went to the trouble to archive a particular email.  These archived emails were not deleted.  In practice, few Transit Authority users ever bothered to archive any of their emails, and so after 60 days almost all were deleted. Nothing wrong with such a system in principle, but the problem here is that it was not suspended when suit was filed. In fact, in what the court stated was “remarkable” and “indefensible”, the defendant continued to destroy all emails for over two years after the suit was filed.

The opinion begins by noting that the “safe harbor” of new Rule 37(f) was not intended to apply to this situation, at least insofar as the emails destroyed after the suit was filed are concerned. The rule requires “routine” and “good faith” operation of a system.  Although it was routine destruction, the court did not consider it to have been carried out in good faith after suit was filed.  That is primarily because we are talking about the destruction of live ESI, namely email still on the system, and not on back-up tapes.  A preservation hold should have prevented this. After the live emails are so destroyed, the only place to find them is on the backup tapes. For that reason, among others, even though the court agreed with defendant that the backup tapes were not reasonably accessible under Rule 26(b)(2)(B), it nevertheless found good cause to order that they be restored and searched at defendant’s expense.  To hold otherwise would reward defendant for destroying relevant emails, leaving the backup tapes as the only remaining source of the evidence.  The court rejected this at *8 with a humorous touch:

While the newly amended Federal Rules of Civil Procedure initially relieve a party from producing electronically stored information that is not reasonably accessible because of undue burden and cost, I am anything but certain that I should permit a party who has failed to preserve accessible information without cause to then complain about the inaccessibility of the only electronically stored information that remains. It reminds me too much of Leo Kosten’s definition of chutzpah: “that quality enshrined in a man who, having killed his mother and his father, throws himself on the mercy of the court because he is an orphan.”

Judge Facciola rejected Defendant’s undue burden and expense inaccessibility arguments, and granted Plaintiff’s motion to compel.  He then ordered the parties to meet and discuss how the backup tapes will be restored, and as mentioned, how to search the restored emails, either through keyword as plaintiff had proposed, or via concept search as the judge suggested might be more efficient. (As a postscript, I understand the parties met, but instead of agreeing on search, they settled the case instead.)

Of course, keyword searches have been around for decades and are familiar to any lawyer who has ever done computer research.  You can, for instance, run a computer search of hundreds of thousands of emails to find all emails that include one or more of a list of names, as plaintiff here proposed.  This takes just seconds, but can produce a high percentage of irrelevant emails; ones that include the names but have nothing to do with the case. It can also omit many relevant emails that just do not happen to include the keywords you guessed a relevant email would have (or perhaps included them, but misspelled them, a problem not often found with computerized legal research).  

The use of complex Boolean connectors (such as directives that one term be within the same paragraph as another, or that an otherwise included email be excluded if it contains certain terms, i.e. “but not”) can sometimes improve on the search.  So too can the use of so called “fuzzy logic.”  But even with the use of Boolean and fuzzy logic, these keyword searches, also known as “theoretical set” searches, are still largely a guessing game.  In practice, they often fail to uncover too many otherwise relevant emails, without significantly reducing the irrelevant ones.  

A search that creates a lot of noise, that is, one that produces too many irrelevant emails, can create very significant time and expense burdens on all the parties, but especially on the producing party.  If for instance the search creates a list of 100,000 emails, the producing party will have to review all of these emails for possibly privileged communication before production.  This is a very expensive undertaking, and although clawback agreements provide some comfort, they cannot obviate the need for, and expense of, the privilege review.  It is also expensive for the receiving party who also has to spend time and money to review the irrelevant emails.

Therefore, if there is a better search method than keyword that can produce a high percentage of relevant hits, and thus less noise and less wasted time for privilege review, it is to the advantage of all parties to use it.  Moreover, it is a potentially very valuable product.  There are several software vendors who have created alternative search algorithms to keyword searches.  All are sometimes lumped together as “concept searches.” They use a variety of methods, involving such things as contextual usages, algebraic modeling and probabilistic categories.  The exact formulas are usually kept secret by the software vendors for obvious reasons, but most are prepared to provide expert testimony in court if necessary to justify the legitimacy of their search methods.

Paul & Baron’s article at pages 26-27 summarizes the existing state of information retrieval science in a “mind boggling,” but eloquent manner:

However, broadly speaking, information retrieval methods fall into three broad classes: set-theoretic (Boolean strings, supplemented by fuzzy search capabilities), algebraic (premised on the mathematical idea that the meaning of a document can be derived from the constituent terms in a document, and thus weighting retrieval by the proximity of a document’s terms in the form of two or higher dimensional maps, as in vector space modeling), and probabilistic (using language models and Bayesian belief networks, the latter of which involves making educated inferences about the relevance of future documents based on prior experience in reviewing documents in a given collection).

In thinking about retrieval problems, one can also supplement all of these methods by focusing on the language used by the creators of the records, which will include using taxonomies and ontologies, essentially synonyms of words and relevant classes of related words to be developed and built in at the front end of a search process to better refine the search, and to maximize both recall and precision. In contrast to strict set-based Boolean techniques, the above algebraic and probabilistic categories of search methods are often broadly termed under various forms of the heading “concept searching.”

A review of the various vendors who offer such marvels will have to await another time, as there are several of them now, and it is a growing field.  Still, this is definitely something that you should look into before agreeing to simple keyword searches, especially if the volume of ESI to be reviewed is high.  The concept search software fees are probably too expensive for small volume or low dollar cases, but they could be a huge money saver for a larger case.  Most vendors will provide you with an idea of the price break point based on the byte size of the ESI.  Of course, it is more than the number of megabytes involved.  You also need to consider the case subject matter, the amount of money involved, and the importance and complexity of issues.

For more information on this subject check out the West Legalworks CLE webinar I did with Jason R. Baron - Director of Litigation, U.S. National Archives and Records Administration, College Park, Maryland; Doug Oard, Ph.D. - Associate Dean for Research, College of Information Studies, University of Maryland; and my law partner at Akerman in Los Angeles, Michael S. Simon. The 1.5 hour audio CLE is entitled The e-Discovery Search Quagmire: New Approaches to the Problem of Finding Relevant Needles in the Electronic Haystack.

Intellectual Foundation of Electronic Discovery

Electronic discovery is a new, multi-disciplinary field of knowledge that arises out of the Law, but goes beyond it to include Information Science and Technology.  This new fractal synthesis is emerging by necessity to meet the many radical challenges imposed upon our system of justice by the rapid acceleration of technology.

Our writings and other forms of communication are being digitized at a dizzying pace. In one generation, we have moved from a primarily paper-based, tangible information culture, to a culture based on intangible binary code. We have moved from a society that stores and retrieves information locally in paper files, filing cabinets, books and libraries, to one that stores information globally in electronic devices and the Internet.

This revolution is necessarily having a profound impact upon the law because information is the foundation of our justice system.  We determine what is just and right based on the evidence.  As Judge Scheindlin stated in Zubulake IV: “Documents create a paper reality we call proof.” Zubulake v. UBS Warburg, LLC, 220 F.R.D. 212 (S.D.N.Y. 2003). For lawyers to adapt to this new world we need to rethink what we mean by documents, to understand and revisualize them as electronically stored information.  Then we will see the electronic reality we call proof. 

As attorney and information expert, Jason Baron, put it in the law review article he recently wrote with attorney George Paul: “Information is fundamental to the legal system. Accordingly, lawyers must understand that information, as a cultural and technological edifice, has profoundly and irrevocably changed.” Information Inflation: can the legal system adapt? 13 Rich J.L. & Tech 10 (2007). Baron and Paul argue that litigators must move beyond the gamesmanship model of paper discovery into a more cooperative search for truth in e-discovery. They point out how the sheer volume of electronic documents makes this new collaborative model imperative. They do not suggest that the adversarial system of dispute resolution be abandoned entirely, just that the discovery model be changed. They also argue for the creative adoption of new technologies and records management systems by lawyers. They observe correctly that:

All this equates to perhaps the biggest new skill set ever thrust upon the profession – a revolution for the practice. What it means to be a lawyer will change rapidly in the years to come.

The new skill set needed to comprehend the electronic reality we call proof does not come easily. Many are unwilling to take the time and effort needed to begin to master the rudimentary elements of  computer technology and information science.  The younger generations have some advantage in familiarity with technology, but still most have never studied information technology and records management in college. Very few move from out of these disciplines into the law. Unfortunately, these subjects are ignored altogether in law school.  Therefore, until academia catches up with these new trends and radically alters curricula to match the new reality, most lawyers will have no choice but to try to learn these new fields on their own, or in CLEs. Unfortunately, most of the CLEs on information management and technology in e-discovery are sponsored by vendors, and have not so hidden agendas.  They often make competing and confusing claims on what technology to use, and how to manage the flow of information. Until you have had a few decades of experience with all this, it can be very difficult to sort out.

I expect (hope) that the quality of continuing education for lawyers in the areas of Information Science and Technology will improve dramatically in the next few years.  Most of this will come in the context of litigation and e-discovery, but other areas of the law are also burdened with the flood of information.  In the meantime, we will all have to struggle and do the best we can on our own, attending seminars given by vendors and others with a healthy dose of skepticism. 

One good place to start is the Sedona Conference, although most of the materials are legal, they do contain a heavy dose of IT.  Other places to look and learn are Computer World, the IEEE Computer Society, the Association of Records Managers and Administrators, and many technology law review periodicals.   For information science webs try: the Digital Library of Information Science and Technology, which is an open access archive for the Information Sciences; the Journal of  Information Science; and, the American Society for Information and Technology.

There are many other more “fun” and entertaining webs and blogs where you can keep up with the latest trends and learn as you go, such as IT Wire, the ever popular WIRED, the venerable PC World Magazine, the hip SlashDotOrg, the GigaOm, the Register, the DIGG,  and Tech Crunch, to name just a few.  A few minutes with Google will uncover many more. 

I am still looking for other good sources, especially good old paper books on the subjects (easier to read after hours in front of a terminal), and from time to time will post information about them here.  If you have any good suggestions, please feel free to let me know by email or post a comment here.  But please spare us the commercials.  I will quickly delete any such comments anyway, so save yourself the time.

Top Ten Reasons e-Discovery is a Major Headache for Most Companies and Lawyers

Most corporate counsel agree that Electronic Discovery is the major problem in litigation today. Here are the Top Ten reasons:

1. The costs to preserve, find and review electronic evidence, including emails, are astronomical, and getting worse every day. See Eg., Kentucky Speedway v. NASCAR, 2006 U.S. Dist. LEXIS 92028 (E. D. Ky. Dec. 18, 2006) ($3,000,000 expenses in 5 months for e-discovery alone).  According to an unconfirmed report by a Microsoft insider, it now spends an average of $20 Million Dollars per case.  It is no surprise that e-discovery is now the hot field for entrepreneurs, and that it has mushroomed into a $2 Billion Dollar a year industry.
2. The unacceptably high risks of losing a case, or being forced to settle a case because of e-discovery, rather than the merits.  The mistakes in e-discovery are pervasive and often disastrous. The biggest of them all is the Coleman v. Morgan Stanley case in Florida, which resulted in a $1.5 Billion Dollar verdict. Coleman (Parent) Holdings, Inc. v. Morgan Stanley & Co., Inc., 2005 WL 674885 (Fla.Cir.Ct.. 2005) (although the decision has preliminarily been reversed, the Florida courts are not through with this yet).  Also see: GTFM, Inc. v. Wal-Mart, 2000 WL 1693615 (S.D.N.Y. Nov. 9, 2000), (sanctions); Exact Software v. Infocon, 2006 WL 34999992 (N.D. Ohio) (Dec. 5, 2006) (more sanctions); Phoenix Four, Inc. v. Strategic Resources Corp., 2006 WL 1409413 (S.D.N.Y. 2006) (still more sanctions).
3. Electronic records are easy to destroy or alter, but the bad emails and instant messages never seem to go away! The smoking guns in court rooms today are found in computers, not filing cabinets. In fact, 98% of all business records are now electronic, and 80% of them are never converted to paper or other tangible form. So if you don’t look for the ESI, you will miss the key evidence.
4. The amount of electronic information stored by most corporations today is staggering.  In the Enron case, they found twice as much information stored in its computers than in the Library of Congress; over 78 Billion pages.  As of 2006 the world is sending 60 Billion e-mails per day.  The volume makes it impossible to retrieve and review everything in most large cases today, and nearly impossible for anyone not an expert to find the needles in the haystack.
5. The computer systems and information storage systems have become extremely complex.  It is difficult for any one expert to understand it all.   The complexity makes mistakes almost inevitable, and explanation to supervising judges and magistrates near impossible.
6. Most companies do not have functional ESI management policies. If they do, they are not monitored, much less enforced. With thumb drives and online accounts today so commonplace, most companies have no idea where all their business records and communications really are, even if they know where they are supposed to be.
7. There have been so many mistakes with e-discovery in the past several years that many judges and magistrates are now upset.  They will no longer tolerate mistakes.  As one judge puts it the “pure heart, empty head” defense will no longer work in his courtroom.  Most judges today are reacting by imposing high standards and duties upon the parties and legal counsel.  See eg. Zubulake v. UBS, 229 F.R.D. 422 (S.D.N.Y. 2004) (”Zubulake V”) as discussed further in the Blog “Duties” page above.
8. The New Federal Rules of Civil Procedure make the problem worse by accelerating all deadlines and prohibiting the avoidance of e-discovery issues until the end of a case.  The only way to comply with the new rules is to be extremely well prepared, even before a law suit is filed, and that requires tech-savvy legal counsel, and well-prepared litigants.
9. Most lawyers and law firms are unprepared for e-discovery. Attorneys need to know the basics of information management and computer technology to handle e-discovery issues correctly, but in point of fact, most do not. They do not even like the subject.  As everyone in the profession knows, most trial lawyers are big talkers, not geeks. If a law firm does have one or two attorneys with some computer-tech expertise, they are typically the youngest with little or no litigation experience. 
10. Most corporations and in-house legal counsel are unprepared for e-discovery.  They may have fine IT departments, and great inside legal counsel, but the two departments speak very different languages and do not work well together.

In my opinion, the only viable solution to the problem of e-discovery is for a company to create its own internal e-discovery readiness response team (”Team”).  The alternative of delegating everything to expensive e-discovery vendors, and dozens of outside counsel around the country, has been tried and does not work.  Of course vendors are still key, and so are outside counsel, but the corporate client needs to be in charge of its own destiny.  The internal e-discovery Team is the best hope to reduce costs, manage risks, and better control quality. 

As discussed further in the Blog “About” page above, the Team is composed of in-house attorneys, IT personnel and management.  It rests on the three pillars of Law, Information Science, and Technology.  The Team functions to implement litigation holds, collect data within the timeline of new Rules, retain e-discovery vendors, supervise local counsel, and improve electronic records systems.  A few companies have done this already and it works: Pfizer, Halliburton, and Merrill Lynch, to name a few. 

But experience shows it is hard to get Law and IT to work together and communicate. Most companies want and need outside help to set up their Team.  That is where I come in, and the services of my law firm.  We make the Team happen.  Our e-discovery program is unique because it is designed to empower the client by helping the company to start and run their own Team.  We serve as Team Coach and Trainer. The Client is the Owner, and has its own Team Captain.  Just like a Coach sometimes has to step up and argue with the referees, we also sometimes appear in court when necessary to advocate the Team’s position, and assist local counsel on these issues.  In that sense, we also serve as a National e-Counsel.  It is a challenging service, but one we love to perform!

Litigation Hold Is Not Enough: Sanctions Imposed Under Rule 26(g) for Negligent Collection and Preservation

Sanctions were recently imposed under Rule 26(g) for errors in the collection and preservation of computer files. Cache La Poudre Feeds, LLC v. Land O’Lakes Farmland Feed, LLC, 2007 WL 684001 (D.Colo. March 2,  2007).  Rule 26(g) requires an attorney to sign all discovery requests, responses, and objections.  The Rule further states that:

The signature of the attorney or party constitutes a certification that to the best of the signer’s knowledge, information, and belief, formed after a reasonable inquiry, the disclosure is complete and correct as of the time it is made.

Moreover, Rule 26(g)(2) provides that by signing, an attorney is certifying that to the

best of the signer’s knowledge, information, and belief, formed after a reasonable inquiry, the request, response, or objection is:    . . . (B) not interposed for any improper purpose, such as to harass or to cause unnecessary delay or needless increase in the cost of litigation. . . . (C) not unreasonable or unduly burdensome or expensive, given the needs of the case, the discovery already had in the case, the amount in controversy, and the importance of the issues at stake in the litigation.

Unfortunately, in my experience, the obligations imposed upon counsel of record by Rule 26(g) are not always followed. The Land O’Lakes case provides a rare example of the imposition of sanctions for its violation.  Even here, the court held that the conduct in question was also sanctionable as spoliation. Further, the sanctions imposed against the defendant whose attorneys violated the rule were minor, only a $5,000 fee award plus court reporter costs.  The mild nature of the sanction is perhaps explained by the court’s obvious irritation at the conduct of both parties, including the plaintiff, who had previously been sanctioned for a Rule 11 violation.  If the moving party had been wearing a “white hat,” I expect that a far harsher sanction would have been imposed.

Even though the sanctions imposed were relatively minor, the case is still important, not only because Rule 26(g) was applied, but also because of the facts found to be sanctionable.  These facts make clear that it is not enough to simply issue a litigation hold to key employees, and then assume they will properly locate, preserve and produce the relevant computer files and other ESI.  Counsel have a duty under the rules to followup on the hold notice, and make reasonable efforts to independently verify that the hold directive has been followed, and the relevant ESI has been preserved and produced. This is part of the so called “Zubulake duties” discussed at length in the “Duties” blog page above. See Zubulake v. UBS Warburg LLC, 229 F.R.D. 422 (S.D.N.Y.2004) (”Zubulake V“).

The defendant in this case, Land O’Lakes, sent out a litigation hold notice to key employees within days after the trademark violation suit was filed. The court found the timing was acceptable, but faulted Land O’Lakes’ in-house and outside counsel for the procedure chosen to preserve and collect the ESI, and for the poor followup to the hold notice. 

After the written hold notice was sent, there were interviews with key witnesses, but the Land O’Lakes employees were essentially on their own to locate and preserve the emails and other files that they considered to be related to the trademark dispute.  The employees looked through their files, and although they located 50,000 pages of documents related to the mark “Profile”, they only found 415 emails. Counsel simply accepted all of this as correct. No attempt was made by either in-house counsel or by outside counsel, who signed the discovery responses under Rule 26, to independently verify their efforts.  Counsel simply took the files they produced and assumed that it was complete and the search was thorough.   Further, no system-wide key word search was ever run on defendant’s systems, or the key employees, as plaintiff argued strenuously should have been done. 

The Court was clearly troubled by this borderline negligent approach under the circumstances.  But the court found even more troubling the failure of counsel to prevent the “wiping” of hard drives from computers of employees who left the company after the suit was filed, at least one of whom was undeniably a “key player.” The court considered this a failure to preserve evidence that constituted spoliation.

The plaintiff argued that this conduct constituted a clear violation of defendant’s Zubulake duties.  Specifically plaintiff argued that:

Defendants failed to comply with the following “duties” set forth in Zubulake V -

• once litigation is commenced or a party reasonably anticipates litigation, it must suspend its routine document retention/destruction policy and put in place a “litigation hold” to ensure the preservation of relevant documents;
• in furtherance of the “litigation hold,” counsel must become fully aware of the client’s document retention policies and data retention architecture;
• counsel must communicate with “the key players”in the litigation in order to understand how they stored information;
• counsel must take reasonable steps to monitor compliance with the “litigation hold” so that all sources of discoverable information are identified and searched; and,
• having identified all sources of potentially relevant information, a party and its counsel are under a duty to retain that information and produce information responsive to the opposing party’s requests.

Under these circumstances the court held:

While instituting a “litigation hold” may be an important first step in the discovery process, the obligation to conduct a reasonable search for responsive documents continues throughout the litigation. See Fed.R.Civ.P. 26(e)(2) (a party is under a duty seasonably to amend discovery responses “if the party learns that the response is in some material respect incomplete or incorrect and if the additional or corrective information has not otherwise been made known to the other parties during the discovery process or in writing”).

A “litigation hold,” without more, will not suffice to satisfy the ‘reasonable inquiry’ requirement in Rule 26(g)(2). Counsel retains an on-going responsibility to take appropriate measures to ensure that the client has provided all available information and documents which are responsive to discovery requests. Sexton v. United States, 2001 WL 649445 (M.D. Fla. 2001). As the Advisory Committee Notes make clear, “Rule 26(g) imposed an affirmative duty to engage in pretrial discovery in a responsible manner that is consistent with the spirit and purposes of Rules 26 through 37.” In this case, I find that Defendants failed to meet this standard.

When Should You Search for Deleted Files?

The District Court in Tampa recently issued a discovery Order involving deleted email. Wells v. Xpedx, 2007 WL 1200955 (M.D.Fla. April 23, 2007). The case raises the interesting issue of when you should search for deleted files, and when you should not because they are “not reasonably accessible” under Rule 20(b)(6). In my view, this often hinges on whether or not the files were “double-deleted.” The order in question granted plaintiff’s motion to compel a 30(b)(6) deposition of defendant’s IT representative, but deferred ruling on the companion motion to compel production of deleted emails because there was an inadequate factual record to decide the issues, and for that reason the deposition was allowed.

The dispute began after defendant Xpedx produced its written records retention policy manual. The manual showed that Xpedx had introduced a new email retention policy in 2003 wherein emails were automatically deleted after 90 days, unless they were specifically designated for retention. The manual explained that if an email was not designated for retention, then after 90 days the automatically deleted email could not be restored, without consent of the company’s legal or tax department.

Not surprisingly, plaintiff sought a deposition to learn what this policy meant, and specifically to explain the procedures for retrieval. Further, plaintiff hoped to discover in that deposition if any emails were destroyed relating to his claims. Plaintiff also sought to compel production of all Xpedx emails related to his claims. During the hearing on the motion to compel, the plaintiff narrowed the scope of production request to any emails containing his name, Joseph Wells, in certain time periods in 2002 and 2003 by seven Xpedx employees.

The defendant opposed the deposition, arguing that it was unnecessary because it had already produced its records retention manual. This argument was not persuasive in view of the vagaries of the wording of the policy, especially as to the approval of restoration of deleted emails by the law or tax department. Further, a plaintiff in this position is normally permitted to inquire whether the actual practices of the company comply with the policy directives. Defendant opposed the motion to compel because it argued that all relevant emails had already been produced. This is a very dangerous argument, as many defendants in the past have discovered, including UBS Warburg and Morgan Stanley.  After you make that kind of representation to a court, it tends to look bad if relevant emails are later discovered, no matter how innocent and understandable the mistake. Finally, defendant argued that if there had been any other emails that were not specifically preserved, then they would have already been deleted under its 90 day policy. Magistrate Judge Elizabeth Jenkins, who is well known in Central Florida for her good analysis and pragmatic results, begins her discussion of the email discovery dispute by noting that:

Deleted emails are, in most cases, not irretrievably lost. Discoverability of Electronic Data Under the Proposed Amendments to the Federal Rules of Civil Procedure: How Effective Are Proposed Protections for “Not Reasonably Accessible” Data? 83 N.C.L.Rev. 984, 988 (2005).  Deleted emails may remain on a computer hard drive, servers or retained on back-up tapes. Id. at 988-90.

So far, so good; but then Magistrate Jenkins goes on to hold that: “The producing party has the obligation to search available electronic systems for deleted emails and files. Peskoff v. Faber, No. 04-526 (HHK/JMF), 2007 U.S. Dist. LEXIS 11623, at *13 (D.D.C. Feb. 21, 2007).”  There I think the opinion goes too far, or should have been better qualified and clarified to apply only to “once-deleted” emails and files, and to exclude “double-deleted” files. 

First let me explain what I mean by these terms.  As everyone knows, if an email in Outlook is deleted, it is not erased; it is just moved into another Outlook folder for deleted files. It is still readily accessible. Any user can change their mind and restore the email. But if an email in the Deleted Folder is deleted again, in other words “double-deleted,” then it is no longer indexed in Outlook, and no longer readily accessible (unless, in some circumstances, as one observant reader pointed out, it is still within a double-deleted email retention period set by the Outlook Administrator; usually only seven days).  Of course, double-deleted files are not truly deleted yet on the hard drive, but the file markers pointing to it and identifying it have been omitted (or, in some cases, will soon be omitted). These double-deleted files still exist, they have not been erased, but, in most cases at least, they are no longer easy to find and retrieve.  It requires some level of forensic examination to locate and try to retrieve them.  In fact, in some circumstances, double-deleted emails may be impossible to retrieve from the hard drive (be it server or PC) because they have been written over by other files. The same process applies to “once-deleted” files in the Windows operating system environment.  They are not removed from an index of files when first deleted; they are simply moved to a new location, the Trash directory.  But if the Trash is emptied, the pointers for that file are removed, and the space is considered to be available for writing new data.    

So, back to the opinion, which states a party has an “obligation to search available electronic systems for deleted emails and files.”  If the phrase “deleted emails and files” is construed to mean “once-deleted,” than I agree completely.  These files are readily accessible.  But, in most circumstances, it would be wrong to try to stretch the meaning of this holding to also include files that have been “double-deleted,” and thus require forensics to locate and restore, or resort to a search of back-up tapes.  

In my opinion, and that of most commentators and courts that have squarely faced the issue, the obligation to search for “double deleted” files should not arise in all circumstances.  This duty should only arise in certain special circumstances, where, for instance, there is evidence that highly relevant emails have been double-deleted, and therefore that there is good cause to go to the extra time and expense inherent in a forensic examination for such files.   Most courts do not require an extraordinary search for deleted files, unless and until special circumstances are shown to warrant such extraordinary efforts. See for example, Hedenburg v. Aramark American Food Services, 2007 U.S. Dist. LEXIS 3443 (W.D. Wash. Jan. 17, 2007) discussed in my blog entry of February 25, 2007.   

The facts alleged by Xpedx to oppose the deposition of its 30(b)(6) IT representative suggest that the emails sought were all “double deleted.”   But it is not clear, and in fact, that was the plaintiff’s whole point.  His motion to compel the deposition sought discovery of the facts surrounding these emails and deletion.  The deposition should reveal whether or not the emails in question were once deleted, and thus reasonably accessible under Rule 26(b)(2)(B).  If they are twice-deleted, and further, if under Xpedx’s computer system that means that they are not readily accessible, as I suspect, then an entirely different legal analysis applies to determine whether the plaintiff is entitled to compel production of these emails.  If they are not reasonably accessible under Rule 26(b)(2)(B),  then the plaintiff would be required to make a showing of good cause sufficient to justify the extra expense of location and recovery of the double-deleted emails. 

For these reasons, Judge Jenkins’ order compelling the deposition makes good sense, especially when you consider she limited the deposition to four hours and permitted it to occur by phone.  It also makes sense to defer ruling on the motion to compel.  This motion cannot be properly determined without evidence as to whether the deleted emails are reasonably accessible, and if not, whether the facts and circumstances show good cause for them to be produced anyway, perhaps with cost sharing as suggested by Rule 26(b)(2)(B).