The past week has been a busy one for us and we’re excited to announce that thanks to a generous data donation we’ve added an additional 7,000 oral arguments to CourtListener. These files are available now and can already be searched, saved, and made into podcasts.
Although 7,000 more oral arguments may not sound like much, I must point out that these files are larger than your average MP3 and this has taken a week for our powerful server to download and prepare. Our collection now has more than 200 continuous days of listening – more than six months of audio.
From here on, we’ll continue getting the latest oral arguments from the Federal Appeals courts that offer them but we are eager for more donations so we can build up our archive. Oyez.org is doing an incredible job with the Supreme Court (and we hope to integrate these eventually), but if you know somebody in the federal appellate courts or if you have a collection of oral arguments you’d like to share, please get in touch! We’re eager to hear from you and to build the largest collection of oral arguments that we can.
It’s really hard to overstate the incredible nature of the open source community, but if you head over to CourtListener, you’ll find that between yesterday and today the entire website has been revamped. This was done over the past month by a volunteer developer who took time off work to give every single page in the entire site a fresh lick of paint.
On top of just plain looking better, the new site has a raft of improvements that we’re excited about:
- The entire site from top to bottom can now be used from your phone, tablet or desktop. Complicated pages will do the right thing to adapt to smaller or bigger screens. (Yes, we now use bootstrap.)
- The accessibility of the site has been vastly improved for people with motility or vision difficulties. Over the next few days we’ll continue rolling out improvements in this area that will trickle down even to keyboard-heavy users.
- Sharing pages on Facebook or Twitter and saving the site to your desktop on iOS, Android or Windows now works properly, with good titles, descriptions and meta data.
This post is supposed to be about the new version of the site, but I can’t help but gush about the open source community and the smart people that it attracts. For our part, we did a little bit of deployment work and a few design sessions, but this developer came in, fixed every page of the site and then gave us a new and beautiful version. That’s incredible and it’s something that we could never do outside of the open source community. It was also particularly nice that, thanks to geographic proximity, we got to have two in-person sessions during the process.
This update makes CourtListener ready for the next several years and modernizes the way it looks so that it feels right regardless of if you visit on your phone, computer, Playstation, or whatever. With thousands of people visiting each day, we couldn’t be more excited to provide this update.
Let us know what you think and if you’re interested in helping build the next feature like this one, get in touch! There’s always more work to do. We’d love to hear from you.
We’re happy to share three pieces of news about oral arguments at CourtListener.
First, the sixth circuit has begun putting oral argument audio on their website and we have begun dishing it up through CourtListener. We briefly spoke to the technology team at the court and their reaction to our questions about their system was, “Oh, is that on our website already?” So this is a very new development, even for them.
Right now their site has oral argument audio back to August 7th and we are in the process of grabbing this audio and putting it in our archive. Unfortunately, the case in the news right now that’s blocking gay marriage in the circuit was argued one day prior to the oldest files they’ve posted, and so we don’t have audio for that case, and possibly never will. This is one big reason we’ve wanted to get into oral arguments on CourtListener and why we’ve been supported with a grant from Columbia Library to do this work: This content is simply going dark as new content is published.
While we’re happy to now be collecting data from the 6th Circuit, it’s still disappointing that we don’t have it from the Second, Tenth, or Eleventh Circuit, as they do not (yet) publish their oral argument audio. We simply have to hope that the content is being stored somewhere and that we’ll get it eventually. (If you’re reading this and have connections in those courts, we’d love to hear from you.)
A second piece of news is that we just added our 1,000th oral argument since beginning eight days ago! This is already more than 40GB of data — a harbinger of complex systems problems to come, but we’re excited to be collecting so much so quickly and we hope you’ll create some podcasts or alerts for what we have.
Our final piece of news today is that our oral argument podcasts are now available on Stitcher. If you have a smartphone and enjoy podcasts, simply search for CourtListener in the Stitcher app, and our podcasts will come right up.
We’re very excited to announce that CourtListener is currently in the process of rolling out support for Oral Argument audio. This is a feature that we’ve wanted for at least four years — our name is CourtListener, after all — and one that will bring a raft of new features to the project. We already have about 500 oral arguments on the site, and we’ve got many more we’ll be adding over the coming weeks.
For now we are getting oral argument audio in real time from ten federal appellate courts. As we get this audio, we are using it to power a number of features:
- Oral Argument files become immediately available in our search results.
- A podcast is automatically available for every jurisdiction we support and for any query that you can dream up. Want a custom podcast containing all of the 9th circuit arguments for a particular litigant? You got it.
- You can now get alerts for oral arguments so you can be sure that you keep up with the latest coming out of the courts.
- For developers, there are a number of new endpoints in both our REST API and our bulk data API for audio files.
- Using the Free Law Seal Rookery, we are enhancing the audio we find on court websites by adding album art and better meta data.
For now, search results and alerts are limited to the data that is provided by court websites, so you cannot (yet) get alerted any time somebody says a certain word in court. Audio is a new area for us though and we’d absolutely love to automatically create transcripts for the courts, enabling such a feature. This would be an incredibly powerful feature, so if you are an expert on audio transcription, we’d love to hear from you and to work together on this.
Beyond all of the great features we’re rolling out today, oral argument data also marks an important turning point for the project because it lays the ground work for adding more types of data to CourtListener. It’s been a large undertaking adding a second type of data to the project, but adding a third will be much easier. Next in our hopper will likely be the content from RECAP so that you can create alerts, have powerful APIs, and do all the other things you expect from CourtListener, except this time, for documents from PACER.
We’re very excited about being able to provide oral argument data today and RECAP data tomorrow. We can’t wait to see what kinds of legal research and innovation these new features bring.
Last Friday, it was reported by the Washington Post and Ars Technica that Chairman of the Judiciary Committee, Senator Patrick Leahy, had sent a letter to Judge Bates, the head of the Administrative Office of the Courts, urging the AOC to put back online the recently-removed PACER documents from five courts. I had not seen the full letter posted anywhere yet, so present it here:
Free Law Project agrees with Senator Leahy that taking these documents offline represents “a dramatic step backwards” and that the Courts’ currently proposed work-around represents “a troubling increase in costs…” We hope the AOC will be open to restoring online access to these documents and stand ready to help make these documents freely available online for the public were that agreeable to the AOC.
Brian and I were guests this week on the live Internet show, This Week in Law on the Twit Network. In the show we cover a number of topics ranging from the history of Free Law Project and the need for innovation in the legal arena to the copyright and trademark issues in the latest Deadmou5 brouhaha.
Also available on YouTube. We hope you’ll enjoy watching.
At least since the destruction of the Ancient Library of Alexandria, the world has known the importance of having a backup. The RECAP archive of documents from PACER is a partial backup of documents taken offline by five federal courts. It is impossible to determine how complete a backup we have, because the problem with missing documents is that you cannot even determine that they are missing without a complete list of what used to be available. No such lists exist for the documents from these five courts.
But as coverage of this surprising and unprecedented action by PACER officials continues (see techdirt), the BBC has an article that takes an interesting approach by pointing out some of the landmark civil rights cases taken off PACER through this action.
The BBC mentions the case Ricci v. DeStefano which was decided at the Second Circuit while Sonia Sotomayor was a Circuit Judge. Sotomayor, now a Supreme Court Justice, had her role in deciding the case closely scrutinized during her Supreme Court confirmation hearings. Many who dug in to Sotomayor’s background during those hearings likely relied on the documents available via PACER related to this and other cases she decided. What will happen when the next individual from the Second, Seventh, Eleventh, or Federal Circuit, is nominated for higher office? Unless PACER officials make some accommodation, such as the one we asked for yesterday, researchers and the public will face the daunting task of requesting these materials individually, possibly in person, at excessive expense and inconvenience for everyone involved.
And as I told Common Dreams, the Administrative Office of the Courts (“AOC”) cannot argue that some of these cases have been closed for “13 years.” Some were closed this year! They also cannot argue that “many had not been accessed in several years” because as we demonstrated in a recent video, it can be very difficult and expensive to use PACER. Thus, PACER usage statistics should tell us nothing about the importance of these files. And often, as was the situation with Judge Sotomayor’s historical record, some cases only become particularly interesting years later and for reasons possibly unrelated to the case itself.
Put another way, libraries should hang on to their copy of Plato’s Crito, even if its been a few years since anyone checked it out.
This PACER situation presses home more than ever the importance of having a backup. Many critics of the PACER system were dissatisfied with many aspects of the AOC’s custodianship of America’s judicial record, but no critic ever thought that after so much work to move to online electronic records, we would return to an offline system of access to court records. If we now cannot even count on the courts to keep the electronic records online, even if behind a byzantine paywall, then we really must insist on a backup.
A recent announcement on the federal PACER website indicated that PACER documents from five courts prior to certain dates (pre-2010 for two courts, pre-2012 for one court, etc.) would no longer be available on PACER. The announcement was reported widely by news organizations, including the Washington Post and Ars Technica. The announcement has now been changed to explain, “As a result of these architectural changes, the locally developed legacy case management systems in the five courts listed below are now incompatible with PACER; therefore, the judiciary is no longer able to provide electronic access to the closed cases on those systems.” See a screenshot of the earlier announcement without this explanation:
This morning, Free Law Project signed on to five letters from the non-profit, Public.Resource.Org, headed by Carl Malamud, asking the Chief Judge of each of these five courts to provide us with access to these newly offline documents. The letter proposes that we be provided access in order to conduct privacy research, particularly with respect to the presence of social security numbers in court records, as Public.Resource.Org has done previously in several contexts. In addition we offer to host all the documents in a free public archive, at the Internet Archive, as we do now with RECAP documents. Free Law Project would also plan, ultimately, to incorporate the documents into our CourtListener platform, for even easier full-text searching and public accessibility.
The National Law Journal collected various tweets from lawyers and academics upset over the PACER announcement, and the general reaction to the surprise decision has been dismay, as many of these practitioners and professors rely on these documents for their work. We hope the Chief Judges we have reached out to today will agree to our request and allow us to get these documents back online for the public as soon as possible.
If you only watch one video about using the federal Public Access to Court Electronic Records (PACER) system, make it this video by Free Law Project’s Brian Carver: “Using PACER: What Could Possibly Go Wrong?”
The video provides a demonstration of what a regular member of the public might experience trying to find a copy of a recent newsworthy federal district court opinion on the court’s website and through the federal PACER system. This example was genuinely chosen because Brian himself had heard about a recent newsworthy case out of the District Court for the District of Maryland. In fact, we’re fairly sure that other examples might cast these sites in an even worse light.
Free Law Project believes that Congress should provide adequate funding to the federal courts so that the financial argument for PACER’s fees would be moot and everyone could agree that public access to court records should be free. But even in the absence of that, we conclude from this demonstration that the non-document related fees in PACER for search results and reports that are charged without an interstitial warning of their magnitude are particularly onerous and should be abolished. The courts could take this step immediately, while continuing to charge for actual court documents by the page, and a major obstacle to public access would be removed. Of course the video demo also illustrates a number of usability concerns that we think are best resolved by allowing third parties, such as Free Law Project, to build better search and access tools on top of these documents, but this is only feasible if at least these third parties are given free or very low cost bulk access to the underlying documents. We hope the Administrative Office of the Courts will be open to such options.
We recommend watching the video Full Screen and/or in High Definition so that the slides are readable.
In 2010, we wrote our first scraper. It was a nasty affair that could do nothing more than download PDFs from a webpage — any webpage. Since that point, we’ve come a long way and today we’re extremely excited to announce that the Juriscraper library now supports every state court of last resort. In most states this is the “Supreme Court”, though some states call their highest court the “Court of Appeals” or similar.
This means that no matter what jurisdiction interests you, no matter what area of the law you work in, you can follow the works of your state’s highest court in real time, getting emails or RSS feeds of the latest cases that interest you. And, of course, as new opinions are issued by these courts, we will keep adding them to our system, continuing to build the biggest repository of cases we can. As of today, Juriscraper has collected more than 400,000 opinions, and we expect that number to grow and grow.
On top of this, we also support dozens of intermediate State appellate courts and all of the federal courts of appeals. Check our list to see if a specific court is supported and let us know if there’s one we should add — user requests are often how we prioritize our work! There are already a few administrative bodies and other specialized courts that we plan to add soon.
Completing our appellate court coverage is a milestone we’ve been working towards for a very long time, and we could never have gotten here without lots of support, both financial and social. The following people have contributed code, presentations and time to help get Juriscraper where it is today (let me know if I forgot you!):
- Asadullah Baig
- Brian Carver
- Ben Cassedy
- Andrei Chelaru
- Alan deLevie
- Bo Jin (Krist)
- Deb Linton
- Andrew McConachie
- Matt Meiske
- Taliah B. Mirmalek
- Polya Pelova
- Raymond Yee
- David Zvenyach
And the following organizations have provided generous support for this project:
Thank you all very much for the great work you’ve done and the support you’ve provided. We now have a complete and free system for efficiently gathering opinions from state courts of last resort.
This is truly a momentous day and you might be wondering what’s next for Juriscraper. The first thing we’ve begun doing already is to collect oral argument audio. An important feature of Juriscraper’s design is modularity and after a few minor re-workings, the code is now well-positioned to archive more than just court opinions. We are currently working on expanding Juriscraper’s reach to audio files and we soon expect to build an archive of oral arguments.
Beyond this, Juriscraper’s design allows it to be easily turned to other document types or jurisdictions entirely. We have long thought that we’d also like to archive open access law journal articles and incorporate those into CourtListener. (Get in touch if you’d like to help!) Or you might have some entirely different archiving project in mind, the Juriscraper code can take much of the heavy lifting out of your way and allow you to focus on the unique aspects of the documents you’re hoping to archive. If you put the code to some such alternative use, please get in touch to let us know!