Latest Updates: Our Blog

How we made DocumentCloud note embeds responsive

Posted
Sep 25th, 2014

Tags
Documents

Author
Ted Han

By Tom Meagher, data editor at The Marshall Project,
Emily Yount, interaction designer at The Washington Post,
Matt DeLong, national digital projects editor at The Washington Post
and Ted Han, lead developer at IRE/DocumentCloud

On Aug. 3, The Marshall Project, a new nonprofit journalism organization focused on criminal justice issues, published an investigation in partnership with The Washington Post that revealed new evidence raising doubts about a high-profile Texas execution.

TOM: Our reporter, Maurice Possley, began working on this story months before most of the rest of our newsroom at the Marshall Project was even hired. By the time we were able to start helping, the story was mostly reported, so we dove into the documents to bring ourselves up to speed.

The case against Cameron Todd Willingham — who was executed in Texas for the murder of his three daughters — had been written about extensively over the last 22 years, but a lot of new information was uncovered, and it was all in the documents. We knew we wanted to be able to explore and highlight the correspondence that cast this case in an entirely new light. DocumentCloud was clearly the answer.

In the course of his reporting, Possley, who has covered this case for more than a decade, was given access to copies of dozens of primary source documents that tell the backstory of Cameron Todd Willingham and the informer who helped convict him. In filing its grievance with the State Bar of Texas against the former prosecutor in the case, the Innocence Project had acquired these documents and assembled them into a series of appendices. They gave us eight PDF files that added up to nearly 400 pages. We used DocumentCloud to stitch them all back together into one large file.

We then combed through the appendices and dozens of other records of court testimony and correspondence. As we saw the various typefaces and handwriting styles that made up the key passages, we knew we wanted to use DocumentCloud notes to present excerpts directly in the story.

Matt: I started working on the story in earnest a couple of weeks before it published. We were very excited about having so many primary-source documents to enrich the narrative. The Post has been using DocumentCloud for years, but we’ve long been frustrated by one of its biggest limitations: it isn’t mobile-friendly. This isn’t really DocumentCloud’s fault; these scanned documents are a set size, so when you scale them down, at some point words will become too small to read.

We had seen how the New York Times addressed the problem, by putting up the text and linking to the original document in DocumentCloud. That’s totally logical and fine if the words are all that you care about, but in this case we have official letters and handwritten notes between the characters in the story. The pages themselves are interesting, and many readers will want to see them with their own eyes.

We decided at the outset that however we ended up displaying the documents we included in the story, they had to be responsive. But this meant we’d have to come up with our own hack. Emily and I had both been thinking about this problem individually for a while, and we had some time to work on it, so we decided to try to figure out a solution that we could use in this project.

Emily: At the time of publication, DocumentCloud’s note embed code already resized and repositioned the note based on the width of the DC-note-container div, so I knew we only needed to solve for when the note is wider than the note embed and the right side of the note is cut off (see image below).

image01

To solve this problem, when the embed first loads, the code stored the coordinates, width and height of the note relative to an image of the page of the annotated document. When the page loads, the browser resizes or the orientation of your device changes, javascript media queries (matchMedia) detect whether the note is wider than the embed and then resizes and repositions the document image.

The original coordinates, width and height allow us to determine how wide the note is in relation to the document image and resize the document just enough to make the note 100% of the embed, instead of the document image 100% of the embed. This helps with readability by making the text as large as it can be. At times, depending on the width of the note and the size of the text, there will still be readability issues, so cropping the annotations carefully and testing to make sure they are readable is really important.

Here’s an example from the Willingham story of a responsive note on an iPhone 5:

image00

Ted: We were thrilled when Ben Chartoff (OpenNews fellow at the Washington Post) reached out to put Emily in touch with us.

We believe deeply in DocumentCloud as an open source project as well as the service to which journalists post documents relevant to the public interest. Emily and Matt’s motivation to extend the behavior which DocumentCloud already provides and to share their code back is exactly the kind of effort we love to see and encourage.

Technology in the world of news is a means toward the end of better reporting. Especially in competitive industries like ours, an open source ethos around the tools we all share is an avenue for us work together to improve the state of all reporting. Anyone who solves an issue for their own needs can help to solve that issue for everyone.

In that spirit we were excited to incorporate Emily’s code into our own. To do so, we spun our note code off into its own repository to make it easier for anyone to contribute (you can find the code on Github as documentcloud-notes). Then with the Washington Post’s & Marshall Project’s stories as a basis we began incorporating the changes. Ultimately, we ended up rewriting much of Emily’s code in the process, but what she had written served as the design criteria to anchor the code we wrote.

Our responsive notes code is already live on DocumentCloud now, and journalists needn’t take any additional steps to use it. Any embedded note from DocumentCloud will now behave responsively.

Congrats to our users, documen…

Posted
Sep 28th, 2012

Tags
Twitter

Author
documentcloud

Congrats to our users, documents on DocumentCloud have been viewed 60 million times. /TH

Nice use of embedded annotatio…

Posted
Sep 28th, 2012

Tags
Twitter

Author
documentcloud

Nice use of embedded annotations to piece together a timeline of the death of a Guantánamo detainee: http://t.co/UZvZISIU by @ProPublica /ja

Trying to integrate the Docume…

Posted
Sep 20th, 2012

Tags
Twitter

Author
documentcloud

Trying to integrate the DocumentViewer with an app in your newsroom? Useful tip: Here’s the “public” API: https://t.co/axjlpe2g /ja

The DocumentCloud API now supp…

Posted
Sep 10th, 2012

Tags
Twitter

Author
documentcloud

The DocumentCloud API now supports CORS responses for public queries. Feel free to use it if JSONP isn’t your cup of tea… /ja

Interested in learning about D…

Posted
Sep 6th, 2012

Tags
Twitter

Author
documentcloud

Interested in learning about DocumentCloud’s story? The @knightfdn is doing a review of the 2009 #newschallenge http://t.co/nmylmn6U

Do you depend on DocumentCloud…

Posted
Sep 5th, 2012

Tags
Twitter

Author
documentcloud

Do you depend on DocumentCloud’s APIs? Sign up to our API announcements list: https://t.co/EDHWfQYn /th

You may have noticed we’re ext…

Posted
Aug 30th, 2012

Tags
Twitter

Author
documentcloud

You may have noticed we’re extra busy today, documents might take a little while to process through. Thanks for bearing with us. /th

Interested in DocumentCloud fo…

Posted
Aug 30th, 2012

Tags
Twitter

Author
documentcloud

Interested in DocumentCloud for your language(s)? Make sure to let us know: https://t.co/SQ59mGLx /th

Thanks for bearing with us, we…

Posted
Aug 23rd, 2012

Tags
Twitter

Author
documentcloud

Thanks for bearing with us, we’re keeping an eye on things, but service is back to normal. /TH

Hey folks, apologies for the d…

Posted
Aug 23rd, 2012

Tags
Twitter

Author
documentcloud

Hey folks, apologies for the difficulties, we’re experiencing some uniquely heavy traffic today. We’ll update you when we can. /TH

DocumentCloud document embeds …

Posted
Aug 15th, 2012

Tags
Twitter

Author
documentcloud

DocumentCloud document embeds just got a little bit smarter about SSL. They’ll now serve via HTTPS on secure pages. /ja

We have a lot of interesting p…

Posted
Jul 16th, 2012

Tags
Twitter

Author
documentcloud

We have a lot of interesting projects in the work and are looking for others to join us! Check out http://t.co/rsgd25Xp /th

DocumentCloud Seeks Developer to Work on Open Source Software Platform

Posted
Jul 16th, 2012

Tags
People

Author
Ted Han

We have a lot of projects going on at DocumentCloud and to serve those goals we’re looking for others to join us! For those who may be unfamiliar with our project, we’ve included the full details below.

DocumentCloud is a web based platform allowing journalists to upload, analyze, annotate, and publish primary source documents. We want give journalists the tools to show their audience their source material, not just tell them about it. In addition to the newsrooms worldwide who use DocumentCloud, our open source software projects, such as Backbone.js, Underscore.js, Docsplit, and Jammit, are relied upon by companies such as LinkedIn, Walmart, Foursquare and more. DocumentCloud is run by Investigative Reporters & Editors.

What DocumentCloud is building

  • DocumentCloud is growing fast, and we’re looking to accelerate that pace by expanding our tools into other languages beyond English. In the next year we’ll adapt our platform to accommodate multi-language OCR, search indexing, and entity extraction tools.
  • DocumentCloud always looks for new ways to present documents and engage readers. We are extending DocumentCloud’s document viewer and annotation tools so that readers can make their own comments and notes on documents.

DocumentCloud is looking for someone with a combination of the following skills

Experience with Ruby and JavaScript; API driven web applications; working on and fostering FOSS; user-centered products; Experience the JVM toolchain; linux administration on Platform as a Service providers such as AWS.

Things we like and hope you like too!

Literate programming; Extracting libraries from app code; Polyglot programming; Web standards; Journalists; Natural Language Processing

Practical Details

Investigative Reporters & Editors is based in Columbia, Missouri, on the University of Missouri’s campus. DocumentCloud is comfortable operating with a distributed team.

You can email us at jobs@documentcloud.org

There are now over 5 million p…

Posted
Jul 12th, 2012

Tags
Twitter

Author
documentcloud

There are now over 5 million pages of primary source documents in DocumentCloud. Keep ‘em coming… /ja

We’re extra popular today, so …

Posted
Jun 25th, 2012

Tags
Twitter

Author
documentcloud

We’re extra popular today, so document processing times may be a little longer than usual. Thanks for bearing with us. /th

Navis DocumentCloud plugin for WordPress

Posted
Jun 22nd, 2012

Tags
Code , ,

Author
Samantha Sunne

Chris Amico had a problem he wanted to solve: when bloggers posted a document from DocumentCloud into WordPress, switching between the Visual and HTML modes destroyed the embed code. So he created Navis DocumentCloud, a plugin designed to protect the javascript from the Visual Editor.

“That crux of it is that it makes it safe to drop a document into WordPress and be certain that it won’t be broken,” said Amico, of the plugin he authored as an application developer for NPR’s StateImpact project.

Users can post their document using a DocumentCloud button on the post’s toolbar in Visual mode, or a shortcode in HTML mode.

documentcloud button

The plugin also allows users to configure the width and height of the document viewer in an administrative panel in the Settings menu. The “Full-width” option is designed to make the document viewer as wide as the post content.

“We wanted to give reporters the ability to make a post that is basically just the document,” Amico said.

wp_plugin2

The plugin is available on GitHub and the Project Argo site. StateImpact is a spinoff project of Project Argo, which are both run by NPR. Amico said he wanted to help bloggers use DocumentCloud because he used it as a reporter for PBS NewsHour. “At some point, my plan is to put it into WordPress’s plugin directory,” Amico said.


Samantha Sunne volunteers with DocumentCloud at its hub in Columbia, Missouri. She studies investigative and multimedia reporting at the University of Missouri.

In our API, the JSON for docum…

Posted
Jun 13th, 2012

Tags
Twitter

Author
documentcloud

In our API, the JSON for document sections and annotations is now sorted — by page number and the location of the note on the page. /ja

State of the Backbone

Posted
Jun 4th, 2012

Tags
Code

Author
Ted Han

It may surprise some to find that Backbone.js was released just over a year and a half ago. In that time, Backbone has gained a remarkable reach, and one that stretched beyond our anticipations. As just one example (and there are many examples to be found in the Backbone documentation), last week brought us to BackboneConf in Boston where several hundred developers gathered to discuss Javascript application development.

Jeremy Ashkenas, engineer emeritus of DocumentCloud, opened the conference with the first State of the Backbone keynote. We’ve recorded it for those who weren’t able to join us at BackboneConf.

When Documents Are Challenged:…

Posted
Apr 5th, 2012

Tags
Twitter

Author
documentcloud

When Documents Are Challenged: http://t.co/fKTTxFyy

When Documents Are Challenged

Posted
Apr 5th, 2012

Tags
Documents,IdeaLab

Author
Mark Horvit

Last week, DocumentCloud received a complaint seeking the removal of a collection of emails posted by journalists with the Australian Financial Review. The emails involved a company called NDS, which hired a law firm to try and have the documents pulled from public view. This kind of thing is rare, but it happens. This case in particular has a couple of wrinkles that make it unusual, and it presents a good opportunity to remind all of our members that DocumentCloud has policies and options in place that allow you to keep all documents processed through our service available to the public for as long as you desire.

I’ll detail those below. But first, a little context.

DocumentCloud was created …

Back up and serving fresh docu…

Posted
Mar 31st, 2012

Tags
Twitter

Author
documentcloud

Back up and serving fresh documents. /th

Hi folks, we’re in the midst o…

Posted
Mar 31st, 2012

Tags
Twitter

Author
documentcloud

Hi folks, we’re in the midst of an interruption. More news shortly /th

Thanks to a suggestion from Ri…

Posted
Mar 26th, 2012

Tags
Twitter

Author
documentcloud

Thanks to a suggestion from Richard Stallman, DocumentCloud embed codes now include “noscript” links to the original PDF and plain text. /ja

The site is back to normal, bu…

Posted
Mar 22nd, 2012

Tags
Twitter

Author
documentcloud

The site is back to normal, but we are experiencing heavy load at the moment. Thanks for bearing with us /th