The New Wave of Review

Digital history is riding a “review wave.” In the fall of 2015, the American Historical Association released its new “Guidelines for the Evaluation of Digital Scholarship in History”. In February 2016, the association’s flagship journal, The American Historical Review, published an exchange titled “Reviewing Digital History” that inaugurated its first venture into digital project reviews. In my own field, the Western Historical Quarterly began printing “Born-Digital Reviews” in the fall of 2015. The Journal of American History first started publishing website reviews in 2001, but in September 2013 changed this section to “Digital History Reviews” (the journal also publishes lengthier reviews of digital research projects in its “Metagraph” section). Moving forward, digital historians will increasingly find their work evaluated in some of the discipline’s major print journals.

What’s odd is the degree to which supposedly hidebound print journals are the ones propelling this recent wave of review. After all, it’s not as if digital historians need print journals to review each other’s work. Blogging, Twitter, and other online platforms have stood at the heart of the field for years. We often tout the speed and openness of these platforms compared to the molasses-slow publishing cycles or gated paywalls of print journals. And yet, with some rare exceptions, we don’t use these platforms to engage in substantive or critical evaluation of the work of our peers. New digital history projects are released all the time. If you’re like me, you stick mostly to virtual high-fives: you tweet a link to the project, offer congratulations and commendations, and maybe add it to a syllabus or workshop. Deeper engagement takes place mainly through informal conversations or behind the doors of classrooms – not exactly the sort of public, rigorous intellectual evaluation that drives a field forward. Our colleagues deserve better.

Digital history’s reticence for critical online evaluation stands in contrast to, say, the lively exchange that unfolded in 2015 over Matt Jockers’s Syuzhet package, a method that Jockers developed for identifying literary plot shapes using sentiment analysis. After Jockers first announced Syuzhet, Annie Swafford wrote a pointed critique of the method, and over the course of roughly one month the two literary scholars debated the validity of the method in a series of back-and-forth posts. Other digital humanities scholars weighed in from across the disciplinary spectrum. Whatever your thoughts on Syuzhet, the entire online exchange was a rigorous, substantive, and transparent evaluation of digital scholarship. So why do digital historians seem to prefer virtual high-fives to this kind of deeply evaluative online engagement?

There are a few reasons for the dearth of online reviews and critiques within the field of digital history. For one, there are real drawbacks to online platforms. The immediacy of writing a blog post affords less time for measured reflection or carefully crafted or revised responses than, say, a review in a print journal. Self-published posts also lack editorial oversight. A good journal editor can vet the qualifications of reviewers, help them improve and refine their critiques, and serve as a mediator between reviewers and the people they’re reviewing. Without an editorial presence or a shared platform, online reviews run the risk of operating on unequal playing fields. One historian might be writing from a position of seniority or have a much larger or more vocal online readership than another. It’s also a lot easier for someone like me to tout online exchanges as “lively” or “freewheeling” when I don’t run the risk of getting denigrated or harassed because of my race or gender. Gatekeeping may be a dirty word, but openness isn’t exactly a panacea.

There’s also the broader challenge of subject specialization and expertise. Digital history’s unifying thread is methodological, not thematic. As a historian of the nineteenth-century United States, just how deeply can I engage with, say, Vincent Brown’s spatial history narrative of Jamaica’s 1760-1761 slave revolt? I might be able to discuss its interactive design or the way it uses a spatial framework to circumvent textual silences in the archive. But am I really capable of evaluating Brown’s interpretation of the revolt as a unified, strategic rebellion rather than a series of haphazard insurrections? Even more importantly, am I qualified to evaluate the significance of this claim in terms of how it changes our understanding of Caribbean history? Probably not. This is why it was so encouraging to see deep, thoughtful reviews of Slave Revolt in Jamaica in recent issues of Social Text and The American Historical Review. The reviews were written by Elizabeth Maddock Dillon, Claudio Saunt, and Natalie Zacek, all of whom combine subject expertise with considerable experience in digital humanities projects. Both Social Text and The American Historical Review also gave Vincent Brown the opportunity to respond to these reviews – exactly the type of substantive, scholarly exchange that seems to be in such short supply for digital history projects.

But, again: these exchanges took place in print journals. Consequently, there was a gap of more than two years between the project’s release and the publication of reviews. This lag doesn’t make the exchanges any less valuable, but it hews far more closely to the way the discipline reviews print monographs. In an alternate scenario, the scholarly exchanges between Vincent Brown and his reviewers might have unfolded in a series of online posts over the course of a few months, rather than a few years, after the project’s release. Moving this back-and-forth out from behind the paywalls of Duke University Press and Oxford Journals could have allowed for other scholars to weigh in, much like what happened after the initial posts between Matt Jockers and Annie Swafford during the Great Syuzhet Debates of 2015.

Ultimately, though, I find the format of this new wave of digital history less interesting than its substance. There are a few different ways to evaluate digital history projects, which I would group loosely under pedagogy and public engagement, academic scholarship, and what historian Fred Gibbs terms “data and design criticism.” Most digital history reviews fall under the first category of public engagement and pedagogy. The Journal of American History’s “Digital History Reviews”, for instance, frames its reviews follows: “The goal is to offer a gateway to the best works in digital history and to summarize their strengths and weaknesses with particular attention to their utility for teachers [emphasis added].” As I write in a forthcoming article for Debates in Digital Humanities 2016, this emphasis reflects the field’s particular genealogy and its roots in public history initiatives. Both the reviewers and the projects themselves continue to position digital history in terms of public engagement rather than academic scholarship.

Some reviewers, of course, do try to evaluate digital history projects as works of academic scholarship, akin to a scholarly monograph. This second approach, conducted in large part by field specialists rather than “digital” historians, often compliment the public-facing dimension of a digital project before ultimately critiquing its shortcomings in terms of historiography and interpretation. In a review of Richard S. Dunn’s website Two Plantations, Kirt von Daacke notes that the site’s archival collections “represent the best of digital media.” He ends the review, however, with a standard complaint: “Frustratingly, Two Plantations never indicates its target audience, only hints at interpretation, and ignores historical literature altogether. Its analysis section never really answers the questions it poses, nor does it situate Dunn’s interpretation in the broader scholarship on slavery.” Without explicit interpretive claims to grab onto, trying to evaluate the scholarly contributions of digital projects can feel like trying to scramble up a smooth wall.

The third approach to reviewing digital history focuses on a project’s design, interface, methods, pipelines, and datasets. These kinds of “data and design criticism”, to borrow Fred Gibbs’s formulation, often make a passing appearance in digital history reviews, such as describing a website’s layout or critiquing the usability of certain features. Few reviewers, however, put it at the center of their evaluations. One recent exception is Joshua Sternfeld’s lengthy review of Digital Harlem in the American Historical Review. In it, Sternfeld offers a prolonged description of the site’s digital infrastructure and features before launching a blistering critique of the project. He questions the representativeness of the project’s archive, criticizes its method of data entry and sampling, and ultimately describes Digital Harlem as “subverting the provenance of the source data.” For his part, the project’s co-creator Stephen Robertson returns serve with an equally blistering counter to Sternfeld’s review. Robertson argues that Sternfeld “misrepresents the design and content of the site” and “only fitfully engages with the spatial orientation of Digital Harlem.” Whatever side of the exchange you come down on, the back-and-forth illustrates how questions of data and design can stand at the center of digital history reviews.

I find myself frustrated by all three kinds of digital history reviews. First, I appreciate the value of evaluating projects in terms of pedagogy and public engagement. But the preponderance of this first kind of review reinforces the (false) notion that digital history does not, in fact, add substantive new academic knowledge to the field. This notion feeds into the second kind of review, one that takes digital projects to task for shortcomings surrounding academic argument and interpretation. I’m actually sympathetic to this kind of review, but they often mistakenly evaluate digital projects in terms of what the reviewer wants them to be (a traditional academic monograph) rather than what they are (an online exhibit, research tool, pedagogical resource, etc.). Finally, I worry that the third strand of digital history review – “data and design criticism” – will further exacerbate what I see as the field’s problematic privileging of method over argument. Data collection, interactivity, visualization and design – all these features should be part of the review process, but they need to be grounded in a frank evaluation of whether and how they lead to new knowledge or interpretations about the past.

Does a digital history project fundamentally change how we understand a particular topic? How does it fit within the existing literature about this subject? What are a project’s methodological strengths or flaws specifically in relation to the project’s historical contributions? As a field, we need to dig deeper into these kinds of questions when we evaluate each other’s work. A call to burrow into the scholarly weeds of historiography and interpretive nuance puts me at odds with one of digital history’s core tenants: cultivating a broad audience. The general public doesn’t necessarily care how a particular scholarly brick fits within the grand edifice of historical knowledge. Neither, for that matter, do literary critics, media theorists, philosophers, or the rest of our colleagues in the broader digital humanities community. Hell, a lot of historians don’t want to wade too deeply into debates and arguments outside their specialization. But that doesn’t mean we shouldn’t do it.

I’m calling for digital historians to seize and shape the current wave of review. Regardless of whether we do so in blogs or print journals, we need to more substantively evaluate the work of our peers. We need to evaluate and critique each other’s work not just in terms of public engagement and pedagogy or data and design, but in terms of new historical knowledge, insight, and interpretations that these projects contribute to the field. In the next few days I’m going to follow my own advice and post a review of a digital project related to my particular sub-field of nineteenth-century U.S. history. Readers who aren’t in this sub-field might find it tedious, but my hope is that it will spark similar evaluations of other digital history projects. Stay tuned…

11 thoughts on “The New Wave of Review

  1. Cameron

    I really appreciated the broader view and challenge you offer in this post. I have a couple of thoughts.

    I’d make a connection between the AHA guidelines and the expanded number of journals reviewing digital history. A review in a journal, rather than as a post on an individual scholar’s blog, will have some value in the evaluation of a digital project as part of the tenure and promotion process. It will literally appear alongside the reviews of monographs that are an accepted part of that process. In that sense, a journal review gives an imprimatur that a digital project is scholarship. The delay before a journal review appears is an issue regardless of what form of scholarship is being reviewed – another way in which journals remain wedded to the restrictions of print long after most of their readers access their contents online that I’ve complained about before . I’ve heard talk for a while about several journals moving to publishing reviews online as they come in. That can’t come soon enough. And for what its worth, the American Historical Review is providing authors with links that provide free access to their reviews, which mean those at least are not completely gated.

    Also, I think there is more going on my exchange with Josh Sternfeld than you note here. Much of Josh’s review – and consequently my response — is concerned with the design and data, but he also takes issue with the historical contribution of the site, and frames much of the review in those terms. On this point, Sternfeld read the site very narrowly within the terms of the existing historiography. He wanted it to speak explicitly to existing debates about law enforcement and community organizations. The site, and the larger project of which it is part, explicitly sought to escape those limited frames, and focus instead on everyday life – to present a view of life in Harlem that went beyond the race leaders, writers, artists, and musicians who dominate the existing historiography. Sternfeld dismissed this picture of everyday life as a “distorting mosaic of unrelated experiences.” I countered by pointing to the gap between the effort to understand racial violence by looking at clashes between the community and the police, which Sternfeld held up as what Digital Harlem should have focused on, and our article “Harlem in Black and White,” which explores not only clashes between police, but a diverse range of interracial encounters in businesses, schools, hospitals, public transport, on the streets, and in various sites of commercialized leisure. Our argument draws on the rich and diverse picture offered by the black press, which Matt Delmont’s new project Black Quotidian is seeking to highlight, not to the narrow slice that dominates older writings on Harlem.

    So in the case of the review of Digital Harlem, there is method and argument – its a very different review than the one in the Journal of American History.

    1. Thanks for the thoughtful comments, Stephen.

      That’s a great point about the credentialing dimension of journal reviews and part of what makes the AHA guidelines important. It’s just interesting to me that digital historians have in some ways embraced a radical shift away from the traditional academic process (ex. eschewing monographs, embracing alt-ac career paths, large-scale project collaboration, blogging/Twitter/wikis, etc.) but that this shift has been much slower to extend to evaluation and review of each other’s work. That is clearly starting to change, but it seems to be taking place through quite traditional medium of the print journal.

      Thank you for expanding on the AHR exchange between yourself and Joshua Sternfeld. You’re absolutely correct that the debate merges “data and design” criticism with historiographical issues. This is, in fact, exactly the kind of blending that I’d love to see more of, and I didn’t mean to conflate it with my concerns over the risks over foregrounding method over argument. Having said that, after re-reading the review and your response, I think the bulk of the conversation hinges on a disagreement over the nature of the site and how it should be evaluated. As you point out at the end of your response: “While Sternfeld points to some of the strengths and weaknesses of Digital Harlem, ultimately his assessment is skewed because he does not recognize the category of digital scholarship into which the site falls…What historians and other humanities scholars gain from being able to access Digital Harlem is an opportunity to think about digital mapping as a research method, not as a means of making or illustrating arguments, but as a way to combine and explore heterogeneous sources.”

      This is where I think my own post runs up against a thorny reality: evaluating a digital history project is in many ways a lot harder than evaluating a book. Will Thomas’s typology certainly helps in that regard, but even those divisions can lead to problems. If Sternfeld evaluated Digital Harlem as a “thematic research collection,” should he have primarily evaluated it on that basis alone, or should he have reviewed it primarily in terms of the peer-reviewed articles and the monograph that came out of it? In the case of the former, this makes it harder to evaluate historical conclusions and data and design since it is primarily meant to be a tool. In the case of the latter, the connections running between the digital project and the peer-reviewed scholarship can be difficult to suss out from the perspective of a reviewer.


  2. I’ve been following this developing debate through teaching my Approaching Digital History seminar and I am struck by two implications of Stephen Robertson’s argument about which I am curious to hear responses:

    First, doesn’t the fact that the articulation of the full interpretive value of Digital Harlem took place in a more traditional peer-reviewed article suggest that the digital project itself is less interpretive historical publication than research tool. There’s nothing wrong with that, but it does leave open the question of how a digital publication itself can express historical argument and interpretation. If it is really more a research tool then we are indeed in Cameron’s “perpetual sunrise” of method. The arguments will have to take place around debates concerning method since that what research tools are, yes? Or are we deciding that the digital context troubles these typologies between publication and research tool? And then, are we fine living with that ambiguity? Or do we, must we, sort it out a bit more?

    This leads to my second question. If we are fine with that ambiguity, then why should we, in a normative sense, distinguish between “A review in a journal, rather than as a post on an individual scholar’s blog.” Yes, I get it: digital history must secure more professional legitimacy within the existing structures of the profession, and a review in a flagship journal secures this, but then doesn’t this undercut the very value of something like Digital Harlem, which itself purposefully shifts away to new territory from the rules and practices of those flagship journals. If projects themselves, or research tools, or whatever in-between things we are creating digitally occupy new ambiguous spaces between the old and new forms of history, why shouldn’t reviews also?

    I really mean these questions sincerely although I realize they sound a bit rhetorical in nature.


  3. It seems to me that another issue here is the nature of print journals and how they inherently do not mesh well with digital media. Journals are limiting in so many ways, particularly when it comes to digital projects–you can’t follow the link to the project, you can’t see the aesthetic/design choices that are (possibly) being critiqued, and you can’t experience the interactivity that is (hopefully) provided by the project. Additionally, in a perhaps ironic twist, the very accessibility that draws many of us to digital history is taken away by the esoteric nature of print journals. Why shouldn’t print journals start meeting DH halfway, instead of failing to integrate it into an incompatible medium? I would love to see journals offer digitally published reviews. (And that isn’t to say that individually published reviews on scholars’ blogs shouldn’t be considered legitimate–I absolutely think they are. But I also think that there is a more ideal middle ground where DH projects, reviews, and journals can–and should–meet.)

Leave a Reply

Your email address will not be published. Required fields are marked *