Expert/Crowdsourcing for the Linked Modernisms Project Scholarly and Research Communication Stephen Ross, Alex Christie, & Jentery Sayers volume 5/ issue 4 / 2014 University of Victoria, Modernist Versions Project Introduction ough “Building Partnerships to Transform Scholarly Publishing” has never officially been part of the mandate of either the Modernist Versions Project (MVP) or Linked Modernisms (LiMo), upon reflection, it has turned out to be among their key functions. Advancing new ways of doing scholarship within the field of modernist studies has driven us to explore novel approaches to publishing, and demonstrated the utter necessity of partnerships to make those approaches effective. In what follows we will provide a quick introduction to the Modernist Versions Project1 and some of the tools we use, consider what sort of inquiries appear at scale, and give an overview of the Linked Modernisms Project. We will attend in particular to how this suite of projects has led us to rethink traditional scholarly publishing practices, and how partnerships have become indispensable to mobilizing our research. CCSP Press Scholarly and Research Communication Volume 5, Issue 4, Article ID 0401186, 11 pages Journal URL: www.src-online.ca Received July 31, 2014, Accepted August 25, 2014, Published December 16, 2014 Stephen Ross is Associate Professor of English and Cultural, Social, and Political ought at the University of Victoria. He is author of Conrad and Empire (2004); editor of Modernism and eory (2009); co-editor of e Modernist World (2015), Dorothy Richardson’s Pointed Roofs, and Dorothy Richardson’s e Tunnel; and General Editor of the Routledge Encyclopedia of Modernism. He is Principal Investigator of both the Modernist Versions Project and Linked Modernisms. Email: saross@uvic.ca Alex Christie is a doctoral research assistant with Implementing New Knowledge Networks (INKE) and the Modernist Versions Project (MVP). He works across the Electronic Textual Cultures Lab (ECTL) and the Maker Lab in the Humanities at the University of Victoria, using new and built media to model the procedural nature of modernist texts. Email: achris@uvic.ca Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. © 2014 Stephen Ross, Alex Christie, & Jentery Sayers. is Open Access article is distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons .org/licenses/by-nc-nd/2.5/ca), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited. 1 Scholarly and Research Communication volume 5/ issue 4 / 2014 Jentery Sayers is Assistant Professor of English and Director of the Maker Lab in the Humanities at the University of Victoria. His work has appeared in American Literature; e-Media Studies; Digital Studies / Le champ numerique; e International Journal of Learning and Media; Kairos; Rhetoric, Technology, and Pedagogy; e Information Society; and Computational Culture, among others. Email: jentery@uvic.ca The modernist versions project e MVP sought to help transform scholarly publishing through partnerships from the start. Funded by a Partnership Development Grant through the Social Sciences and Humanities Research Council (SSHRC), it has played a role in the increasing practice of putting scholarly edited and curated materials online as open access digital editions. Our most prominent publication of this nature is the first edition of James Joyce’s Ulysses, which we published over the course of a year from June of 2012–June of 2013 (Joyce, 2012). We supplemented the presentation of PDF page images and OCRcorrected text with a series of Twitter chats led by international Joyce scholars and a series of video podcasts of lectures by prominent Joyceans2. From the outset, the mindset was that of leveraging the partnerships upon which the MVP was founded – in this case the Modernist Journals Project, Fairleigh Dickinson University, the Electronic Textual Cultures Laboratory (ETCL), and Implementing New Knowledge Environments (INKE) – to help rethink what it would mean to present the first edition of Ulysses in a scholarly setting that was equally compatible with a public humanities ethos. e production of digital editions was in the first instance a matter of necessity for the MVP as it sought to pursue its more focused goal of digitally versioning texts. In the spirit of genetic criticism’s pursuit of the avant-texte3, the MVP again ventured into unfamiliar scholarly publishing terrain. As Stephen Ross (2012) wrote early on: “e aim is to read the gap between one version and the next, following Derrida’s imperative to attend to difference itself as the source of meaning rather than presuming that it is a merely negative product of the non-coincidence of two entities: text A and text B” (n.p.). We sought to publish the difference between two existing witnesses, and to say what that difference meant. In one sense, this is just literary criticism as it has long been practiced. In another sense, it nuances the notion of the avant-texte significantly by showing how differential texts produce “ghost texts” that are perhaps even more significant than those tied to material artifacts. Pursuit of this admittedly evasive objective required that the MVP build its partnerships with Editing Modernism in Canada, Islandora and DiscoveryGarden, and Juxta (through NINES). Doing so allowed us to draw upon enormous expertise and deep experience with some of the same issues we were just embarking on. It also gave us access to state-of-the-art digital collation and versioning applications with which to experiment as we developed various workflows for generating new critical insights via digital collation of texts. We experimented with conceptual versioning and various levels of Text Encoding Initiative (TEI) encoding both before and aer collating texts (e.g., Tanigawa, 2013a, 2013b, 2014). Over time, though, our focus shied away from textual variation per se and onto other ways we might conceive of “modernist versions.” Most recently, this shi has taken two decisive forms. First, Jana Millar Usiskin, working through our partnership with Compute Canada, has built a repository of modernist novels for topic modelling and other statistical modes of inquiry, as well as producing network visualizations of some key modernist texts (see Millar Usiskin & Moa, 2013, 2014a, 2014b). Adapting Matt Jockers’ (2013a, 2013b) work with Victorian novels to the different demands of modernist texts, Millar Usiskin and Moa have refined Jockers’ recommended workflow to conform more accurately to modernist novels’ preoccupation with verbs, sentiment, and time. Second, Katie Tanigawa, Alexander Christie, and Adèle Barclay, working closely with Jentery Sayers, have developed what we are calling Z-Axis Scholarship. 2 Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. is approach involves geo-referencing (with TEI) modernist novels and then using the resulting data to warp historical maps of cities such as Paris and Dublin (see Barclay, 2014; Christie, Ross, Sayers, Tanigawa, & the INKE-MVP Research Team, 2014). e results, though still preliminary, have been enticing, suggesting some new lines of inquiry regarding class, religion, and sexuality in Ulysses, Nightwood by Djuna Barnes, and Quartet by Jean Rhys. Scholarly and Research Communication volume Finally, working with these tools and partners helped us discover what sorts of intervention we needed that we would have to build for ourselves, provoking an editorial and critical enterprise like the MVP to take on the unwonted task of designing and building soware. Enter Susan Schreibman and Tanya Clement’s Versioning Machine (Figure 1). Having identified what worked for us – and what did not – through our experiments with a number of other tools (outlined in Huculak & the MVP Research Team, 2011), and with the invaluable creative and technical contributions of MVP team member Daniel Carter, the Tools Team of the MVP has now produced Versioning Machine 5.0 (Carter, 2014), which is due for release in beta in the fall of 2014. e new Versioning Machine presents an intuitive user interface and plentiful features for versioning work in both text and oral formats, as well as across formats. Figure 1: Screen capture of the Versioning Machine 5.0 Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. 3 5/ issue 4 / 2014 Scholarly and Research Communication volume 5/ issue 4 / 2014 Looking ahead, the MVP will continue all three lines of work as it develops existing partnerships and evolves new ones to continue exploring innovative approaches to scholarly publishing. Inquiries appearing at scale ough the MVP began with a narrow interest in how multiple witnesses of modernist texts varied among themselves, it has quickly embraced Alan Liu’s (2012) claim that “Scale is a new horizon of intellectual inquiry” and his follow-up question: “What kinds of humanistic phenomena appear only at scale?” (p. 21). Until very recently, the MVP lacked a sufficiently large body of clean or reliable texts to work with. Largely for reasons of copyright, scholars have been slow to digitize modernist texts. As we began building our own repositories, then, we also began exploring the larger possibilities of Linked Open Data. is precipitated a shi in our objects of inquiry, away from individual texts or avant-textes and toward the much larger world of modernist cultural production. To ensure sufficient complexity, we also wanted to consider how that world itself is produced as a field of study. We began to consider how we could productively study both individual texts and the much larger field of modernist cultural production as networks that could usefully be visualized and explored with digital methods. By eavesdropping on the INKE team’s developing paper on “e Emergence of Linked Data in the Humanities” (Simpson et al, n.d.) and investigating projects such as linkedjazz.org and europeana.eu, we quickly realized the potential afforded by Resource Description Framework (RDF) and other means of faceted searching of linked databases. When we came across the work of Colin Allen, Mathias Niepert, and Cameron Buckner on the Indiana Philosophy Ontology (Allen, Niepert, & Buckner, 2013), we knew we had found the ideal precedent and model for our own work. e Indiana Philosophy Ontology (InPhO) uses the open access Stanford Encyclopedia of Philosophy (SEP) as its database, and has devised various approaches to making the material in it searchable and discoverable in sophisticated ways. is has involved constructing an information ontology for the terms and entities in the SEP, and devising methods of dealing with dynamic ontologies as a way of responding to an ever-growing and shiing body of knowledge. at is, as the SEP is updated and expanded, the ontology that allows for its navigation must also expand and change, retroactively establishing connections with terms that derive from different moments in the SEP’s production. Additionally, the InPhO researchers have been able to use natural language processing (NLP) and a combination of machine learning with human domain expertise checking to automate the process of updating the dynamic ontology as much as possible. ese approaches, combining intensive processing capacity with light-touch human intervention, have been extremely productive. ey have allowed experts to correct or refine the machine findings, and they have led experts to new insights based on how machine inferences connect terms. As Buckner, Niepert, and Allen (2010) put it, such approaches are crucial to the ongoing viability of encyclopedic enterprises in the twenty-first century: “Scholars and students don’t just need the reference works – they also need the means to search and navigate them effectively. To preserve the utility of encyclopedias as they grow, we must also improve our ability to represent their contents in meaningful ways accessible to novice and expert alike” (n.p.). 4 Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. Perhaps even more to the point of the concerns presented here, InPhO have pioneered a hybrid model of scholarly publishing that we believe to be exemplary for digital humanities projects. ey have taken a commercially published resource (SEP) and made it open access through academic investment. ey then partnered with Indiana University to support the ongoing development and refinement of InPhO’s ontology and faceted browsing capabilities. Central to this project have been RDF, NLP, and inferential algorithms of precisely the sort required by Linked Modernisms as it develops a similar project in relation to the Routledge Encyclopedia of Modernism (REM). Scholarly and Research Communication volume Linked modernisms Linked Modernisms emerged from thinking about how we could best leverage RDF and faceted searching for big data inquiries into modernist cultural production. Linked Modernisms aims to allow researchers to discover (perhaps serendipitously) relationships among versions of modernism; visualize and plumb those versions; refine queries across subjects, objects, and predicates; and develop nuanced understandings across disciplinary, artistic, temporal, linguistic, or geographical articulations of modernism. In effect, LiMo seeks to be able to ask questions of and characterize relationships across all conceivable entities associated with modernist cultural production wherever, whenever, and however they occur. It is a tall order. To fill it, LiMo envisions a four-tier information matrix that also describes a new approach to scholarly publishing through a range of partnerships. First, information included in the REM itself; second, metadata about the terms in the REM provided to us by domain experts; third, metadata gathered through NLP and inferential algorithms; and fourth, metadata about those who provided both the initial data (the entries) and the first round of metadata. ough no one would claim such a matrix to be complete, we will have at our disposal both the most expansive and inclusive body of knowledge about modernism and the production of the field of modernist studies as well as a sophisticated means of analyzing that data. e resulting information matrix emphasizes discoverability, allowing for serendipitous encounters with the datasets, queries driven by user curiosity, and user-generated itineraries through the datasets. It presents an innovative approach to scholarly publishing inasmuch as it fuses a proprietary dataset published by a commercial press with an open access metadataset that is partially derived from the processing of the proprietary materials. Both datasets are analyzed by grant-funded research projects in Canada and the USA, with the results appearing in both traditional peer-reviewed and non-traditional open access fora. e whole can be achieved only through innovative partnerships among academic researchers, institutional units, and commercial interests. Tier I of LiMo’s information matrix is the REM itself, produced under contract for Routledge by General Editor Stephen Ross with the understanding that while copyright over the contents remains with the press, they may be used by Ross for nonconsumptive or non-expressive purposes (a factor that will become especially significant in using NLP to infer relationships among terms in the data and metadata). Moreover, though Routledge owns the copyright to the content, they do not own any of the metadata we harvest by other means. We have agreed to share our metadata with Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. 5 5/ issue 4 / 2014 Scholarly and Research Communication volume 5/ issue Routledge in the spirit of open access, but have insisted that the metadata must remain open access. 4 / 2014 With eight Subject Editors, 60 Editorial Board Members, and approximately 1,200 contributors, the REM aims to include everything that may be called modern(ist) across the arts and around the world, with little concern for strict periodization or narrow aesthetic criteria. It features nearly 3,000 terms and runs to approximately 1.5 million words, all of them written by domain experts. It constitutes a new venture for Routledge itself, which has taken on the task of producing a large online resource of curated scholarly material rather than contracting it out (as with the Routledge Encyclopedia of Philosophy). When the REM launches in 2016, it will be the most expansive source of information on modernism in the world. It is the bedrock of Linked Modernisms. Tier II of Linked Modernisms gathers metadata directly from domain experts through a survey. Each term in the REM has abundant implicit metadata; it carries with it an information ecology that may or may not be embedded in the entry itself, and which affords a rich source of data for Linked Modernisms’ Linked Open Data (LOD) program. Most of this data, however, is not recorded in the entries themselves, and was not at the time routinely collected by Routledge4. e first order of business for Linked Modernisms was to determine what metadata we would ideally like to gather (all of it), versus how much we could reasonably expect our contributors to provide. is entailed taking the first steps toward building an ontology that would encompass all aspects of a wide range of entities, so we established a core framework with fundamental classes and sub-classes of entity. In this respect, LiMo faced a challenge larger than that presented to either the designers of linkedjazz.org or InPhO: where they focus only on one subject or discipline, LiMo had to be able to accommodate a range of disciplines, with a range of entities – including people, movements, techniques, works and artifacts, locations, languages, and events. Our framework seeks to capture any term related to modernism in dance, film, literature, theatre, music, architecture, design, visual arts, and intellectual currents – anywhere around the world. As we transform this core framework into an official ontology, we will use existing ontologies as much as possible – only devising our own features where necessary – to preserve maximum interoperability with other projects and to ensure maximum usefulness in the LOD environment. We plan to publish the ontology in early 2015 as both a useful resource and a scholarly contribution to taxonomizing the field of global modernism. With the framework in place, we built a survey on SurveyMonkey5 designed to harvest the relevant metadata about the entries in the REM from the contributors themselves. at is, we are crowdsourcing information from domain experts, achieving a level of accuracy and reliability not normally associated with crowdsourcing, and a breadth of information not normally associated with reliance upon domain experts. e survey is currently in circulation with over 1,200 contributors to the REM. e results of the survey, distributed using SurveyMonkey, are human-readable and allow us to gain an impressionistic overview of the metadata breakdown. ey are exportable in spreadsheets, CSV, XML, HTML, summary form, or question by question. We have had the most success in exporting the results in CSV and using a free online 6 Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. converter to produce JSON objects from the results. e JSON objects must be cleaned up either manually or by running a simple Python script to automate, before they can be loaded into a triple-store such as Apache Jena or Neo4J, which can then be queried using SPARQL. When thus exported into RDF, the survey results – the metadata for the REM – assume a machine-readable format that enables more complex analysis, including options for faceted searching, network visualizations, and querying degrees of separation between entities in the domain. Scholarly and Research Communication volume e key enabling fact in terms of innovations in scholarly publishing and new models of partnership is that though the contents of the REM are under copyright by Routledge, the metadata and survey results are not. As such, both the raw metadata and the analyzed results are open access and public. We note with some wry (dis-)satisfaction that Routledge appears to have learned from this experience, and perhaps others like it: at least some contributor contracts issued by Routledge now include a clause requiring authors to complete a metadata survey in addition to providing the content for which they have been contracted. Tier III takes Linked Modernisms into the realm of Natural Language Processing (NLP) and algorithmic harvesting of metadata. With our data output in machinereadable RDF, Linked Modernisms will trigger a partnership with InPhO to draw upon their expertise in automated and semiautomated data analysis. Colin Allen, Mathias Niepert, and Cameron Buckner are leaders in using NLP, algorithmic identification of collocated terms and topic modelling, and human-machine interaction to ensure highquality results. Our collaboration with them will include analyzing both the results of the metadata survey and the content of the REM. rough our partnership we will follow the protocols devised by InPhO in analyzing the content of the Stanford Encyclopedia of Philosophy. e academic partnership between Linked Modernisms and InPhO, drawing upon material – some of which is copyright protected, some of which is not – produced for a for-profit publisher will help chart new territory in scholarly publishing. Equally, Linked Modernisms will draw upon its partnership with Susan Brown’s Canadian Writing Research Collaboratory (CWRC) and the Orlando Project6 (Brown, Clements, & Grundy, 2010) to pool resources and efforts. For some time, the CWRC and Orlando have been pursuing many of the same methodological questions that Linked Modernisms is now engaging, with significantly overlapping material. Continuing the theme of developing partnerships to transform scholarly publishing, this collaboration will be informed by InPhO’s contributions and will aim to establish interactive forms of scholarly publishing that let researchers benefit from copyrighted material without necessarily having access to that material. at is, where copyright restricts the material produced by the Orlando Project for Cambridge University Press and by the REM for Routledge, metadata surveys and NLP approaches to the content – non-consumptive or non-expressive uses – allow researchers to analyze materials without necessarily being able to read them directly. In one respect, this may be one of the most significant benefits to distant reading, as proprietary repositories can still be used to yield scholarly insights even when they remain off-limits to researchers, or at least difficult for them to access. Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. 7 5/ issue 4 / 2014 Scholarly and Research Communication volume 5/ issue 4 / 2014 One area in which Linked Modernisms will collaborate most fruitfully with CWRC, Orlando, and INKE is the relatively new (to humanities) terrain of sophisticated visualizations of research results. e manifold partnership of these projects with InPhO’s advanced algorithmic tools will help develop John Saklofske’s (2013) NewRadial visualization application as a form of publishing. at is, when researchers execute queries on data that yield significant results, they will be able to publish them in scholarly form as visualizations that demonstrate graphically the arguments they are pursuing. is may in fact be the most exciting aspect of the kind of work being done by the Orlando Project, INKE, and Linked Modernisms, as it affords scholars not just new ways to understand and present their research results, but innovative ways to query their fields. Tier IV of the Linked Modernisms information matrix takes a self-aware approach to the very construction of the field of modernist studies. Supplementing the content (REM), first level of metadata (survey), and second level of metadata (NLP and algorithmically produced) of Linked Modernisms is a third level of metadata. It concerns how, by whom, and with what invisible biases the field of global modernist studies (as represented by the REM and its associated metadata) is constructed. Proceeding from the knowledge that no field of knowledge is neutrally constructed, we have constantly wondered how we can make legible the invisible or latent factors governing the production of the field of global modernist studies as the REM and Linked Modernisms will capture it. at is, if the knowledge contained in the REM’s entries and the metadata gathered about those entries comes from particular individuals – even if they are domain experts – then how can we trace the particularity of those individuals? How, that is, can we prevent the field of knowledge from posing as neutral and merely factual? How can we reconstruct the argument latent in that field of knowledge? In an effort to answer these questions, we have devised a second information ontology framework that seeks to capture all relevant metadata about the individuals who provided the metadata in the first survey (meta-metadata?). is framework has become the basis of another survey that asks demographic questions of the domain experts with an eye to capturing an overview of those characteristics that most oen remain unmarked, even though they can profoundly affect the information proffered: sex, gender, sexuality, ethnicity, religion, language, location, professional status, training, etc. We have received ongoing Human Research Ethics Board approval 7 to conduct this research, and intend to implement the survey in the coming year. As with the first two levels of metadata derived from the REM and its first survey results, this survey will also be supplemented by NLP and algorithmic processing to infer relationships that may otherwise remain obscure. at is, even where respondents believe themselves to be completely honest about their backgrounds and partiality, inferential algorithms may be able to detect further occulted aspects of how they produce and present information, which will be significant for understanding modernist studies as itself a field of data produced in specific circumstances. Hybrid publishing and partnerships In terms of the present publication, perhaps the most interesting element of Linked Modernisms is its reliance upon fusing multiple publication models through 8 Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. partnerships with a commercial press and other academic initiatives. e REM itself is a commercial publishing venture by Routledge on behalf of the global corporation Taylor and Francis. Linked Modernisms will harvest metadata based upon the content of the REM. Simultaneously, the metadata will feed into the faceted searching apparatus being devised by Routledge as the primary element of value added to the REM, setting it apart from other online encyclopedias. e metadata will be gathered for open access viewing and use by Linked Modernisms, which is housed at the University of Victoria. It will be published online through mvp.uvic.ca – an academic domain. e analysis of the metadata will be done by Linked Modernisms in collaboration with InPhO, housed at the University of Indiana, and CWRC and the Orlando Project, housed at Guelph University and the University of Alberta, respectively. e results will be disseminated in a further hybrid format, with raw and preliminary results openly accessible online, and with more in-depth analyses submitted for peer-reviewed publication in both online and print formats. Moreover, as the REM expands and undergoes revision, so Linked Modernisms will continue to gather metadata and analyze it in partnership with InPhO and Orlando, creating a recursive publishing environment in which those contributing to the REM aer the results of Linked Modernisms are made public may be doing so in light of – or at least with an awareness of – the initial findings. is awareness will add a layer of selfreflexivity to the production of knowledge about global modernist studies and the field of modernist studies itself, which can in turn be analyzed as part of the ongoing analysis of the metadata. Scholarly and Research Communication volume As Linked Modernisms matures, we remain keen to develop modes of scholarly publishing and dissemination that are suited to the dynamic nature of the knowledge field under consideration – both data and metadata – and that conform to the principles of open access and creative commons sharing. We welcome any suggestions along these, or indeed any other, lines. Notes 1. e MVP would like to acknowledge and thank its partners, without whose aid very little of what is described here would have been possible: Networked Infrastructure Nineteenth-century Electronic Scholarship (and Juxta), the Modernist Journals Project, Islandora/DiscoveryGarden, Editing Modernism in Canada, Implementing New Knowledge Environments, the Humanities Computing and Media Centre at the University of Victoria (UVic), and Fairleigh Dickinson University. We are grateful to the Social Sciences and Humanities Research Council of Canada (SSHRC) and the Maker Lab in the Humanities at UVic for their support of this project, and to the Electronic Textual Cultures Lab at UVic, and Ray Siemens, in particular, for facilitating it. 2. A summary of this activity is now published online on the University of Victoria Library’s site, UVicSpace (see Krecsy & Modernist Versions Project Team, 2014). 3. is concept assumes that all materials from notes through manuscripts, typescripts, corrections, proofs, rejected versions, deletions, additions, all print editions, and any corrected or collected editions – taken together – constitute the literary object of study. e avant-texte is a textual ecosystem in which the official, Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. 9 5/ issue 4 / 2014 Scholarly and Research Communication volume 5/ issue first, final, or approved edition is but one element, held to be of no greater importance than any other. 4 / 2014 4. is has apparently changed, as Routledge now circulate a metadata survey to all contributors to their publications. 5. Our preference now is to use Fluid Surveys or another Canadian-based company, due to concerns about data security with American companies, but it was prohibitively complicated to migrate the survey from one platform to the other, so we continue to use SurveyMonkey for this project. 6. Like Linked Modernisms, the Orlando Project aims for open access but is constrained by its relationship with Cambridge University Press, which owns the copyright to its material. 7. HREB Approval Certificate 12-372. References Allen, Colin, Niepert, Mathias, & Buckner, Cameron. (2013). The Indiana philosophy ontology project. URL: https://inpho.cogs.indiana.edu/ [July 31, 2014]. Barclay, Adèle. (2014). Reading and georeferencing Rhys (Sayers, Jentery, Wilson, Karly, & the Maker Lab in the Humanities, Eds). URL: http://web.uvic.ca/~mvp1922/reading-and-georeferencingrhys/ [July 31, 2014]. Brown, Susan. (2009). The Canadian writing research collaboratory/Le Collaboratoire scientifique des écrits du Canada. URL: http://www.cwrc.ca/projects/infrastructureprojects/pilot-projects /women-and-words-canadian-feminist-literary-collectives-1980-2000/ [July 31, 2014]. Brown, Susan, Clements, Patricia, & Grundy, Isobel. (2010). The Orlando project. URL: http://www .artsrn.ualberta.ca/orlando/ [July 31, 2014]. Buckner, Cameron, Mathias Niepert, & Colin Allen. (2010). From encyclopedia to ontology: Toward dynamic representation of the discipline of philosophy. Synthèse. URL: https://inpho.cogs.indiana .edu/papers/2010BucknerNiepertAllen.pdf [July 31 2014]. Carter, Daniel. (2014). The versioning machine for audio: Introducing VM 5.0. The Modernist Versions Project. URL: http://web.uvic.ca/~mvp1922/the-versioning-machine-for-audiointroducing-vm-5-0/ [July 31, 2014]. Christie, Alex, Ross, Stephen, Sayers, Jentery, Tanigawa, Katie, & the INKE-MVP Research Team. (2014). Z-Axis Scholarship: Modeling how modernists wrote the city. The Modernist Versions Project. URL: http://web.uvic.ca/~mvp1922/z-axis/ [July 31, 2014]. Huculak, J. Matthew, & the MVP Research Team. (2011). MVP White paper on collation tools. The Modernist Versions Project. URL: http://web.uvic.ca/~mvp1922/mvp-white-paper-on-collationtools-2011/ [July 31, 2014]. Jockers, Matt. (2013a). Macroanalysis: Digital methods and literary history. Champaign, IL: University of Illinois Press. Jockers, Matt. (2013b). “Secret” recipe for topic modelling themes. URL: http://www.matthewjockers.net /2013/04/12/secret-recipe-for-topic-modeling-themes/ [July 31, 2014]. Joyce, James. Ulysses. (2012). The first edition. The Modernist Versions Project (Huculak, Matthew J., Curator). URL: http://web.uvic.ca/~mvp1922/portfolio/ulysses-shakespeare-co-1922-1st-edn/ [July 31, 2014]. 10 Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. Krecsy, Stefan, & Modernist Versions Project Team. (2014). Year of Ulysses. URL: https://dspace .library.uvic.ca/handle/1828/5484 [July 31, 2014]. Liu, Alan. (2014). The state of the digital hmanities: A report and a critique. Arts and Humanities in Scholarly and Research Communication volume Higher Education,11(8), 8-42. Millar Usiskin, Jana, & Belaid Moa. (2014a). Making models of modernism. The Modernist Versions Project. URL: http://web.uvic.ca/~mvp1922/mmm/ [July 31, 2014]. Millar Usiskin, Jana, & Belaid Moa. (2014b). Visualising communities in Mrs. Dalloway. The Modernist Versions Project. URL: http://web.uvic.ca/~mvp1922/dallowaycommunities/ [July 31, 2014]. Millar Usiskin, Jana, & Belaid Moa. (2013). Making modernism big. The Modernist Versions Project. URL: http://web.uvic.ca/~mvp1922/big/ [July 31, 2014]. Ross, Stephen. (2012). Theory of versioning. The Modernist Versions Project. URL: http://web.uvic.ca /~mvp1922/theory-of-versioning/ [July 31, 2014]. Saklofske, Jon, & the INKE Modelling and Prototyping Team. (2013). NewRadial. http://inke.acadiau .ca/newradial/ [July 31, 2014]. Simpson, John, Brown, Susan, Sayers, Jentery, Bath, Jon, Quamen, Harvey, Saklofske, Jon, Barclay, Adèle, Christie, Alex, Elliott, Mandy, & the INKE Research Group. The emergence of linked data in the humanities. [Unpublished paper] Tanigawa, Katie. (2014). Visualizing a changing Nostromo. The Modernist Versions Project. URL: http://web.uvic.ca/~mvp1922/visualizing-a-changing-nostromo/ [July 31, 2014]. Tanigawa, Katie. (2013a). Returning to the periodical context. The Modernist Versions Project. URL: http://web.uvic.ca/~mvp1922/returning-to-the-periodical-context/ [July 31, 2014]. Tanigawa, Katie. (2013b). Warping the city: Joyce in a mudbox. Maker Lab in the Humanities. URL: http://maker.uvic.ca/mudbox/ [July 31, 2014]. Ross, Stephen, Christie, Alex, & Sayers, Jentery. (2014). Expert/Crowdsourcing for the Linked Modernisms Project. Scholarly and Research Communication, 5(4): 0401186, 11 pp. 11 5/ issue 4 / 2014