After the success of the second edition in Utrecht, the ADHO-Special Interest Group “Digital Literary Stylistics” (SIG-DLS) organizes a new version of its Tool criticism workshop for DH2020 Ottawa. This year, it will be open to contributions of all interested researchers and it will also set a focus on historical tool criticism. The main goal of the workshop is to provide more critical awareness on the use of tools in DLS.This type of tool criticism can only be stimulated in two ways: (a) by a direct confrontation with user feedback and reports,which usually remain “in the background” of research papers; (b) via reconstructions of the historical evolution of tools and methodologies (which are never a given, but are built on top of complex –and sometimes unpredictable– research paths). The workshop is a natural extension of the DLS Tool Inventory (DLS-TI), which gathers information on the practices of the various traditions present in DLS. The workshop is scheduled for a pre-conference slot (the date pending confirmation). It will be structured as a mini-conference, where participants will be asked to present short critical reports (20 minutes each) on their use of tools in DLS. We encourage submissions on the following topics:
user interface criticism;
criticism of implied/hidden epistemologies (anatomy of tools);
theoretical tool criticism (e.g. issues in modeling);
historical tool criticism;
biases in DLS studies;
Proposals should be sent by 15 April 2020 to email@example.com. They should contain a title and a brief abstract (max 300 words). Also feel free to enter contributions to the DLS Tool Inventory (DLS-TI), if relevant. Acceptances will be notified by 15 May, 2020.
Organizers: Simone Rebora (University of Verona & University of Basel), J. Berenike Herrmann (University of Basel), Francesca Frontini (UPVM & Praxiling), Geoffrey Rockwell (University of Alberta), and Thierry Poibeau (CNRS & LATTICE)
Scientific committee: Anne-Sophie Bories (University of Basel), Julia Flanders (Northeastern University), Francesca Frontini (UPVM & Praxiling), J. Berenike Herrmann (University of Basel), Thomas C. Messerli (University of Basel), Thierry Poibeau (CNRS & LATTICE) , Simone Rebora (University of Verona & University of Basel), Geoffrey Rockwell (University of Alberta), Jan Rybicki (Jagiellonian University), and Peter Verhaar (Leiden University).
Please note that all participants and attendees will need to be registered for DH 2020. Full-day workshop registration for participants costs $70CAD and includes lunch. Conference registration information can be found here: https://dh2020.adho.org/registration/
When researching real-world contexts, scholars need a valid procedure for identifying and analyzing metaphorically used language. This means that they need a handle for systematically gauging the reliability of annotations, possibly for ensuing machine learning, but also a way of spotting patterns and principles behind creative and otherwise ‘messy’ cases. One such procedure for metaphor identification is MIPVU, the Metaphor Identification Procedure Vrije Universiteit (Steen et al. 2010), which celebrates its tenth anniversary in 2020. Originally developed at VU Amsterdam, it is now being widely used in research projects around the globe, being adapted for different languages (see Nacey et al. [in press] Metaphor identification in multiple languages: MIPVU around the world) and different types of authentic data, including spoken and multimodal discourse (see VISMIP for visual metaphor [Sorm & Steen 2018], and FILMIP for filmic metaphor [Bort-Mir 2019]. However, many scholars are challenged by developing their own computational environment for applying MIPVU and analyzing the annotations, including reliability tests. Some use an XML editor, others MS Excel or MS Word. So far, a tool is missing that facilitates an intuitive annotation and analysis environment.
Our hands-on tutorial introduces metaphor scholars to a computational tool for applying MIPVU called CATMA, which stands for Computer Assisted Text Markup and Analysis, see https://catma.de. CATMA was developed at the Hamburg University and is currently used by over 60 research projects worldwide. It supports collaborative annotation and analysis as well as explorative, non-deterministic practices of text annotation. It integrates text annotation, text analysis, and visualization in a web-based working environment, combining the identification of textual phenomena with their investigation in a seamless, iterative fashion. The workshop is endorsed by the Special Interest Group “Digital Literary Stylistics” (SIG-DLS; https://dls.hypotheses.org/) of The Alliance of Digital Humanities Organizations (ADHO).
Workshop format: The workshop will be hands-on. After a short run-through of the basics of MIPVU, participants will learn how to work with CATMA. This introduction to CATMA includes the core annotation and analysis functionalities, text upload, annotation and specification of annotation categories, as well as text queries of source text and its annotations. Participants will also generate the visual output of query results and learn the export of markup data in XML format. In a last section, we will discuss several methods of annotation validation, including inter-annotator agreement.
Participants will be asked to acquaint themselves with the English version of the annotation manual MIPVU in preparation. Access to language-specific MIPVU protocols from Nacey et al. (in press) may be provided upon request. Participants will, however, need no prior knowledge of computational text annotation and can work with their own laptop computers. CATMA runs on Laptop or PC (Windows, Unix or MacOS) with a current web browser (MS Explorer or Edge; Firefox, Chrome, Safari) with a mouse or touchpad (touchscreen navigation is not yet supported).
About the organizers
J. Berenike Herrmann, Basel University, firstname.lastname@example.org. Berenike is Assistant Professor (‘Oberassistentin’) at the DHLab Basel. She is a literary/linguistics digital humanist with a track record in metaphor studies, computational stylistics, and cognitive stylistics. Among her research topics are modernist and realist literature from a comparative perspective, the epistemology and methodology of computational textual studies, and evaluative discourse on the web 2.0. She is co-developer of the original and the German MIPVU.
Jan Horstmann, Hamburg University, email@example.com. Jan is a digitally working literary scholar and currently coordinates the forTEXT project (https://fortext.net) for the dissemination of digital routines, resources, and tools for text annotation and analysis. He has taught CATMA in many seminars and workshop in recent years and uses the tool especially for the annotation and analysis of German literary texts. His particular interests lie in narratological categories as well as renunciation and irony in Goethe.
Aletta G. Dorst, Leiden University, firstname.lastname@example.org. Lettie is an Assistant Professor in English Linguistics and Translation Studies at Leiden University Centre for Linguistics. Her research interests and main publications are in the fields of metaphor studies, stylistics, translation studies, genre analysis, contrastive linguistics and corpus linguistics. She was part of the team that developed MIPVU for English and Dutch, and co-author of German MIPVU. She has applied MIPVU to texts from different genres and registers and in different languages.
Nils Reiter, Stuttgart University, email@example.com, has a background in natural language processing and works on the analysis of literary texts. He is principal investigator in several DH-related projects concerning dramatic, narrative and historic texts and has supervised several workshops, tutorials and classes on statistical methods for text analysis, including annotation and metaphor detection. Starting in September 2019, he will be a visiting professor at Cologne University.
It has become a tradition for the stylometric community to organize a small get-together whenever a DH conference takes place near one of the labs. In 2014 the crowd had its meeting in the Australian wilderness of Wollombi near Newcastle, and in 2016 they enjoyed the mountain seclusion of Żabnica near Kraków. This year we met in the heart of the urban jungle, at Netherlands Institute for Advanced Study in the Humanities and Social Sciences (NIAS), invited by the Riddle of Literary Quality team and Karina van Dalen-Oskam.
Karina van Dalen-Oskam opens the meeting. Photo by Tomoji Tabata
The meeting gathered some of the experts in the field and newcomers both from affiliated Federation of Stylometry Labs and ones pursuing solo stylometric projects. The workshop served as a great opportunity to get to know each other in a more relaxed setting and as an occasion for community building. Over the course of three days (15-17 July) we discussed our projects in development, good practices in the field and making it more accessible to newcomers, juggled ideas and enjoyed each other’s company over lovely dinners.
In relation to recent replication debates, we discussed the need for registering experimental setups as well as the exact shape of data (e.g. which particular editions were used, and what changes, such as normalization, introduced). While we noted the need for specifying the experimental setup in detail in advance, it was also observed that in practice we cannot foresee all the problems on our way and may need to adapt the method in the process.
Peter Boot presenting research on types of impact of text on readers. Photo by Tomoji Tabata
We also discussed technical problems with working on digitized texts, especially historical ones – dealing with issues related to versioning, accuracy of preserved copies and search for the version closest to original intended by the author, lack of proper descriptions of digitized files in some collections – and the need for separating the development of reliable methods from answering actual research questions with a particular dataset.
Floor Buschenhenke discussing tracing how writers create. Photo by Tomoji Tabata
Our interest was also sparked by a few more literary problems: the distinctiveness of authorial voice across various media (literary vs non-literary text, literature reviews, speech, audiovisual data) and manner of creating (e.g. typing oneself or dictating). Introducing the dimension of reader reception of literary texts to the discussion, researchers from Amsterdam addressed differences between stylometrists and readers in their perception of ‘style’, as well as the topic of stylistic development over time as observed in genetic criticism projects.
Finally, we also discussed the development of the Federation of Stylometry Labs and explored ideas for making it a more useful resource for aspiring stylometrists. You will notice the results of this discussion on our website https://stylometry.org in the upcoming months. This is surely of interest to SIG DLS members and observants, so stay tuned!
Most of us are back from the post-conference summer break: Time to revisit! Seven weeks have passed since the DH2019 conference in Utrecht: presented as the biggest DH conference so far (with about 1,100 participants and more than 400 presented papers), it also distinguished itself for the number and quality of contributions in digital literary stylistics. With this post, through highlights and summaries, we try to provide an overview of the most recent outcomes of stylistics research in DH. Well aware that (as our colleague José Calvo Tello already highlighted when reporting on DHd2018) these notes cannot be considered but a preliminary, incomplete, and subjective report on a much more complex and stratified event. As usual, more information can be found in the conference programme and book of abstracts, both available on the conference website. To extend the variety of documentation, then, the #dh2019 and #sig_dls hashtags were remarkably active on Twitter, while participants like Geoffrey Rockwell even shared their conference notes online.
But well, if you would like to know the specific point of view of the SIG-DLS (and, more specifically, of the member who’s writing this report), here are a few notes.
The first step, inevitably, is the pre-conference workshop organized by the SIG-DLS steering committee. Held on the morning of 9 July, DLS Tool Criticism. An Anatomy of Use Caseswas conceived as a moment of self-reflection (and of self-criticism) on the tools and methodologies most frequently used in digital literary stylistics. During the workshop, Clémence Jacquot presented her use of the TXM software in the study of Apollinaire’s poetry (stimulating a discussion on limits and potentials of pre-compiled, user-friendly tools for the study of style); Geoffrey Rockwell presented a case study and a theoretical reflection on the concept of replication (thus rekindling the still lively discussion sparked by Nan Da’s criticism, but going beyond, querying ‘replication’ as a more flexible concept); and Steffen Pielström resisted the temptation to advertise his efficient topic modeling tool, inciting on the contrary an acute disputation on the pros and cons (and on the aleas) of such a technique. One of the major achievements in the final discussion was the shared impression that the quantitative/qualitative dichotomy is simply misleading, and digital literary stylistics will have to move beyond it, looking for a more organic integration of approaches.
When reaching the bulk of the conference, the impressions we can record here become inevitably more random and scattered. A general trend was evident, however, with digital stylistics contributions divided roughly in two main categories: methodological analyses, providing very advanced and stimulating discussions of (more or less) shared approaches; and applications on complex and engaging case studies. This can be considered as a reflection of the state of things in stylometry, where methodological research is needed to ground more and more steadily the scientific rigor of the approaches (with the risk of an excessive technicism), and new applications even more so, if our research field is to gain recognition beyond the boundaries of DH.
Among the methodological papers, some titles call for a specific mention. In “Comparing Assonance and Consonance for Authorship Attribution”, Lubomir Ivanov proposed prosodic elements as possible new features in stylometric classification, considering the extremely relevant role of prosody in 18th century prose (and beyond). His experiments showed how the combination of assonance/consonance with traditional stylometric methods generates significant improvements in the classification. Further research on the subject is definitely advised! In “Feature Selection in Authorship Attribution: Ordering the Wordlist”, Maciej Eder and Joanna Byszuk worked on the effects of re-ordering the (traditional) list of features. Main discovery: a combination of term frequency (the widely-used MFW) and a coefficient of variation slightly improves the results. Finally, in “Identifying Similarities in Text Analysis: Hierarchical Clustering (Linkage) versus Network Clustering (Community Detection)”, a team of four researchers led by Jeremi Ochab tried to provide an answer to a still under-studied–but ever-present–question in stylometric classification: how do I cluster the texts to visualize similarity/dissimilarity? Results showed that network clustering tends to underestimate the actual number of clusters, while hierarchical clustering overestimates it (and of course, which one is better will depend on the research question!). Traditional algorithms (Ward, Louvain) were confirmed as the most efficient.
Placing itself between method and application, David L. Hoover proposed his “Invisible Translator Revisited”. Given the (supposed) invisibility of the translator in stylometry, Hoover proposed a method for making her visible again, by filtering out the words preferred by the author. The approach showed surprisingly effective, but will it work for all pairs of authors-translators? As it happens, the idea found an indirect confirmation in “Attributions Of Early German Shakespeare Translations”, where a team of three researchers led by David Lassner was equally successful in revealing the translator (here, by combining advanced classification methods based on sub-word-level features). Shifting the focus towards themes and genres, Fotis Jannidis and his colleagues (in “Thematic complexity”) used two approaches, LDA and Zeta, to model genre distribution and to measure thematic complexity (via the “Gini index”) for dime novels and high-brow genres. Their results support their hypothesis of high literature comprising a broader spectrum of subject areas than pulp fiction genres. On a similar line of enquiry, a team of seven researchers led by J.D. Porter used stylometric approaches to identify passages in novels that adopt stylistic traits from other genres (in the paper “Microgenres”).
Moving then to the applications, it is worth mentioning “Challenging Stylometry: The Authorship of the Baroque Play La Segunda Celestina”, by Laura Hernández Lorenzo and Joanna Byszuk (who wins our award of the most prolific stylometrician, with three co-authored papers, and who on top of that also has contributed a SIG-DLS blog post!). That of La Segunda Celestina is a real “challenge” for authorship attribution: in fact, the results of the analysis did not limit themselves to “solving” the problem, but suggested possible new lines of enquiry. In “Stylometric Analyses of Character Speeches in French Plays”, then, Ioana Galleron provided an effective example of “scalable reading”, combining multiple approaches to distinguish the style of male and female characters. Finally, Massimo Salgaro and Simone Rebora (i.e. myself, in the panel “Digital Humanities for the Study of Social Reading”) showed how stylometric approaches can also be used to distinguish professional and non-professional book reviews.
A conclusive note should be dedicated to “Stylometry for Noisy Medieval Data: Evaluating Paul Meyer’s Hagiographic Hypothesis” by Ariane Pinche, Jean-Baptiste Camps, and Thibault Clérice. The work stands out as a perfect realization of a processing pipeline (from handwritten text recognition to stylometric classification, with advanced techniques to reduce noise and manage spelling variants) that instantiates a fruitful dialogue with traditional literary research, by confirming (and in part disputing) Paul Meyer’s hypotheses about the composition of the French saint’s Lives. The paper was awarded the Paul Fortier Prize for the best young scholar paper of the conference. A further success that confirms the increasingly central position of digital literary stylistics in DH research.
The SIG-DLS steering committee reiterates their congratulations to Ariane, Jean-Baptiste, and Thibault, but also to the Programme Committee chairs Elena and Fabio, and to the local organizers Joris and Franciska, for making such a spectacular conference possible. Looking forward to meeting you all again in Ottawa (hopefully, with even more stylometry to discuss)!
Since spring 2019, we’ve all been aware of the debate triggered by Nan Z. Da’s “Computational Case against Computational Literary Studies”. The article addresses methodological dimensions and among other things revealed that we still need to work on an awareness of the field in its entire breadth. Some time ago, the SIG-DLS steering committee published a statement in Cultural Analytics. You can read it here.
After all, we are A maturing field that is now ripe for some benevolent peer criticism
Although Da’s article has various problems, we think it can actually serve as a valuable pointer to discuss issues of replicability and the theory-method fit of our tools. After all, we are maturing field that is now ripe for some benevolent peer criticism!
With a critical look at our main types of inquiry, Mike Kestemont (Stylometry), Clémence Jacquot (Textométrie), and Steffen Pielström (Semantic Text Mining) will each ‘dissect’ one of our typical methods with regard to theoretical modelling and research questions. The ws will also address the future shape of tool inventory (DLS-TI), one of our initiatives.
Are you going to be at Utrecht? You can still register! In case you have trouble registering for the workshop post hoc (bc you’re already registered for the conference) please let us know.
Plotting Poetry – Machiner la poésie gathers scholars applying computational methods to the analysis of poetic texts. They focus on metrics, rhyme, stylistics, poetics, and they pursue their hermeneutical agenda through mechanically enhanced reading. The group started as an international conference organised in Basel in 2017 by Anne-Sophie Bories, Hugues Marchal and Gérald Purnelle, with keynotes by Franco Moretti and Valérie Beaudouin, followed by a workshop organized in Berlin in 2018 by Anne-Sophie Bories and Burkhard Meyer-Sickendiek. The group will meet again in Nancy in 2019 (CfP), hosted by Véronique Montémont and Anne-Sophie Bories, then in Prague thanks to Petr Plecháč.
Let’s create our inventory of Digital Literary Stylistics methods and tools!
This summer at DH2018, the members of SIG-DLS in situ came up with some new activities for the new season. Among these was a DLS Tool Inventory of our own, reflecting our practice, with tested and tried methods, software, tools.
Please go ahead and enter the tool/method that you know well in a one-line report. Mind you, that shall be tools/methods that are real and recent, tested and tried within the last five years. Please don’t enter reports on tools created by yourself. Where possible, your one-line report shall also include a brief and constructive report on strengths and weaknesses of the tool/version that you used; on usability with regard to your research question.
The SIG-DLS is planning to organise a “scavenger hunt” initiative in collaboration with Patrick Juola. The goal of the initiative will be that of discovering new datasets for authorship attribution/stylometry, through a competition (PAN-like) approach.
As we are still in the early stages of planning, we are looking
first of all for anybody who would like to join the organizing committee. We
will have to brainstorm on some preliminary issues (some very practical, like
deciding where to store the data, or looking for possible sponsors) and to
start sharing the tasks.
If you are interested, please contact Simone Rebora and he will tell you all the details.
It features annotation tutorials on manual Metaphor Identification (MIPVU), a vector-based exploration by Nils Reiter and his team from Stuttgart, and the annotation and corpus tool CATMA. The workshop is a 1.5-day event.
In addition there’s going to be two full lectures by Lettie Dorst (Leiden) and Jan-Christoph Meister (Hamburg), which may be attended independently of the workshop.
A few months ago, our COST Action Distant Reading for European Literary History (CA16204) was approved and started running. Even more recently, our Working Group ‘Scholarly Resources’, aka its corpus backbone, met in Prague, for the first time! We are very excited about getting started and would like to briefly report on our work in this blog.
The COST Action Distant Reading has the goal to “develop the resources and methods necessary to change the way European literary history is written” (Memorandum of Understanding). To approach this goal, the Action’s Working Group ‘Scholarly Resources’ aims to create a big open source benchmark corpus of novels from 1850-1920, called ELTeC (European Literary Text Collection). During the course of the Action, we will examine ELTeC with different computational distant reading methods such as topic modelling, authorship attribution, network analysis, stylistic analysis, and different types of character analysis.
The Action’s Working Group ‘Scholarly Resources’ will coordinate the task of creating the ELTeC; It consists of members from all over Europe, from literary studies, computational and corpus lingustics, and from information science. The COST Action is a great opportunity to collaboratively work with researchers and bring together expertise for different languages as well as computational methods.
A challenging task has been defining the corpus selection criteria, a common annotation model and potential workflows for corpus creation for the ELTeC which can be applied to European novels from different languages as well as various publication contexts.
During our first Working Group meeting, we developed corpus selection and balancing criteria which follow a simple but consistent corpus design approach and represent the variety of novels in this period. Among other things, we do not want to solely rely on normative canon-based selection criteria and set a bias to our understanding of novels. The focus of the ELTeC encoding scheme is to uniformly represent historical novels from different languages with a basic TEI mark-up. The standard markup is necessary for applying different types of distant reading methods rather than representing the original text structure. ELTeC and the encoding scheme will be freely available at our GitHub Organization.
In this way, we would like to contribute a big chunk to the creation of a digital basis for cross-national and cross-language analysis of European literature!
In fact I have no idea why I’m blogging about this, since the whole event was covered by “La Repubblica” (so nice to see that there are still countries in this world where literature is taken seriously by mainstream meadia) and the local press. They all published relatively sensible accounts of the event, also with diagrams; and a photo of some of the participants (I don’t want to brag, but…) made it into Greek press as well! Better than that: you can still (I think) watch the proceedings on Livestream!
Still… The speakers were either already members of our SIG, or will be soon, or are our close cousins from Quantitative Linguistics. Arjuna Tuzzi and Michele Cortelazzo (University of Padova), Jacques Savoy (University of Neuchâtel), Jan Rybicki (Jagiellonian University), Maciej Eder (Polish Academy of Sciences), Patrick Juola (Duquesne University), George K. Mikros (National and Kapodistrian University), Pierre Ratinaud (University of Toulouse II) all agreed that stylometric evidence overwhelmingly points to Italian writer Domenico Starnone (he has always been one of the main suspects anyway) rather than his wife, translator Anita Raja; yet her participation of some sort cannot be ruled out. This is very uncharacteristic agreement among people who use different methods and come from different backgrounds; and suggests this is a result that should be reckoned with.
I think this event was noteworthy for two things: first of all, DLS is becoming more and more visible in the media, but there is still a long way to go. Second, it ended in a spontaneous discussion on the ethics of the whole thing. Do anonymous writers have a right to privacy, especially when they’re (probably) alive? Starnone’s and Raja’s privacy had already been trampled upon by journalists who claim to have traced Ferrante’s royalties. Should they not be left alone?
My own answer is the following: I don’t really care who wrote which book, any book, and who took the money, as long as I learn something interesting about the process of literary creation. I think I did this time. While it is quite plausible that both Raja and her husband are in it together, the stylometric fingerprint is that of the latter. If it’s a collaboration, then this is a very valuable insight that might lead us further on: what is, if any, the “silent partner’s” contribution? In what way might Raja’s work as translator of Christa Wolf contribute to the Ferrante phenomenon? If they are, or one of them is, Ferrante, are they becoming Ferrante themselves?
This blog is going to become the web page of the Special Interest Group Digital Literary Stylistics (SIG-DLS), whose constitution has recently received official approval by ADHO (adho.org/sigs). We are going to publish news about the SIG-DLS initiatives and related events, such as workshops and conferences, as well as blog posts about the research done by the SIG members. The full proposal SIG-DLS (as approved by ADHO), with a description of the organisation and its aims, can be found here: https://github.com/DLS-SIG/SIG-Proposal/wiki/DLS-SIG—Proposal