MA Seminar in Stylometry @ Uniwersytet Jagielloński, Kraków, Poland

This is a two-year course taught (or, rather, loosely coordinated by Jan Rybicki), which introduces students of the University’s Translation Studies programme to quantitative approaches in the study of – you’ve guessed it – translation. The course’s main deliverable is a ca. 80-page dissertation in English.

The course’s first year begins with an introduction to the field and to its methods, and ends in a specified subject of later work. The students also produce a short “teaser” of what they will be writing about. Some examples are presented below.


Sara Chodorowicz

Genres according to Project Gutenberg

The subject of this experiment is 83 novels that, according to the Project Gutenberg, belong to one of the three genres: adventure, crime fiction and detective fiction. The presented samples are both originally written in English and translated into English, for example from French (such as Jules Verne, Alexander Dumas père). The majority of the selected authors appear only in one category; however, there are three names (Morrison, Orczy and Wallace) that have works that belong to both adventure and detective fiction; thus, to those books that belong to adventure section, a letter A has been added after the name of the author. Moreover, the name Stevenson appears in adventure as well as detective fiction; yet the name does not represent the same person. Adventure is represented by Robert Louis Stevenson and detective fiction by Burton Egbert Stevenson. In order to avoid confusion, the letter B has been added to the latter man’s work.

The aim of the experiment is to determine whether the novels that Gutenberg categorised into one group can be distinguished from those belonging to two other groups. Furthermore, it ought to be established whether translations from the same language show any similar traits. Another issue that is of concern is the matter of authorial signal, for there are novels written by the same author that belong to two categories. The methods employed to answer all those questions are consensus cluster analysis (Figure 1) and network analysis (Figure 2).

Figure 1. Cluster analysis

Cluster analysis illustrates the relationship between particular samples; however, if one was to see connections between all the texts, network analysis is needed.

Figure 2. Network analysis

In order to distinguish what category does a particular novel belong to, the sample has been coloured according to their category: red shades for adventure, green shades for crime fiction and blue shades for detective fiction.

The adventure books appear to exhibit similarities between one another, as the samples are grouped closely together. The only novel that is separated from the rest is Arthur Morrison’s  The Dorrington Deed-Box (morrisonA_box). It is instead grouped with other Morrison’s books, those that belong to the detective fiction category, thus showing a strong authorial signal of Morrison. Detective fiction novels, though not as neatly as the adventure ones, are mostly grouped in one place. They are more interconnected with crime fiction novels, which illustrates the similarity between the two categories. Authors that are particularly close include Sax Rohmer and Arthur Rees or Joseph Smith Fletcher, Harrington Strong and H. Beam Piper. Crime fiction novels do not seem to compose a strong group, as only a few samples are present in a particular place on the network. As previously stated, this category is interlaced with detective fiction novels.

Translated samples do not show any apparent similarities, as they are not universally connected one to another or even placed in close proximity on the network.

Taking all the evidence into consideration, it can be concluded that the Gutenberg division proves to have some merit. Adventure books clearly compose one group. Detective and, particularly, crime fiction are not clustered so closely but, as they are genres similar enough, the connection between them can be expected.


Kamil Różański

Discovering The Dirty Old Man’s Identity

My very first research that I did more for fun than on spec for a serious discovery turned out to be more fascinating than I thought of. It all started after 1834 when Adam Mickiewicz, one of the most important figures in Polish literature, wrote an epic poem considered the national epic of Poland, Pan Tadeusz  (Sir Thaddeus). The poem consists of twelve chapters called books, all written in Polish alexandrine (13 syllables divided into two half lines). There is, however, yet one chapter called the 13th Book that describes the wedding night of the title character Tadeusz with Zosia. The book resembles Mickiewicz’ style in its use of the Polish Alexandrine but on the other hand is packed with words that are commonly treated as vulgar. There was ongoing speculation about the authorship of the 13th Book of Pan Tadeusz as no writer owned up to writing it. There were three authors who were taken into account: Tadeusz Boy-Żeleński, Włodzimierz Zagórski and Alexander Fredro; the latter being most frequently suspected.

In my research I decided to have a first look at the romantic writers contemporary to Mickiewicz and do a cluster analysis of their works based on 100 most frequent words. My corpus consists of 5 books of Mickiewicz (one being a translation of Byron’s The Giaur), 4 books of Fredro (with the assumption that he was the author of the 13th Book), 2 of Słowacki and 1 of Krasicki.

I expected the 13th Book to occur somewhere near Pan Tadeusz as both of them were written in Polish alexandrine as epic poems. The results however proved me wrong and rewarded me with a very satisfying discovery.

 

Figure 1. Fredro exposed

In Table 1 we can see the results of cluster analysis based on 100 most frequent words with the use of Classic Delta distance. We can clearly see that the 13th Book is closer to Fredro’s works than Mickiewicz ones. It might have been an insufficient evidence for Fredro’s authorship and that’s what I thought at first too. From all the books that my corpus consists of,  only 3 are dramas and the 13th Book seems to like them more than other, more even than Mickiewicz model Pan Tadeusz. What’s more, all of dramas included in the corpus were written by Fredro. I decided that a consensus tree based on 100-1000 most frequent words will resolve all my remaining doubts on who the title dirty old man was.

Figure 2. It’s definitely Fredro

When I saw the results of the consensus tree that is presented in the Table 2, all of my remaining doubts vaporised like a wispy Cretan cloud on my honeymoon that I send you my warm regards from.  The 13th Book still is sticks to Fredro’s dramas and is none like other epic poems from the corpus.

There is only one conclusion that I drew from my research. The resemblance that the 13th Book and Fredro’s dramas have is so apparent that we can assume that he really was the author of the filthy continuation of Mickiewicz’ national epic.


 

Computer Assisted Text Markup and Analysis (CATMA) – An undogmatic approach to corpus analysis and Germany’s literary super heroes

Do you know CATMA? Not yet? Then read the following blog. Do you think you know CATMA? It’s still a good idea to read the following blog…

CATMA is a web-application for text markup and analysis. Its central function is annotating, analyzing and visualizing one or multiple texts. CATMA has been around since 2008.

When the implementation of the web-application CATMA started in 2008, the focus was to create a tool for digital close reading. Although there always has been the possibility to run some standard queries as e.g. word frequencies automatically, the focus of the application was entirely on the annotation functionality of single texts. Ten years later, CATMA provides the possibility to store, manage and analyse not only single texts, but also bigger corpora – using both close and distant reading methods. Although CATMA still can be used for manual annotation of single texts, in this article, we want to show you how the web-application can be used on corpora.

Goethe and Schiller, Germany’s literary super heroes

Let’s say you would like to explore the authorial styles of Germany’s literary super heroes, Goethe and Schiller. As you want to make the two author corpora comparable, you choose 12 dramatic and 5 prosaic texts of each author. Just upload those texts to CATMA and organise them into different corpora, one with texts by Goethe and one with texts by Schiller. Once your corpora have been created in CATMA, you can easily export them, or, more important, share them with your team.

Without any further preparation you can now start analysing your corpora. Using the Analyze modul you can generate a word list with frequencies. This list will show you that there are around 800.837 words in your selection of Goethe’s texts with around 46.840 types. Schiller’s texts contain 35.351 word types and a total of 498.790 words. These numbers are not very meaningful yet, because, obviously, the texts do not have the same length. Looking at the word frequency list, from a narrative perspective, the first person pronoun “ich” (I) is probably most interesting as it points to a predominant first-person perspective. Or is it just the dramatic characters? Other highly frequent (but narratologically possibly less relevant) words in the Goethe corpus are “und” (and), “die” (the/who/which – female), “zu” (to), and “der” (the/who/which – male), much as in the Schiller corpus (albeit in the order “der”, “und”, and “die”). At this top tier of frequency, there seems to be a similarity.

Word frequencies in Goethe’s and Schiller’s texts

But how about the other end of the lexical spectrum? Goethe is known to have had a comparably large vocabulary, so we may quickly check the least frequent words in the corpus – those which are used only once. Are there many of these hapax legomena?

In the Goethe corpus they sum up to 28.792 words altogether. Schiller uses 21.025 words only once in the entire corpus. This means that Goethe uses 61.4% of all the types of this corpus only once. Schiller uses 59.4% of all types in the corpus only once. Maybe this similarity suggests that Schiller after all was closely following Goethe in variability of word use. However, nobody will ever know how large the vocabulary of Schiller’s works might have become had he reached the same old age as Goethe in the end…

If we turn away from the low frequencies now and have another look at the high ones, we get hold of another interesting phenomenon. Among the most frequently used signs by Schiller are question- and exclamation marks. They appear at ranks 4 (exclamation mark) and 8 (question mark). In the Goethe corpus they only make it to ranks 11 (exclamation mark) and 27 (question mark). A look at the distribution of exclamation and question marks in our corpora shows that the distribution of exclamation marks ist always higher than the one of question marks in the Goethe corpus. Schiller on the other side has written five texts in which there are more question marks than exclamation marks. Is it maybe that Schiller’s authorial signature is characterized by an unusual amount of questions? Especially his play “Don Carlos. Infant von Spanien” is characterized by question marks, as you can see in the CATMA small multiples view below:

Small multiples view of distributions of question- and exclamation marks in the works of Schiller

Our case study so far has applied some handy distant reading functions, which are often used in one way or another in stylistics. But CATMA offers also a method for what we call scalable reading. Starting off with distant reading as we did, you may now scale down a bit: Simple double clicking on one point in the distribution graph or one keyword in the keyword in context table will take you to the specific position in the text where you find either the keyword or the accumulation of a word or tag you found in your small multiple distribution graphs. So, you can go from corpus to multiple texts in one visualization, on to single text view and even to the very position at which you find one word in one text – changing dynamically from distant to close reading and back just as you wish. This scalability also allows to develop different kinds of interpretation from data analysis to more context-oriented interpretation of certain passages in single texts.

I leave you here to start your own case study now, be it on the style of Goethe and Schiller or other literary phenomena. But just before you do that, do know that there are more functions for working on corpora in CATMA, among which are automatic annotations of part of speech (POS) tagging, as well as that of verbal tense and temporal signals (in German texts).

And of course you can also use the central CATMA function, which is annotating one text, or a whole corpus, with your very own categories, analyzing and visualizing them. CATMA ist running as version 5.0 right now, but in 2019, CATMA 6.0 will be launched. Users can look forward to the new design of the graphical user interface, the optimization of workflows and the addition of some features. 

As we focused on the corpus-specific functions in CATMA in this article, you might want to have a look at our tutorials for a complete overview of the application’s functionalities: http://catma.de/documentation/tutorials/

Have fun annotating, exploring, and scaling!

Harry Potter, computational fun and sexy gains

About the workshop on the Recreation of Harry Potter, endorsed by the SIG-DLS at DH2018 on 25 June 2018, led by Mike Kestemont and Enrique Manjavacas; and developed in cooperation with Greta Franzini and Marco Büchler, part of the 2018 Digital Humanities conference in México City

By Corina Koolen

Harry Potter novels, not one of the first topics one might think about when performing stylometry. But, as Mike Kestemont argues, popular literature is one of the unrightfully overlooked areas of digital literary studies, which still often focuses on the classics. With the workshop run by him and Enrique Manjavacas (many thanks for all your help!) during the international Digital Humanities Conference 2018 in México City, they show us exactly why this is undeserved. I am impressed by the setup chosen, which combines computational analyses with thoughtful reflection on a number of humanistic issues and interests.

One of them is the legal aspect of researching contemporary novels. After a discussion on the differences per country, where the basic conclusion is that there it is very hard to determine what exactly is legal in which country, Kestemont remarks: “We always talk about the author’s rights. I believe I have rights, too, as a researcher.” I could not agree more, this is going to be ammunition for legal discussions in the future.

Then the stylometry. Generally, stylometrists are known for performing authorship recognition. J.K. Rowling herself found out about the discipline when she was unmasked by Patrick Juola as the real author behind Robert Galbraith. There are other cool things to do, however, through stylometry. And this is where it becomes interesting for me as a researcher of popular literature: we are going to look at stylistic similarity between the novels and HP fan fiction. There are two cases that we will test, both of which follow the original structure of the novels. The first case is Aidan Chase. This fan fiction author, of whom little biographical information is available, reframed the originals to create a story world where Harry’s parents never died, but stayed true to the main story line. The second is Norman G. Lippert. He created new novels, based on the original characters, because his children were so disappointed that the series had ended. And indeed, the tools we apply show that Lippert deviates more from the originals than Chase. By using text-matcher by Jonathan Reeve, a not-yet-completely-polished but impressive tool nonetheless, in combination with the visualisation tool Bokeh, it is possible to visualize the overlap (see image); including the opportunity to make a line-by-line comparison of the sentences that have similar word usage. Chase is proven to copy large parts of the originals literally; showing how we can also apply stylometric methods to examine intertextuality, including reuse of materials.

Visualization in Bokeh - credit: Christof Schöch

Bokeh visualizes the overlap between two texts – size and location in the text. Credit: Christof Schöch

It gets even more interesting when we start to examine a larger body of Harry Potter fan fiction. Kestemont uses the database Archive of our Own to mine metadata and texts of fan fiction novels. This gives the researcher information that is never as easy accessible as it is here: who are the main characters, who have relationships, what are the fandoms that the author chooses to include — fan fiction authors provide this meta information because potential readers can use it to more easily to select fiction on the relationships and fandoms they prefer. The creativity is astounding: Lord of the Rings combined with Harry Potter is not as rare as I had expected it to be; and there are TV shows being crossed with Harry Potter that no one in the room has ever heard of and appear to not even be decently Google-able.

When we dive into the contents of these fan fiction stories, of course, there is sex. Lots of sex. (I for one never thought I would hear the term ‘elf porn’ in an academic context.) A computational topic model of fan fiction versus the original novels shows that topics specific to fan fiction are pornography, transportation and modern technology — that last one, interestingly, is also a topic The Riddle project team found as more typical (pdf) of ‘popular fiction’ as opposed to ‘literary’ fiction. But apart from the giggles the porn topic generates, it also shows something about how readers engage with characters. ‘Slash’ is, as fanfic researchers know, an important genre within fan fiction. Central characters, especially male ones, are paired in a romantic and often pornographic relationship. One of the first pairings was Kirk/Spock; the ‘/’ sign gives slash its name. Kestemont focuses on how attention to certain characters deviates from the original novels. Draco Malfoy and Severus Snape for instance, are much more present in fan fiction than in the Harry Potter novels, whereas Ron Weasley has the opposite effect. As Kestemont states: it gives us the possibility to research reception in a different way, to see how readers/writers engage with the original materials and characters.

That, I would stress, is an important academic outcome of this workshop, but I would like to end on another: fun. What always strikes me about Mike Kestemont’s work, is the joy he appears to get from his materials, the humour he brings to it. As he stresses, this is enhanced by working with bright and motivated colleagues such as co-tutor Enrique Manjavacas. But it is also partially explained by the type of material. As popular fiction is as much part of our cultural heritage as Great Literature is, this serves a dual purpose. First, we get a better view of fiction in general. Second, with that fun and humour comes creativity —  something we could use a little bit more of every now and then. Because that, I think, is where the magic happens. Bombarda Maxima!

Differing appreciation for male and female writers

Male authors are in general appreciated more than females, and Corina Koolen, working at the Huygens Institute, applies stylometry to make plausible that this greater appreciation is not based on obective elements of style or plot, but presumably rather on the prejudices of readers, critics and juries in literary prize competitions.  I think that this is a new application of stylometry — even allowing for the work in author profiling, which concerns differences in style without attendingto the question of quality.  Koolen also shows, for example, that genre influences the descriptions of men and women more than the gender of the author does.

There’s a sympathetic review (in Dutch) of her dissertation in the one
of the leading Dutch newspapers, the NRC:
The thesis itself is in English and is distributed by the University of Amsterdam:

John Nerbonne, Freiburg

A missing president?

Analysis of The President is Missing, by James O’Sullivan of University College Cork.

The answer is quite conclusive: ‘The accompanying graph represents the novel on the x-axis, broken into segments: the thicker the bottom line, the more certain the proximity to the relevant author’s style. Considering Patterson’s fingerprint is represented by green, it is plain to see that, contrary to our previous study, this is a co-authored novel in which he was the scribe.’

Of course, as O’Sullivan points out himself, we need to stay wary where the genre signal comes in to make life complicated (no fictional writing of Clinton’s could be used for the study, simply because there isn’t any). However, there is good reason to assume that the authorial signal here trumps the genre signal.

Finally, O’Sullivan points out another important dimension of that particular collaboration: that of the market. “What better way to sell a book, than to have a mogul of commuter fiction combine with a former US president?”

Interestingly, according to both Lane and O’Sullivan, it is the former US president who does raise his voice towards the end of the novel, contributing a finishing touch, or rather, a finishing strike to the tale:

“…there’s the chutzpah with which Clinton (Patterson, I would suggest, may have stepped aside at this stage) waits until the twilight of the novel and then, like Tolstoy, squares his shoulders and expounds, in fiction-free form, his politico-historical thoughts.” (Lane)

… and this is what it sounds like:

“I want the United States to be free and prosperous, peaceful and secure, and constantly improving for all generations to come.”

I’d like to conclude that the current state of world history is clearly up for political debate, while the novel is naturally subject of an aesthetic one. Meanwhile, O’Sullivan’s article is a great stylometry story –  and we look foward to more of those.

Distant Reading for European Literary History

A few months ago, our COST Action Distant Reading for European Literary History (CA16204) was approved and started running. Even more recently, our Working Group ‘Scholarly Resources’, aka its corpus backbone, met in Prague, for the first time! We are very excited about getting started and would like to briefly report on our work in this blog.

The COST Action Distant Reading has the goal to “develop the resources and methods necessary to change the way European literary history is written” (Memorandum of Understanding). To approach this goal, the Action’s Working Group ‘Scholarly Resources’ aims to create a big open source benchmark corpus of novels from 1850-1920, called ELTeC (European Literary Text Collection). During the course of the Action, we will examine ELTeC with different computational distant reading methods such as topic modelling, authorship attribution, network analysis, stylistic analysis, and different types of character analysis.

The Action’s Working Group ‘Scholarly Resources’ will coordinate the task of creating the ELTeC; It consists of members from all over Europe, from literary studies, computational and corpus lingustics, and from information science. The COST Action is a great opportunity to collaboratively work with researchers and bring together expertise for different languages as well as computational methods.

A challenging task has been defining the corpus selection criteria, a common annotation model and potential workflows for corpus creation for the ELTeC which can be applied to European novels from different languages as well as various publication contexts.

During our first Working Group meeting, we developed corpus selection and balancing criteria which follow a simple but consistent corpus design approach and represent the variety of novels in this period. Among other things, we do not want to solely rely on normative canon-based selection criteria and set a bias to our understanding of novels. The focus of the ELTeC encoding scheme is to uniformly represent historical novels from different languages with a basic TEI mark-up. The standard markup is necessary for applying different types of distant reading methods rather than representing the original text structure. ELTeC and the encoding scheme will be freely available at our GitHub Organization.

In this way, we would like to contribute a big chunk to the creation of a digital basis for cross-national and cross-language analysis of European literature!

Drawing Elena Ferrante’s Profile: International Workshop at University of Padova

Dear DLS-SIGroupies (is that what we are?),

Some of us took part this month in a very interesting workshop in Padova, Italy, part of the International Quantitative Linguistics Associations‘ biennial summer school, organized by our dear friend Arjuna Tuzzi from Padova University’s Gruppo Interdisciplinare di Analisi Testuale. It was devoted to solving Italy’s current greatest authorship riddle: who is hiding behind the pseudonym of Elena Ferrante, best-selling author of, among others, L’amica geniale. 

In fact I have no idea why I’m blogging about this, since the whole event was covered by “La Repubblica” (so nice to see that there are still countries in this world where literature is taken seriously by mainstream meadia) and the local press. They all published relatively sensible accounts of the event, also with diagrams; and a photo of some of the participants (I don’t want to brag, but…) made it into Greek press as well! Better than that: you can still (I think) watch the proceedings on Livestream!

Still… The speakers were either already members of our SIG, or will be soon, or are our close cousins from Quantitative Linguistics. Arjuna Tuzzi and Michele Cortelazzo (University of Padova), Jacques Savoy (University of Neuchâtel), Jan Rybicki (Jagiellonian University), Maciej Eder (Polish Academy of Sciences), Patrick Juola (Duquesne University), George K. Mikros (National and Kapodistrian University), Pierre Ratinaud (University of Toulouse II) all agreed that stylometric evidence overwhelmingly points to Italian writer Domenico Starnone (he has always been one of the main suspects anyway) rather than his wife, translator Anita Raja; yet her participation of some sort cannot be ruled out. This is very uncharacteristic agreement among people who use different methods and come from different backgrounds; and suggests this is a result that should be reckoned with.

I think this event was noteworthy for two things: first of all, DLS is becoming more and more visible in the media, but there is still a long way to go. Second, it ended in a spontaneous discussion on the ethics of the whole thing. Do anonymous writers have a right to privacy, especially when they’re (probably) alive? Starnone’s and Raja’s privacy had already been trampled upon by journalists who claim to have traced Ferrante’s royalties. Should they not be left alone?

My own answer is the following: I don’t really care who wrote which book, any book, and who took the money, as long as I learn something interesting about the process of literary creation. I think I did this time. While it is quite plausible that both Raja and her husband are in it together, the stylometric fingerprint is that of the latter. If it’s a collaboration, then this is a very valuable insight that might lead us further on: what is, if any, the “silent partner’s” contribution? In what way might Raja’s work as translator of Christa Wolf contribute to the Ferrante phenomenon? If they are, or one of them is, Ferrante, are they becoming Ferrante themselves?