Chapter 9 Assessment

9.1 Literature Reviews

A literature review is intended to communicate to the reader the academic importance of a research problem. For the highest grades in a literature review, your work will either:

• Demonstrate the originality and importance of a question to which there is currently no answer in existing work on the subject, or
• Provide an original critique of academic work on an existing question

In both cases, you are not expected to have an answer to the question yourself!

It is important to distinguish between the process of writing a literature review, and the end product. The end product (e.g. what you submit for assessment) is a 2500 word piece of work that should enable an intelligent but uninformed reader to understand the importance of a research problem, its academic importance, and the key academic debates that constitute current enquiry into the subject. This means that you will have to make a number of design decisions, notably which debates and authors to include, and those to exclude, which of those included are central, and those that can be relegated to a footnote.

A literature review in the sense of the product presented for assessment is slightly artificial. Usually literature reviews are integrated into research articles. To get an understanding of how this assessment fits within general academic work, read key journals in the field that you are working. Usually, in something like Security Studies or similar, an author will start with an introduction to a problem or issue, and then situate that issue within existing academic work on the topic, and in the process identifying a key question to answer.5454 Here are some good examples of this:@@ They’ll then go on to provide a reasoned method for answering the question, and answer it. What we’re focused upon in this assessment is the first two steps.

You should title your literature review as a question. For example:

• Why do historians disagree about the starting point of the military revolution in Europe?
• Does technological determinism persist in military innovation?
• How are generations of military planes constructed?

If you are stuck for something to write about, a good formula for generating potential topics is to do some preliminary research. Ask yourself “How have X analysed Y?” where X = self-selected members of an academic discipline,5555 Historians, strategists, political theorists, etc. and Y = a case study (conflict)5656 For your own benefit, try to avoid those used as case studies on the course, it’s better to use this to expand your knowledge into a new area. or an element of a case study (important event/debate),5757 In the context of this course, there are no shortage of key events. Often a single, infamous, war crime forms a cornerstone for ongoing discussions about key theoretical questions. or disciplinary tool (ticking time bomb scenario, key theoretical discussion relevant to the course).

After you have found something that looks interesting, ask yourself “Why is that important?” in the sense that you should be primarily focused upon academic importance in this assessment. Policy relevance is optional.5858 Outside universities this is likely to be the other way around, but you paid to take an academic course. Lastly, you should be keeping in mind “Is there something important that they have missed?” because this last question is where you will find the critical engagement/originality elements that I mentioned at the outset.

You are free to stick within a single discipline, but sometimes it is interesting to compare the approaches of two disciplines to the same topic. In the end, pick a topic that interests you, and that has some demonstrable academic importance. You don’t get extra marks for picking a cutting-edge or vitally important question, but without demonstrable academic importance, it is hard to score high marks in this assessment.

9.2 Research Essay

If the prospect of a 5000 word research essay worries you, please don’t panic - there are effectively 10 teaching sessions to support you towards this in term 2. The basic structure of a research essay is similar to that of a research article that you will find in an academic journal, but it is likely to be shorter (most academic articles are 7000-9000 words). In short, you will need an introduction, to explain your research question, explain how and why you’re going to answer it in a given way, and then provide an answer.

There are four general components for a successful research essay: Identifying a research area, identifying an interesting research puzzle, constructing a theoretical framework, and posing an answerable research question. We will be covering this in detail in the lecture series in term 2. Identifying a research area is much the same as what you do in a literature review.

Identifying research puzzles is important, because they are a good way to sharpen your thinking, and to avoid research questions with obvious answers (which means it is hard to develop original engagement with the topic). As proposed by Karl Gustafsson and Linus Hagström, research puzzles can be framed in this way:

‘Why x despite y?’, or ‘How did x become possible despite y?’3 A puzzle thus formulated is admittedly a research question, but one requiring much closer familiarity with the state of the art than a ‘why x-question’. The researcher considers the phenomenon x puzzling since it happens despite y – that is, previous knowledge that would seem contradicted by its occurrence.5959 Gustafsson and Hagström (2018)

However a good research puzzle might not be answerable. This is a big problem for a 5000 word essay - you don’t necessarily have the space to engage at depth with some kinds of questions. One important problem is too much novelty. Here I will borrow from Michael Horowitz, a professor at the University of Pennsylvania.6060 His twitter handle is mchorowitz Horowitz had a great piece of advice for choosing PhD dissertation topics that I think is also applicable to graduate-level research in general. In essence: either pick a new body of theory to analyse a pre-existing case study or substantive issue, or use pre-existing theory to analyse a new case study or substantive issue. Old theory/old case is unlikely to get you anywhere interesting, and (particularly with 5000 words) attempting to explain a new body of theory and apply it to a new case study for which there isn’t much agreed evidence is the equivalent of a moonshot. Horowitz frames this as “High risk/high reward”, here I frame it as a unicorn, because at 5000 words successful examples are pretty much figments of the imagination.

Where a 5000 word essay extends on a literature review is that you are then expected to answer the question. This means that you will need to construct a theoretical framework. As above, you can pick old or new theory, but a good theoretical framework for answering a research question usually involves two competing theories or explanations, which can be used to evaluate evidence or explain events. Here it’s good to research to the point where you can identify key competing explanations/authors, prior to selecting a couple to use in your essay. An important consideration here is the existence of prior work. If there is no prior work in the area, then you are going to have a really tough time. If a theory or argument is so left-field that it doesn’t really connect to existing academic research, how are you going to be able to make those necessary connections and answer the question in 5000 words? Similarly, if the case study that you want to examine has very little written about it by reputable authors, how are you going to establish the facts of the case within the word limit? My advice is that you pick a research puzzle where there are plenty of related pre-existing disagreements, or one that sits at the intersection of two fields/disciplines.

The last step is to consider what kind of question can be answered in 5000 words. This is primarily an issue of scoping questions. Set questions are often quite broad or vague, because part of the art of answering a set question essay is to be able to re-scope the question to something answerable in the introduction. Bear in mind when reading around for suitable questions that you are not assessed upon your ability to produce work comparable to people with a minimum of 3-5 years of professional training, but you are assessed on your ability to select a question that can be answered within 5000 words without substantial original research. To navigate this, let us turn to Greek mythology.6161 Bet you weren’t expecting that line.

Per Wikipedia:

Scylla and Charybdis were mythical sea monsters noted by Homer; Greek mythology sited them on opposite sides of the Strait of Messina between Sicily and the Italian mainland. Scylla was rationalized as a rock shoal (described as a six-headed sea monster) on the Italian side of the strait and Charybdis was a whirlpool off the coast of Sicily. They were regarded as maritime hazards located close enough to each other that they posed an inescapable threat to passing sailors; avoiding Charybdis meant passing too close to Scylla and vice versa. According to Homer, Odysseus was forced to choose which monster to confront while passing through the strait; he opted to pass by Scylla and lose only a few sailors, rather than risk the loss of his entire ship in the whirlpool.

You face two dilemmas in scoping your research question. First, whether the answer to the question is either too obvious, or frankly impossible. Second, whether the argument required to answer the question is simple, or obscenely complex. By “complex” I mean that it involves far too many factors to be able to pull them all together in a coherent manner. Per Homer, I suggest that you err on the side of difficulty and complexity, but not too much.

To give some explanation, let’s say I want to write a 5000 word essay about the design of nuclear command and control systems. I pose the following question:

Did culture affect the design of nuclear command and control systems?

The problem with the question is that it’s quite clear the answer is yes. When we look at the history of nuclear command and control systems, we find lots and lots of different systems in place. The US President carries around a nuclear “football” allowing them to issue orders anytime, anywhere, while the British Prime minister writes a set of letters that go into a safe in a submarine. The Soviets tried out a “dead man’s hand” system to automate nuclear apocalypse in the event of incoming ICBMs. Fundamentally, the question as framed is so general that the answer is likely to be straightforward. A question at this level of abstraction is going to produce an answer full of generalities. Okay, attempt 2:

How did culture affect the design of nuclear command and control systems?

Okay, now we’ve gone in completely the other direction. The scope of this question is such that we’re now trying to explain how culture might affect the design of nuclear command and control systems across 8+1 nuclear weapon states.6262 Israel is widely suspected to have nukes, but decided being a real-life Schrodinger’s Cat was better than publicly confirming this. How many different ways did culture influence the command and control setups of nine different states? I don’t know. Maybe you could answer this question in a book, but it is far beyond the scope of a 5000 word essay. Oh, and the question as posed would also have to account for the influence of culture in the evolution of all of these systems. Again, you could answer this in a broad brush fashion, but not really in the detail you’d need to say something original on the topic. Time for round 3:

How did security cultures affect the design of American nuclear command and control systems?

Better, but still not perfect. In contrast to the previous question, we’ve now rescoped from nine countries to one, and cut down “culture” into “security culture” - sets of cultural and institutional ideas dealing with security. However this is still too complex. The American nuclear deterrent consists of a huge variety of overlapping systems, embedded in a variety of institutional context. The question as it stands would force you to consider an extremely wide range of cases to try to provide an answer. Onto round 4:

How did security culture affect the design of Permissive Action Links on American nuclear warheads in Europe?

This is far, far, more answerable than the previous questions. This might seem strange - the question itself is a lot longer than the one we started with. At the same time, the question is dealing with a very specific issue - Permissive Action Links that prevent the unauthorised use or detonation of a nuclear warhead - and a specific context to focus upon. It might scare you to learn that the US had nuclear weapons stationed in Europe outside their full control, protected in some cases with a mechanical lock, but looking at the transition to computerised locks is an interesting case study. This kind of question is the ‘sweet spot’ for a 5000 word essay, but please don’t feel that you have to write on this topic, or even from a historical perspective - this is just here for an example.

Okay, so once you have a question, then you have to answer it. See the previous chapter for advice on this. But again, we’ll be talking about constructing research projects in detail during term 2.