9

Developing a Critical Analysis of a Text

Keywords

Critical Analysis exercise; Critical Analysis Questions

This chapter focuses on how you can use your mental map in developing an in-depth analysis of any text from the front-line literature. The framework we put forward and exemplify in use is an elaboration of the Critical Summary based on the five Critical Synopsis Questions that you met in Part One. Completing a Critical Analysis of a text takes a lot of effort. But you will reap some very valuable rewards if you make that effort for the texts that are of most central significance for your work. First, you will get to know the texts extremely well and will have quite comprehensively evaluated them. Second, you will have assembled, in a structured format, the basis for writing an incisive Critical Review of each text individually, or a Comparative Critical Review of multiple texts (to be discussed in Chapter 11). Most importantly, the more Critical Analyses you do, the more familiar you will become with the key and components of your mental map, and with the Critical Analysis Questions that can be asked of a text. Eventually, using the map and asking the Critical Analysis Questions will become automatic. Then you will be in a position to use your mental map and Critical Analysis Questions selectively, without necessarily having to check whether you have forgotten to ask any questions, or needing to write your responses down.

We now introduce our structured approach for undertaking a Critical Analysis of a text. At the end of the chapter, once you have read through these ideas, we invite you to conduct your own full Critical Analysis of Wallace’s article in Appendix 2, referring as you go along to the various sources of guidance we have provided. (In the following chapter, we will offer our own Critical Analysis along with comments on our reasoning at each step, so that you can compare your responses with ours.)

From five Critical Synopsis Questions to ten Critical Analysis Questions

The five Critical Synopsis Questions introduced in Chapter 3 encouraged you to:

• think why you are investing your time in reading a particular text;
• get a sense of what the authors have done to convince their target audience;
• summarize what they have to say that is of relevance to you;
• consider how convincing their account is; and
• draw a conclusion about how you might use the text for your purposes, in the light of its content and your evaluation of the authors’ argument.

The ten Critical Analysis Questions do the same job (Table 9.1), but in more detail. The first expansion, in Critical Analysis Questions 2 and 3, helps you analyse what the authors are doing (and so alerts you to potential limitations of their work that might affect how convincing you find their claims). The second expansion, in Critical Analysis Questions 5 to 9, helps you evaluate the claims in a more sophisticated way.

We will presently introduce a form that is completed as part of the process of conducting the structured Critical Analysis. The form contains ideas to guide your critical thinking at three levels:

1. The Critical Analysis Questions, numbered 1–10, to ask yourself when reading and analysing a text.
2. For most of these Critical Analysis Questions, one or more sub-questions, lettered (a), (b) and so on, that help to highlight aspects of the question.
3. Prompts, enclosed in brackets, to draw your attention to possible details you could look out for in working towards your answer to any Critical Analysis Question or sub-question.

We suggest you carry out your Critical Analysis at the same time as you read a text, rather than afterwards. The Critical Analysis Questions are grouped to form a sequence:

Table 9.1 Linking Critical Synopsis Questions with Critical Analysis Questions

Critical Synopsis Question

Associated Critical Analysis Question(s)

A  Why am I reading this?

1  What review question am I asking of this text?

B  What are the authors trying to do in writing this?

2  What type of literature is this?

3  What sort of intellectual project is being undertaken?

C  What are the authors saying that is ? relevant to what I want to find out?

4  What is being claimed that is relevant to answering my review question?

D  How convincing is what the authors are saying?

5  To what extent is there backing for claims?

6  How adequately does any theoretical orientation support claims?

7  To what extent does any value stance affect claims?

8  To what extent are claims supported or challenged by others’ work?

9  To what extent are claims consistent with my experience?

E  In conclusion, what use can I make of this?

10  What is my summary evaluation of the text in relation to my review question?

• Critical Analysis Question 1 encourages you to think about why you have selected the text and how your Critical Analysis of it may contribute to your enquiry.
• Critical Analysis Questions 2 and 3 guide you in determining what the authors are attempting to do and alert you to potentially fruitful lines of critical questioning.
• Critical Analysis Question 4 encourages you to summarize whatever content of the text is of significance to you.
• Critical Analysis Questions 5, 6, 7, 8 and 9 are complementary. Together they help you to examine critically different aspects of this content to see to what extent you find it convincing.
• Critical Analysis Question 10 invites you to form a conclusion, in the light of your Critical Analysis, based on your informed judgement about the extent to which any claims relating to the focus of your enquiry are convincing, and why.

Below, we set out all the Critical Analysis Questions, sub-questions and prompts in the order that they appear on the blank Critical Analysis form. Beneath each of the ten Critical Analysis Questions, we have offered our rationale (shaded) for why we consider it important to ask this question of the text.

We suggest you now read carefully through the explanations, checking that you understand the rationale for each Critical Analysis Question.

Advice on making effective use of Critical Analysis Questions

1  What review question am I asking of this text?

(e.g., What is my central question? Why select this text? Does the Critical Analysis of this text fit into my investigation with a wider focus? What is my constructive purpose in undertaking a Critical Analysis of this text?)

Rationale for Critical Analysis Question 1. It is crucial to begin by identifying a review question. In an essay, this question may map onto a central question, while in a longer piece of work it will probably reflect one aspect of the central question. The review question provides you with a rationale for selecting a particular text and a constructive purpose for reading it critically. Any text you select should potentially contribute to addressing your review question.

2  What type of literature is this?

(e.g., Theoretical, research, practice, policy? Are there links with other types of literature?)

Rationale for Critical Analysis Question 2. Identifying the main type of literature that the text belongs to will help you to predict what its features are likely to be. The type of literature will indicate the main kind of knowledge embodied in any claim, enabling you to check whether typical limitations of claims to this kind of knowledge may apply. (See the section in Chapter 8 on types of literature, including Table 8.1.)

3   What sort of intellectual project for study is being undertaken?

Rationale for Critical Analysis Question 3. Establishing the authors’ intellectual project will clue you in to what they are trying to achieve, why and how. You will be aware of whom they are seeking to convince of their argument and associated claims to knowledge. You will then be in a good position to evaluate what they have done. (See the section in Chapter 8 on different sorts of intellectual project, including Table 8.2.)

Sub-questions

(a)  How clear is it which intellectual project the authors are undertaking? (i.e., Knowledge-for-understanding, knowledge-for-critical evaluation, knowledge-for-action, instrumentalism, reflexive action?)

(b)  How is the intellectual project reflected in the authors’ mode of working? (e.g., A social science or a practical orientation? Choice of methodology and methods? An interest in understanding or in improving practice?)

(c)  What value stance is adopted towards the practice or policy investigated? (e.g., Relatively impartial, critical, positive, unclear? What assumptions are made about the possibility of improvement? Whose practice or policy is the focus of interest?)

(d)  How does the sort of intellectual project being undertaken affect the research questions addressed? (e.g., Investigation of what happens? What is wrong? How well a particular policy or intervention works in practice?)

(e)  How does the sort of intellectual project being undertaken affect the place of theory? (e.g., Is the Investigation informed by theory? Generating theory? Atheoretical? Developing social science theory or a practical theory?)

(f)  How does the authors’ target audience affect the reporting of research? (e.g., Do the authors assume academic knowledge of methods? Criticize policy? Offer recommendations for action?)

4  What is being claimed that is relevant to answering my review question?

Rationale for Critical Analysis Question 4. As a basis for considering whether what the authors have written is convincing, you will need to identify any argument that they are putting forward in the text and establish what main claims to particular kinds of knowledge underlie it. Concentrate on identifying a small number of major ideas by summarizing the content of the text. Try to avoid getting distracted by minor details. (See the section in Chapter 8 on kinds of knowledge, including Figure 8.1.) As further preparation for a critical consideration of the authors’ claims, it is helpful to work out the degree of certainty with which any knowledge claim is asserted and the degree to which the authors generalize beyond the context from which the claim to knowledge was derived. (See the section in Chapter 7 on dimensions of variation among knowledge claims, including Figure 7.1.)

Sub-questions

(a)  What are the main kinds of knowledge claim that the authors are making? (e.g., Theoretical knowledge, research knowledge, practice knowledge?)

(b)  What is the content of each of the main claims to knowledge and of the overall argument? (e.g., What, in a sentence, is being argued? What are the three to five most significant claims that encompass much of the detail? Are there key prescriptions for improving policy or practice?)

(c)  How clear are the authors’ claims and overall argument? (e.g., Stated in an abstract, introduction or conclusion? Unclear?)

(d)  With what degree of certainty do the authors make their claims? (e.g., Do they indicate tentativeness? Qualify their claims by acknowledging limitations of their evidence? Acknowledge others’ counter-evidence? Acknowledge that the situation may have changed since data collection?)

(e)  How generalized are the authors’ claims – to what range of phenomena are they claimed to apply? (e.g., The specific context from which the claims were derived? Other similar contexts? A national system? A culture? Universal? Is the degree of generalization implicit? Unspecified?)

(f)  How consistent are the authors’ claims with each other? (e.g., Do all claims fit together in supporting an argument? Do any claims contradict each other?)

5  To what extent is there backing for claims?

Rationale for Critical Analysis Question 5. It is important to check the extent to which the main claims to knowledge upon which any argument rests are sufficiently well supported to convince you, whether through evidence provided by the authors or through other sources of backing. (See the section in Chapter 7 on dimensions of variation amongst knowledge claims, including Figure 7.1, and the section in Chapter 8 on types of literature, including the potential limitations of claims to knowledge listed in Table 8.1.)

Sub-questions

(a)  How transparent are any sources used to back the claims? (e.g., Is there any statement of the basis for assertions? Are sources unspecified?)

(b)  What, if any, range of sources is used to back the claims? (e.g., First-hand experience? The authors’ own practice knowledge or research? Literature about others’ practice knowledge or research? Literature about reviews of practice knowledge or research? Literature about others’ polemic? Is the range of sources adequate?)

(c)  If claims are at least partly based on the authors’ own research, how robust is the evidence? (e.g., Are there methodological limitations or flaws in the methods employed? Do the methods include cross-checking or ‘triangulation’ of accounts? What is the sample size and is it large enough to support the claims being made? Is there an adequately detailed account of data collection and analysis? Is there a summary of all data that is reported?)

(d)  Are sources of backing for claims consistent with the degree of certainty and the degree of generalization? (e.g., Is there sufficient evidence to support claims made with a high degree of certainty? Is there sufficient evidence from other contexts to support claims entailing extensive generalization?)

6  How adequately does any theoretical orientation support claims?

Rationale for Critical Analysis Question 6. Any text must employ certain concepts to make sense of whatever aspect of the social world is being discussed. Many texts will feature an explicit theoretical orientation as a framework for understanding and possibly as a basis for the authors’ recommendations for improvement. You will need to decide whether the claims being made are clear and coherent, and whether you accept the assumptions on which they rest. To assist your critical reflection, check which concepts and other tools for thinking have been used, what they are taken to mean and how they frame the claims being made. (See the section in Chapter 6 on tools for thinking, the section in Chapter 8 on types of literature, including the potential limitations of claims to knowledge listed in Table 8.1, and the section on different sorts of intellectual project, including Table 8.2.)

Sub-questions

(a)  How explicit are the authors about any theoretical orientation or conceptual framework? (e.g., Is there a conceptual framework guiding the data collection? Is a conceptual framework selected after the data collection to guide analysis? Is there a largely implicit theoretical orientation?)

(b)  What assumptions does any explicit or implicit theoretical orientation make that may affect the authors’ claims? (e.g., Does a particular perspective focus attention on some aspects and under-emphasize others? If more than one perspective is used, how coherently do the different perspectives relate to each other?)

(c)  What are the key concepts underpinning any explicit or implicit theoretical orientation? (e.g., Are they listed? Are they stipulatively defined? Are concepts mutually compatible? Is the use of concepts consistent? Is the use of concepts congruent with others’ use of the same concepts?)

7  To what extent does any value stance adopted affect claims?

Rationale for Critical Analysis Question 7. Since no investigation of the social world can be completely value-free, all claims to knowledge will reflect the value stance that has been adopted. So it is important to check what values have guided the authors of a text, how these values affect their claims and the extent to which the value stance makes the claims more or less convincing. (See the section in Chapter 6 on tools for thinking, the section in Chapter 8 on types of literature, including the potential limitations of claims to knowledge listed in Table 8.1, and the section on different sorts of intellectual project, including Table 8.2.)

Sub-questions

(a)  How explicit are the authors about any value stance connected with the phenomena? (e.g., A relatively impartial, critical or positive stance? Is this stance informed by a particular ideology? Is it adopted before or after data collection?)

(b)  How might any explicit or implicit value stance adopted by the authors be affecting their claims? (e.g., Have they pre-judged the phenomena discussed? Are they biased? Is it legitimate for the authors to adopt their particular value stance? Have they over-emphasized some aspects of the phenomenon while under-emphasizing others?)

8  To what extent are claims supported or challenged by others’ work?

Rationale for Critical Analysis Question 8. It is unlikely that any study of an aspect of the social world will be wholly unrelated to others’ work. One valuable check is therefore to examine whether authors make links with other studies. Another is to consider, from your knowledge of other literature, how far the claims being made are supported by work that others have done. So you may wish to refer to other texts that address phenomena related to the text you are analysing.

Sub-questions

(a)  Do the authors relate their claims to others’ work? (e.g., Do the authors refer to others’ published evidence, theoretical orientations or value stances to support their claims? Do they acknowledge others’ counter-evidence?)

(b)  If the authors use evidence from others’ work to support their claims, how robust is it? (e.g., As for 5(c).)

(c)  Is there any evidence from others’ work that challenges the authors’ claims and, if so, how robust is it? (e.g., Is there relevant research or practice literature? Check any as for 5(c).)

9  To what extent are claims consistent with my experience?

Rationale for Critical Analysis Question 9. Your own experience of the social world will probably not be identical to that being studied in the text but it is still relevant. In considering how convincing the claims made in a text may be, it is worth checking whether these claims have significant similarities with your experience and evaluating whether they sound feasible or unrealistic, given what you know from experience.

10  What is my summary evaluation of the text in relation to my review question?

Rationale for Critical Analysis Question 10. What you have learned from your answers to Critical Analysis Questions 2–9 provides the basis for your overall, well-informed and balanced judgement about how convincing are the claims being made that relate to your review question (Critical Analysis Question 1). All your answers will now be available for you to draw upon selectively as you write an account of the text when addressing the review question that has driven your critical reading activity.

Sub-questions

(a)  How convincing are the authors’ claims and why?

(b)  How, if at all, could the authors have provided stronger backing for their claims?

Appendix 3 is a blank Critical Analysis form. You may wish to photocopy it and then complete one form for each text that you analyse in detail. If you have access to a computer, you may prefer to create a master file by typing in the content of the blank form, then using it as a template. (You can also download a Critical Analysis template from the SAGE website: www.sagepub.co.uk/wallaceandwray.) You will find it useful to save each completed Critical Analysis form as a separate file on your computer. Computerizing the form enables you to write as much as you like in answering each question. If you print out a completed Critical Analysis form, keep it with the original text if possible. Then you can quickly refer back to the text if necessary.

Your Critical Analysis of an article reporting research

To make the exercise work, we will specify the two review questions that you should ask of Wallace’s text. (We have done this so that you can compare your responses with ours, which we will provide in the next chapter.) The review questions are:

1. What does this text suggest may be key factors promoting or inhibiting the effectiveness of a particular aspect of educational leadership and management practice?
2. To what extent are the factors identified applicable to the leadership and management of my organization or one known to me?

Remember that you can refer, as necessary, to:

• the discussion in Chapters 6–8 relating to the key and components of your mental map;
• Table 8.1 for a list of potential limitations of each type of literature that you can look out for;
• the advice in this chapter on making effective use of each Critical Analysis Question.

(We have indicated above that knowledge of other relevant literature is needed to complete Critical Analysis Question 8, sub-question (c). However, if our example paper is not within your subject area, you do not need to refer to other texts in this exercise.)

Students embarking on a detailed Critical Analysis like this for the first time often encounter difficulties in finding answers to one or more questions, but it is important not to give up too soon. Always think carefully about how the text might, in fact, contain the information, perhaps implicitly, that you need. Expect to read the text with great attention in order to detect some of the indicators that you are looking for. Now complete your own Critical Analysis of Wallace’s article in Appendix 2 (for which you may wish to use the blank form in Appendix 3).

Once you have completed your Critical Analysis, turn to the next chapter. You will be able to check your responses to each Critical Analysis Question or sub-question against ours, to see what our rationale was for each of our responses and to decide whether you agree or not.