WU

Draft and test data collection tools

Write any data collection tools that you plan to use and validate them. For example, you might write a questionnaire, test it with a sample of respondents from the target population, and then revise it, going through the process until it works.

In your report, say exactly what you did and how you did it, and any modifications needed. It should show that they are now appropriate for your planned research. The report should be suitable for eventual inclusion in the methodology chapter of your dissertation. You also need to decide how much data will you need to get for your results to be valid.

Other than improving your tools, you should ask several otehr questions:

What kinds of other problems might you face collecting data? For example, if you use on-line forms, does the technology work as planned? Do people find it easy to use?
What is feasible and what is not? For example, consider your time-frame and budget. How much do you need to travel? If you need meeting rooms, are they available?
How effective were your methods of recruiting participants? Is it easy to get enough people?
Now that you have some data, how good are your processes for analyzing it?

Why you need to pre-test questionnaires

Even if a question is very well written, it does not mean that respondents will automatically understand it straight away. All questionnaires need testing with people before you use them widely. You might understand your questions perfectly, but you can't know how others might interpret them. Time spent in testing is well-spent. Imagine the frustration of having to start over from scratch because your questionnaire was found to have a serious flaw late in the fieldwork.

Even some very simple questions cause confusion. They are so simple that people don't take them on face value. They look like a trap or seem to ask for some kind of hidden, deeper answer. For example, How much is a one-dollar candy? The answer is so obvious that it can be frustrating, and it seems like you're fishing for another answer.

Expect to be surprised that some people will not understand even the most obvious (to you) of questions. Then, when you ask, they produce the most logical and obvious responses that had never occurred to you. Your questions could:

use unfamiliar terms
sound complicated
sound vague
sound foreign (if you're using another language)
encompass more than one issue
make people feel embarrassed, frustrated or stupid. For example, Have you stopped beating your wife? (Whether you answer yes or no, it still means that you have been beating your wife.)

Other causes of confusion might be:

unclear layout
innocent questions that are not intended to indicate confusion
re-reading things to figure out what the materials say
unexplained assumptions
language, literacy or numeracy difficulties
out-of-sequence explanations
not enough instructions on what to do
look like you are trying to trap them into giving a particular answer
confusing procedures
too much or too little information
incorrect information
boredom: lack of options
frustration: activities that are too difficult for the target population.

Questionnaires with open questions are much more flexible, especialy if you use them as a basis for interviews in which you can spontaneously add follow-up questions or talk people around in a circle. It might also be possible to let the questionnaire evolve during fieldwork so that you can follow up on emerging trends.

Questionnaires with closed questions, however, are very different. They need thorough, extensive testing because respondents must choose between the answers you provide. Consequently, your list of answers must allow all possibilities. Don't even think of using the questionnaire until you know it works very well.

Steps for testing questionnaires

Your goal is that respondents can read and understand each question straight away, and give an honest answer.

After proof-reading, test your first draft with colleagues to eliminate the most obvious glitches.

Then do your testing with persons of the target population who have not yet seen the materials nor been asked your questions in any other form. People who already know the subject matter or saw previous drafts of materials won't be tripped up as if they were doing the questionnaire from scratch.

Obviously, you will also ask for verbal feedback, but the non-verbal feedback can just as useful. If possible, observe them for non-verbal clues. If something is unclear, some will stop and re-read it. Others will be clearly frustrated. Others will just decide to skip that bit, and might come back to it later. Some might even give up and choose random answers.

If the topic causes embarrassment, you will need culturally-appropriate questions, and some indirect questions might work better. Some people will try to guess the answers you want. In any case, thorough testing will indicate the quality of your data.

Note how much extra help you give people. Even your unwitting use of body language or perceptible attitude may alter people's responses. Don't help people if they must be able to do the questionnaire with no help at all.

You should see how long it takes them. Respondents normally should be able to do a written survey questionnaire in 15-30 minutes. Some are less.

Then make corrections and test your questionnaire again on a new group of people from the target population who have never seen the questions before, and collate suggestions for improvement. You need to use different people each time, because they need to be doing the questionnaire for the first time; they can no longer give their first impression of the meaning of the questions. If necessary, you can repeat this kind of test more widely with other new groups of people from the target population.

Decide when to stop. You can keep improving forever, so stop when your questionnaire is good enough, that is, when the incoming suggestions are trivial and you are satisfied with it.

See also: https://www.youtube.com/watch?v=DRL4PF2u9XA

Gather first round of data

Gather your first round of data and present a report on it. With some kinds of methodologies, this will be a pilot project, that is, a small-scale project to test the metholodogy.

In your report, include:

a description of the amount and quality of data collected
the recording technique
an assessment of your methodology and any adaptations you needed to make
any threats to validity of your research
anything odd or unexpected, such as adverse incidents or counter-intuitive findings
any emerging trends
any required further research for the project to meet its research goals. (Some methodologies permit changes of direction partway through, while others disallow it.)

If you find deficiencies in your methodology, report any corrections or modifications you had to make or plan to make.

Note: If you are giving interviews, some supervisors recommend that you use two separate audio recorders and keep both running so you have a backup.

Midway review

The midway review is your supervisory committee's check on your progress so far. A positive assessment indicates that you are making good progress and are on track to complete. The committee may also decide to set any conditions or corrective actions that it believes are necessary for you to achieve satisfactory completion.

As you learn more about the topic, you might find that the question you started with is not the best question. You find that the real problem is not what you thought it was, despite being based on your literature review. This is quite normal in some kinds of research, and doesn't necessarily mean that you made a mistake. It is more likely a sign of progress, because the new understanding of the issue is a result of your research. You could use the better question, and either simply go back to the fieldwork stage and do another iteration to get more data, or plan to do new research fieldwork.

A more concerning eventuality is the publication of research that solves your research problem, making your topic obsolete as a contribution to knowledge. Ask your supervisor. (In the best case, it is simply an addition to your literature review.)

You might be required to provide a written report to your supervisory committee, indicating what you have done so far. Typical questions that you might need to answer are:

To what extent has your research complied with the original proposal?
Are you on schedule?
Is your original definition of the research question still adequate?
How effective has your methodology plan been?
How much raw data have you already gathered?
Does your raw data so far indicate that you will be able to draw conclusions?
Have any patterns, trends or anomalies already arisen in the data? If so, describe them.
What major obstacles do you currently face?

Fieldwork

Continue gathering data in the field, and start on a rough draft. You don't have to send it in, but you should make regular progress reports.

Implement your method(s). Get out there and meet people. If you're naturally a little afraid, this will be a major challenge, and procrastinating won't help.

Keep complete field notes as you go. Note down observations either at the time you make them or immediately after. (Contemporaneous is the key word.) DON'T trust your memory. If you collect enough information it is inevitable that you'll forget something important.

Keep records of:

Exactly what you do, why, and information that you get
When, where, who, why, etc. of interviews
Assumptions arising or that you must make
Factors that help define your project more precisely
Changes to planned method (and the reasons for those changes)
Ethical issues and how you handled them
Special factors that you had to consider
Terms with special meanings
Data that were unexpected or don't seem to fit
Patterns that appear
In fact, all your ideas, observations and anything else that might be relevant.

You probably won't use them all in the end, because your thinking will develop over the time of the research. But even the ones you don't use will be valuable in helping your thinking to develop. And of course, some will survive into the final work. The way to have them is to write them down and you'll be able to quote them directly later on.

Safeguard your notes. Many good projects never get finished because the notes are lost. Keeping them organized will make them more useful.

Start on a write-up. You'll probably want to start a rough draft that is a couple of chapters behind what you're doing in the field. You might then try to polish some earlier chapters while you're still working on drafts of the later chapters.

A gentle warning: be careful not to get so busy writing that you become less productive in the field. On the other hand, don't get so busy researching that you procrastinate on writing.

When to stop collecting data

You know when stop collecting data when you meet three criteria:

The data adequately represents the population.
Further data-gathering would not uncover any new information.
You have enough good data to draw valid conclusions.

In quantitative research, your sampling system should ensure that you meet meet these conditions.

In qualitative research, it is not quite so simple. Respondents have almost certainly given you a variety of different answers. Do you really know why, or are you just guessing? If you feel like this, you still need to keep collecting data. So how do you know when do you stop?

As you collect more data, a clear pattern should eventually emerge in the data regarding the phenomenon being researched. Just noticing a pattern is not adequate; you need enough data to confirm the pattern and incontravertably answer the original research question. In this case, more data would not improve, change, strengthen, nor add to your conclusion. In theory, it mightn't be the only possible pattern, but in practice there's usually only one. Later on, demonstrating the pattern in the data is a large part of your analysis.

In ethnography, it is usually possible to keep collecting more and more valuable data, but the three criteria still apply. You can stop collecting data when you have enough to answer your research question and to confirm your conclusions.

The point of enough data depends on:

The clarity of the research question. It is easier to gather enough data if the research question is very clear, the definition of the population is very clear, and your data-gathering approach is focussed. (Put another way, it will be easier to know when you have found something if you have a clear idea of what you're looking for.) In practice, you keep collecting data until it produces no new information, and then you can identify the saturation point in hindsight.
The capacity of the method to give sufficient detail. For example, a survey is good for collecting smaller items of data from a large number of respondents, while detailed interviews are normally best used with a smaller number of participants.
The quality of data (detail or richness) in interviews. Your interviewees might not have given enough detail for you to answer aspects of your research question.
The complexity of the phenomena under investigation. A phenomenon with more different aspects requires more data.
Whether data is triangulated, that is, you collect data through different methodologies on the same topic.
Whether the phenomenon under investigation is common or rare. The rarer the phenomenon, the less likely it is to occur in any one respondent, so you need more data.
The exent of homogeneity in the population. It will be easier to reach saturation in an homogenous group, but a very heterogenous group might give a random selection of very different answers. Consequently, you will probably need more respondents in a heterogenous group to get enough data to support a specific conclusion.

Grounded theory and saturation

In grounded theory, the point when of enough data is called saturation. The rule of thumb is that data saturation occurs when three consecutive interviews do not reveal any new data. However, this is only a rule of thumb, and some data topics might still be unexplained, necessitating more interviews.

Identifying the saturation point is normally a matter of judgement; there is no mathematical formula for saturation, although many have tried. In a dissertation, it will to some extent depend on your supervisor or committee chair. When you have believe you have reached saturation, present your case and see whether he/she agrees. If so, move on. If not, why not?

The role of iteration

The Research Cycle answers the question: I think I might have done enough fieldwork, but do I need to go back and do more?

For this reason, qualitative research is called iterative; researchers can often to do another iteration of data collection. A theoretical sample is a sample that evolves according to findings as they emerge. If a particular line of inquiry is fruitful, the researcher can add more repondents that will help to further explore it. If you want to use a theoretical sample, you should specify it in your proposal.

❓ What if the data does not lead to a firm conclusion?
You have these options:
• Did you fail to notice the pattern in the data?
• Should you do another iteration of data collection and analysis?
• Should you qualify your conclusions rather than make definitive statements. For example, you might say that something tends to ...
• Should you suggest further research with a different or larger population, or with a different sampling technique?

❓ What should I do? I don't have enough data because I can't see any pattern.
Your problem might be that you can't see the pattern, not insufficient data. It might be more difficult to see a pattern if it is quite different from what you expected. Speak to your supervisor. (Note: If your research is examining the correlation of two variables, the pattern could be a null hypothesis, that is, the two variables do not correlate.)

❓ Do I also have to explore thoroughly all emergent themes?
You should explore them if they indicate new findings that affect your conclusions or the direction of your research. Otherwise, it might be enough to confirm that they exist and define them accurately.

❓ I've collected all the data that I'd planned. But the data still has gaps and raises questions that I can't answer. Should I collect more data or should I simply accept it and stop collecting data?
Collecting more data would be appropriate if you have a reason to believe that more data fill would fill those gaps. Otherwise, you should probably accept that it represents the current state of knowledge out there. Analyze the data you have and consider suggesting topics for further research. (By the way, if you have already noticed gaps in the data, then you have already identified a pattern. That's progress.)

References PLoS One. 2018; 13(6): e0198606. Published online 2018 Jun 20. doi: 10.1371/journal.pone.0198606.
Fusch, P. I., & Ness, L. R. (2015). Are We There Yet? Data Saturation in Qualitative Research. The Qualitative Report, 20(9), 1408-1416. Retrieved from https://nsuworks.nova.edu/tqr/vol20/iss9/3
Ray Galvin How many interviews are enough? Do qualitative interviews in building energy consumption research produce reliable knowledge? Just Solutions Cambridge Working Paper 2014B. Published in The Journal of Building Engineering, 2015.
https://www.researchgate.net/profile/David_Morgan19/post/Whats_is_ideal_sample_size_in_qualitative_research/attachment/59d6301679197b807798e33b/AS%3A360974889570309%401463074528665/download/Galvin%2B15%2BHow%2Bmany%2BInterviews.pdf. (Viewed 1 October 2018)

About outliers and anomalies

Your data must be ready for the next stage of your thesis or dissertation. This includes accounting for any outliers and anomalies, which are items of data that do not fit the data pattern. Some sources refer to these as discrepant data, that is, data that is inconsistent with either the assumptions of the research, the literature, or any emerging hypotheses.

While it can arise through good research methods, it can also arise from methodological mistakes where the researcher only looks for what he/she wants to find.

Whatever the case, you will need to account for them. How these are handled depends greatly on the kind of research and the particular field:

They might indicate an error in methodology or data collection, which must be corrected for the dataset to be considered valid.
They might be normal and not problemmatical. For example:

In ethnography and sociology, they might indicate individuals who come from an unusual set of circumstances that was not anticipated in the definition of the population.
In linguistics, residue items are fairly normal. They might indicate idioms, archaeisms, or sociolinguistic effects.

In some statistical methods, small numbers of outliers are insignificant and inconsequential, so researchers can simply delete them.
Exceptions to the rule might indicate an unusual phenomenon that invites further research. Some of these surprises turn the research in another direction, or create new topics for further resesarch. This is a good thing and not unusual; it is especially the case when some data items are counter-intuitive.

New, completely unexpected themes in a qualitative study might emerge during the fieldwork or data analysis. These themes might even appear unrelated to the research problem and conceptual framework, and it would be dishonest to re-interpret the data to make it fit a set of expected conclusions. It can be especially annoying if something shows that the research problem/question should be reframed differently. If the loose end prevents you from drawing any conclusions, your research might be stuck with nowhere to go.

The question is then What you do to tie up such a major loose end?

My preferred option would be to explore the topic in an implications chapter, and perhaps revise definitions and create further research questions. That's high value. If it’s not possible, consider these other options:

Check the data for errors.
Take the whole research through another iteration.
Do some further fieldwork to explore those particular themes in enough depth to draw conclusions. (Common in ethnography.)
Go back to the literature review and add anything written about those themes. (Not a good option; it presumes that the surprise was foreseeable in the literature, and the whole point is that it wasn’t.)

In most cases, this will blow out the time it takes to finish the research.

Tip

When outliers are quite common and normal, the more accurate conclusion might be to state a tendency: X tends to be Y. In these cases, a claim that All X is Y invites suspicion because it does not allow any exceptions at all.

Writing as you go

In many institutions, students hand in their work to their supervisors part by part, often in chapters. You will normally need to have edited each chapter into good enough shape so that the supervisor will want to read it. While it doesn't need to be perfect, it needs to be free of grammmatical mistakes, typographical errors, and layout glitches. Research supervisors generally want to discuss the research aspect of your dissertation and don't want to talk about annoying writing errors. In fact, if they find lots of writing errors, they are quite entitled to return your draft and ask you to make those corrections before they will even read it.

However, the main part of the editing is the last stage before submission, where you will read and re-read the whole dissertation many times.

When writing a first draft, your main task is to get everything into words on the screen. Many advisors say that you shouldn’t edit at all during this stage, although I don’t mind it a little. The problem with editing at this stage is that it is easy to get distracted by imperfections and then have difficulty getting the first draft written.

When you are satisfied, put it away for at least three days, and preferably a week. This will give you the opportunity to look at it with fresh eyes. Then re-read it. You will probably see small errors that you missed before. (It is difficult to check your own writing, because you read what you think you meant or what you remember, not what is actually written.) You might find yourself saying Why did I say that?, What did I mean by that?, or That's not what I meant to say. or This seems jumbled and confusing. This gives you a chance to fix these errors while you can still remember enough and before anybody else sees them.

Note: Keep backups, because computer data can be lost very easily.

For more information on editing, see the separate link.

Processing data for analysis

How will you store data securely and maintain confidentiality?

Quantitatve data normally needs cleaning. This involves deleting duplicates and correcting any spelling errors that would affect computer processing.

Does anything need translation?

Do you need to transcribe interviews of voice recordings in preparation for analysis, such as interviews done in qualitative research. You can pay a transcription service to do it for you, but some students do their own. Although time-consuming, it gives more opportunity to understand the data well, and might spark insights and analyses that would otherwise more difficult. If you are writing in English and your time is more valuable than the fees for commercial software applications, you can try nvivo, temi.com, rev.com, Trint, or Otter. Google Chrome also has free plugins for transcription. Here’s a tip from Tοm Granοff. Zoom webinars has an automatic transcription feature for its cloud recordings. The transcription can be exported into Notepad, then into your word processor, and then presumably into your Qualitative Data Analysis package. See https://www.youtube.com/watch?v=3dk7xk3WtzE.

Analysis and statement of findings

The analyis is basically the bridge between your data and your findings.

Collate and arrange your data in a neat, legible, systematic layout. This will require interpreting the data, and the way you do it will depend on your methodology and the kind of data.

Think about your data and how it will solve your research problem and answer your research questions. Draw conclusions, and check that they relate to the main research problem as most recently defined. This is the most abstract step of all, and perhaps hardest to define. How it's done will depend on your specific topic and the kind of data you are using. (The importance of alignment will become immediately clear; the method must produce data that will enable you to solve the original research problem.)

If data is extremely consistent, your research might get onto the fast-track. However, your research could get very interesting if your data is diverse or inconsistent. You need to find out why that is so. Perhaps your method was good, but other particular factors caused inconsistencies. It might also have been aspects of your methodology that could not have been anticipated during planning.

Different ways to analyze data

In a quantitative analysis, researchers use statistical measures to evaluate hypotheses.
Some qualitative studies convert their data into numerical information. For example, they might count the number of times an idea came up in interviews and use that information to suggest a conclusion.
Another way is to qualitatively evaluate qualitative evidence. For example, the researcher might draw attention to the main themes that have arisen in a set of interviews and analyze the transriptions as if they were a documentary source. This is the approach taken below.

A general procedure for interviews, focus groups, program evaluations, and ethnography

Before you start

Distinguish between two kinds of topics:

People’s perception of objects
The reality/essence of the object itself

Check that the collected data perfectly matches the research question and research problem.
Read your field notes notes as many times as you need to feel you have mastered them.

Identify the main themes in the data

Can you state or define the themes clearly?
What do all instances of a theme have in common? Is a theme always the same thing or is it actually several different things that only look similar?
Consider variations, because a single theme might not always be exactly the same.
Is the theme confirmed strongly enough to draw a conclusion? If not, can you to draw a qualified conclusion?

Identify and confirm patterns

Identify patterns in the data. (Data normally forms an identifible pattern.) The sooner that you identify and confirm the pattern, you sooner you finish.
Describe those patterns in words, E.g.

distinct tendency to ...
All respondents gave answers consistent with ...
Responses were quite polarized ...

Do you have enough data to confirm those patterns as facts (i.e. more than just personal impressions)? If you have identified and confirmed a pattern in the data, the analysis is largely demonstrating it. If you do not have enough data to confirm the pattern, go back to the field and collect more data.
The analysis should lead to findings. In many fields, the statement of the pattern is the finding.

Present the patterns in the data as your findings.
Base your findings directly on the data.
Avoid overstatements when expressing your findings. For example, it might be more accurate to say that X tends to affect Y rather than imply that X always affects Y.

Create an outline

Group or categorize the themes so that they form a clear outline that is easy for your readers to understand. It must represent the patterns that you have confirmed.

Edit it into a coherent document

Explain from the respondent's point of view.
In your explanation, include direct quotations from your respondents. This has several advantages:

You can illustrate your points with real examples.
They brings your writing to life, making it much more personal and readable.
It minimizes the gap between respondents and your readers, who can read the actual evidence rather than only your interpretation.

Give your conclusion. This should directly answer your initial research question.
Write it up as your analysis chapter. Wherever possible, it should enlighten the topic to the reader, e.g. with direct quotes and examples.

Keep editing

Your outline will evolve as you get more data and as you understand it better.

⚠ Take care not to insert your personal attitudes or opinions into the analysis.

⚠ Some mistakes are easy to make if you make incorrect assumptions about your respondents:

A common comment might represent many respondents, but it might not be very insightful. Similarly, a rarely occurring comment might be very insightful.
A short comment might be valuable.
A long comment might not be valuable.
An inarticulate respondent might be very insightful.
An unintelligent or poorly educated respondent might still make a very helpful contribution.
An unexpected response or theme does not mean it is an aberration. On the contrary, it might be valuable.

❓ How can I know that the data will answer my research question?
It will if you have gathered data that addresses the research question. This is why alignment is so valuable.
However, the answer you get might not be the answer you anticipate. Some students even think that their data is wrong if it leads to conclusions that they didn't expect.

❓ How good does my data have to be?
The data must have enough rich detail to lead to confirmed conclusions, and lots of analysis will not make a poor data-set adequate. Your data is still inadequate if most occurences are only short comments with little information. In that case, you might still need to gather more data, that is, hold more interviews or focus groups to get more detail.

❓ The best students identify patterns quite quickly, almost as soon as they emerge. Other students don't notice them and think that they don't have enough data yet.

Methodology report

The method chapter should be very easy to write:

Tell it as story. Chronological order will make it easier for your readers, and it will probably make it a lot easier for you to write.
If you followed your plan closely, some of this chapter might be simply putting it into past tense, reporting what you actually did, rather that what was planned. For example, if you planned to have between 120 and 150 survey respondents, and then had 127, you simply report that you had 127 respondents.
If you kept good notes as you went, much of the work writing this chapter will simply be typing them up and polishing the language. (That is easy work.)

Unfortunatey, if you didn't follow your plan and kept inadequate notes as you went, you will have to do a lot of difficult work to reconstruct your method accurately, and might be at risk of failing the program.

Method chapters also have some strict rules:

Accuracy and honesty are essential. In some fields, it is quite common that experiments don't work the first time. If an experiment didn’t work, say why and say what you had to do to fix the problem. You need not humilate yourself, but you do have to be honest.
Give enough detail for someone else to replicate your study.

In some fields, it is quite common for other researchers to test your research by replicating it to see if gives the same results.
In a few cases, it is not possible to replicate research, such as action research and some ethnographies. Even so, the methodology statement is essential for accountability.

If you used software to transcribe or analyze data, say what it was (including the particular edition of the software), why you chose it, and how you used it.

Discussion

In some institutions or departments, the discussion is part of the analysis chapter. In others, however, it is a separate chapter, and this is a probably a little easier for students, because it makes it into a separate stage of writing with a separate purpose. In some institutions, it is called Implications.

Despite little consensus as to what precisely should be in it, there is considerable agreement that it should present at least the immediate implications of the findings. In this view, its purpose is to compare the results of the current research with other related research.

Make a start on your discussion

First, briefly re-state your findings.
State your thesis or core idea, and evaluate how your findings address it. Do they give support or not?
Are there any plausible alternative explanations for your findings?
Were any findings unexpected, counter-intuitive, controversial, or disruptive? If so, why?
What were the limitations or weaknesses of your study?

⚠ Make sure your discussion does not undermine your research or bring it into doubt.

Compare your findings

The most common and straightforward way to explore implications is to compare your results with those of other researches on similar or related topics. This kind of comparison demonstrates how you have added to knowledge on the topic, and indicates progress. (This assumes that your research is closely related to exisiting literature and that you are adding to a body of knowledge that is to some extent monolithic.)

When comparing your findings with the studies in your literature review, did your work:

confirm them?
expand or broaden them by filling a gap in the literature?
disrupt them by producing results that were inconsistent with previous research? (If so, do you know why?)
show that specific changes are necessary?
show that increased sophistication is necessary? (For example, some new differentiations or definitions might be essential to the future study of the topic.)

Most or all of these other sources should be in your literature review. However, be careful not to write another literature review; this discussion of the literature should be different because it has a different purpose.

What could change?

The second kind of implication is to examine what would or should change as a result of your research. Your study might also have other interesting and important wider implications, and this is also the place to explore them. In this section, and only here, you may wander off track a little because implications often go wider than the specific research topic.

In any case, you should ask:

How should existing theories be modified?
How should existing practices be modified?

Future research

What future research do you suggest?* For example:

Expand the size of the population
Consider a different population
Use another methodology
Explore emerging issues
Evaluate an alternative conclusion if it has some compelling evidence
Explore and test findings that were unexpected, counter-intuitive, controversial or disruptive
Respond to weaknesses or limitations in your study
Test the results of your qualitative study with quantitative studies
Explore the meaning of the findings of your quantitative study with qualitative studies.

* Some dissertation handbooks instruct writers to place suggestions for further research in the conclusion.

Conclusion

In a thesis or dissertation, the conclusion is a separate chapter. It shows that you argued for a particular conclusion and recaps on how you've reached it. The core of the conclusion (the thesis for which you have argued, and which is the answer to the main research question) should be expressed in a single sentence. Some advisors also expect that you will provide summarized answers to other subsidiary research questions if you have asked them. The last sentence is usually the most difficult sentence of all to write; it should usually give a general application.

I was asked, Does the discussion of findings need any references? Is the writing supposed to be critical like a literature review? Or do you have to restate the references from the literature review?

My answer, It depends. If it is a statement of findings and immediate implications, there is no aspect of a literature review. In other cases, the statement of findings and immediate implications is followed by a comparison with the related literature. This discusses whether the reseach findings confirm, modify or contradict other research on the same topic. In this case, there is an aspect of literature review and references are essential.

Tricks and traps in writing conclusions

• Keep it brief.
• Avoid at all costs bringing up new issues that could be interpreted to challenge the thesis for which you have argued.
• Descriptive conclusions tend to be sentences that encapsulate the description, and are a little tricky because they can easily become vague and rather meaningless.
• In some kinds of studies, the conclusion might also mention leftover topics that arose such as side issues that need further separate research.

Finish rough draft of relevant sections

Describe your methodology in fieldwork, that is, what you actually did, not just what you planned to do. This will go into your methodology chapter.
Set out your data. This will be the basis of the relevant chapter or appendices.
Give your analysis of the data. This will be the first draft of the analysis chapter.
Write a full draft of your implications section. Implications might be theoretical, or apply to professional practice.

When possible, tell a story; put the information in chronological order and make it flow. It will engage your readers and make reading more enjoyable. It will also demonstrate your point, which is equivalent to providing evidence to prove your point. However, be very cautious of bias, which an easy trap if you choose a particular perspective from which to present your story.

If you are working with a particular theory, you will normally state it in your introduction, either as an assumption or as a theoretical framework. During the dissertation, keep in discussion with the theory. You might find that your evidence sometimes agrees with the theory and sometimes does not. In doing so, you are contributing to the theory, by either supporting it or modifying it.

Update literature review

You should update your literature review at this stage. It is probably a while since you first wrote it, and new publications since then might have changed the face of your topic. However you handle it, you will need to include them in your literature review, so that your dissertation is up to date when submitted.

Write other sections

Add any appendices

Some theses and dissertations don't need any appendices, although one institution requires all students to include the ethical clearance as Appendix A.

In certain cases they are necessary; they normally include information or documents that are necessary to the research but would disrupt the reader if included in the main text. The two main categories are raw data and ancillary documents.

If you need to know whether to include something as an appendix, consider these questions. If in doubt, discuss them with your supervisor:
• Is it necessary to the research?
• Would it disrupt the reader if included in the main text?

Write the introduction

Write the introduction, which will be Chapter 1. The introduction gives the reader a clear direction on where the whole thesis or dissertation is going, explains the topic, and gives information that readers will need to interpret it correctly.

Most of the introduction has already been written, but you still need to edit it, so that you can put in it any assumptions and new directions that come up during research. You can also add any delimitations, definitions, and assumptions that you identified since writing the proposal. (You can also clarify assumptions where they are most pertinent in the body of the text.)

The parameters of the work need to be clear, even if they are implicit. Poorly defined boundaries have the potential to cast doubt on the conclusion. Parameters may be defined in terms of specific populations, programs, movements, periods, bodies of literature, etc. Sometimes parameters are purely arbitrary, but you should explain what they are and that they are arbitrary.

Beware. Do not give definitions for ordinary words which give no cause for concern. Include definitions only for terms that are specific to the field of study and are either ambiguous, vague, or used inconsistently. (You might find competing definitions of some key terms in the literature.)

Fine-tune your title

You already have a working title, but now you need a final title. Do this last to make sure you get it exactly right. As you write the dissertation, you might have better understoond what it is actually about, so your final title might differ from your original working title.

Write the preliminaries

Write the preliminaries. At the very least, you need a title page and a table of contents. Most institutions also require an approval page and an abstract. See the style guide for more assistance, for example, lists of tables and figures. Do not give a dedication.

Your software might automatically generate the table of contents and lists of tables and figures. However, you will still need to check the formatting.

Include a preface if your personal experience is necessary to understand the content of the dissertation.

Abstract

The main purpose of an abstract is to inform other researchers what you did and what you found in order for them to know whether they need a full copy for their own research purposes. Consequently, it must usually be able to function as an independent document with its own identifiers and explanation of any acronyms used. Some abstracts are published as speparate documents. Either way, abstracts should be written only after the body of the text has been approved. They must usually have Abstract at the top as the main title.

Length

Institutions and publishers of abstracts usually set limits in word totals. Simply follow the rules of your institution. Consider these variations:

Abstracts must be brief, never more than one page.
Most institutions and publishers of abstracts set limits of 200 words, although some publishers allow up to 350 words for Ph.D. dissertations.
Some institutions allow abstracts that run into several pages.
Some institutions require abstracts to be included in the preliminaries while others require them to be submitted afterward and appended inside the front cover.

Abstracts increasingly follow a fixed format comprising the following parts:

Title section. Give the title and year of the dissertation, your name, the qualification for which the dissertation or thesis was submitted, and your institution.
Problem. What problem does this work attempt to solve? What is the scope of the project? What is the central research question or hypothesis? What is the main argument, thesis or claim?
Reason for writing. What is the importance of the research? Why would a reader be interested in the larger work?
Methodology. An abstract of a scientific work might include specific models or approaches used in the larger study. Other abstracts might describe the types of evidence used in the research.
Results. Describe the results. This is usually the most important section for your readers.
Implications. How does this work add to the body of knowledge on the topic? Are there any practical or theoretical applications from your finding? What are the implications for future research?

A common wording for abstracts is as follows:
The purpose of this study is ...
The scope of this study ....
The methodology ...
The findings ...
Conclusions reached are ...
Limitations of this study include ...
This study contributes ...

Collate the whole document

Collate all the parts and edit it into one harmonious document. You should now have the following:

Preliminaries
Chapter 1: Introduction

A topic statement, written clearly, concisely and exactly. It could be expressed as a research problem, a question that cannot yet be answered, or as a hypothesis (perhaps with sub-hypotheses).

Statement of scope or delimitations of your topic
Definitions of any terminology that is either unique or has multiple meanings

Statement of significance
Possible findings or outcomes

Chapter 2: Literature review

Full literature review
Theoretical framework with a statement of any specific assumptions

Chapter 3: Methodology

Plan

Defined population(s)
Method for selecting respondents
Plan for developing and trialling any data collection tools
Planned schedule
Defined ethical compliance
A budget (if needed)

Implementation

Testing and modification of data-gathering tools
Comments or adjustments arising from implementation

Chapter 4: Analysis
Chapter 5: Conclusion
Appendix A: Permisssion
Appendix B: Data collection instruments
Appendix C: Raw data

This outline is not obligatory. For example:

The introduction may contain a statement of purpose if not clear elsewhere.
The introduction may contain a statement of assumptions.
The methodology plan and implementation may be integrated.
If the discussion or implications section is long enough, it can also be a separate chapter.
Some chapters can be so long in a large dissertation that they can beome multiple chapters. For example:

The literature review can be split into separate chapters for different disciplines or topics, especially in interdisciplinary studies.
Some kinds of research, especially ethnography, require a separate chaper for context description, that is, a description of the population.
The data and analysis are combined in ethnographic studies, and often split into chapters on separate topics.

Where should you put long verbatims? Unless your supervisor allows otherwise, view the full text of verbatims as raw material and put them in appendices. The text should predominantly contain your description and analysis supported by brief, pertinent quotations from the verbatims. The post-modern trend, however, is put long verbatims in the main text. The rationale has been that they are valid research because they let the speakers speak, without the researcher’s interpretation and editorializing.

Edit and proofread

With thanks to Dαwιe Vαn Vuuνen

The standards for dissertation presentation is that of a published book, and academic standards are based on publishers’ standards. In fact, one of the purposes of a dissertation program is for the student to demonstrate that he/she could write an original academic book. This is no longer simply a standard; most dissertations in accredited institutions are now published, even if non-commercially in a research repository on the internet.

You might feel that re-writing and editing are a frustrating waste of time, but an essential part of the task is polishing the manuscript and gettting the details right. This is the stage to get meticulous. Get details in logic and terminology correct, and make sure your spelling, punctuation and references are as close to perfect as you can make them. It is a basic academic skill, and is almost the same as writing a book for a publisher. According to one source, theses and dissertations are frequently rejected for small errors that are overlooked at the editing and proofreading stage. (Lyn Wαlden, 2017.)

Unless you notice a serious mistake, it is now too late to add any new discovery, late data, or interesting discussion topic. You really have to go with what you have, and can only make minor changes.

Your readership

Theses and dissertations have two main kinds of readership:

A dissertation for a research degree is written for a readership of researchers. (This includes non-PhD degrees that are equivalent to a PhD, such as many Doctor of Education programs.) As a result, the writer can assume that readers have a certain amount of background knowledge. It is not possible to write dissertations in some fields without making these assumptions, for example, the use of mathematical notation.
A dissertation for a professional degree is usually written for a professional readership. For example, a Doctor of Ministry dissertation is written for other senior Christian leaders to read.

What do supervisors and assessors generally prefer to read?

Supervisors and assessors generally want to sit down and enjoy reading something that is easy to read, informative, and interesting. They don't want reading to feel like hard, grinding work. Enjoyable dissertations works generally have the following characteristics:

The grammar, typing, layout, and punctuation are correct so readers are not annoyed by frequent errors.
The language follows the conventions of good written expression (e.g. sentence length, economical number of words, clarity, transitional paragraphs, use of common words when they are adequate).
The structure is logical so that readers can easily follow the train of thought. This might mean telling something as a story.
Complex, new ideas are presented element by element, not as confusing verbiage.
It is often helpful to include direct quotes when analysing first-hand accounts and interviews. This has the advantage of humanizing the research, as well as exposing readers to the primary information, rather than only your interpretation.
Admittedly, a few sections might be harder to read, such as routinized reporting, or necessary but uninteresting details. (Readers usually skim these sections quickly.)

Different kinds of editing

The editing stage can comprise at least five different kinds of activities, and students tend to be much better at one kind than the others.

Developmental editing
Line editing
Copy editing
Feedback
Final proof reading

Developmental editing

By the end of this stage, the logic of the whole dissertation should be sound. It has to make sense to someone who reads it afresh, who has a good background on the field of study, but in not your particular topic.

The primary concern is the flow, structure, and logic. Edit these at this early stage because there is little value in perfecting the details of a large section that you might later delete.

You might need to improve the structure, for example, shrinking long-winded sections and expanding sections that are too brief. Don't worry if it reduces the page count; it's progress even if it doesn't feel like it. For each thing that needs to be condensed or cut, you will probably have to add more to fill gaps that you hadn't seen before. You can learn something through each of these changes so that the final product is better.

For example:

Is the sequence the most logical for your readers? Do you need to move sections?
Do you have evidence to support all your statements and conclusions? If it is not your own empirical evidence, you need facts from recent sources. Beware, however, that you need to evaluate your sources; a quotation from a credible source is not necessarily conclusive.
Do you need to delete sections? (Some sections might turn out to be irrelevant to the actual topic. It is always frustrating and disappointing to find out that you have to delete sections, especially if they are very long and you have worked hard on them.)
What should you do to sections that say the same thing in different ways or are redundant? Do you need to amalgamate them or delete one of them?
Are there contradictions or inconsistencies? These can be subtle and difficult to spot.
Are all sections at the right level of section heading? For example, do you have any major points hidden under minor headings?
Do the amount of words given to a point reflect its significance to the whole dissertation?
Have your statements been qualified or have you over-generalized?

⚠ Omissions are particularly dangerous; everything written might be good but the things left out might be the weaknesses. Your supervisor should alert you of significant omissions during the writing process.

💡 Hint. Keep the outline simple so it is easy to see the bigger picture. If the whole dissertation is in one word processor file, use the table of contents to see the outline. Updating the view of the outline is then as simple as clicking the update button.

Line editing

This is editing at the level of writing. Simple, direct, concise is still good advice; even academics want to read something engaging and interesting.

You might have noticed that some of the language in dissertations is quite complex. The question is then, “How complex should the language be?” The answer is, “Only as complex as the topic requires.” At times you might need to be complex because you use specialist terminology and deal with difficult topics or necessary details. But that is no excuse for convoluted writing that is unpleasant to read. It is still best to use simple, clear, concise writing. Do not use lengthy verbiage to conceal a lack of real thought or to increase the word total. Instead, use good language to make your point more clearly and engage your readers.

Is there any clumsy, unclear, imprecise, or long-winded language that you should re-write?
Do sentence lengths vary enough?
Do you need to make the text flow for your readers?
Is it the same voice speaking throughout the dissertation? Does it follow a consistent academic tone, or is it sometimes informal or inconsistent?
Could something be better expressed as a story rather than a collation of concepts?

Copy editing

Your dissertation might be looking good, but you need to go through it looking for errors in grammar, punctuation, and layout. This calls for attention to detail and this is the stage to get meticulous. Get details in logic and terminology correct, and make sure your spelling, punctuation and references are as close to perfect as you can make them.

You need to do this manually, even though you should also use software.
Save yourself some work by using the styles feature in your word processor.
Take particular care with citations and references, which are very easy to get wrong. The punctuation and layout requirements are very detailed, and some requirements vary greatly between different style guides. Using a service like Zotero can save time and ensure greater accuracy.
Let some software do checking for you. This will save you lots of time and effort, and will pick up errors that you simply didn't see. You'll normally find that the software is imperfect, so you should be willing to overrule its suggestions at times:

The first is a good spellchecker and grammar checker. MS Word will do a basic check, but it is reportedly not as good as Grammarly.
The second is a program to check your language style. Wordrake is available for a free seven-day trial. Use it when you have checked the manuscript as thoroughly as you can. ProWritingAid can check grammar, spelling, and language style. Its dictionary and thesaurus also checks word selection. A free trial version is available.
The third kind of software will check your references. See reciteworks.com. Colleagues have recommended it, but it does not guarantee perfect accuracy. Some experts recommend you do them by hand and use software as a checker. (If you use software such as Zotero to generate them, your references will probably be more accurate anyway.)

💡 Hint. It can be easier and more accurate to edit logic, spelling, grammar, style, and layout separately. Try going though the whole manuscript many times, each time looking for a different kind of problem. In the final stages of editing, a full edit might take only an hour or two.

💡 Hint. Print out your thesis or dissertation and check a paper copy at least once. If you find too many errors, make the corrections, print another hard copy, and check again. Why? Many people find it easier to see errors in hard copy; computer screens seem to hide them for some reason.

Feedback

Have someone read the whole manuscript and give you honest feedback. Tell them what you want them to do, because some expect to be only a proof-reader rather than a general critic.

At first, it might be someone close whom you trust to be constructive and not devastatingly critical.
Then get feedback from people of the kind of you the target readership who have some distance from you and your work. For example, if you are doing original research, then it will be other researchers. If you are writing for a professional degree, then your audience might be professionals.

Final proof reading

Readers will be annoyed if they keep finding small mistakes in an otherwise good dissertation. Go through looking for the little errors you've missed so far. These might be typographical errors that the spellchecker missed, grammar that’s not quite right, punctuation errors, and clumsy sentences.

💡 Hint. It is a good idea to have two people, one after the other, proof-read your dissertation.

💡 Hint. Check your institution's writing policy. In many cases, it is quite permissible to get help, but not to use the help as ghost writers.

How long should it take someone else to proof-read your dissertation?

First, you need to get it done soon if a deadline is closing in on you. Second, it depends on the reader’s expertise and attention to detail. A careless reader achieves little, but some people are quite accurate and still quite fast.

For the most part, the goal of a writer is to have something that is a quick, pleasant read where your readers feel, I learned something new and helpful. Consequently, proofreaders need not be so conscientious that they get bogged down with details. They don't need to let the task take forever or become a stressful burden. Proof-readers are not co-authors and it's not their role to find solutions to problems. They only have to tell you if they see something wrong.

In most cases, a dissertation is in bad shape if reading it is a slow and laborious task. The exceptions are those parts where the writer's priority is to plod through lengthy (a.k.a. boring) details and get them right.

Final stages

When should you stop? The purpose of the final editing is to get the details right, but don't let perfectionism or lack of confidence prevent you from graduating. It is always easy to do one more edit or to look for one more journal article. However, you have to draw the line somewhere. It is better to have a dissertation that is not completely perfect and to graduate, than to have a good dissertation that never gets submitted.

⚠ Warning. Some students are so obsessed with perfection that they have difficulty finishing. In order to graduate, they have to learn that A good dissertation is a done dissertation.

When you are ready, run your paper through turnitin.com and show your supervisor the output.

If your supervisor agrees, you can now print off the final version and submit it to the committee. For fully online institutions, you'll more likely submit a pdf software version.

Oral examination

The oral examination will follow the procedure published in your institution's handbook, and you should familiarize yourself with it. Procedures for oral examinations, however, vary greatly between institutions and some call oral examinations oral defense, while others use the term viva voce, which is Latin for live voice.

Is an oral examination required?

Some US states require them by law for doctoral candidates in non-religious schools.
Some institutions do not require them at all, often because they depend on assessors from other institutions.
Some institutions only require an oral examination if the student's work is of borderline standard.

If an oral examination is required, the arrangement depends on the institution:

Some have a private meeting with a committee of academics, who try to keep the meeting as informal and non-threatening as its importance can allow.
Some have a formal public meeting. The candidate is sat facing a committee, and allowed only a copy of the thesis. The chairperson of the committee acts as moderator.
Some have a formal meeting, but it is not public and only a small number of persons are present, for example, other faculty members and other doctoral students.
Some allow on-line video meetings.
Some also require students to give formal presentations, perhaps with a time limit. For example, some institutions require candidates to give an oral presentation of 30-45 minutes, preferably with a PowerPoint, and then answer questions.

Institutions also vary greatly in the resources you may use. The traditional standard is that you may have only a copy of your dissertation. Some online institutions also allow you have your textbooks with you, which is very useful if you want to quote anything on methodology.

Getting some information

Your supervisors will only schedule an oral defense because they think you are ready. You already know what you need to know, so be encouraged. It is almost unknown for candidates who reach this stage to fail and not get a degree at all. (It would reflect badly on the supervisor to approve a defense for a student who would not pass.)

Ask your supervisor how long in advance the committee members will need to read the whole dissertation before the final defense. It will probably be at least 2-4 weeks, but they might also want to think it over for a little longer.

How to prepare

If you are required to have an oral examination, your objective is to successfully defend your dissertation. You should be very familiar with all contents of your dissertation, and should be able to find any sections or topics without undue delay and ready to field any questions on them. The oral defense should be easier if you are accustomed to discussing your research. You might already have considerable experience in these skills if you are experienced in giving presentations at formal conferences and responding to colleagues' questions.

Some supervisors even conduct a mini-defense with their students beforehand to give confidence and accustom them to the process; in the best cases, students have no surprises later on during the actual defense.

If you can, attend oral defenses or ask people about their experiences. Some institutions record them, and you might be able to listen to or watch the recordings.

If you did a quantitative study, check your statistics system. Most doctoral candidates have done only one course in statistics, and the statistician is the dark horse in an examination committee. You shouldn't really have to answer questions like: Why didn't you use the ABC statistical method instead? (mentioning an obscure method that you have never heard of), and the moderator should disallow it. But you must be able to explain the system you used, show that it was fit for purpose, and show that it produced reliable results.

More (Link opens new window.)

At the actual defense

Be confident and talk as the expert. Make sure your methods and results discussion are solid.

Spend the minimum time recapping your proposal and the majority of your time explaining your findings.

Examiners will ask about things that you didn’t include in your presentation, although some of these might already be listed in your methodology chapter or in your suggestions for further research. Try to anticipate what they might be, such as:
“What alternative methodologies did you consider, and what effects might they have had?”
“Did you experience any setbacks during data collection?”
“Does your research have any particular gaps or limitations?”
“In hindsight, what would you have done differently?”

Examiners will probably also ask some questions you hadn't expected. For example, they might pick out what seems to be subtle weaknesses, gaps, or contradictions in your work. Don't let it fluster you; just give honest and fair answers.

Afterwards

The committee might require some minor corrections. It is quite normal, so just accept it and do not be discouraged. If you are required to re-submit with corrections, there should be a deadline given in the assessment report. The committee should also give you a written list of exactly what corrections they require; ask for it if they don't. (You don't want to be in a situation where they say that you didn't make all required corrections.)

Theses and dissertations

From fieldwork to submission

As you go ...

Recruiting participants

Pre-fieldwork