“Pre-grading” papers

Ross Woods, 2022

One of the major challenges to massifying higher education is grading papers, because so far, all professors still have to manually read papers and assign grades.

A few paper-grading software programs have been written but they make the same mistake of expecting certain topics to come up in the essay, which software can identify as vocabulary items. However, this does not work when an innovative writer creates a better answer without plodding through the same expected information. The other weakness of this software is that, although papers might mention the correct items, the software cannot judge what is said about them.

Another approach

“Pre-grading” means that software checks papers before a human assessor. Papers that do not meet specific standards are automatically failed and do not need a human assessor to read them.

  1. Check the file type. (The correct file type is required to upload a paper, but the software might record an incorrect file type.)
  2. Check the date of submission.
  3. Check the length in number of words.
  4. Check readibility.*
  5. Check the spelling, grammar and style.
  6. Check plagiarism.
  7. Check outline, if a specific template is mandatory.

* Readability systems give numerical scores for items of written work based on factors such as sentence length, number of words with three or more syllables, characters per word, or high-frequency vocabulary. Examples are: Flesch Reading-Ease, Flesch–Kincaid Grade Level, Dale-Chall, Automated readability index (ARI), SMOG grade (Simple Measure Of Gobbledygook), Gunning fog index, Fry readability formula, Coleman–Liau index, Lexile Framework for Reading, and Spache Readability Score.

Instructor inputs

Most of these depend on specific criteria. For example, the instructor must specify a submission date if papers can be failed for late submission. The instructor might determine minimum and maximum readability scores, and set maximum plagiarism checker scores. Other kinds of instructor adjustments include the date for announcing assessment results and the reasons for those results.

In a few cases, the assessor might want to do other adjustments. For example, those who teach introductory writing might want to see scores for readability, spelling, grammar and style, but not to have the paper immediately failed on that basis. As another example, someone teaching more advanced students might not care about readability, spelling, grammar and style.

The checking order

The choice of checking order could be based on several criteria. The first possibility is the server load, so that items with a light server load are checked first. The second possibility is “sudden death” items, that is, items that are easy to check but result in instant fails. It is probable that these two criteria are different expressions of the same thing.

Necessary and “nice to have” features

Necessary:

  1. Useful for pre-assessment at server
  2. Open source code

“Nice to have”

  1. Fully available online to students for checking beforehand, and giving feedback
  2. Use multiple readability scales
  3. Multiple languages
  4. Specific feedback to students on their errors, e.g., nature of error, suggests corrections
  5. Feedback to instructors and program reviewers, e.g., class/cohort averages of different kinds of errors
  6. Custom dictionaries
  7. Choice of American or British English
  8. Style: Passive voice, Sentence length, Transitions
  9. Sentence rephrasing
  10. Error analysis
  11. Academic style
  12. Downloadable

References
See opensource.com/article/20/3/open-source-writing-tools
https://languagetool.org/ https://github.com/languagetool-org
http://proselint.com/
LibreOffice has a spelling and grammar checker
https://rigorousthemes.com/blog/best-open-source-grammarly-alternatives/
Natural Language Toolkit: https://github.com/nltk/nltk