A student essay is more than a piece of writing; it can contain names, family stories, health disclosures, and opinions that students never expected anyone outside the classroom to read. When a school adopts an AI grading tool, that material moves into a new system, and the district remains responsible for protecting it. Commentary on AI detection tools in higher education has already highlighted how uploading student work to the wrong service can create exposure under federal student privacy rules. The same caution applies to grading and feedback products.

Privacy review often falls between departments, with teachers focused on usefulness and technology staff focused on security. A shared list of questions helps both sides speak the same language and makes vendor comparisons fair. The questions below are not legal advice, and your district counsel should have the final word, but they cover the ground that most privacy reviews begin with.

Ask for answers in writing and keep them with your procurement records, since vague reassurance in a sales call is not a substitute for specific commitments in a contract or data privacy agreement. If a vendor cannot or will not answer a question clearly, that tells you something important about how the product will be supported once your students are using it. Treat unanswered questions as open risks rather than minor details to sort out later.

Questions about storage and access

Begin with where student writing is stored, in what form, and for how long it is kept after an assignment ends. Ask who at the vendor can view identifiable essays, under what circumstances, and how that access is logged and reviewed. Districts should also know whether the vendor uses subcontractors that process student data, because every additional party widens the area that has to be protected, audited, and contractually bound to the same standards.

• Where are essays stored, and in which country or region?
• How long is student work retained, and how is it deleted on request?
• Which vendor staff can see identifiable student writing, and is access logged?
• Which subcontractors process student data, and under what agreements?
• Does the vendor sign a district data privacy agreement before any pilot begins?

Questions about model training and use of data

One of the most important questions is whether student essays are used to train or improve models, either the vendor's or a third party's. Many districts require that student work never be used for training without explicit consent, and that requirement should be written into the contract. Ask also whether the vendor sends data to external model providers and what protections apply to that transfer.

Ask how the tool handles identifying details inside essays, since students may include names, addresses, or sensitive experiences in their writing without a second thought. Some products offer redaction or recommend that teachers avoid assignments likely to elicit personal disclosures, and a few let districts configure what is collected. Understanding these safeguards helps teachers design prompts and workflows that respect students' privacy and reduces the chance of a difficult surprise later in the semester.

Questions about teacher control and transparency

Privacy is also about control, and a tool that lets teachers review and edit every score and comment before release protects students from errors as well as from exposure. Ask whether students ever see raw model output, whether the teacher can override any result, and whether the system keeps a record of edits. These features support accountability if a family later questions a grade.

Request documentation of how the product handles accuracy limitations and bias, including any testing with diverse student populations and multilingual writers. Recent research found that AI detection tools disproportionately affect non-native English writers, a reminder that fairness and privacy issues often travel together in school technology. Vendors that have thought carefully about these problems will usually have clear, specific answers, while those that have not will tend to fall back on general claims about accuracy.

Making the review routine

Turn the questions into a standard checklist that every new tool must complete before staff use it with student work. Assign a named person in the district to own the review, and publish the list of approved products so teachers know what is allowed. Surveys show that fewer than half of principals report having any AI policy, so a simple approved-tools list is a meaningful improvement on its own.

Revisit approvals annually, since vendors change their terms, add features, and switch infrastructure providers without much fanfare. A short renewal questionnaire keeps commitments current without repeating the entire review, and it gives the district a dated record of what each vendor promised at each stage. Over time, this routine protects students and builds the kind of trust that allows schools to adopt useful tools with confidence instead of anxiety.