AI Changes Everything: Why Student Data Review Can't Wait Another Semester
October 15, 2025
Jessica Arciniega, Katherine Kalpos, Morgan Sexton, and Amelia Vance
CC BY-NC 4.0
As schools embrace the potential of AI, a critical question demands attention: How do sophisticated AI models alter what it means to protect student data? For starters, it underscores the need for schools to reassess the data they collect from students and the duration for which they retain it.
AI makes students’ sensitive information more vulnerable than ever before. Earlier this month, the Texas Attorney General filed a lawsuit against edtech giant PowerSchool, a student information system, for a breach that allegedly exposed the sensitive information of more than 60 million students and 10 million teachers, sometimes including their Social Security Numbers (SSNs). What makes this particularly dangerous in the AI era is that once algorithms scrape this data from the dark web, they can connect it with other datasets to create comprehensive profiles—turning today's attendance records into tomorrow's identity theft schemes. There is no telling where it may end up or how it may be used. Additionally, a recent study revealed that a freely available AI training set, which had been used for multiple years, contained hundreds of millions of images containing personally identifiable information.
In an age of AI and persistent cyber threats, holding less student data can be a powerful way for schools to increase student privacy protections. Schools must collect only the truly necessary information, use it only for its intended educational purpose, and retain only what they need for as long as they need it, before securely destroying it. Every day we delay robust data minimization practices, we increase the risk that students will become victims of AI-enabled identity theft and fraud.
The AI-Powered Risk Explosion
Today's schools don't just track grades and attendance; they also monitor student progress. Edtech platforms and data analytics tools may capture extensive details about our students' school experiences, including their quiz scores, assignment completion rates, medical conditions, counseling records, and family income information for meal programs.
When used appropriately, this data collection helps schools identify struggling students early, personalize learning experiences, and comply with state and federal reporting requirements that ensure all students receive equitable educational opportunities. When misused, the risks expose students to harm that can last for a decade.
Unfortunately, student data is under constant attack. The Cybersecurity and Infrastructure Security Agency (CISA) reported in 2023 that “For K-12 schools, cyber incidents are so prevalent that, on average, there is more than one incident per school day.”
At the same time, AI models are growing increasingly more sophisticated in their ability to connect the dots across datasets. AI can link what might seem like harmless learning analytics data today to other information sources tomorrow, create inference records, and generate detailed profiles that may follow students throughout their lives. The risk isn't just theoretical, it's mathematical. The more data (especially sensitive data) that schools collect and store, the greater the potential is for misuse when that data inevitably faces security threats.
This is particularly true for SSNs. One study found that 97% of people whose names, addresses, and SSNs were exposed in breaches and traded on the dark web became victims of attempted identity theft. For children, this risk only compounds over time.
Children: Targets in an AI World
Unlike adults with established financial histories, children often have clean credit histories that make them perfect targets for AI-enhanced identity theft schemes. When kindergartners' SSNs are compromised, theft may go undetected for years until the child applies for their first credit card, student loan, or job.
For example, imagine that a third-grader's data is exposed in a data breach today. By the time they’re ready for college, AI systems may have already combined that information with data from other sources and identified unique patterns, creating synthetic identities to open credit lines, file fraudulent tax returns, or even commit crimes under the child's identity. The student who should be focused on choosing a major instead spends months untangling a web of financial fraud that began when they were eight years old.
Students may experience vastly different levels of data exposure based on their backgrounds and lived experiences. For example, justice-involved youth, youth with disabilities, foster youth, immigrant youth, and youth from families receiving public benefits could potentially appear in several overlapping systems that may collect and share extensive data about them without fully accounting for the privacy risks this can create. These unique concerns underscore the need to center data minimization practices informed by educators and policymakers working with vulnerable youth.
AI-Ready Data Minimization
Schools have made significant progress in data privacy. There was a time when schools printed student SSNs on ID cards. This would be unthinkable today! In fact, several state and district laws and policies explicitly regulate schools’ collection and use of SSNs. Our companion article, “Protecting Student Privacy: A Critical Back-to-School Priority” includes examples of states with these restrictions. Progress like this shows that meaningful change is possible when school leaders prioritize student privacy.
But the AI revolution demands we go much farther, much faster. The solution necessitates a fundamental shift in how we approach student data in an AI-powered world. Data minimization is now schools’ strongest defense. The most secure data is the data that schools don't have.
Data minimization principles are critical in the AI era. Schools must collect only what they need, use it solely for its intended purpose, and retain it for only as long as necessary before securely destroying it.
We have released an accompanying resource, "A District Guide to Data Minimization in the Age of AI," to help schools systematically review their data practices, identify quick wins, and implement data minimization frameworks that protect students without compromising their educational missions or AI ambitions. Start with the quick wins outlined in our implementation guide—removing SSN fields from enrollment forms, training front office staff, and clearly marking required versus optional data fields.
Leading the AI Era
As school leaders, you have the power to make AI-ready data minimization a district-wide priority. Data minimization must be a core value driving every data collection decision, especially as we increasingly embrace AI tools that can make data exponentially more powerful and potentially dangerous.
The AI revolution has begun. Take action now to limit the data you collect and retain, thereby better protecting your students from AI-fueled identity theft.
*Special thanks to Michael Klein for their substantial contributions to the creation of this resource.