Understanding the Complex Landscape of Student AI Use in Writing
AERA 2025 Roundtable Presentation
American Educational Research Association (AERA) Annual Meeting 2025
Roundtable Presentation: "The Influence of Digital Tools on Literacy, Learning Engagement, and Student Development"
April 2025
TBD
Background & Motivation
This research examines how students integrate AI tools into their writing processes, moving beyond the typical focus on plagiarism and academic integrity concerns. Key background factors include:
- Increasing AI adoption in writing, with approximately 13% of teens having used ChatGPT for school work
- Prevalent concerns among educators about plagiarism and academic integrity, creating uncertainty about AI's legitimate role in classrooms
- A significant research gap in understanding how students integrate AI into their writing process beyond surface-level use
- Mixed student attitudes ranging from uncritical AI use (risking "AIgiarism") to skepticism without effective strategies
Theoretical Framework
The research is grounded in multiple theoretical perspectives:
- Cognitive perspective: Writing as goal-directed problem solving that relies on planning, strategy, and working memory (Flower & Hayes, 1981)
- Sociocultural perspective: Writing as a socially situated act shaped by audience, peers, tools, and context (Prior, 2006)
- Integrative view: Argumentative writing research bridges both views—writers operate within cognitive processes and community contexts (Alexander et al., 2023; Graham, 2018)
- AI as a writing partner: Generative AI functions as a collaborator or "surrogate author," blurring the cognitive/social divide and potentially redefining writing practices
Research Questions
RQ1: What patterns of engagement emerge from student-AI interactions with a specialized writing assistant?
RQ2: How do these engagement patterns correspond to psychological profiles of student intent and goal-directed writing behavior?
RQ3: What is the relationship between these interaction profiles and writing outcomes (e.g., rhetorical structure, argument quality, degree of AI integration)?
Methodology
Participants & Data
- Participants: 1,581 students (high school & college levels) using Editory AI
- Dataset: Data drawn from 3,101 log files (April-July 2024), yielding 2,460 writing sessions and 8,720 recorded AI interactions
- Platform: Editory AI - A custom academic writing assistant with structured prompts for eight essay components
- Features: Users can adjust tone, length, and component type for each AI generation; logs detailed interaction metadata
Analysis Approach
Each AI interaction was classified by:
- Timing: Delay between requests (short: <45s, medium: 45s-2min, medium-long: 2-5min, long: >5min)
- Parameter changes: Whether students adjusted AI settings between requests (static: no change; dynamic: one or more parameter changes)
Clustering Method
- Feature extraction: Computed patterns for each student/session (e.g., proportion of time intervals, frequency of query types)
- K-means clustering: Applied to engagement features to group similar usage profiles
- Cluster validity: Four clusters showed good separation (average silhouette score ≈0.66)
Results: Four Engagement Profiles
| Profile | Characteristics | Interpretation |
|---|---|---|
| 1. Static Deliberative | Longer pauses between AI queries with minimal setting changes | Careful, goal-driven use; students wait, revise, then ask AI occasionally |
| 2. Static Rapid | Short intervals between queries with minimal changes | Students rapidly generate content in succession, using default settings consistently |
| 3. Dynamic Exploratory | Frequent parameter changes between interactions and moderate timing | Iterative experimentation and refinement using AI in bursts |
| 4. Rapid & Superficial | Very short intervals between AI calls with no adjustments | Rapid-fire requests and less reflective integration of AI outputs |
Implications & Applications
This research provides valuable insights that can inform:
- Development of more effective AI writing tools that support different types of student engagement
- Educational practices for integrating AI into writing instruction in ways that promote critical thinking
- Policies regarding AI use in educational settings that acknowledge the complex reality of student engagement
- Future research on how AI tools can be designed to encourage more deliberative and exploratory writing processes