Introduction
As AI tools become increasingly prevalent in coding education, educators are rethinking how to assess students' true understanding of programming concepts. This tutorial will guide you through creating a project-based assessment system that leverages AI tools like GitHub Copilot while maintaining the integrity of coding skills evaluation. You'll build a system that evaluates both code quality and conceptual understanding through interactive coding challenges.
Prerequisites
- Intermediate Python knowledge
- Basic understanding of machine learning concepts
- Access to a GitHub account with Copilot enabled
- Python libraries:
numpy,scikit-learn,pytest - Basic familiarity with Git version control
Why this matters: This approach helps educators move beyond simple code generation to focus on understanding how students think through problems and apply programming concepts.
Step 1: Setting up the Project Structure
Create your assessment framework
First, we'll establish a project structure that separates assessment logic from student code. This ensures that our evaluation system can be reused across different coding challenges.
assessment_project/
├── assessment_framework/
│ ├── __init__.py
│ ├── evaluator.py
│ └── rubric.py
├── challenges/
│ ├── __init__.py
│ ├── challenge_1.py
│ └── challenge_2.py
├── tests/
│ ├── __init__.py
│ └── test_evaluator.py
└── README.md
Initialize the project
Create the directory structure and initialize a Git repository:
mkdir assessment_project
mkdir -p assessment_framework challenges tests
cd assessment_project
git init
Step 2: Implementing the Evaluator
Create the core evaluation logic
The evaluator will analyze student submissions for code quality, logic correctness, and adherence to best practices:
# assessment_framework/evaluator.py
import ast
import inspect
from typing import Dict, List, Tuple
class CodeEvaluator:
def __init__(self):
self.rubric = self._load_rubric()
def evaluate_submission(self, code: str, challenge: str) -> Dict:
"""Evaluate a student's code submission"""
try:
tree = ast.parse(code)
# Check for code complexity
complexity_score = self._calculate_complexity(tree)
# Check for best practices
best_practices_score = self._check_best_practices(code)
# Check for logical correctness
logic_score = self._check_logic(code, challenge)
return {
'complexity': complexity_score,
'best_practices': best_practices_score,
'logic_correctness': logic_score,
'total_score': (complexity_score + best_practices_score + logic_score) / 3
}
except Exception as e:
return {'error': str(e)}
def _calculate_complexity(self, tree) -> float:
# Simple complexity calculation based on function count
functions = [node for node in ast.walk(tree) if isinstance(node, ast.FunctionDef)]
return min(len(functions) / 5.0, 1.0) # Normalize to 0-1 scale
def _check_best_practices(self, code: str) -> float:
# Check for common good practices
checks = [
'for' in code and 'range' in code,
'def ' in code,
'return' in code,
]
return sum(checks) / len(checks) if checks else 0.0
def _check_logic(self, code: str, challenge: str) -> float:
# This would be implemented based on specific challenge requirements
return 0.8 # Placeholder for demonstration
def _load_rubric(self) -> Dict:
return {
'complexity': 0.3,
'best_practices': 0.4,
'logic_correctness': 0.3
}
Step 3: Creating Challenge Examples
Define a sample challenge
We'll create a challenge that requires students to implement a simple sorting algorithm:
# challenges/challenge_1.py
def bubble_sort(arr):
"""
Sort an array using bubble sort algorithm
Args:
arr (list): List of integers to sort
Returns:
list: Sorted list in ascending order
"""
# Student's implementation would go here
pass
# Expected solution
def expected_bubble_sort(arr):
n = len(arr)
for i in range(n):
for j in range(0, n - i - 1):
if arr[j] > arr[j + 1]:
arr[j], arr[j + 1] = arr[j + 1], arr[j]
return arr
Step 4: Integrating AI Tools
Using Copilot for code generation
With Copilot enabled, students can generate code, but we want to ensure they understand what they're generating. We'll create a system that shows both generated and manually written code:
# assessment_framework/rubric.py
import json
class AssessmentRubric:
def __init__(self):
self.criteria = {
'code_quality': {
'weight': 0.4,
'description': 'How well does the code follow Python best practices?'
},
'algorithm_correctness': {
'weight': 0.3,
'description': 'Does the algorithm produce correct results?'
},
'understanding_explanation': {
'weight': 0.3,
'description': 'Can the student explain their approach?'
}
}
def generate_feedback(self, evaluation_result: dict, student_code: str) -> str:
"""Generate detailed feedback based on evaluation"""
feedback = []
if evaluation_result['total_score'] < 0.7:
feedback.append("Consider reviewing fundamental concepts in sorting algorithms.")
if 'complexity' in evaluation_result and evaluation_result['complexity'] < 0.5:
feedback.append("Try to break complex problems into smaller functions.")
if 'best_practices' in evaluation_result and evaluation_result['best_practices'] < 0.6:
feedback.append("Remember to use descriptive variable names and proper docstrings.")
return "\n".join(feedback)
Step 5: Testing Your System
Write unit tests for your evaluator
Testing ensures our evaluation system works correctly and consistently:
# tests/test_evaluator.py
import pytest
from assessment_framework.evaluator import CodeEvaluator
def test_complexity_calculation():
evaluator = CodeEvaluator()
code = '''
def func1():
pass
def func2():
pass
'''
result = evaluator.evaluate_submission(code, 'test')
assert 'complexity' in result
assert result['complexity'] == 0.4 # 2 functions / 5 = 0.4
def test_best_practice_checking():
evaluator = CodeEvaluator()
code = '''
def my_function():
return 42
'''
result = evaluator.evaluate_submission(code, 'test')
assert 'best_practices' in result
assert result['best_practices'] == 1.0 # All checks pass
Step 6: Running the Assessment
Putting it all together
Create a main script that demonstrates how to use your assessment system:
# main.py
from assessment_framework.evaluator import CodeEvaluator
from assessment_framework.rubric import AssessmentRubric
from challenges.challenge_1 import bubble_sort, expected_bubble_sort
def main():
evaluator = CodeEvaluator()
rubric = AssessmentRubric()
# Simulate student submission
student_code = '''
def bubble_sort(arr):
n = len(arr)
for i in range(n):
for j in range(0, n - i - 1):
if arr[j] > arr[j + 1]:
arr[j], arr[j + 1] = arr[j + 1], arr[j]
return arr
'''
# Evaluate the submission
evaluation = evaluator.evaluate_submission(student_code, 'bubble_sort')
print("Evaluation Results:")
print(f"Complexity Score: {evaluation['complexity']}")
print(f"Best Practices Score: {evaluation['best_practices']}")
print(f"Logic Score: {evaluation['logic_correctness']}")
print(f"Total Score: {evaluation['total_score']}")
# Generate feedback
feedback = rubric.generate_feedback(evaluation, student_code)
print("\nFeedback:")
print(feedback)
if __name__ == "__main__":
main()
Summary
This tutorial demonstrates how educators can create effective assessment systems that work alongside AI tools like Copilot. By focusing on evaluation criteria that emphasize understanding and best practices rather than just code generation, we can better assess students' true programming abilities. The system evaluates code quality, algorithmic thinking, and conceptual understanding, providing detailed feedback that helps students improve their skills while leveraging AI as a learning tool rather than a replacement for understanding.
The key insights from this approach are:
- Move beyond simple code output to assess thinking processes
- Use AI tools to enhance learning, not replace critical thinking
- Implement comprehensive evaluation rubrics that cover multiple skill areas
- Provide actionable feedback that helps students grow
