Skip to content

enable feedback step#269

Merged
ArthurCRodrigues merged 5 commits intomainfrom
268-autograder-enable-feedbackstep-in-the-web-pipeline-service
Apr 11, 2026
Merged

enable feedback step#269
ArthurCRodrigues merged 5 commits intomainfrom
268-autograder-enable-feedbackstep-in-the-web-pipeline-service

Conversation

@matheusmra
Copy link
Copy Markdown
Member

@matheusmra matheusmra commented Apr 11, 2026

This pull request refactors the grading workflow to use a new GradingRequest dataclass for improved clarity and maintainability. It also restructures the grading logic into smaller helper functions for better error handling and separation of concerns.

API and Data Structure Improvements:

  • Introduced a GradingRequest dataclass to encapsulate all parameters needed for grading a submission, replacing the previous use of multiple function arguments. (web/service/grading_service.py)
  • Updated the API endpoint (create_submission) to create and pass a GradingRequest object to the grading task, instead of passing individual parameters. (web/api/v1/submissions.py) [1] [2] [3]

Grading Logic Refactoring:

  • Refactored the main grading function (grade_submission) to accept a GradingRequest and split the logic into helper functions: _run_pipeline, _persist_success, and _persist_failure, improving readability and error handling. (web/service/grading_service.py)
  • Enhanced error handling during grading: now logs errors with more detail and handles both pipeline and unexpected failures more robustly. (web/service/grading_service.py)

Feedback and Result Handling:

  • Improved feedback generation for preflight failures and ensured result trees and focus are serialized properly for storage. (web/service/grading_service.py)

Minor Cleanups:

  • Cleaned up and clarified the _node_to_dict utility function for serializing result trees. (web/service/grading_service.py)

Copilot AI review requested due to automatic review settings April 11, 2026 20:10
@matheusmra matheusmra linked an issue Apr 11, 2026 that may be closed by this pull request
2 tasks
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enables the Autograder pipeline’s Feedback step for web-initiated grading jobs, so web submissions will now produce and persist user-facing feedback alongside scores and result trees.

Changes:

  • Enable feedback generation in the web grading pipeline (include_feedback=True).
  • Provide an (empty) feedback configuration dictionary to satisfy the feedback subsystem.
  • Pass locale into build_pipeline().

@matheusmra
Copy link
Copy Markdown
Member Author

@ArthurCRodrigues

Copy link
Copy Markdown
Member

@ArthurCRodrigues ArthurCRodrigues left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

include_feedback should a optionable flag.

It was False by default previously because FeedbackStep wasn't actually ready yet.

But now that we have it, requests should come in with include_feedback toggled true or false.

external_user_id: str
submission_files: dict
locale: str = "en"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically add the include feedback attr here

"""Build the autograder pipeline and run it in a thread."""
pipeline = build_pipeline(
template_name=request.template_name,
include_feedback=True,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and then use

        include_feedback=request.include_feedback,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Important: This will cause changes in api contract!

@matheusmra
Copy link
Copy Markdown
Member Author

Ok, I'm also implementing the feedback config

Introduce feedback configuration and targeted learning resources throughout the grading stack. DefaultReporter now filters online resources so that resources with linked_tests are only shown when related tests fail; the reporter API was adjusted to pass failed test names. ReporterService converts incoming online_content dicts to LearningResource objects. Persist feedback options by adding feedback_config (JSON) and include_feedback (bool) to the GradingConfiguration model and wiring those fields through the API (create submissions / grading_configs) and GradingRequest. Added an Alembic migration to add the new DB columns and updated API documentation and Pydantic schemas to document the feedback_config and include_feedback fields.
@matheusmra
Copy link
Copy Markdown
Member Author

@ArthurCRodrigues added support for customizable feedback reports in the grading system. It introduces new fields (feedback_config and include_feedback) to the grading configuration, updates the database schema and API to handle these fields, and enhances the feedback report logic to include targeted online learning resources based on failed tests. Documentation is also updated to describe the new configuration options.

Feedback report configuration and logic:

  • Added feedback_config and include_feedback fields to grading configuration models, schemas, and API endpoints, allowing users to control feedback generation and customize report content. [1] [2] [3] [4] [5] [6] [7] [8]
  • Updated the feedback report generation logic to support "targeted" online resources: resources are only shown if their linked tests failed, otherwise only global resources are displayed. [1] [2]

Database migration:

  • Added an Alembic migration to add the feedback_config (JSON) and include_feedback (boolean) columns to the grading_configurations table.

API and documentation updates:

  • Updated API documentation to describe the new feedback_config and include_feedback fields, including a detailed reference for feedback configuration options and examples. [1] [2] [3] [4] [5] [6]
  • Updated API endpoints and response models to handle the new feedback fields. [1] [2] [3]

Other technical improvements:

  • Minor code cleanups, such as removing unused imports and fixing parameter names for consistency. [1] [2] [3] [4] [5]

Copy link
Copy Markdown
Member

@ArthurCRodrigues ArthurCRodrigues left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

include feedbac

@ArthurCRodrigues ArthurCRodrigues merged commit b67d351 into main Apr 11, 2026
3 checks passed
@ArthurCRodrigues ArthurCRodrigues deleted the 268-autograder-enable-feedbackstep-in-the-web-pipeline-service branch April 11, 2026 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Autograder] Enable FeedbackStep in the Web Pipeline Service

3 participants