Skip to content

feat: test creation and commit in /tests #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
haddybhaiya merged 4 commits into from
Mar 31, 2026
Merged

feat: test creation and commit in /tests #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6e25a0b
feat: test creation and commit in /tests
haddybhaiya Mar 31, 2026
b8bc9dd
fix : fetch head Pr branch
haddybhaiya Mar 31, 2026
8776a5a
Merge b8bc9dd67c44ce037ab56714be21125d905d9b63 into e3634abac733335e0…
haddybhaiya Mar 31, 2026
dcd9d38
Add AI-generated tests
actions-user Mar 31, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/agent.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ jobs:
runs-on: ubuntu-latest
permissions:
pull-requests: write
contents : read
contents : write
Comment on lines 8 to +10

Copilot AI Mar 31, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing contents permission to write enables pushing commits from this workflow. On pull_request triggers this has two concrete risks: (1) the push can retrigger the workflow and create a commit loop unless you add explicit safeguards, and (2) for fork PRs the token won’t be able to push anyway, so the job will repeatedly attempt/fail. Consider adding if: guards (e.g., only run on non-fork PRs and skip when github.actor is the actions bot), and ensure checkout/push targets the PR head branch if committing is intended.

Copilot uses AI. Check for mistakes.
steps:
- name: Checkout repo
uses : actions/checkout@v3
Expand Down
19 changes: 19 additions & 0 deletions agent/github/committer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import os
import subprocess

def commit_tests():
try:
branch = os.getenv("GITHUB_HEAD_REF") # PR branch

subprocess.run(["git", "config", "user.name", "github-actions"], check=True)
subprocess.run(["git", "config", "user.email", "actions@github.com"], check=True)

subprocess.run(["git", "add", "tests/"], check=True)
subprocess.run(["git", "commit", "-m", "Add AI-generated tests"], check=True)

subprocess.run(["git", "push", "origin", f"HEAD:{branch}"], check=True)

print("Tests pushed to PR branch")

except Exception as e:
print("Commit Failed", e)
7 changes: 6 additions & 1 deletion agent/llm/test_generator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,12 @@ def generate_tests(diff:str,context:list =None)->str:
}
],
)
return response.choices[0].message.content.strip()
content =response.choices[0].message.content.strip()
if content.startswith("```"):
content = content.split("```")[1] # Extract code from markdown
if content.startswith("python"):
content = content[len("python"):] # Remove language specifier
return content.strip()
Comment on lines +37 to +42

Copilot AI Mar 31, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Markdown code-fence stripping is fragile here: content.split("```")[1] will raise/behave incorrectly when the response contains multiple fenced blocks, leading text before the fence, or an unclosed fence. Also, the PR description mentions handling bash, but this code only strips python and will leave a leading bash token that will make the saved file invalid Python. Consider robustly parsing a single fenced block (and rejecting/handling non-Python languages) before returning.

Copilot uses AI. Check for mistakes.
except Exception as e:
return f"Error generating tests:{str(e)}"

12 changes: 10 additions & 2 deletions agent/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
from agent.indexing.embedder import get_embeddings
from agent.indexing.vector_store import store_embeddings, query_embeddings
from agent.llm.test_generator import generate_tests

import subprocess
from agent.github.committer import commit_tests

Copilot AI Mar 31, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commit_tests is imported from agent.github.committer, but the new module added in this PR is agent/github/committer.py (double “t”). As-is, the agent will crash with ModuleNotFoundError at startup. Update the import to match the actual filename/module, or rename the module for consistency.

Suggested change
from agent.github.committer import commit_tests
from agent.github.commiter import commit_tests

Copilot uses AI. Check for mistakes.
import subprocess , os
from agent.llm.groq_client import generate_review
from agent.github.commenter import post_comment
def get_pr_diff():
Expand Down Expand Up @@ -72,6 +72,10 @@ def main():
# 8. Generate tests
print("Generating tests...")
tests = generate_tests(diff, context=relevant_chunks)
os.makedirs("tests", exist_ok=True)
with open("tests/test_generated.py", "w", encoding="utf-8") as f:
f.write(tests) # Save generated tests to a file for potential commit
print("Generated tests saved to tests/test_generated.py")
Comment on lines 74 to +78

Copilot AI Mar 31, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated test content is written directly to tests/test_generated.py even when generate_tests() returns an error string (it returns "Error generating tests:..." on exceptions). This can lead to committing a non-test error message into the repository. Consider checking for the error sentinel (or otherwise validating that the output is Python test code) before writing/committing.

Copilot uses AI. Check for mistakes.

# 9. Combine output
final_output = f"{review}\n\n---\n\n### Suggested Tests\n{tests}"
Expand All @@ -83,6 +87,10 @@ def main():
print("Posting comment...")
post_comment(final_output)

# 11. Commit tests
print("Committing tests...")
commit_tests()
Comment on lines +90 to +92

Copilot AI Mar 31, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commit_tests() is called unconditionally in the PR workflow. Combined with the workflow trigger on pull_request and a constant commit message, this can create repeated commits / reruns, or attempts to commit when nothing changed. Add a guard (e.g., only commit when tests/test_generated.py differs, and/or skip when the actor is the actions bot / commit message matches).

Copilot uses AI. Check for mistakes.

print("Done")

if __name__ == "__main__":
Expand Down
26 changes: 26 additions & 0 deletions tests/test_generated.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import pytest
import os
from agent.github.committer import commit_tests
from agent.main import get_pr_diff

def test_commit_tests():
commit_tests()

def test_get_pr_diff_empty():
with pytest.raises(subprocess.CalledProcessError):
get_pr_diff()

def test_commit_tests_exception():
try:
commit_tests()
except Exception as e:
assert str(e)

def test_get_pr_diff_no_diff():
diff = get_pr_diff()
assert diff.strip() == ""

def test_commit_tests_push():
subprocess.run(["git", "config", "user.name", "github-actions"], check=True)
subprocess.run(["git", "config", "user.email", "actions@github.com"], check=True)
commit_tests()