AI Code Tutor

This GitHub Action uses AI to provide personalized feedback for student assignments in C/C++ and Python. It analyzes test results and code, identifying errors, suggesting optimizations, and explaining concepts clearly. Ideal for GitHub Classroom, it saves instructors time and ensures consistent, on-demand feedback.

The AI tutor processes JSON test reports from pytest-json-report, generated by pytest tests wrapping C/C++ or Python code. It detects logic errors, recommends efficient algorithms, and links to relevant documentation.

Key Features

• AI-powered feedback for C/C++ and Python assignments.
• Supports multiple JSON test reports from pytest-json-report.
• Analyzes multiple student code files (.c, .cpp, .py).
• Flexible LLM selection (Claude, Gemini, Grok, Nvidia NIM, Perplexity) with Gemini fallback.
• Customizable feedback language (e.g., English, Korean).
• Excludes common README content to optimize API usage.

Prerequisites

• Python Dependencies:
  • Install required packages:

    pip install pytest pytest-json-report pytest-xdist requests

  • Generate JSON reports with:

    python -m pytest --json-report --json-report-indent=4 --json-report-file=report.json tests/test_file.py

  • See pytest-json-report documentation.
• API Key: At least one API key for supported LLMs (Claude, Gemini, Grok, Nvidia NIM, Perplexity), set as repository secrets (e.g., INPUT_CLAUDE_API_KEY, INPUT_GOOGLE_API_KEY).
• Docker: For C/C++ testing, use a Docker image with clang, cmake, and pytest.

Usage

1. Add a workflow file (e.g., .github/workflows/classroom.yml) to your repository.
2. Configure it to run tests and invoke the AI tutor. Example for C/C++ assignments:

name: Grade Assignment
on: [push, pull_request, workflow_dispatch]
jobs:
  grade:
    runs-on: ubuntu-latest
    env:
      CONTAINER_WORKSPACE: /app/workspace
      CONTAINER_TESTS: /tests
      CONTAINER_SRC: /app/workspace/src
      C_FILENAME: main.c
      WORKSPACE_OUTPUT: ${{ runner.temp }}/output
      CONTAINER_OUTPUT: /output
    steps:
      - uses: actions/checkout@v4
      - name: Set up environment
        run: pip install pytest==8.3.5 pytest-json-report==1.5.0 pytest-xdist==3.6.1 requests==2.32.4
      - name: Create output folder
        run: mkdir -p ${{ env.WORKSPACE_OUTPUT }}
      - name: Run C/C++ tests
        run: |
          docker run --rm \
            --volume ${{ github.workspace }}:${{ env.CONTAINER_WORKSPACE }}:ro \
            --volume ${{ env.WORKSPACE_OUTPUT }}:${{ env.CONTAINER_OUTPUT }}:rw \
            --workdir ${{ env.CONTAINER_TESTS }} \
            ghcr.io/kangwonlee/edu-base-cpp:4e0d6d8 \
            /bin/sh -c "cmake . -DCMAKE_BUILD_TYPE=Debug -DSTUDENT_DIR=${{ env.CONTAINER_WORKSPACE }} && make && python3 -m pytest --json-report --json-report-indent=4 --json-report-file=${{ env.CONTAINER_OUTPUT }}/report.json test_dynamic.py"
      - name: AI Code Tutor
        uses: kangwonlee/[email protected]
        if: always()
        with:
          report-files: ${{ env.WORKSPACE_OUTPUT }}/report.json
          student-files: ${{ env.CONTAINER_SRC }}/${{ env.C_FILENAME }}
          readme-path: ${{ env.CONTAINER_WORKSPACE }}/README.md
          explanation-in: English
          model: gemini
          INPUT_CLAUDE_API_KEY: ${{ secrets.INPUT_CLAUDE_API_KEY }}
          INPUT_GOOGLE_API_KEY: ${{ secrets.INPUT_GOOGLE_API_KEY }}
          INPUT_GROK_API_KEY: ${{ secrets.INPUT_GROK_API_KEY }}
          INPUT_NVIDIA_API_KEY: ${{ secrets.INPUT_NVIDIA_API_KEY }}
          INPUT_PERPLEXITY_API_KEY: ${{ secrets.INPUT_PERPLEXITY_API_KEY }}
        timeout-minutes: 10

Notes

• C/C++ Testing: Tests can run in a Docker container with pytest wrapping C/C++ code (e.g., via ctypes for shared libraries, as in test_dynamic.py). Ensure JSON reports are generated.
• Model Selection: Set model to prefer an LLM (e.g., gemini). If its key is unavailable, the action falls back to Gemini if INPUT_GOOGLE_API_KEY is set, or uses any one of available key.
• Secrets: Store API keys as repository secrets with INPUT_ prefix (e.g., INPUT_GOOGLE_API_KEY) in Settings > Secrets and variables > Actions.
• README Optimization: Exclude common README content with:
  • Start: From here is common to all assignments.
  • End: Until here is common to all assignments.
  • Use double backticks (``).

Optimizing pytest for AI Feedback

• Use descriptive test names (e.g., test_sum_range_for__valid_input).
• Include clear assertion messages (e.g., assert result == 10, f"Expected 10, got {result}").
• Keep tests focused for accurate AI interpretation.

Inputs

Input	Description	Required	Default
report-files	Comma-separated JSON report files	Yes	report.json
student-files	Comma-separated student code files (.c, .cpp, .py)	Yes	exercise.py
readme-path	Path to assignment instructions (README.md)	No	README.md
explanation-in	Feedback language (e.g., English, Korean)	No	English
model	Preferred LLM (e.g., gemini, claude)	No	None
INPUT_CLAUDE_API_KEY	Claude API key	No*	None
INPUT_GOOGLE_API_KEY	Google Gemini API key	No*	None
INPUT_GROK_API_KEY	Grok API key	No*	None
INPUT_NVIDIA_API_KEY	Nvidia NIM API key	No*	None
INPUT_PERPLEXITY_API_KEY	Perplexity API key	No*	None

*At least one API key is required.

Example with Multiple Files

with:
  report-files: 'report1.json,report2.json,reports/*.json'
  student-files: 'src/main.c,src/utils.c'
  readme-path: README.md
  explanation-in: English
  model: gemini
  INPUT_GOOGLE_API_KEY: ${{ secrets.INPUT_GOOGLE_API_KEY }}
  INPUT_CLAUDE_API_KEY: ${{ secrets.INPUT_CLAUDE_API_KEY }}

Outputs

• Feedback: Markdown written to $GITHUB_STEP_SUMMARY, visible in the GitHub Job Summary, and saved as feedback.md in artifacts.

Limitations

• Primarily supports C/C++ and Python assignments via pytest-json-report.
• Requires at least one valid API key.
• C/C++ feedback relies on pytest tests wrapping compiled code.

Future Enhancements

• Auto-detect feedback language.
• Support additional programming languages.
• Add verbose mode for detailed feedback.

Troubleshooting

Check GitHub Actions logs for details.

Common Errors

• API Key Issues: "No API keys provided" – Ensure at least one API key is set in secrets.
• Report File Issues: "Report file not found" – Verify JSON report exists.
• Student File Issues: "Student file not found" – Check file paths and extensions.

Debugging Tips

• View logs in the "AI Code Tutor" job.
• Test locally with act.
• Use INPUT_FAIL-EXPECTED=true for debugging expected test failures.

Contact

Questions? Contact https://github.com/kangwonlee.

License

BSD 3-Clause License + Do Not Harm.
Copyright (c) 2024 Kangwon Lee

Acknowledgements

• Built using python-github-action-template by Vincent A. Cicirello (MIT License).
• Gemini 2.0 Flash and Grok 3 assisted with code and documentation.
• Registered as #C-2024-034203, #C-2024-035473, #C-2025-016393, and #C-2025-027967 with the Korea Copyright Commission.