Home

Warning

This project is still under active development and subject to change. While the goal is to change the pawtograder config format as little as possible, it is not yet stable.

This is the documentation for the Pawtograder adapter of the pyret-autograder system.

See the pyret-autograder wiki for documentation of the core design of the autograder, and how to build upon the default graders provided by the library.

Background

pyret-autograder uses a directed acyclic graph (DAG) based execution model where each node's runner can return the following outcomes:

• block with a corresponding reason for the block
• noop
• emit with either a score or an artifact
• internal-error

Notably, a runner should never raise an error.

While any runner can produce any of these results without restriction, currently the grader's exported by pyret-autograder--the only graders currently used by pyret-autograder-pawtograder--fall into one of the following conceptual groups:

• a guard: which can either block further execution, or produce noop
• a scorer: which should only emit a score
• an artist: which would only emit an artifact

Usage

pyret-autograder-pawtograder only provides a programmatic interface; for a more convenient experience, use either pawtograder itself[^1] or the pyret-autograder-cli package.

[^1]: still a work in progress

Either interface expects a top-level pawtograder.yml file conforming to the following format:

grader:
  pyret # tells pawtograder to use pyret-autograder-pawtograder to
  # interpret the file
default_entry:
  <file_relative_to_student_submisson> # optional, but recommended
  # if all parts of the assignment use the same file
graders: # key-value pairs where each key represents the id of the grader, and
  # each key conforms to a certain grader (as described below)
  # for example:
  <key>: # this can be called anything, just a key in the DAG
    type: ... # this is the type of grader, see the graders page
    # other values, again see the graders page

See the Graders page for a full list of graders in addition to all of their options.

For a more concrete (and possibly more up-to-date 😳) specification see the lib/schema.ts file.

Pawtograder

Note

TODO: write documentation once fully implemented.

pyret-autograder-cli

The pyret-autograder-cli interface can use used easily on any machine running a modern (v22+) version of Node.js using npx.

All pawtograder specific commands live under the pawtograder subcommand so running:

npx pyret-autograder-cli pawtograder --help

will print a help menu.

Tip

If you create a node project, and you add pyret-autograder-cli as a depedency, you can simply run npx pyret-autograder or just pyret-autograder if you add npm executables to your path using direnv

Assuming you have the following file structure your your current working directory:

.
├── solution
│   └── pawtograder.yml
└── submission

you can run run the submission against the solution using:

npx pyret-autograder-cli pawtograder ./submission -s ./solution