Autograde a notebook by running it
Options
-------
Arguments that take values are actually convenience aliases to full
Configurables, whose aliases are listed on the help line. For more information
on full configurables, see '--help-all'.
--create
Create an entry for the student in the database, if one does not already exist.
--quiet
set log level to CRITICAL (minimize logging output)
--debug
set log level to DEBUG (maximize logging output)
--no-execute
Don't execute notebooks and clear output when autograding.
--force
Overwrite an assignment/submission if it already exists.
--db=<Unicode> (NbGrader.db_url)
Default: ''
URL to the database. Defaults to sqlite:///<course_directory>/gradebook.db,
where <course_directory> is another configurable variable.
--notebook=<Unicode> (NbGrader.notebook_id)
Default: '*'
File glob to match notebook names, excluding the '.ipynb' extension. This
can be changed to filter by notebook.
--student=<Unicode> (NbGrader.student_id)
Default: '*'
File glob to match student IDs. This can be changed to filter by student.
Note: this is always changed to '.' when running `nbgrader assign`, as the
assign step doesn't have any student ID associated with it.
--assignment=<Unicode> (NbGrader.assignment_id)
Default: ''
The assignment name. This MUST be specified, either by setting the config
option, passing an argument on the command line, or using the --assignment
option on the command line.
--log-level=<Enum> (Application.log_level)
Default: 30
Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
Set the log level by value or name.
--course=<Unicode> (NbGrader.course_id)
Default: ''
A key that is unique per instructor and course. This MUST be specified,
either by setting the config option, or using the --course option on the
command line.
--course-dir=<Unicode> (NbGrader.course_directory)
Default: ''
The root directory for the course files (that includes the `source`,
`release`, `submitted`, `autograded`, etc. directories). Defaults to the
current working directory.
Class parameters
----------------
Parameters are set from command-line arguments of the form:
`--Class.trait=value`. This line is evaluated in Python, so simple expressions
are allowed, e.g.:: `--C.a='range(3)'` For setting C.a=[0,1,2].
AutogradeApp options
--------------------
--AutogradeApp.answer_yes=<Bool>
Default: False
Answer yes to any prompts.
--AutogradeApp.assignment_id=<Unicode>
Default: ''
The assignment name. This MUST be specified, either by setting the config
option, passing an argument on the command line, or using the --assignment
option on the command line.
--AutogradeApp.autograded_directory=<Unicode>
Default: 'autograded'
The name of the directory that contains assignment submissions after they
have been autograded. This corresponds to the `nbgrader_step` variable in
the `directory_structure` config option.
--AutogradeApp.config_file=<Unicode>
Default: ''
Full path of a config file.
--AutogradeApp.config_file_name=<Unicode>
Default: ''
Specify a config file to load.
--AutogradeApp.course_directory=<Unicode>
Default: ''
The root directory for the course files (that includes the `source`,
`release`, `submitted`, `autograded`, etc. directories). Defaults to the
current working directory.
--AutogradeApp.course_id=<Unicode>
Default: ''
A key that is unique per instructor and course. This MUST be specified,
either by setting the config option, or using the --course option on the
command line.
--AutogradeApp.create_student=<Bool>
Default: False
Whether to create the student at runtime if it does not already exist.
--AutogradeApp.db_assignments=<List>
Default: []
A list of assignments that will be created in the database. Each item in the
list should be a dictionary with the following keys:
- name
- duedate (optional)
The values will be stored in the database. Please see the API documentation
on the `Assignment` database model for details on these fields.
--AutogradeApp.db_students=<List>
Default: []
A list of student that will be created in the database. Each item in the
list should be a dictionary with the following keys:
- id
- first_name (optional)
- last_name (optional)
- email (optional)
The values will be stored in the database. Please see the API documentation
on the `Student` database model for details on these fields.
--AutogradeApp.db_url=<Unicode>
Default: ''
URL to the database. Defaults to sqlite:///<course_directory>/gradebook.db,
where <course_directory> is another configurable variable.
--AutogradeApp.directory_structure=<Unicode>
Default: '{nbgrader_step}/{student_id}/{assignment_id}'
Format string for the directory structure that nbgrader works over during
the grading process. This MUST contain named keys for 'nbgrader_step',
'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
'notebook_id', as this will be automatically joined with the rest of the
path.
--AutogradeApp.feedback_directory=<Unicode>
Default: 'feedback'
The name of the directory that contains assignment feedback after grading
has been completed. This corresponds to the `nbgrader_step` variable in the
`directory_structure` config option.
--AutogradeApp.force=<Bool>
Default: False
Whether to overwrite existing assignments/submissions
--AutogradeApp.generate_config=<Bool>
Default: False
Generate default config file.
--AutogradeApp.ignore=<List>
Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__']
List of file names or file globs to be ignored when copying directories.
--AutogradeApp.log_datefmt=<Unicode>
Default: '%Y-%m-%d %H:%M:%S'
The date format used by logging formatters for %(asctime)s
--AutogradeApp.log_format=<Unicode>
Default: '[%(name)s]%(highlevel)s %(message)s'
The Logging format template
--AutogradeApp.log_level=<Enum>
Default: 30
Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
Set the log level by value or name.
--AutogradeApp.logfile=<Unicode>
Default: '.nbgrader.log'
Name of the logfile to log to.
--AutogradeApp.notebook_id=<Unicode>
Default: '*'
File glob to match notebook names, excluding the '.ipynb' extension. This
can be changed to filter by notebook.
--AutogradeApp.permissions=<Int>
Default: 0
Permissions to set on files output by nbgrader. The default is generally
read-only (444), with the exception of nbgrader assign, in which case the
user also has write permission.
--AutogradeApp.release_directory=<Unicode>
Default: 'release'
The name of the directory that contains the version of the assignment that
will be released to students. This corresponds to the `nbgrader_step`
variable in the `directory_structure` config option.
--AutogradeApp.source_directory=<Unicode>
Default: 'source'
The name of the directory that contains the master/instructor version of
assignments. This corresponds to the `nbgrader_step` variable in the
`directory_structure` config option.
--AutogradeApp.student_id=<Unicode>
Default: '*'
File glob to match student IDs. This can be changed to filter by student.
Note: this is always changed to '.' when running `nbgrader assign`, as the
assign step doesn't have any student ID associated with it.
--AutogradeApp.submitted_directory=<Unicode>
Default: 'submitted'
The name of the directory that contains assignments that have been submitted
by students for grading. This corresponds to the `nbgrader_step` variable in
the `directory_structure` config option.
NbGrader options
----------------
--NbGrader.answer_yes=<Bool>
Default: False
Answer yes to any prompts.
--NbGrader.assignment_id=<Unicode>
Default: ''
The assignment name. This MUST be specified, either by setting the config
option, passing an argument on the command line, or using the --assignment
option on the command line.
--NbGrader.autograded_directory=<Unicode>
Default: 'autograded'
The name of the directory that contains assignment submissions after they
have been autograded. This corresponds to the `nbgrader_step` variable in
the `directory_structure` config option.
--NbGrader.config_file=<Unicode>
Default: ''
Full path of a config file.
--NbGrader.config_file_name=<Unicode>
Default: ''
Specify a config file to load.
--NbGrader.course_directory=<Unicode>
Default: ''
The root directory for the course files (that includes the `source`,
`release`, `submitted`, `autograded`, etc. directories). Defaults to the
current working directory.
--NbGrader.course_id=<Unicode>
Default: ''
A key that is unique per instructor and course. This MUST be specified,
either by setting the config option, or using the --course option on the
command line.
--NbGrader.db_assignments=<List>
Default: []
A list of assignments that will be created in the database. Each item in the
list should be a dictionary with the following keys:
- name
- duedate (optional)
The values will be stored in the database. Please see the API documentation
on the `Assignment` database model for details on these fields.
--NbGrader.db_students=<List>
Default: []
A list of student that will be created in the database. Each item in the
list should be a dictionary with the following keys:
- id
- first_name (optional)
- last_name (optional)
- email (optional)
The values will be stored in the database. Please see the API documentation
on the `Student` database model for details on these fields.
--NbGrader.db_url=<Unicode>
Default: ''
URL to the database. Defaults to sqlite:///<course_directory>/gradebook.db,
where <course_directory> is another configurable variable.
--NbGrader.directory_structure=<Unicode>
Default: '{nbgrader_step}/{student_id}/{assignment_id}'
Format string for the directory structure that nbgrader works over during
the grading process. This MUST contain named keys for 'nbgrader_step',
'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
'notebook_id', as this will be automatically joined with the rest of the
path.
--NbGrader.feedback_directory=<Unicode>
Default: 'feedback'
The name of the directory that contains assignment feedback after grading
has been completed. This corresponds to the `nbgrader_step` variable in the
`directory_structure` config option.
--NbGrader.generate_config=<Bool>
Default: False
Generate default config file.
--NbGrader.ignore=<List>
Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__']
List of file names or file globs to be ignored when copying directories.
--NbGrader.log_datefmt=<Unicode>
Default: '%Y-%m-%d %H:%M:%S'
The date format used by logging formatters for %(asctime)s
--NbGrader.log_format=<Unicode>
Default: '[%(name)s]%(highlevel)s %(message)s'
The Logging format template
--NbGrader.log_level=<Enum>
Default: 30
Choices: (0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL')
Set the log level by value or name.
--NbGrader.logfile=<Unicode>
Default: '.nbgrader.log'
Name of the logfile to log to.
--NbGrader.notebook_id=<Unicode>
Default: '*'
File glob to match notebook names, excluding the '.ipynb' extension. This
can be changed to filter by notebook.
--NbGrader.release_directory=<Unicode>
Default: 'release'
The name of the directory that contains the version of the assignment that
will be released to students. This corresponds to the `nbgrader_step`
variable in the `directory_structure` config option.
--NbGrader.source_directory=<Unicode>
Default: 'source'
The name of the directory that contains the master/instructor version of
assignments. This corresponds to the `nbgrader_step` variable in the
`directory_structure` config option.
--NbGrader.student_id=<Unicode>
Default: '*'
File glob to match student IDs. This can be changed to filter by student.
Note: this is always changed to '.' when running `nbgrader assign`, as the
assign step doesn't have any student ID associated with it.
--NbGrader.submitted_directory=<Unicode>
Default: 'submitted'
The name of the directory that contains assignments that have been submitted
by students for grading. This corresponds to the `nbgrader_step` variable in
the `directory_structure` config option.
Examples
--------
Autograde submitted assignments. This takes one argument for the
assignment id, and then (by default) autogrades assignments from the
following directory structure:
submitted/*/{assignment_id}/*.ipynb
and saves the autograded files to the corresponding directory in:
autograded/{student_id}/{assignment_id}/{notebook_id}.ipynb
The student IDs must already exist in the database. If they do not, you
can tell `nbgrader autograde` to add them on the fly by passing the
--create flag.
Note that the assignment must also be present in the database. If it is
not, you should first create it using `nbgrader assign`. Then, during
autograding, the cells that contain tests for the students' answers will
be overwritten with the master version of the tests that is saved in the
database (this prevents students from modifying the tests in order to
improve their score).
To grade all submissions for "Problem Set 1":
nbgrader autograde "Problem Set 1"
To grade only the submission by student with ID 'Hacker':
nbgrader autograde "Problem Set 1" --student Hacker
To grade only the notebooks that start with '1':
nbgrader autograde "Problem Set 1" --notebook "1*"
By default, student submissions are re-executed and their output cleared.
For long running notebooks, it can be useful to disable this with the
'--no-execute' flag:
nbgrader autograde "Problem Set 1" --no-execute
Note, however, that doing so will not guarantee that students' solutions
are correct. If you use this flag, you should make sure you manually
check all solutions. For example, if a student saved their notebook with
all outputs cleared, then using --no-execute would result in them
receiving full credit on all autograded problems.