Automated Judging

From PC2wiki
Jump to: navigation, search

Automated Judging is a feature where PC² automatically selects a run and using a Validator gives the run a judgement. This is also known as Computer Judging.

Only Problem definitions with a Validator enabled/used can be automatically judged.

The Validator produces an output file that conforms to the International Collegiate Programming Contest Validator Interface Standard. See the article International Collegiate Programming Contest Validator Interface Standard for examples that PC² expects the validator to create.

Version 8

todo: describe in general Version 8 Automated judging.

Details

When Version 8 Auto Judging was enabled, the following judgements were added to the judgement list:

  1. No - Compilation Error
  2. No - Time-limit Exceeded
  3. No - Other - Contact Staff

In Version 8 these judgements were added to insure that the conditions above were in the judgement list, because only the judgements in the judgement list could be displayed to the team. These three judgements insured that those conditions would be sent/shown to the team.

Version 9

An overview to setup a PC² auto judging

  1. Create a judge account - create a judge account using Generate Accounts
  2. Create a problem and assign a validator using Edit Problem
  3. Assign the judge account to auto judge runs for the problem using Assign Auto Judge
  4. Login to the judge module and start autojudging, login as the judge (Judge 3), see Starting Auto Judging

Other Details

Unlike Version 8, the 3 judgements are no longer added to the judgement list.

The Version 9 auto judge feature will use value for the outcome tag produced by a Validator. To see how the outcome tag is used see the examples in the article International Collegiate Programming Contest Validator Interface Standard.

See Also