Difference between revisions of "Problem.yaml"

From PC2wiki
Jump to: navigation, search
m (Data File Locations: added more detail)
m (Stop of first failure: add stub divisions)
 
(12 intermediate revisions by 2 users not shown)
Line 2: Line 2:
 
compose a problem description and data files.
 
compose a problem description and data files.
  
Each problem.yaml defines a single problem.
+
See the article [[Problem Format]] for a complete
 +
list of files that describe a problem.
  
Each problem is defined in its own directory (folder), in that
+
== Contents of problem.yaml ==
folder is the problem.yaml file and the other problem specific
 
files.  The folder name is referenced in the contest.yaml file.
 
  
==Sample problem.yaml==
+
{|border="1" cellpadding="1"
 +
|+ problem.yaml keys
 +
! Key !! Comments !! Example
 +
|-
 +
| source || optional || ICPC World Finals 2011
 +
|-
 +
| author || optional, defaults to "Unknown"  ||
 +
|-
 +
| license || optional, defaults to "cc by-sa"  ||
 +
|-
 +
| rights_owner || mandatory || ICPC
 +
|-
 +
| keywords || optional ||
 +
|-
 +
| difficulty || optional ||
 +
|-
 +
| limits ||problem limitations ||
 +
|-
 +
| validator
 +
| optional, see details below| validator options
 +
| case_sensitive    space_change_sensitive
 +
|}
  
  # Problem configuration
+
 
  source: ICPC Mid-Atlantic Regional Contest
+
 
  author: John von Judge  
+
{|border="1" cellpadding="1"
  rights_owner: ICPC
+
|+ PC^2 problem.yaml keys
 +
! Key !! Comments !! Example
 +
|-
 +
| name||Problem Title|| Sumit Sample Problem
 +
|-
 +
|}
 +
 
 +
The problem name will typically be fetched from the [[Problem.tex]] file
 +
rather than using the name key.
 +
 
 +
 
 +
=== Problem Time Out Limits ===
 +
 
 +
#Run-time Limit for the problem, in whole seconds
 +
limits:
 +
    timeout: 2
 +
 +
=== PC^2 Validator ===
 +
 
 +
validator:
 +
    validatorProg: 'edu.csus.ecs.pc2.validator.clicsValidator.ClicsValidator'
 +
    validatorCmd: '{:validator} {:infile} {:ansfile} {:feedbackdir} '
 +
    usingInternal: false
 +
    validatorOption: 1
 +
    use-internal-validator: false
 +
 
 +
=== CLICS Validator ===
 +
 
 +
The CLICS validator operates by breaking the input into space-separated tokens. It then checks to see whether a token represents a floating-point value; this is done by using the Java method <code>Double.parseDouble(String)</code> -- any String which this method understands to be a valid Double is considered a floating-point token.  If a float tolerance has been specified (either float_absolute_tolerance or float_relative_tolerance or both), then the input tokens are checked for compliance with the specified tolerance. 
 +
 
 +
  #Example CLICS Validator configuration for the problem
 +
 +
<code>validator_flags: case_sensitive space_change_sensitive float_absolute_tolerance 0.001 float_relative_tolerance 1.</code>
 +
 
 +
# case_sensitive - true if this flag is present.
 +
# space_change_sensitive  - true if this flag is present
 +
# float_absolute_tolerance d.ddd  - the floating-point absolute tolerance to be enforced
 +
# float_relative_tolerance d.ddd - the floating-point relative tolerance to be enforced
 +
  # defaults if corresponding flag is omitted: NOT case-sensitive, NOT space-sensitive, NO float tolerances
 +
 
 +
=== Judging Types ===
 +
 
 +
   
 +
#How the system handles judging for the problem
 +
judging-type:
 +
 
 +
    #Whether or not there will be an "Auto-Judge" used for this problem
 +
    computer-judged: true
 +
   
 +
    #Whether or not there will be manual review of submissions following auto-judging
 +
    manual-review: true
 
   
 
   
name: Sumit Sample Problem
+
    #Whether or not teams should be sent a "Preliminary Notification" of the initial (computer) judgement
 +
    send-prelim-judgement: true
 +
 
 +
=== Input data source ===
  
The source, author, and rights_owner are optional.
+
#How the system manages input sent to team submissions
 +
input:
 +
 +
    #Whether the team submission is to read its input from stdin: yes if "true", no (meaning "open and read from a file") if false or missing
 +
    readFromSTDIN: true
  
name: is followed by the title of the problem.
+
Read input from stdin
  
==Data File Locations==
+
input:
 +
    readFromSTDIN: true
  
Each problem author needs to create a problem.yaml file and
+
Read input from a file
the judge's data and answer files.
 
  
Below is an example of where pc2 expects data files to be
+
input:
located.  This example uses the problem short name 'sumit'.
+
    readFromSTDIN: false
  
The input data files and expected output files (answer files)
+
=== Stop of first failure ===
are created in the data/secret directory.
+
   
 +
#Whether (true) or not (false) to stop execution upon the first occurrence of a failed test case
 +
stop-on-first-failed-test-case: true
  
Example file locations are:
+
Default is false, the system will test all data sets
  sumit/problem.yaml
 
  sumit/data/secret/sumit.ans
 
  sumit/data/secret/sumit.in
 
  
Requirements:
+
=== Divisions ===
* Problem input data files end with the extension .in
 
* Judge's answer files end with the extension .ans
 
* For every .in file there must be an .ans file with the same name.
 
* data files must be in the data/secret directory
 
  
==Solution source locations==
+
groups: 'Hawaii - BYUH, Laie, Oahu D1; Hawaii - BYUH, Laie, Oahu D1; N. California - UC Berkeley D1 ; Western Washington - University of Puget Sound D1 ; Oregon - George Fox University D1 ; Canada - Simon Fraser University D1 ; Eastern Washington - Whitworth University D1'
  
These are optional source code files solutions to the problem.
+
==Sample problem.yaml files==
  
sumit/submissions/accepted/Sumit.java
+
=== Minimal sample with problem name ===
sumit/submissions/accepted/sumit.c
 
sumit/submissions/accepted/sumit.cpp
 
  
==Sample data file location==
+
# Problem configuration
 +
source: ICPC Mid-Atlantic Regional Contest
 +
author: John von Judge
 +
rights_owner: ICPC
 +
 +
# pc2-specific key
 +
name: Sumit Sample Problem
  
This sample data is for the teams, usually it is the same data
+
The source, author, and rights_owner are optional.
that is printed/listed in the problem description.
 
  
sumit/data/sample/sample.in
+
=== Sample 2 ===
  
==Examples==
+
name: 'Sumit file comp + manual file pc2val'
 +
source:
 +
author:
 +
license:
 +
rights_owner:
 +
 +
validator_flags:
 +
 +
limits:
 +
    timeout: 30
 +
 +
judging-type:
 +
    computer-judged: false
 +
    manual-review: true
 +
    send-prelim-judgement: false
 +
 +
input:
 +
    readFromSTDIN: false
 +
  
In the 9.3 distribution there are examples of contest.yaml
+
=== Sample 3 ===
and problem definitions under the samps/contests directory.
 
  
Here is the example of the files under samps/contests/sumithello/.
+
# Problem configuration, version 1.0
  contest.yaml
+
  # PC^2 Version: Version 9.3 20130117 (Thursday, January 17th 2013 06:10 UTC) Java ver 1.6.0_22 build 2591 Windows 7 6.1 (x86)
  hello/data/secret/hello.ans
+
# Created: 2013-03-15 14:42:06 PDT
  hello/data/secret/hello.in
+
---
  hello/problem_statement/problem.tex
+
  hello/problem.yaml
+
name: Practice
  sumit/data/secret/sumit.ans
+
source:
  sumit/data/secret/sumit.in
+
author:
  sumit/problem_statement/problem.tex
+
license:
  sumit/problem.yaml
+
rights_owner:
 +
   
 +
  limits:
 +
    #Run-time Limit for the problem, in whole seconds
 +
    timeout: 10
 +
 +
  # CLICS Validator configuration for the problem
 +
#(defaults if corresponding flag is omitted: NOT case-sensitive, NOT space-sensitive, NO float tolerances)
 +
validator_flags: case_sensitive space_change_sensitive float_absolute_tolerance 0.001 float_relative_tolerance 1.0
 +
   
 +
  #Whether (true) or not (false) to stop execution upon the first occurrence of a failed test case
 +
  stop-on-first-failed-test-case: true
 +
   
 +
  #How the system handles judging for the problem
 +
  judging-type:
 +
    #Whether or not there will be an "Auto-Judge" used for this problem
 +
    computer-judged: true
 +
    #Whether or not there will be manual review of submissions following auto-judging
 +
    manual-review: true
 +
    #Whether or not teams should be sent a "Preliminary Notification" of the initial (computer) judgement
 +
    send-prelim-judgement: true
 +
   
 +
#How the system manages input sent to team submissions
 +
input:
 +
    #Whether the team submission is to read its input from stdin: yes if "true", no (meaning "open and read from a file") if false or missing
 +
    readFromSTDIN: true
  
 
==See Also==
 
==See Also==
* [[Contest Control System]]
 
 
* [[contest.yaml]]
 
* [[contest.yaml]]
 +
* [[Problem Format]]
 +
* [[problemset.yaml]]
  
[[Category:CCS|CCS]]
 
  
 +
[[Category:CLICS]]
 
[[Category:Version 9.3]]
 
[[Category:Version 9.3]]
 
[[Category:Import and Export]]
 
[[Category:Import and Export]]

Latest revision as of 22:50, 8 November 2019

The problem.yaml is one part of the set of files that compose a problem description and data files.

See the article Problem Format for a complete list of files that describe a problem.

Contents of problem.yaml

problem.yaml keys
Key Comments Example
source optional ICPC World Finals 2011
author optional, defaults to "Unknown"
license optional, defaults to "cc by-sa"
rights_owner mandatory ICPC
keywords optional
difficulty optional
limits problem limitations
validator validator options case_sensitive space_change_sensitive


PC^2 problem.yaml keys
Key Comments Example
name Problem Title Sumit Sample Problem

The problem name will typically be fetched from the Problem.tex file rather than using the name key.


Problem Time Out Limits

#Run-time Limit for the problem, in whole seconds
limits:
    timeout: 2

PC^2 Validator

validator: 
    validatorProg: 'edu.csus.ecs.pc2.validator.clicsValidator.ClicsValidator'
    validatorCmd: '{:validator} {:infile} {:ansfile} {:feedbackdir} '
    usingInternal: false
    validatorOption: 1
    use-internal-validator: false

CLICS Validator

The CLICS validator operates by breaking the input into space-separated tokens. It then checks to see whether a token represents a floating-point value; this is done by using the Java method Double.parseDouble(String) -- any String which this method understands to be a valid Double is considered a floating-point token. If a float tolerance has been specified (either float_absolute_tolerance or float_relative_tolerance or both), then the input tokens are checked for compliance with the specified tolerance.

#Example CLICS Validator configuration for the problem

validator_flags: case_sensitive space_change_sensitive float_absolute_tolerance 0.001 float_relative_tolerance 1.
# case_sensitive - true if this flag is present.
# space_change_sensitive  - true if this flag is present
# float_absolute_tolerance d.ddd  - the floating-point absolute tolerance to be enforced
# float_relative_tolerance d.ddd - the floating-point relative tolerance to be enforced
# defaults if corresponding flag is omitted: NOT case-sensitive, NOT space-sensitive, NO float tolerances

Judging Types

#How the system handles judging for the problem
judging-type:
    #Whether or not there will be an "Auto-Judge" used for this problem
    computer-judged: true

    #Whether or not there will be manual review of submissions following auto-judging
    manual-review: true

    #Whether or not teams should be sent a "Preliminary Notification" of the initial (computer) judgement
    send-prelim-judgement: true

Input data source

#How the system manages input sent to team submissions 
input:

   #Whether the team submission is to read its input from stdin: yes if "true", no (meaning "open and read from a file") if false or missing
   readFromSTDIN: true

Read input from stdin

input:
    readFromSTDIN: true

Read input from a file

input:
    readFromSTDIN: false

Stop of first failure

#Whether (true) or not (false) to stop execution upon the first occurrence of a failed test case
stop-on-first-failed-test-case: true

Default is false, the system will test all data sets

Divisions

groups: 'Hawaii - BYUH, Laie, Oahu D1; Hawaii - BYUH, Laie, Oahu D1; N. California - UC Berkeley D1 ; Western Washington - University of Puget Sound D1 ; Oregon - George Fox University D1 ; Canada - Simon Fraser University D1 ; Eastern Washington - Whitworth University D1'

Sample problem.yaml files

Minimal sample with problem name

# Problem configuration
source: ICPC Mid-Atlantic Regional Contest
author: John von Judge 
rights_owner: ICPC

# pc2-specific key 
name: Sumit Sample Problem

The source, author, and rights_owner are optional.

Sample 2

name: 'Sumit file comp + manual file pc2val'
source: 
author: 
license: 
rights_owner: 

validator_flags: 

limits:
    timeout: 30

judging-type:
    computer-judged: false
    manual-review: true
    send-prelim-judgement: false

input:
    readFromSTDIN: false

Sample 3

# Problem configuration, version 1.0 
# PC^2 Version: Version 9.3 20130117 (Thursday, January 17th 2013 06:10 UTC) Java ver 1.6.0_22 build 2591 Windows 7 6.1 (x86) 
# Created: 2013-03-15 14:42:06 PDT
--- 

name: Practice
source: 
author: 
license: 
rights_owner: 

limits:
   #Run-time Limit for the problem, in whole seconds
   timeout: 10

# CLICS Validator configuration for the problem
#(defaults if corresponding flag is omitted: NOT case-sensitive, NOT space-sensitive, NO float tolerances)
validator_flags: case_sensitive space_change_sensitive float_absolute_tolerance 0.001 float_relative_tolerance 1.0
   
#Whether (true) or not (false) to stop execution upon the first occurrence of a failed test case
stop-on-first-failed-test-case: true

#How the system handles judging for the problem
judging-type:
    #Whether or not there will be an "Auto-Judge" used for this problem
    computer-judged: true
    #Whether or not there will be manual review of submissions following auto-judging
    manual-review: true
    #Whether or not teams should be sent a "Preliminary Notification" of the initial (computer) judgement
    send-prelim-judgement: true
   
#How the system manages input sent to team submissions 
input:
   #Whether the team submission is to read its input from stdin: yes if "true", no (meaning "open and read from a file") if false or missing
   readFromSTDIN: true

See Also