Difference between revisions of "Test Harness"

From PC2wiki
Jump to: navigation, search
m (Output Data Set: put output in table with source and command lines)
m (Input Data Set: added runs source files section)
Line 9: Line 9:
 
* [[contest.yaml]] with problems data and [[problem.yaml]]
 
* [[contest.yaml]] with problems data and [[problem.yaml]]
 
* [[runs.tsv]] - runs to be submitted
 
* [[runs.tsv]] - runs to be submitted
* team/run source files  
+
* team/run source files see Run Source Files section for more details
 
* [[groups.tsv]] - group/region info
 
* [[groups.tsv]] - group/region info
 
* [[teams.tsv]] - team info
 
* [[teams.tsv]] - team info
Line 22: Line 22:
 
* [[scoreboard.tsv]]
 
* [[scoreboard.tsv]]
 
* [[results.tsv]]
 
* [[results.tsv]]
 +
 +
===Run Source Files==
 +
 +
To submit a run information from [[runs.tsv]] and run source files are needed.
 +
Each run submission (source file) is created under its own run directory.
 +
The run directory is the word <tt>run</tt> followed by the run id (4 digits
 +
zero left padded)
 +
 +
Here is an example of run files in their run directories.
 +
 +
run0001/BadHello.java
 +
run0002/Hello.java
 +
run0003/BadHello.java
 +
run0004/Hello.java
 +
run0005/BadHello.java
 +
run0006/BadSumit.java
 +
run0007/BadSumit.java
 +
run0008/Hello.java
 +
run0009/BadSumit.java
 +
run0010/BadSumit.java
 +
run0011/BadSumit.java
  
 
==Output Data Set==
 
==Output Data Set==

Revision as of 09:49, 1 February 2013

The CCS Test Harness is a system to test a CCS by configuring a contest, submitting runs and evaluating the output.

Testing

Input Data Set

Input data list is:

Optional

These input files are also used to evaluate the same files created by the CCS.

=Run Source Files

To submit a run information from runs.tsv and run source files are needed. Each run submission (source file) is created under its own run directory. The run directory is the word run followed by the run id (4 digits zero left padded)

Here is an example of run files in their run directories.

run0001/BadHello.java
run0002/Hello.java
run0003/BadHello.java
run0004/Hello.java
run0005/BadHello.java
run0006/BadSumit.java
run0007/BadSumit.java
run0008/Hello.java
run0009/BadSumit.java
run0010/BadSumit.java
run0011/BadSumit.java

Output Data Set

Standings
Filename Source Command
runs.tsv Extractor pc2extract submission
scoreboard.tsv Extractor pc2extract scoreboard
results.tsv Extractor pc2extract results
Full Event Feed Event Feed Server nc localhost 4713
Frozen Event Feed Event Feed Server nc localhost 4714

See Also