Test Harness

From PC2wiki
Revision as of 08:49, 1 February 2013 by Doug Lane (talk | contribs) (=Run Source Files: fixed typos)
Jump to: navigation, search

The CCS Test Harness is a system to test a CCS by configuring a contest, submitting runs and evaluating the output.

Testing

Input Data Set

Input data list is:

Optional

These input files are also used to evaluate the same files created by the CCS.

Run Source Files

To submit a run information from runs.tsv and run source files are needed. Each run submission (source file) is created under its own run directory. The run directory is the word run followed by the run id (4 digits zero left padded)

Here is an example of run files in their run directories.

run0001/BadHello.java
run0002/Hello.java
run0003/BadHello.java
run0004/Hello.java
run0005/BadHello.java
run0006/BadSumit.java
run0007/BadSumit.java
run0008/Hello.java
run0009/BadSumit.java
run0010/BadSumit.java
run0011/BadSumit.java

Output Data Set

Standings
Filename Source Command
runs.tsv Extractor pc2extract submission
scoreboard.tsv Extractor pc2extract scoreboard
results.tsv Extractor pc2extract results
Full Event Feed Event Feed Server nc localhost 4713
Frozen Event Feed Event Feed Server nc localhost 4714

See Also