bild
School of
Computer Science
and Communication

Test cases

Student testcases collection: student-tests-2014.tar.gz (Last updated 2014-04-28 22:41)

As part of the compiler course project, students should write their own test cases and make them available to the course leader. Writing test cases is a natural part of writing a compiler, here we formalise that task by asking each project to contribute their tests.

You should write tests in each of the first 3 categories. Systematic tests, checking the envelope of the language, are encouraged. Please keep the number of live variables below 10 at all times (since we don't require register spilling for any target, i.e. handling of register allocation failures). (If you have SPILL, please do write some tests which exercise that, see below.)

Test cases come in 4 categories:

"compile" -- valid Minijava code with or without course extensions

"noncompile" -- invalid Minijava code that should not pass a correct compiler

"execute" -- like "compile", but these also execute and output some result.

"nonexecute" -- like "compile", but performs some nasty operation, typically an array-out-of-bounds operation (i.e., relies on an extension).

Each test case needs to be annotated with the extensions it requires or expects to be unavailable. These annotations are read automatically by the test system, and since the test system has a simple-minded parser (does this guy not know how to write a proper parser??) the notation needs to follow a rather simple-minded syntax:

// EXT:EXTCODE1
// EXT:EXTCODE2
// EXT:!EXTCODE3

I.e., at the start of some line, write "// EXT:" and then the extension code, from the table at the project main page.

E.g., a testcase that relies on inheritance should have

// EXT:ISC

at some place in the code. There should be a newline directly after the last visible character.

E.g., a testcase that relies on the absense of if without else in order to fail should have

// EXT:!IWE

at some place in the code.

If your test case relies on that int is really 32 bits, i.e., if it explores the overflow semantics, then please use the special pseudo-extension code INT32.

The SPILL extension is a difficult case. Please declare testscases with the SPILL keyword iff they have ≥ 35 simultaneously live values at some point. A live value is a bool, a scalar int, or a reference. If you have implemented spilling, please make sure (one of your) execute testcases have at least 35 simultaneously live values, and declare it with the SPILL keyword. (Creating silly live registers should not be hard, as part of some silly computation.)

When all code is annotated, please put it into 4 directories: named "compile", "noncompile", "execute", and "nonexecute". Please create all four directories, if you don't have tests for each category, then just leave that directory empty.

The "execute" tests should run for < < 1 second using the Java compiler on a modern computer.

For an "execute" test Foo.java, please include Foo.out with the expected output. I.e., use the same basename, with .java and .out as file name ends. (Tests should not be self-checking, since self-checking is not reliable for buggy compilers.)

Do not include class files or assembly files with your test cases.

Making the test cases available

For now, submission of test cases will be done through the AFS file system at CSC. We will make releases by using a script, a script which collects the test cases directory from your directory DD2488-test-cases.

This is how to set things up:

  cd ~
  mkdir DD2488-test-cases
  cd DD2488-test-cases
  fs setacl -clear . `whoami` all
  fs setacl . YOUR_BUDDYS_ID rwl
  fs setacl . tege:dd2488 rl
  mkdir {,non}{compile,execute}

Each test case should be put in one the the four specified sub-directories. Do not make any directories under the four specified sub-directories.

If you work with somebody else, please create just one DD2488-test-cases; the collection system knows about your group buddy.

Tests tend to have bugs, just like compilers. If you find a bug in the tests, please let pedrodcg@csc.kth.se and austrin@csc.kth.se know about it. We will report it to the culprit. We will also store a hash of the buggy test in the test collection system, making it disappear until it is fixed by you.

After you've received a bug report, you are expected to fix the bug in a very timely fashion, and update your AFS directories.

Copyright © Published by: Torbjörn Granlund <www-kurs@csc.kth.se>
Updated 2014-05-30