The TIGRIS testing system
Submissions to TIGRIS are made via mail, results are mailed back as two separate messages (one with compilation results and, if compilation of the compiler and tests went well, another with results from running the execute/nonexecute tests).
You need an account in order to use TIGRIS. This is not your KTH account, but a separate account system. Students of DD2488 will be given such an account at project start.
The TIGRIS test system for your compilers has been implemented. This system checks that compilers correctly accept just correct Minijava code, and that they generate correct code for well-defined testcases. It also checks that some run-time errors are handled when such feature are claimed.
After you have mailed your submission, it is queued for compilation of the
compiler and compilation of the tests. If TIGRIS cannot even unpack your
archive, you will get an error message as a mail bounce, or in some cases at
the address registered and deduced from the ID in the
You will get mail when compilation of your compiler and compilation of the tests using your compiler are completed. Expect at least 3 minutes, and up to several hours if the queue is long. (There is a limit for how slow compilers are allowed to be; currently 8 minutes for compiling and running all tests. This is plenty, please don't worry about your compiler's performance.)
If you make several submissions in a short period of time, only the last will actually be handled by TIGRIS; older submissions will be archived but otherwise ignored.
Iff compilation went well, you will get a 2nd mail, with execution results. This will be quick for targets JVM and X86; expect it to arrive within 2 minutes after a compilation success mail. Other targets are currently much slower, they require 10-20 minutes.
Here is how to submit your compiler to the test system:
There is a simple shell script that submits
compilers to TIGRIS. (This is also available from Nada's machines at
Replace [DIRECTORY-JUST-ABOVE-COMPILER-SOURCE-DIR] and [COMPILER-SOURCE-DIR] as appropriate.
In [COMPILER-SOURCE-DIR], you need these files:
Note that you should not include any
TIGRIS has its builtin runtime.c, matching the one on the course pages. If you don't like that version, write your own and put it into your src directory. TIGRIS will find it there.
| ||JVM compiler|
| ||x86-64 compiler|
| ||ppc32 compiler|
| ||sparc32 compiler|
| ||arm compiler|
| ||mips32 compiler|
Y.jar are files from your
subdirectory. The test system will assume all
.jar files from
lib subdirectory are needed.
(For compilers not written in Java, the generated compiler should be called
mjc. The invocation changes accordingly, TIGRIS will pass the
target (JVM, ARM, etc) as the first argument. A typical invocation will look
mjc JVM /some/path/foo.java -S
-S option tells your compiler to generate assembly code (for a
JVM backend jasmin code) where the filename should be
foo.s if the
source code was in
/some/path/foo.java, i.e., just keep the
basename for the output, i.e., place the file in ".".
For the target JVM, you actually need to generate one file per class, to adhere
to the JVM execution environment. Use either
.j as extension, TIGRIS handles both. The files need to be
placed in ".".
Generating anything but the assembly code (in one file for non-JVM targets, or
one file per class for the JVM target) will result in errors. No directories,
no class files! (But without
-S, your compiler should perhaps
generate class files, object files, or whatnot; TIGRIS will not explore the
If the test case is valid, your compiler should exit with System.exit(0). If the test case is invalid, your compiler should exit with System.exit(1). The validity criteria should be syntax, typing, etc.
Your compiler is run on a Unix-type system, just like any process. It is not a limited environment. The system is however run using a throw-away disk image. Misbehaved processes should just lead to judging failure, but risk to slow down TIGRIS for other students.
Processes programmed for intentional misbehaviour will not be tolerated.
TIGRIS was developed by Torbjörn Granlund. The Xen based re-installation of TIGRIS for komp14 was done in February 2014 in close cooperation with Ragnar Sundblad of the CSC System Group.
This section is mainly for the curious. But if you experience a problem with TIGRIS which you cannot reproduce, the versions listed here might provide some guidance.
Incoming submissions are received by the master system and queued, and the same system then asynchronously takes judging tasks. An incoming compiler is first sent to a compilation virtual system where the compiler is compiled and the newly compiled compiler is run over the testsuite. The result is assembly code for the execute and nonexecute test categories, one set for each supported target. Each of the assembly code sets is then sent to an appropriate execute virtual system.
Each judging phase uses a pristine system disk image. This is part of the Tigris security model.
The virtual systems use a combination of Xen and Qemu. Xen is used whenever X86 is usable, Qemu is used for ARM, PPC, MIPS. Previous versions of TIGRIS used Qemu for everything, and the switch to Xen as well as other improvements made TIGRIS 20 times faster.
The master system runs NetBSD 6.1 with Xen 4.2.3. It employs Qemu 1.7.0. The hardware is a low-end rackmount system with an Intel Xeon E31220 CPU running at 3.10GHz, and two mirrored S3500 SSD drives.
Compilers written in Java, C, and C++ are handled by
NetBSD 6.1.3. This system's
gcc invokes gcc 4.5.3. This system's
gcc48 invokes gcc 4.8.2. This system's
invokes apache ant 1.9.2.
Compilers written in Haskell are handled by
comp2 running FreeBSD
9.2. This system's
ghc command invokes Glasgow Haskell compiler
X86 assembling/linking and target execution is handled by
running NetBSD 6.1.3. This system's
commands come from binutils 2.21.1.
ARM assembling/linking and target execution is handled by
running GNU/Linux Debian 7.3 under Qemu.
MIPS assembling/linking and target execution is handled by
execmips running GNU/Linux Debian 7.3 under Qemu.
PPC assembling/linking and target execution is handled by
execppc running GNU/Linux Debian 7.4 under Qemu.
SPARC assembling/linking and target execution is handled by
execsparc running NetBSD 6.1.3 under Qemu.