Skip to content

Go module for executing programs against test cases in a sandboxed environment. One of the components for CodeIIEST Online Judge.

License

Notifications You must be signed in to change notification settings

codeIIEST/test-runner

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

38 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation


Test-Runner
Go Report Card

Test-Runner is a Go module which can be used to execute programs against particular test cases and get corresponding outputs. The goal is to provide an easy module for testing programs for their correctness. This is one of the many modules required to create a completely functional backend for an "Online Judge".

  • Runs programs in a completely isolated environment using containers
  • Set and record memory and time limits using cgroups
  • Easy to use interface

Table of Contents

  1. Installation
  2. Example
  3. To-do
  4. Contributing
  5. Working

1. Installation

go get github.com/codeiiest/test-runner

2. Example

import (
	"io/ioutil"
	"log"
	"github.com/codeiiest/test-runner/runner/run"
)

func test(code string, lang string, filename string){
	in := []string{"2", "4", "5"}
	out := []string{"4", "16", "25"}
	timeLimit := 2              // Time in seconds
	memLimit := 500*1024*1024   // Memory in bytes

	res := run.Evaluate(code, lang, filename, in, out,
        len(in), timeLimit, memLimit)
	log.Print(res)
}

3. To-do

  • Limit Cpu count through cgroups (perhaps choose CPU?)
  • Support for more languages (compilation and run scripts)
  • Unit Tests
  • Makefile

4. Contributing

Please use the issue tracker. All contributions are more than welcome :)

5. Working

The gist of this module is that it runs the code passed to it in a docker container, evaluates it by comparing the output of the sent program against the expected output.


  • Code is first compiled using their respective compilers here in the function Compile(), or if it is interpreted, directly Run()

  • The compilation takes place inside the container via a bind mounts and the executable is produced

  • That executable is then run with the evaluate script with parameters of time limit, memory limit, number of test cases, and finally the runner script, which depends on whether the language is compiled or interpreted (for example, directly run python scripts, but C/C++ needs compialtion)

  • For a thorough understanding of how to use this module, have a look at the tests created for each language.

About

Go module for executing programs against test cases in a sandboxed environment. One of the components for CodeIIEST Online Judge.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published