Code Defenders A Mutation Testing Game

CodeDefenders is a web-based game about testing code. It turns mutation testing into a simple, competitive challenge between two roles: defenders and attackers.

Defenders write tests for a small piece of code, the Class Under Test (CUT). Attackers then change the CUT in tiny ways, creating mutants. If a test fails on a mutant, that mutant is killed. If all tests still pass, the mutant survives and scores for the attacker.

Active Battleground Games

Creator	Class	Attackers	Defenders	Level
DELETED	Lift	48	37	Hard
theCEO	Lift	2	1	Easy
theCEO	Lift	10	8	Easy
rahulkumar14581	Lift	5	7	Hard
umang	Elevador	2	4	Hard
glauber	Lift	1	1	Hard
mgebre8	Lift	1	2	Hard
theCEO	Lift	1	1	Easy
theCEO	Lift	21	12	Easy
theCEO	Lift	2	1	Easy
umang	Lift	1	2	Hard
jsk71945	BankAccount	4	4	Hard
theCEO	Lift	3	5	Easy
sverrir	Lift	1	3	Easy
ajl5uu	Lift	81	74	Easy

Active Melee Games

Creator	Class	Players	Level
josemi	Lift	6	Hard
josemi	Lift	4	Hard
josemi	Lift	5	Hard
rulands2	Lift	78	Hard
amitapavuloori	Cal	4	Hard
kJac	Lift	18	Hard
nickk2002	Lift	11	Hard
yellowmellow	Lift	2	Easy
ran5tcw	Lift	11	Hard

Research

Philipp Straubinger, Alexander Degenhart and Gordon Fraser. “An Empirical Evaluation of Manually Created Equivalent Mutants,” In Proc. of the International Workshop on Mutation Testing, IEEE, 2024. [Bibtex]

@inproceedings{Mutation24,
  title={An Empirical Evaluation of Manually Created Equivalent Mutants},
  author={Straubinger, Philipp and Degenhart, Alexander and Fraser, Gordon},
  booktitle={IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)},
  pages={237--246},
  year={2024},
  organization={IEEE}
}}

Gordon Fraser, Alessio Gambi, Marvin Kreis and José Miguel Rojas. “Gamifying a Software Testing Course with Code Defenders,” In Proc. of the ACM Technical Symposium on Computer Science Education (SIGCSE), ACM, 2019. [Bibtex]

@inproceedings{SIGSCE_2019_CodeDefenders,
    author = {Gordon Fraser and Alessio Gambi and Marvin Kreis and Jos{\'e} Miguel Rojas},
    title = {Gamifying a Software Testing Course with Code Defenders},
    booktitle = {Proc. of the ACM Technical Symposium on Computer Science Education (SIGCSE)},
    series = {SIGCSE'19},
    publisher = {ACM},
    year = {2019}
}

Gordon Fraser, Alessio Gambi, José Miguel Rojas. “A Preliminary Report on Gamifying a Software Testing Course with the Code Defenders Testing Game,” In Proc. of The European Conference of Software Engineering Education, pp. 50-54, ACM, 2018. [Bibtex]

@inproceedings{ECSEE2018_CodeDefenders,
    author = {Gordon Fraser and Alessio Gambi and Jos{\'e} Miguel Rojas},
    title = {A Preliminary Report on Gamifying a Software Testing Course with the Code Defenders Testing Game},
    booktitle = {Proc. of The European Conference of Software Engineering Education (ECSEE) 2018},
    series = {ECSEE'18},
    pages = {50--54},
    publisher = {ACM},
    year = {2018}
}

Benjamin Clegg, José Miguel Rojas, Gordon Fraser. “Teaching Software Testing Concepts Using a Mutation Testing Game,” In Proc. of the International Conference on Software Engineering: Software Engineering and Education Track (ICSE-SEET), pp. 33-36, IEEE Press, 2017. [Bibtex]

@inproceedings{ICSE_SEET2017_CodeDefenders,
    author = {Benjamin Clegg, Jos{\'e} Miguel Rojas and Gordon Fraser},
    title = {Teaching Software Testing Concepts Using a Mutation Testing Game},
    booktitle = {Proc. of the International Conference on Software Engineering : Software Engineering and Education Track (ICSE-SEET) 2017},
    series = {ICSE-SEET'17},
    pages = {33--36},
    publisher = {IEEE Press},
    year = {2017}
}

José Miguel Rojas, Thomas White, Benjamin Clegg and Gordon Fraser. “Code Defenders: Crowdsourcing Effective Tests and Subtle Mutants with a Mutation Testing Game,” In Proc. of the International Conference on Software Engineering (ICSE), pp. 677-688, IEEE, 2017. [Bibtex]

@inproceedings{ICSE2017_CodeDefenders,
    author = {Jos{\'e} Miguel Rojas, Thomas White, Benjamin Clegg and Gordon Fraser},
    title = {{C}ode {D}efenders: Crowdsourcing Effective Tests and Subtle Mutants with a Mutation Testing Game},
    booktitle = {Proc. of the International Conference on Software Engineering (ICSE) 2017},
    pages = {677--688},
    publisher = {IEEE},
    year = {2017}
}

José Miguel Rojas and Gordon Fraser. “Teaching Software Testing with a Mutation Testing Game,” In Proc. of the Annual Workshop of the Psychology of Programming Interest Group, 2016. [Bibtex]

@inproceedings{PPIG16_TeachingTesting,
    author = {Jos{\'e} Miguel Rojas and Gordon Fraser},
    title = {Teaching Software Testing with a Mutation Testing Game},
    booktitle = {Psychology of Programming Interest Group 2016 (PPIG)},
    year = {2016}
}

José Miguel Rojas and Gordon Fraser. “Teaching Mutation Testing using Gamification,” In Proc. of The European Conference of Software Engineering Education, 2016. [Bibtex]

@inproceedings{ECSEE16_MutationEducation,
    author = {Jos{\'e} Miguel Rojas and Gordon Fraser},
    title = {Teaching Mutation Testing using Gamification},
    booktitle = {Proc. of The European Conference of Software Engineering Education (ECSEE)},
    year = {2016},
    publisher = {Shaker Publishing},
    note = {To appear}
}

José Miguel Rojas and Gordon Fraser. “Code Defenders: A Mutation Testing Game,” In Proc. of The 11th International Workshop on Mutation Analysis, pp. 162-167, IEEE, 2016. [Bibtex]

@inproceedings{Mutation16_CodeDefenders,
    author = {Jos{\'e} Miguel Rojas and Gordon Fraser},
    title = {Code Defenders: A Mutation Testing Game},
    booktitle = {Proc. of The 11th International Workshop on Mutation Analysis},
    year = {2016},
    publisher = {IEEE},
    pages = {162--167}
}

Use CodeDefenders for your lessons

CodeDefenders is an open source software developed and maintained at the Chair of Software Engineering II at the University of Passau. The source code is available along with install instructions on GitHub.
We also provide docker containers for all major releases. You can find links to these containers and more information in our docker documentation.

While the public instance at code-defenders.org is available for everyone to try out CodeDefenders, we strongly recommend setting up a private instance for use in the classroom to ensure a consistent performance.