When I was running tests in very large and sophisticated system I found very interesting problem. During integration tests run in parallel there was port conflicts and port collision prevents tests to run smoothly and in the end they failed. To avoid this behavior I suggested to use very simple solution. My colleagues confirm the need and implementation was done in very short time. Just before you read the solution - please reconsider situation on your own:
- Tests runs on many JVMs
- You don't want to share state between mentioned above JVMs
- You don't want to manually change every test to enter code
The solution suggested was to use random port in range and repeat this until success. Really simple and it just works.
From technical perspective please focus on this closure from tests:
Server server = new AvailablePortScanner(minPort, maxPort).tryToExecuteWithFreePort(new Closure<Server>(this) { @Override public Server call(Object... args) { Integer freePort = (Integer) args[0]; Server server = new Server(freePort); System.setProperty("port", freePort.toString()); return server; } });
Checkout on branch:
origin/groovy/closure_port_scanner
to play with it. Have fun
Won't this blow up when you run tests in parallel? You're setting a system property so another test can override it in the meantime can't it?
ReplyDeleteThis trick was used in spock spec setup. Therefore it can be run in parallel only if running IDE/build tool supports this kind of functionality. I found part of Gradle documentation:"
ReplyDeleteTests are executed in a separate JVM, isolated from the main build process.
(...)
You can specify whether or not to execute your tests in parallel. Gradle provides parallel test execution by running multiple test processes concurrently. Each test process executes only a single test at a time, so you generally don't need to do anything special to your tests to take advantage of this. The maxParallelForks property specifies the maximum number of test processes to run at any given time. The default is 1, that is, do not execute the tests in parallel."
If everything was implemented the way described in docs there is no risk to overwrite port by accident because at a single point in time only one test is executed within process (System.properties of different processes does not collide). Of course this trick does not prevent multiple service instances on different ports.