Nirvana for tester — Nerrvana

More than four years ago, we attended the creation of a convenient system to automatically run the Selenium tests of our products on different browsers. Then no one has heard of the Sauce Labs, and all came up with something of their own. We also came up — did my system, which is able to independently run the Selenium tests in the selected browsers and OS on schedule. Well, and stores the execution results, of course.

At some point we decided that the system can be useful not only to us, and decided to make the service available to everyone.

Not so long ago this service is Nerrvana was opened to all comers.


the

That is able to do service

For each run of tests generates the right amount of Selenium RC/node in the cloud, which are combined into a Selenium Grid s. In some ways the service is similar to Sauce Labs, but has a key difference: it not only provides the Selenium node, and performs user tests on them. At the same time, to run tests on different browsers, the code tests nothing should be changed — we just have to put ticks in the web interface.

Work with the service looks like this: you have Selenium tests, you FTP download in Nerrvana. Through the web interface (or via API) you choose (put a tick) on which browsers the tests to run, and choose whether this will be a one-off run or the tests need to run on a schedule.
If your tests can run in multiple threads — you can specify how many Selenium nodes you want to use for each browser.

Your work ends, then it all makes Nerrvana.

It will run your tests in the selected browsers at scheduled intervals, storing the results of each run, that you can later view. The tests will run simultaneously for the selected browsers (but, of course, independently from each other).

If you wish, you can easily let your tests report Nerrvana the error — then Nerrvana will be able to notify you, for example, by mail.

You can also use a plugin for Jenkins which can update the tests and run them in Nerrvana for every new build. You can still add in Selenium IDE Nerrvana formatter who will create the tests that are optimized to work with Nerrvana (read more).

the

How it works

As it so happens, you can see for yourself, if you have five minutes of free time.
(click images for full resolution)

the
1. Get in Nirvana (need email, you will receive account details).
the
2. Click Add new test run
   Screenshot

   
the
3. Fill in the field "Test run name", put a check next to the desired platform (temporarily, you can only choose two platforms at a time, or two Node on the platform — gebrettert)
the
4. is selected as the executable file habratest/rebuildAndRun.sh. I wrote a basic test for Selenium1*, which opens the Habr, and what it does, and then shows in Nerrana results.
the
5. Choose "Option 1", "Run" (in option 2, i.e. the run on a schedule, the ability to specify how often disabled because of garraffello too).
   Screenshot

   
the
6. Click "Go ahead", and on the Dashboard to see how the test is performed.
   Screenshot

   
the
7. you Can navigate to the page with the results of the implementation and see that the test wished to inform them before proceeding. You can click on the eye icon (will need a login and password that came in the mail) and see what report he made. The report is in _files_out/habratest/logs/html/habraReport.html to create it using our small framework.
   performance Report

   
   FTP Results

   
   Generated tests report

   

About the reports you can read more here (example, example), and if you want, you can use it for their tests.
In the end, you can look at the code of the test itself.

Here are the screenshots made the tests in different browsers: FF, Chrome, Safari, IE10.

If you still want to play — FTP change the test code or put to another test. In tests you can use as Selenium1 and WD — simple_test_java in the folder you will find examples.

the

What's next


Actually, we don't quite represent where we go, and how to use our differences from Sauce Labs. For our purposes Nerrvana enough for the eyes, but perhaps you will be able to say, the service could be useful to you, whether to develop it and how? Some of our thoughts on the matter:

the
the
• Nerrvana supports the execution of tests in Java, PHP, Ruby, Python, Perl, in particular may work with tests that use frameworks like Сucumber, Robot Framework, Thucydides and the other (we plan to write about how to use these frameworks in Nerrvana).
the
• Tests to Nerrvana, i.e. you can run them from anywhere, or it can make your client.
the
• So as a test run in Nerrvana schedule, it can help to monitor some key features of your websites.
the
• Servers Nerrvana can be spread around the world and give the opportunity to choose where to run the tests.
the
• Tests run much faster than tests using Sauce Labs, since there is no delay when communicating with a Selenium test node.
the
• Nerrvana instance can be deployed inside the corporate network, if safety or other reasons do not allow to run the tests outside
the
• from the Shortcoming that the tests we have — the set of software is limited by the fact that pre-installed by us (unless it is a library jar, of course). We are considering the use Chef to be able to run the tests needed by the configuration version of OS, version of Java/PHP, the Selenium, etc configurable machine will run longer but will be more flexible.
the
• If you implement the previous paragraph, Nerrvana can become a CI server in the cloud (like TravisCI), is able to perform not only the Unit, but the Selenium tests.


In General, you can do a lot, but it's not clear what will really be interesting and relevant.

I apologize in advance if the same happens gebrettert and it will cause some problems in the service. In addition to us, now the real user is almost there, and because we use only one physical server. If need be, in the future we will add servers.

We will be glad, if Nerrvana will be useful for some of you.
And I'll be glad to hear comments and answer questions. Perhaps some answers are already there here.

<sub>______
* Selenium1 in the test used to show our framework for the report, we have not yet had time to write for WD. Found that my test Selenium1 somehow doesn't work well in Opera. For WD in Opera, the same thing works OK if you try, keep in mind. Anyway, in my experience, WD is already running the first stable Selenium. Example test on the WD lies in simple_test_java.</sub>

Article based on information from habrahabr.ru