Nirvana for tester — Nerrvana

More than four years ago, we attended the creation of a convenient system to automatically run the Selenium tests of our products on different browsers. Then no one has heard of the Sauce Labs, and all came up with something of their own. We also came up — did my system, which is able to independently run the Selenium tests in the selected browsers and OS on schedule. Well, and stores the execution results, of course.

At some point we decided that the system can be useful not only to us, and decided to make the service available to everyone.

Not so long ago this service is Nerrvana was opened to all comers.


the

That is able to do service


For each run of tests generates the right amount of Selenium RC/node in the cloud, which are combined into a Selenium Grid s. In some ways the service is similar to Sauce Labs, but has a key difference: it not only provides the Selenium node, and performs user tests on them. At the same time, to run tests on different browsers, the code tests nothing should be changed — we just have to put ticks in the web interface.

Work with the service looks like this: you have Selenium tests, you FTP download in Nerrvana. Through the web interface (or via API) you choose (put a tick) on which browsers the tests to run, and choose whether this will be a one-off run or the tests need to run on a schedule.
If your tests can run in multiple threads — you can specify how many Selenium nodes you want to use for each browser.

Your work ends, then it all makes Nerrvana.

It will run your tests in the selected browsers at scheduled intervals, storing the results of each run, that you can later view. The tests will run simultaneously for the selected browsers (but, of course, independently from each other).

If you wish, you can easily let your tests report Nerrvana the error — then Nerrvana will be able to notify you, for example, by mail.

You can also use a plugin for Jenkins which can update the tests and run them in Nerrvana for every new build. You can still add in Selenium IDE Nerrvana formatter who will create the tests that are optimized to work with Nerrvana (read more).

the

How it works


As it so happens, you can see for yourself, if you have five minutes of free time.
(click images for full resolution)
    the
  1. Get in Nirvana (need email, you will receive account details).
    2. the
  2. Click Add new test run
    Screenshot
    3. the
  3. Fill in the field "Test run name", put a check next to the desired platform (temporarily, you can only choose two platforms at a time, or two Node on the platform — gebrettert)
    4. the
  4. is selected as the executable file habratest/rebuildAndRun.sh. I wrote a basic test for Selenium1*, which opens the Habr, and what it does, and then shows in Nerrana results.
    5. the
  5. Choose "Option 1", "Run" (in option 2, i.e. the run on a schedule, the ability to specify how often disabled because of garraffello too).
    Screenshot
    6. the
  6. Click "Go ahead", and on the Dashboard to see how the test is performed.
    Screenshot
    7. the
  7. you Can navigate to the page with the results of the implementation and see that the test wished to inform them before proceeding. You can click on the eye icon (will need a login and password that came in the mail) and see what report he made. The report is in _files_out/habratest/logs/html/habraReport.html to create it using our small framework.
    performance Report
    FTP Results
    Generated tests report
    image

    8. About the reports you can read more here (example, example), and if you want, you can use it for their tests.
    In the end, you can look at the code of the test itself.

    Here are the screenshots made the tests in different browsers: FF, Chrome, Safari, IE10.

    If you still want to play — FTP change the test code or put to another test. In tests you can use as Selenium1 and WD — simple_test_java in the folder you will find examples.

    the

    What's next


    Actually, we don't quite represent where we go, and how to use our differences from Sauce Labs. For our purposes Nerrvana enough for the eyes, but perhaps you will be able to say, the service could be useful to you, whether to develop it and how? Some of our thoughts on the matter:

    the
      the
    • Nerrvana supports the execution of tests in Java, PHP, Ruby, Python, Perl, in particular may work with tests that use frameworks like Сucumber, Robot Framework, Thucydides and the other (we plan to write about how to use these frameworks in Nerrvana).
      • the
    • Tests to Nerrvana, i.e. you can run them from anywhere, or it can make your client.
      • the
    • So as a test run in Nerrvana schedule, it can help to monitor some key features of your websites.
      • the
    • Servers Nerrvana can be spread around the world and give the opportunity to choose where to run the tests.
      • the
    • Tests run much faster than tests using Sauce Labs, since there is no delay when communicating with a Selenium test node.
      • the
    • Nerrvana instance can be deployed inside the corporate network, if safety or other reasons do not allow to run the tests outside
      • the
    • from the Shortcoming that the tests we have — the set of software is limited by the fact that pre-installed by us (unless it is a library jar, of course). We are considering the use Chef to be able to run the tests needed by the configuration version of OS, version of Java/PHP, the Selenium, etc configurable machine will run longer but will be more flexible.
      • the
    • If you implement the previous paragraph, Nerrvana can become a CI server in the cloud (like TravisCI), is able to perform not only the Unit, but the Selenium tests.


    In General, you can do a lot, but it's not clear what will really be interesting and relevant.

    I apologize in advance if the same happens gebrettert and it will cause some problems in the service. In addition to us, now the real user is almost there, and because we use only one physical server. If need be, in the future we will add servers.

    We will be glad, if Nerrvana will be useful for some of you.
    And I'll be glad to hear comments and answer questions. Perhaps some answers are already there here.

    ______
    * Selenium1 in the test used to show our framework for the report, we have not yet had time to write for WD. Found that my test Selenium1 somehow doesn't work well in Opera. For WD in Opera, the same thing works OK if you try, keep in mind. Anyway, in my experience, WD is already running the first stable Selenium. Example test on the WD lies in simple_test_java.
Article based on information from habrahabr.ru

Комментарии

Отправить комментарий

Популярные сообщения из этого блога

Integration of PostgreSQL with MS SQL Server for those who want faster and deeper

Изображение
Recently habré was already published integration PostgreSQL and MSSQL. But, items there is categorically not enough. Therefore, the objectives of this publication are the following: the the to broaden and deepen publicly available information about FDW for MSSQL-called tds_fdw to tell the difference in major versions and describe basic compatibility issues; the to talk about the possibilities of optimization of queries that use the foreign table; the to address the subject of caching external data in a materialized view; the to say a few words about the exotic ways to integrate PostgreSQL and MSSQL. the Installing and configuring TDS FDW The guys from PostgresPro already sufficiently said about this process, I will not repeat. Will leave a few links to the official PostgreSQL documentation and examples from tds_fdw: the the creating an extension (extension) in PG: off.dokumentacija ; the server: off.documentation , example of tds_fdw ; the creating
Далее...

Custom database queries in MODx Revolution

This topic will certainly be useful for those who have quite a lot of projects on MODx Revolution, as with the usual sites of business cards is enough and the standard working methods of a La $modx- > getObject() , $modx->getCollection() , etc., and these methods are inherently not just a job database, but also with the objects of MODx. When working with large projects methods a La $modx- > getCollection() for us is not the best solution for two reasons: 1. The overexpenditures. These methods are not just getting data from DB, but also create instance of the retrieved objects. In this case, getting information about 10000 documents, we get 10000 objects of modResource that it is not very cool. 2. The task is complicated by the count of records. In addition to the direct difficulties of counting even at the query level, even if you get 10 records of the same document (for example), MODx will return as a result only one object modResource. Although often that will su
Далее...

Parse URL in Zend Framework 2

Objective: the Have a method to disassemble the components of the string containing the URL. The string can contain both absolute and relative URLS and both these options should be correctly parsed. the And in the drain, let's say the "wrong" format absolute links without "http://". Further in the text links to the "wrong" format will be called partial absolute references . the Implement support for the "RF" domain. the Example: site.ru/page.php /page.php site.ru/page.php the scheme http the host site.ru site.ru the path page.php page.php page.php Implementation of paragraphs 1 and 2 of our problem In this we should help the class Zend\Uri\Http . He has we need methods parse($uri) , getHost() , getPath() , etc. But! When we parse the URL and type " site.ru/page.php " (without the " http:// ") getHost() will return an empty string and getPath()
Далее...