Behat Mink: 00 - Considerations

  • must be ran on same server drupal is built/deployed on
    • NOT good for production deployment/testing
  • Actually really bad period
    • this tightly binds the testing product with the website product
    • can't target multiple environments from a test drone/vm
    •  

 


Given the interest shown at Drupalcon Portland with both a crowded session and BOF, the next step of fostering a growing use of Behat is to have a group of interested users/developers/etc. Behat, if embraced by enough Drupal folks, has the potential to vastly improve the way we test Drupal websites.

Behat IS NOT unit testing nor a specification testing tool. Behat is a Scenario-oriented BDD framework with functional testing capabilities as part of a communication process between stake-holders and developers. Behat was created and is evolving as an agile planning and functional testing methodology. However, if you want to use it as unit testing or spec testing tool, you can, but keep in mind, that it wasn't intended to do ideologically. http://groups.drupal.org/unit-testing is a better place for discussion of unit-testing. Yes, there is some overlap, but that is because of the spectrum in the testing pyramid with the various types of testing: unit tests (base), functional/acceptance tests (middle), (G)UI tests (peak)

Behat was created to address black-box like, functional, user-observable behavior testing.

 


Behat Drupal Limitations

http://www.codepositive.com/articles/easy-effective-testing-drupal-8-sites-behat-3

So the major limitation that we came up against is that to really make use of all the available functions the Behat Drupal extension provides, you need to run tests from the same server that the site being tested is running on.

Now, dependent on your testing process this may not actually be a limitation.

For us it was, we had multiple environments setup online that we wanted any of our developers to be able to run tests against without requiring them to go through the steps of setting up the site, getting the latest tests and then running them.

To simplify the testing process we used Jenkins to automate [spinning up a virtual server with everything required to run Behat and a copy of the site being tested]. Jenkins is an Open Source automation server written in Java that can automate many different manual processes and it's a great fit tasks like this which require lots of repetative technical steps.

Automating the testing process with Jenkins means that developers only need to click a button and wait until they receive a message once tests had completed with the results.


Behat Drupal Module can call Drush CLI

  • since Behat is running on the same server as Drupal, drush commands can be used to setup test bed
  • however for E2E or UAT tests, most functions automated by drush commands should be performed as actual User Actions as to test these functions as a real-world user
  • better practice might be to create server scripts that provide a library that can be called remotely
    • again, in a production environment seems these scripts should not be present, forcing any such testing in production to be done via 'black box' end-user actions that interact with the web ui ( not the back end ).
  •  

 


Given the interest shown at Drupalcon Portland with both a crowded session and BOF, the next step of fostering a growing use of Behat is to have a group of interested users/developers/etc. Behat, if embraced by enough Drupal folks, has the potential to vastly improve the way we test Drupal websites.

Behat IS NOT unit testing nor a specification testing tool. Behat is a Scenario-oriented BDD framework with functional testing capabilities as part of a communication process between stake-holders and developers. Behat was created and is evolving as an agile planning and functional testing methodology. However, if you want to use it as unit testing or spec testing tool, you can, but keep in mind, that it wasn't intended to do ideologically. http://groups.drupal.org/unit-testing is a better place for discussion of unit-testing. Yes, there is some overlap, but that is because of the spectrum in the testing pyramid with the various types of testing: unit tests (base), functional/acceptance tests (middle), (G)UI tests (peak)

Behat was created to address black-box like, functional, user-observable behavior testing.

 

 

 

 

 

 

Tags