- Java vs JavaScript for testing
- PLEASE READ IF YOU THINK YOUR ARE DOING TESTING WITH BDD - ODDS ARE YOU ARE NOT
I screwed up by using Behat, and its elegant syntax, as a testing tool rather than what it actually is — an acceptance tool. Rather than using it only to validate Lightning’s user stories (i.e., “does Lightning deliver what users want?”), I used it to test virtually everything, including bug fixes, obscure corners of functionality, edge cases, and regressions. Bad move.
We ended up with ugly, complicated tests...
Why can’t I write regression tests in Behat?
Well, technically, you can. (I did it in Lightning for 2+ years.) You can also technically throw a thousand monkeys into a building, give them two thousand keyboards, and wait for them to crack prime factorization. But recall the wisdom of Jeff Goldblum in Jurassic Park: just because you can doesn’t mean you should, and although you can write regression and deep functionality tests in Behat, it is an express train to hell. I know because I (and others) have been there.
Conclusions
No matter you are using Codeception or not it is a good idea to understand how to perform browser based acceptance testing using just the php-webdriver by itself. Php-webdriver library provides very clean and flexible APIs you will enjoy working with.
PHP is a language popular for web development, but not for web testing. Test automation engineers prefer Java and Ruby over it. And there is a serious reason for that. There is no such thing like “official Selenium bindings” for PHP, i.e. there is no Selenium client library for PHP created by Selenium team. Developers of php-webdriver get very close to the official Selenium client APIs. That’s why you should use php-webdriver - it really feels and works like native and official Selenium libraries. That is especially important if you have an experience writing acceptance tests in Java or Ruby. Moving to PHP is not that hard, when all the APIs are the same.
PHP is not used outside context of a webserver
PHP
- Uses
- PHP is designed to be ran on a webserver...and encapsulated in HTML for a product
- PHP is not seen by the end user..ever.. because of the SSI or server-side-include
- Javascript can be seen by the end user, but not always
- While PHP is growing into other uses & products, it lags decades behind Java for ambiguous run-time environments
- PHP is designed to be ran on a webserver...and encapsulated in HTML for a product
- Selenium
- the best selenium.jar PHP language binding is the Facebook\php-webdriver api
- this is simply a PHP library that makes calls to the java.jar
- php-webdriver is NOT maintained by the selenium UG so will not only lag behind selenium api
- but no guarrantee ANY api call will be available
- DEV
- PHP should be used for whitebox testing of PHP back-end logic
- Not only can PHPUnit Tests be created simultaneously with dev work...
- ...but good frameworks invite TDD & BDD adoption
- making life SO MUCH EASIER on the developer by automating execution
- tweaking code & running unit/bdd test gives immediate feedback and reduces dev time
- access to native methods allows for precise & controlled debugging by using mocks or
- PHPUnit to produce issues with 1000s of iterations or permutations of test data
- can circumvent javascript & browser interfaces to isolate symptoms and their causes
- IN SAME REPO
Java
- Uses
- Java runs everything from the web, to enterprise back-end, to dishwashers and wrist watches.
- Test products are not black-box extensions of whitebox frameworks
- Selenium
- Selenium JAR is simply a zipped archive of java classes with a main() entry point
- Java will ALWAYS have the complete API available to it
- DIRECT calls to the jar library can even be made
- the JAR can be extended in its native language
- QA
- Compiled languages are the pro's choice for QA Automation - Java & C# have massive lead
- Test products have more 'real-world' interface demands and might have to drive business applications & 3rd party tools in addition to web pages
- Reporting & Test Data requirements are more demanding
- Task types are infinitely more diverse
- community is arguably the largest, making for an environment conducive & inviting to QA testers that are early in their automation career
- learning curve for new staff is easier for same reason
- availability of staff greater
- design patters are more numerous & better vetted
- QA is not testing php code.. testing browser interactions with a product including
- php backend
- javascript front end
- java business application APIs
- php not well suited for all these activities
- SEPARATE REPO/OWNER
| Java | PHP | |
| strict typing |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| selenium webdriver |
|
|
| language hooks |
|
|
|
|
|
| runtime libs |
|
|
| language community |
|
|
| whitebox testing |
|
|
| Audience |
|
|
|
||
|
||
| browser driver |
|
|
|
|
Weak Typing Hell
Great for developers, worst case scenario for testers
Even worse for distributed dev groups
Comparison of Language Handing for compiled & interpreted languages
- one will never see php error until the code runs.. which could be months or years...
- the error will only be found at runtime
- triaging a bug that has been there long past the designers availability makes for a miserable and expensive excursion into debugging.
- likely that bugs like this will have dependencies, each relying on the other's conflicting data type
- large dev teams increase the possibility of encountering confusing code and bad logic, taking debugging into reverse-engineering mode with horrific consequences
Precision instrumentation is critical to precision testing...
Strongly typed languages force better coding practices and are more self descriptive
- More easily understand the intention of author & what is taking pace
- unable to commit or build these types of logic errors
- developer can address immediately, not after a JIRA ticket is opened & deliberated & possibly assigned to another developer for remedy.
- 30s can easily turn into 8hrs
<?php
namespace App\Models;
class WeakTypingHell
{
private $text;
private $element;
private $driver;
public function __construct(){
}
public function stringOrInteger()
{
//make assignments
$a = 1;
$b = '2';
$c = $a . $b; // NEED COMPILE ERROR HERE NOW!
$d = '1' . '2';
$e = 1 . 2;
//print out the text & eval results
print "\n";
print "==============================\n";
print "==============================\n";
print "a: " . $a . "\n";
print "b: " . $b . "\n";
print "c: " . $c . "\n";
print "d: " . $d . "\n";
print "e: " . $e . "\n";
print "so.....\$c = \$a . \$b". "\n";
print "and....\$d = '1' . '2'". "\n";
print "and....\$e = 1 . 2". "\n";
print "then \$d + \$c should cause compile error and NOT evaluate as EQUALS '='". "\n";
print "and what should \$e be?....";
if ($c == $d ) { //
print "PASS?....How does c equal d?". "\n";
} else {
print ("FAIL!.. GREAT... PHP caught that c doesn't equal d");
}
package com.cs.cmsauto.testcases;
public class StrongTypingHeavenTest {
public void strongTypingHeaven() {
int a = 1;
String b = "2";
String c = a + b;
String d = "1" + "2";
String e = 1 + 2;
}
}
}
}
SDLC
- Requirements
- Architect
- Developer
- QA
- User
Role
- DevOps
- Automation with native language developers use
- whitebox & unit testing
- api calls
- integration tests
- call methods directly
- BDD/TDD
- CI/CD
- QA Automation
- QA Automation with compiled language: Java/C#
- blackbox
- headfull
- UAT
- Regression
- Functional
- Sanity
- Smoke
- BDD ( BDD can handle Sanity/Smoke/UAT/E2E & nothing else )
- BDD is for Behavior Driven Development and meant for Biz/Dev/QA to understand a how a feature to work
- Dev creates test FIRST & uses it as an automated test runner for fast iterations as feature implemented
- QA creates independent counterpart that serves as the contract with Dev & Biz on feature requirements
- QA will then create test suites to address 'all' possibilities and error conditions - possibly 1000's of tests(ie DataValidation for api calls or Forms )
- BDD cites one example
- Also known as "specification by example"
- BDD is for Behavior Driven Development and meant for Biz/Dev/QA to understand a how a feature to work
- QA Test Plan
- test suites
- test cases, steps, & data
- scheduling
- environment specs
-----blocking issues------
At a cross roads
Mink won't cut it: basically curl + HTTP GET & POST
Sophisticated Drupal implemenation, not a simple out-of-box CMS
Phantomjs crashes
No content creation,
selector catch/transform: no CSS, hovering flaksy, limited options
[FIX] phpunit/php-webdriver framework that behat calls
Carousel NEXT/PREV inaccessible for click()
Behat ok for limited step defs, but needs test framework under it
Already have 30 defs for 100 tests
300 methods for 1000 tests
!!!
when meganav changes.....
when element changes?
when landing page changes?
!!!!
Each Feature/FeatureContext method is a framework unto itself
[FIX] phpunit/pom framework required
Jenkins not ready for prime time
pulling demo data-base which has redirects not in prod - test fail en masse
workaround is a manual process - can't do 3x daily
writing tests for unknown environment/product
Tests
No Coverage Matrix, visibility
Tests have cross pollination
Test make reference to different environments
Tests need tagging overhaul
Plan needs to be in Excel first
Tests call outdated content - 10 content items gone....
Unnecessary setup() steps
Reporting Output adequate for 100 tests: 1000?
[FIX] phpunit test base should provide logging & report handling
- Already moving past php into javascript-driven functionality....is php a good choice for this?
- who's using & maintaining
- what features first
- what is generating the features
- visual & presentation?
- "do an excellent job, but let's remove 20% of the tools available
-----blocking issues------
- Log in to post comments