PHP Sticking Points
* Strong Typing
* False positives
* Interpreting Code Intent
* Debugging tests difficult, primary with larger teams
* swapping int/string function arguments doesn't produce any IDE errors, so must detect/debug runtime.... could go undetected
* Framework changes can create a multitude of undetected problems
* Return types limited
* can't return Array of Objects, so IDE intellisense breaks
* broken intellisense can't detect logic errors
* trial and error required for using object functions since code hints not available.
* Enums & other constructs not available
Publication, Community & In-house SME Resources
* no in-house expertise with PHPUnit or PHP Test Frameworks ( Unit or Functional )
* PHPUnit Publications are limited and don't address Functional Testing
* PHP Essentials has a quality section addressing Selenium, but uses Unit Test design patterns, directly referencing selectors in tests ( anti-pattern )
* Community support is sparse and PHP Selenium is an edge case - where developers are also managing test functions & using BDD as acceptance & functional test
* in this scenario, PHPUnit is already in use for Unit Testing and Selenium/BDD are extensions to this framework.
CI/CD
* Proof-of-Concept accomplished & resources are bottleneck to meaningful progress
* Behat tests target I&I, T&I, but Demo is only sanctioned build by Jenkins Owner
* Recommend creation of some lightweight BehatMink or PHPUnit curl touch tests for DemoDB with LinkChecks, API services, and configuration checks
* There will some time before Jenkins will be able to spawn multiple Drupal sites and TestAuto is getting bogged down with forcing tests before the targets exist.
* TEST environment is available and provides all required SUTs for Regression
* Content Authoring support already available & can be used once sites are unified....
* Advocate standing up QA team with core automation framework for immediate work on a PageObjectLibrary for all platforms
* Can assign QA with RENDERING support while the AUTHORING is ported
* non-coders can be cleaning up tests, discovering XPath strings for hand-off to coders, and run/evaluate tests/results
* Framework will require some upfront work, but will make test creation/maintenance cost effective
* With AUTHORING & RENDING support, QA to construct Smoke Test
* Dev Ops inherits Smoke Test tailored for Demo?
* When available, QA can create custom suites for multiple DevOps environments ( I&T/T&I/About )
* OnBoard more QA coders for larger Functional Test Suites
* ADD API test libraries for Adobe & Schwab WS
* Create Integration Suites & Provide to DevOps
* Create Functional Suites for testing entire API
* Create QA portal, possibly using an Aquia Drupal instance for Status, Reporting,
Behat Initiative ( Server-Side Non-UI Diags/Test )
* Impressed with Behat implementation
* reads data from GoogleDocs
* uses 2 BehatStepDefs
* FeatureContext has all the test framework heavy lifting
* Should push Context & Utility logic to src/Framwork
* StepDefs make simple calls to Framework
* PHPUnit tests can also leverage functions
* Devs can build up src/Framework utilities
* Drupal/Drush library
* DB/Acquia funcs
* PHP is the best choice for these type of server-side activities
Selenium Initiative ( UI/API Functional & Systems Test )
* Create a multifaceted test platform, with Selenium POM as a Plug-In Library
* Should be equally nimble with API testing and leverage same reporting/logging framework components
* TestNG provides the Functional Test runner & Summary Reporting output
* JUnit also possible and is commonly used with Cucumber
* Cucumber provides Gherkin / StepDef support
* Extent Reports provides a reporting layer that can be called throughout the framework for providing descriptive output
* Log4J provides a logging layer
* Maven provides Composer-style dependency management
* ADCATT is an advanced test framework in use & under active development
* has all design patters & lessons learned from many years of use
* SauceLabs
* depending on how tightly coupled it is with the SUT, possible that framework could be ported to CMS
Steward
* https://github.com/lmc-eu/steward
* This framework appears to be making a very promising step in providing design patterns for functional test.
* It does require an advanced & intimate knowledge of PHPUnit & Unit test patterns to be able to parse out the functional test patterns
Oracle Test Automation Architecture
https://www.oracle.com/technetwork/articles/entarch/shrivastava-automated-frameworks-1692936.html
N-Tiered
https://www.sciencedirect.com/science/article/pii/S1877050914001045
https://github.com/lmc-eu/steward
- Log in to post comments