Behat Mink: 000 - Blocking Issues in Mink for Drupal

  • Mink is probably a great tool for basic websites that don't implement a heavy back-end system.
  • Mink COULD be a winner IF
    • migrates to selenium3 webdriver
    • exposes the entire selenium3 api
    • provides a means to send XPATH directly to selenium api
      • currently catches, parses, translates, THEN sends to api
    • provides a means to send CSS directly to selenium api
      • currently catches, parses, translates, THEN sends to api

Behat front end is a solid Cucumber BDD Engine

  • Still requires a sensible & well-designed test framework to hook into
  • Design the framework around PHPUnit
  • Use Behat to provide BDD translation for feature files
  • BDD isn't test... dev tool
    • one scenario could easily have 100 test cases
  • Cucumber BDD ports are designed to make TDD accessible to Biz stakeholders in an agile environment
    • BDD is not suited for Functional/Regression Testing
    • By design, the cost paid for BDD 'readability' is framework 'functionality' and flexibility
    • restricting test logic naturally reduces capability
    • BDD does not allow for large & complex data sets
  • BDD & Out of Box Mink does not fill the need for a test framework
    • it provides a Cucumber runner
    • the test framework is still a requirement

Test Data ( without a custom test framework underneath Mink... )

  • Test Data must reside in the feature file
    • so duplication is unavoidable
  • no access to data providers

 

Forced selector translation

  • CSS selectors translated to XPATH - no option
  • XPATH selectors translated to native - no option
  • Selenium2 binding... buggy & lacking features
 

JavaScript support lacking

  • Pop-Up Div menus don't seem to be accessible - primary if javascript triggered
  • Mink will detect webserver responses, but JavaScript routines go unseen
  • Must use JavaScript executioner for ANY JS validation/interaction
    • requiring complete rewrite of most stepdefs
  • Mink is not aware of JavaScript calls so JS validation will require an additional Context that duplicates the 'out of box' stepDefs.
    • Its best practice to use a consistent locator schema - namely around XPATH because includes the benefits of CSS selectors PLUS structural notation with selection functions that CSS lacks.
    • Mink translates CSS into XPATH, so there is no real CSS support & implementation is beholden to Mink capabilities & defects.
    • https://github.com/minkphp/Mink/issues/600

Drupal has a complex & sophisticated UI

  • Body tag cannot be written to
    • therefore ANY CKEditor field cannot be written to
    • Mink is limited to where text can be entered, further abstracting the test developer from control of the webdriver.
      • no Drupal CKEditor content creation
  • iframes can't be hooked without id/name attributes
    • due to XPATH/CSS catch, parse, translation
  • CSS has no provision for 'starts-with' or 'ends-with' making hooking impossible/brittle
    • Multiple-entry fields have 'auto-generated' id's with a numeric suffix
      • keyword-tag-ealudmkj1kj-1
      • keyword-tag-ealudmkj1kj-2
      • keyword-tag-ealudmkj1kj-3
    • Even if it did, Mink translates CSS to XPATH so no control over handling

Mink Framework is good for small & basic sites/test repos

  • having tons of non-used stepDefs makes it hard to manage & select appropriate defs
  • a custom framework to mediate between behat & mink is critical to ease of management
  •  

No POM built in

  • POM must be used for testing web pages
  • especially with large, sprawling and complex sites
  •  
  • No Object Repository so each test has hard-coded selectors
    • if/when element selectors change - each calling test will have to be updated.
    • Updating Main Menus can cause catastrophic destruction of tests, possibly requiring every test to be triaged & updated, instead of applicable selectors needing only update in one location, one time.
    • best to use Enums for object library

No TestBase built in

  • @Before/@After steps should hook into a TestBase to provide additional runtime tools like logging, assertions & reporting

PHPUnit still required for Assertions/Evaluations

  • No built in assertion functions
  • Framework should provide its OWN Eval Classes that leverage PHPUnits Assertions
    • Since Assertion routines must be developed... alluding to the assumption that a TestBase framework will be present

PhantomJS crashes

  • PhantomJS crashes during longer sessions
  • might need a better headless

 

@dev
Feature: Xpath Validation
As an anonymouse user and
check xpath selectors

    Scenario: Verify Xpath Selectors Are Interpreted Correctly
    Given I am on "/"
    Then get element by xpath '//ul[@id="main-menu"]/li/a[@href="/uc/"]'
    Then get element by xpath '//ul[@id="main-menu"]/li/a[@data-drupal-link-system-path="<front>"]'
    Then get element by xpath '//ul[@id="main-menu"]/li/a[contains(@data-drupal-link-system-path,"front")]'
    Then I should see "Linux: CentOS7: PHP7 Install"
    Then I click "Linux: CentOS7: PHP7 Install"
    Then I should not see "Product Dev-Deploy-Test Environment Ecosystem"
    Then I click element by xpath '//ul[@id="main-menu"]/li/a[contains(@data-drupal-link-system-path,"front")]'
    Then I should see "Product Dev-Deploy-Test Environment Ecosystem"
    Then Find Anchors Without href
    
    
@xpath1    
    Scenario: Verify NEW Xpath Style Works Better
    Given I am on "/"
    Then find element by js "//a[@id='Insights']"
    Then get element by xpath '//ul[@id="main-menu"]/li/a[@href="/uc/"]'
    #Then get element by xpath '//ul[@id="main-menu"]/li/a[@data-drupal-link-system-path="<front>"]'
    #Then get element by xpath '//html/*/ul'
    #Then get element by xpath '/html/**/img[@src="/media/images/logos/UCTextLogoClear4.png"]'
    Then generate xpath for 'Home'
    Then get element by xpath '//a[@href="/uc/"]'
    Then get element by xpath '//a[contains(text(),"Testing")]/../../li[3]'
    Then find element by js :locator
    
#    Scenario: Invalid User Cannot Login
#    Given I am on "/user/login"
#    When I fill in "edit-name" with "test"
#    And I fill in "edit-pass" with "test"
#    And I press "edit-submit"
#    Then I should see "Forgot"
    
    
 Then generate xpath for 'Home'                                          # Util\MinkUtilsContext::generateXpathFor()
      │ (//html/.//a
      │ [./@href][((./@id = 'Home' or normalize-space(string(.)) = 'Home' or ./@title = 'Home' or ./@rel = 'Home') or .//img[./@alt = 'Home'])] | //html/.//*
      │ [translate(./@role, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz') = 'link'][((./@id = 'Home' or ./@value = 'Home') or ./@title = 'Home' or normalize-space(string(.)) = 'Home')])[1]

 

 


Level Set


BDD
Dev Tool Only - Simply an aggrement between parties
Integration Test on a Good Day
Near impossible to be comprehensive
Test Data out of the picture
ObjectLibrary out of the picture


Behat good Cucumber BDD Engine
Faithful Java port
not replacement for TestBase/Framework

 

Mink
Probably fine for simple implementations
Schwab is pushing the envelope
Miserable selector experience
Must think like selenium, then translate through Mink, using PHP Bindings
Feels like a toy
1/2 a solution = no solution
Ditch it ASAP OR use it only for goutte/curl 'read only' capabilities


Blockers
Menu & Other JS callbacks
Duplicate links... no way to differentiate....because
Forced CSS/XPATH/NamedSelectors Translation & No way to workaround/bypass it
No Typing into non-field elements
No CKEditor fields
Not Aware of JS activity
Selenium2WebDriver binding
Complete API not exposed
JS Workarounds could regain some functionality, but JS DOM doesn't have as much capability
If JS JQuery XPATH/CSS selectors are solid, this is a workaround, but will require complete rewrite of selector logic
 

 

 


Framework
Behat doesn't replace test framework
Behat extends a test framework
PHPUnit should be framework - Provides Assertions
Facebook/PHP gets out of the way & just provides access
Selenium3+
Logging
Setup/Teardown


POM
Need TestBase/Framework
OBject Library ( preferable Enum )
PageContexts with Enum Objects

 

Owner

  • Need Use Cases & Workflows
  • What's Mgmt Vision
  • How will it be used
  • inputs & outputs
  • Dump Mink -> Facebook/PHP & PHPUnit <- BehatBDD Module
  • Need time to architect backbone
  • Select headless/full webdrivers
  • Define Smoke Test
  • Define Integration Test

 

Contributor

  • Need JIRA Tickets prioritized & assigned
  • Who is leading project
  • Assist with Writing BDD
  • What's gameplan

 

Advisor

  • Create Docs
  • TOI
  • Assist with Writing BDD

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Copyright 2018· All rights reserved