AWS Fargate Automated Testing with Cucumber, Gherkin, Java, Maven, Selenium Grid Hub and Chrome and Firefox Nodes

Submitted by nigel on Monday 9th December 2019

Tutorial for creating an automated testing AWS Fargate task that will include Docker containers for a full testing framework:

  • Maven for build automation
  • Java Runtime to enable the execution of maven and the junit unit testing framework
  • Java Cucumber - a tool for BDD testing
  • Gherkin to define behaviours in the English language
  • Selenium Hub being the central loading point for the tests
  • Selenium Chrome - one of the nodes
  • Selenium Firefox - a second node
  • Apache / PHP - the web app and web server.

This is not a trivial exercise but by the end of the blog I will have a working automated testing environment which can be slotted into a DevOps pipeline. Please note this blog is not a tutorial on how to write Cucumber / Gherkin tests, however I will create a few sample Gherkin behaviours and their respective Java Cucumber stepdefs as a starting point. These can then be expanded out by your QA team. 

Starting Point

The starting point will be the simple PHP/Apache web app I put together in my previous blog here - this is a single url website which merely shows the phpinfo() status page. But it's ideal for our automated testing because our testing framework is plenty complex already, and we can still run a few tests against this simple landing page such as whether the additional PHP extensions I included in the Dockerfile are loaded correctly. 

Now we need to build out our testing tree structure. 

At a sibling level to the docroot directory I created in the previous blog, create the following:
$ mkdir -p mytests/src/test/java/mytests
$ mkdir -p mytests/src/test/resources/mytests
Now it's time to create the files I need
$ cd mytests/
$ touch pom.xml
$ touch src/test/java/mytests/RunCucumberTest.java
$ touch src/test/java/mytests/Stepdefs.java
$ touch src/test/java/mytests/SetupTestDriver.java
$ touch src/test/resources/mytests/home_page.feature
This will create a tree structure like this:
$ tree
.
|-- pom.xml
`-- src
    `-- test
        |-- java
        |   `-- mytests
        |       |-- RunCucumberTest.java
        |       |-- SetupTestDriver.java
        |       `-- Stepdefs.java
        `-- resources
            `-- mytests
                `-- home_page.feature
 
6 directories, 5 files
Now we need to flesh out those files we created. Firstly the pom file which contains a list of the project's dependencies. Careful inspection will show you that I used the hellocucumber template as my starting point.
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
 
    <groupId>hellocucumber</groupId>
    <artifactId>mytests</artifactId>
    <version>1.0.0-SNAPSHOT</version>
    <packaging>jar</packaging>
 
    <properties>
        <cucumber.version>4.2.6</cucumber.version>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>
 
    <dependencies>
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-java</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>
 
        <dependency>
            <groupId>io.cucumber</groupId>
            <artifactId>cucumber-junit</artifactId>
            <version>${cucumber.version}</version>
            <scope>test</scope>
        </dependency>
 
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
 
    <!-- https://mvnrepository.com/artifact/io.github.bonigarcia/webdrivermanager -->
    <dependency>
        <groupId>io.github.bonigarcia</groupId>
        <artifactId>webdrivermanager</artifactId>
        <version>3.7.1</version>
    </dependency>
 
	<!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-java -->
	<dependency>
    		<groupId>org.seleniumhq.selenium</groupId>
    		<artifactId>selenium-java</artifactId>
    		<version>3.141.59</version>
	</dependency>
 
    <!-- https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-chrome-driver -->
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-chrome-driver</artifactId>
        <version>3.141.59</version>
    </dependency>
 
    <!-- https://mvnrepository.com/artifact/org.apache.maven.plugins/maven-surefire-plugin -->
    <dependency>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>2.22.2</version>
    </dependency> 
 
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>1.5.6</version>
    </dependency>
 
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
        <version>1.5.6</version>
    </dependency> 
 
    </dependencies>
 
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.7.0</version>
                <configuration>
                    <encoding>UTF-8</encoding>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>
Now my Gherkin tests themselves. These are rather simple - I am checking for whether my GD and Memecached PHP extensions have been loaded, i.e. they are listed on the front page which just shows the entire PHP configuration. Under normal circumstances Gherkin test tend to be GIVEN..WHEN..THEN constructs, but since I am not actually doing anything (e.g. clicking a button to go somewhere) I only need WHEN..THEN
home_page.feature
Feature: Are we on the Home Page?
 
 
  Scenario: On the home page
    When User is on the Home Page
    Then Message Displayed Memcached
 
    When User is on the Home Page
    Then Message Displayed GD
Next is the StepDef file where I convert the Gherkin into Cucumber Java containing the actual business logic behind the quasi-English Gherkin. I don't pretend to be an expert Java coder - I have the sum total of about 4 days experience. Therefore my code is probably sub-optimal, but it does the job and an in-house Java professional could whip it into shape. I have defined two browsers - Firefox and Chrome as per our Selenium configuration, although please note there is currently no semblance of parallel running of the tests in here. That again has been left to more talented Java practitioners than myself.
StepDefs.java
package mytests;
 
import base.SetupTestDriver;
 
import java.util.concurrent.TimeUnit;
 
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import io.github.bonigarcia.wdm.WebDriverManager;
 
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;
 
import java.net.MalformedURLException;
 
import static org.junit.Assert.*;
 
 
public class Stepdefs {
 
	public static WebDriver[] drivers;
	public static String[][] browsers;
 
    @When("User is on the Home Page")
    public void user_is_on_Home_Page() throws Throwable {
       	browsers = new String[][] {
       		{"linux", "firefox", "http://127.0.0.1", "http://127.0.0.1:4444"},
            {"linux", "chrome", "http://127.0.0.1", "http://127.0.0.1:4444"}
        };
 
		drivers = new WebDriver[browsers.length];
        for (int i = 0; i < browsers.length; i++) {
        	drivers[i] = new SetupTestDriver(browsers[i][0], browsers[i][1], browsers[i][2], browsers[i][3]).getDriver();
        }
    }
 
    @Then("Message Displayed Memcached")
    public void message_displayed_memcached() throws Throwable {
        for (int i = 0; i < drivers.length; i++) {
        	Boolean exists = !drivers[i].findElements(By.name("module_memcached")).isEmpty();
        	drivers[i].quit();
        	assertTrue(exists);
        }
    }
 
	@Then("Message Displayed GD")
	public void gd() throws Throwable {
        for (int i = 0; i < drivers.length; i++) {
			Boolean exists = !drivers[i].findElements(By.name("module_gd")).isEmpty();
        	drivers[i].quit();
        	assertTrue(exists);
        }
	}
}
The StepDef file doesn't contain any Firefox or Chrome webdriver startup configuration. This has been placed in a separate Java class.
SetupTestDriver.java
// src/main/java/base/SetupTestDriver.java
package base;
 
import org.openqa.selenium.Platform;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.firefox.FirefoxOptions;
import org.openqa.selenium.firefox.FirefoxProfile;
import org.openqa.selenium.firefox.FirefoxDriverLogLevel;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.openqa.selenium.Dimension;
 
import java.net.MalformedURLException;
import java.net.URL;
import java.util.concurrent.TimeUnit;
 
public class SetupTestDriver {
    private WebDriver driver = null;
    private String browser = null;
    private String baseUrl = null;
    private String os = null;
    private String node = null;
 
    public SetupTestDriver(String os, String browser, String baseUrl, String node) throws MalformedURLException {
        this.browser = browser;
        this.os = os;
        this.baseUrl = baseUrl;
        this.node = node;
 
        Platform platform = Platform.fromString(os.toUpperCase());
        if(browser.equalsIgnoreCase("chrome")) {
            ChromeOptions chromeOptions = new ChromeOptions();
            chromeOptions.addArguments("--headless");
            chromeOptions.setCapability("platform", platform);
            this.driver = new RemoteWebDriver(new URL(node + "/wd/hub"), chromeOptions);
        } else if (browser.equalsIgnoreCase("firefox")) {
            FirefoxOptions firefoxOptions = new FirefoxOptions();
            firefoxOptions.setProfile(new FirefoxProfile());
            firefoxOptions.setCapability("platform", platform);
            firefoxOptions.setHeadless(true);
            this.driver = new RemoteWebDriver(new URL(node + "/wd/hub"), firefoxOptions);
        }
 
        this.driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
        // Maximizing the screen doesn't appear to work.
        this.driver.manage().window().setSize(new Dimension(1600,900));
        this.driver.get(baseUrl);
 
    }
 
    public String getOs() {
        return this.os;
    }
 
    public String getBrowser() {
        return this.browser;
    }
 
    public String getBaseUrl() {
        return this.baseUrl;
    }
 
    public String getNode() {
        return this.node;
    }
 
    public WebDriver getDriver() {
        return this.driver;
    }
}
Finally there's some code to perform the glue between Gherkin and Cucumber.
RunCucumberTest.java
package mytests;
 
import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
 
@RunWith(Cucumber.class)
@CucumberOptions(plugin = {"pretty"})
public class RunCucumberTest {
}
Build the Docker Codebase + PHP/Apache

In my previous blog here I showed how to add the codebase to the php:7.2-apache image. The codebase has now changed, and I now have two top level sibling directories docroot and mytests - which contains the automated test code I've just added. There is a change we need to make to the Dockerfile in this revision, and the inclusion of an entrypoint script called entry.sh. The entry.sh file will copy the test code suite into a shared Fargate non-persistent drive so that the Maven container can access the codebase and run the tests against it. Once the copy of files has been completed, Apache must be run in foreground mode to prevent the container from quitting. Below we have the new version of the Dockerfile, and the entry.sh file. The image needs to be rebuilt, but since that was covered in the previous blog, it is omitted here. 

Dockerfile
FROM php:7.2-apache
RUN apt-get update && apt-get install -y libmemcached-dev zlib1g-dev \
    libfreetype6-dev \
    libjpeg62-turbo-dev \
    libpng-dev \
    libwebp-dev \
    && pecl install memcached \
    && docker-php-ext-enable memcached \
    && docker-php-ext-configure gd --with-gd --with-webp-dir --with-jpeg-dir \
       --with-png-dir --with-zlib-dir --with-freetype-dir \
    && docker-php-ext-install gd
COPY --chown=www-data:www-data ./ /var/www/html/.
COPY entry.sh /usr/local/bin/.
ENV AH_SITE_ENVIRONMENT devops
ENV APACHE_DOCUMENT_ROOT /var/www/html/docroot
RUN sed -ri -e 's!/var/www/html!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/sites-available/*.conf
RUN sed -ri -e 's!/var/www/!${APACHE_DOCUMENT_ROOT}!g' /etc/apache2/apache2.conf /etc/apache2/conf-available/*.conf
RUN /bin/bash -c 'chmod +x /usr/local/bin/entry.sh'
entry.sh
#!/bin/bash
set -e
 
# Copy the automated tests into a shared areaaccessible to maven
cp -R /var/www/html/mytests  /var/automated_tests/.
 
# Run Apache in the foreground to prevent the container from quitting
apache2-foreground
Maven:3.6.2-jdk-13

The Java runtime is bundled with the Maven image if the correct tag is selected. I have opted for maven:3.6.2-jdk-13 which is up to date and supported. This requires a small amount of customisation, so again a Dockerfile will be needed. In this instance I want to put a wait loop in the entrypoint script to ensure that there's no attempt to run the tests before the non-persistent shared drive containing the test suite has been mounted by the d8codebase container. At the end I exec the mvn clean install command to run the tests. 

The easiest way of achieving this is to edit the mvn-entrypoint.sh script which is in /usrlocal/bin. I copied out of the image to edit and the Dockerfile copies it back into place. 

Using my previous blog I built this, tagged and pushed to ECR using the image name d8maven. The instructions aren't copied here for brevity reasons - please refer to my previous blog if you need to. 

Dockerfile
FROM maven:3.6.2-jdk-13
COPY mvn-entrypoint.sh /usr/local/bin/mvn-entrypoint.sh
RUN /bin/bash -c 'chmod +x /usr/local/bin/mvn-entrypoint.sh'
mvn-entrypoint.sh
#! /bin/sh -eu
 
# Copy files from /usr/share/maven/ref into ${MAVEN_CONFIG}
# So the initial ~/.m2 is set with expected content.
# Don't override, as this is just a reference setup
 
copy_reference_files() {
  local log="$MAVEN_CONFIG/copy_reference_file.log"
  local ref="/usr/share/maven/ref"
 
  if mkdir -p "${MAVEN_CONFIG}/repository" && touch "${log}" > /dev/null 2>&1 ; then
      cd "${ref}"
      local reflink=""
      if cp --help 2>&1 | grep -q reflink ; then
          reflink="--reflink=auto"
      fi
      if [ -n "$(find "${MAVEN_CONFIG}/repository" -maxdepth 0 -type d -empty 2>/dev/null)" ] ; then
          # destination is empty...
          echo "--- Copying all files to ${MAVEN_CONFIG} at $(date)" >> "${log}"
          cp -rv ${reflink} . "${MAVEN_CONFIG}" >> "${log}"
      else
          # destination is non-empty, copy file-by-file
          echo "--- Copying individual files to ${MAVEN_CONFIG} at $(date)" >> "${log}"
          find . -type f -exec sh -eu -c '
              log="${1}"
              shift
              reflink="${1}"
              shift
              for f in "$@" ; do
                  if [ ! -e "${MAVEN_CONFIG}/${f}" ] || [ -e "${f}.override" ] ; then
                      mkdir -p "${MAVEN_CONFIG}/$(dirname "${f}")"
                      cp -rv ${reflink} "${f}" "${MAVEN_CONFIG}/${f}" >> "${log}"
                  fi
              done
          ' _ "${log}" "${reflink}" {} +
      fi
      echo >> "${log}"
  else
    echo "Can not write to ${log}. Wrong volume permissions? Carrying on ..."
  fi
}
 
# Wait until we see the test files that are being copied from the PHP/Apache/Codebase container to a shared volume.
while [ ! -d /var/automated_tests/mytests ]
do
  sleep 2
done
 
owd="$(pwd)"
copy_reference_files
unset MAVEN_CONFIG
 
cd "${owd}"
unset owd
 
exec mvn -f /var/automated_tests/mytests -q clean install
Define the Fargate Task

I now have all parts I need apart from a shell script to actually define the Fargate task which will contain all the config for the containers. I have made extensive use of the heredoc convention in the shell script - one per container definition - which makes it easier to read, and also means when I come to make up other tasks (such as static analysis, and persistent playground) I can simple swap out those containers I don't need.

There are a few comments here to note. 

  • You cannot create a Fargate Task without the IAM ecsTaskExecutionRole, but that will not be created for you. It is however created when you manually create your first Fargate task in the AWS console. Of course ecsTaskExecutionRole can be created programmatically too - and will be the subject of a block at another time.
  • The Fargate task definition will not create your CloudFront log group for you! Therefore I do a check in the script to see if it already exists, and if not then create.
  • I have defined a mount point to where I copy the test suite. This is because the test suite is in the code repository which is copied into the PHP/Apache container, but is needed by the Maven container also.
  • The Selenium Chrome browser is using a non-standard port 5556. This is because it would clash with Firefox otherwise which also defaults to port 5555. You must remember to use an environmental variable NODE_PORT to change the port as well as the hostPort and containerPort.
  • I have defaulted the task's compute to extremely high cpu and memory. They should be reduced using trial and error to avoid incurring high compute fees.
#!/bin/bash
 
# Rewrite branch name so no feature/ hotfix/ forward slash
BRANCH="selenium-testing"
REPO="d8codebase"
 
# Repo account and repo
ECR_ACCOUNT="XXXXXXXXXXX.dkr.ecr.eu-west-2.amazonaws.com"
ECR_REPO="${ECR_ACCOUNT}/${REPO,,}"
ECS_CLUSTER="default"
 
# Get the Task Role ARN for executing the task
TASK_ARN=`aws iam get-role --role-name ecsTaskExecutionRole | jq .Role.Arn -r`
 
 
# Execution ARN appears to be the same as the Task ARN
EXECUTION_ARN="${TASK_ARN}"
 
 
# Determine if the log group already exists: NOTE: "aws ecs register-task-definition" will not create the log group for you!
LOG_EXISTS=`aws logs describe-log-groups | jq ".logGroups[] | select(.logGroupName == \"/ecs/"${BRANCH}"\") | .logGroupName" -r | wc -l`
if [[ "${LOG_EXISTS}" -eq 0 ]]; then
	aws logs create-log-group --log-group-name "/ecs/${BRANCH}"
fi
 
CONTAINER_CODEBASE=$(cat <<EOF
	{
		"dnsSearchDomains": [],
		"logConfiguration": {
			"logDriver": "awslogs",
			"options": {
				"awslogs-group": "/ecs/${BRANCH}",
				"awslogs-region": "eu-west-2",
				"awslogs-stream-prefix": "ecs"
			}
		},
		"entryPoint": [
			"/usr/local/bin/entry.sh"
		],
		"portMappings": [{
			"hostPort": 80,
			"protocol": "tcp",
			"containerPort": 80
		}],
		"command": [],
		"cpu": 0,
		"image": "${ECR_REPO}",
		"name": "d8codebase",
		"mountPoints": [{
			"sourceVolume": "automated_tests",
			"containerPath": "/var/automated_tests",
			"readOnly": false
		}]
	}
EOF
)
 
CONTAINER_SELENIUM_HUB=$(cat <<EOF
	{
		"dnsSearchDomains": [],
		"logConfiguration": {
			"logDriver": "awslogs",
			"options": {
				"awslogs-group": "/ecs/${BRANCH}",
				"awslogs-region": "eu-west-2",
				"awslogs-stream-prefix": "ecs"
			}
		},
		"entryPoint": [],
		"portMappings": [{
			"hostPort": 4444,
			"protocol": "tcp",
			"containerPort": 4444
		}],
		"command": [],
		"cpu": 0,
		"environment": [{
			"name": "SE_OPTS","value": "-debug"
		}],
		"image": "registry.hub.docker.com/selenium/hub:3.141.59-uranium",
		"name": "selenium-hub"
	}
EOF
)
 
CONTAINER_SELENIUM_FIREFOX=$(cat <<EOF
	{
		"dnsSearchDomains": [],
		"logConfiguration": {
			"logDriver": "awslogs",
			"options": {
				"awslogs-group": "/ecs/${BRANCH}",
				"awslogs-region": "eu-west-2",
				"awslogs-stream-prefix": "ecs"
			}
		},
		"entryPoint": [],
		"portMappings": [{
			"hostPort": 5555,
			"protocol": "tcp",
			"containerPort": 5555
		}],
		"command": [],
		"cpu": 0,
		"environment": [{
			"name": "NODE_MAX_INSTANCES", "value": "5"
		},{
			"name": "NODE_MAX_SESSIONS", "value": "5"
		},{
			"name": "HUB_HOST", "value": "127.0.0.1"
		},{
			"name": "HUB_PORT", "value": "4444"
		}],
		"image": "registry.hub.docker.com/selenium/node-firefox:3.141.59-uranium",
		"name": "SeleniumNodeFirefox"
	}
EOF
)
 
CONTAINER_SELENIUM_CHROME=$(cat <<EOF
 	{
 		"dnsSearchDomains": [],
 		"logConfiguration": {
 			"logDriver": "awslogs",
 			"options": {
 				"awslogs-group": "/ecs/${BRANCH}",
 				"awslogs-region": "eu-west-2",
 				"awslogs-stream-prefix": "ecs"
 			}
 		},
 		"entryPoint": [],
 		"portMappings": [{
 			"hostPort": 5556,
 			"protocol": "tcp",
 			"containerPort": 5556
 		}],
 		"command": [],
 		"cpu": 0,
 		"environment": [{
 			"name": "NODE_MAX_INSTANCES", "value": "5"
 		},{
 			"name": "NODE_MAX_SESSIONS", "value": "5"
 		},{
 			"name": "HUB_HOST", "value": "127.0.0.1"
 		},{
 			"name": "HUB_PORT", "value": "4444"
 		},{
 			"name": "NODE_PORT", "value": "5556"
 		}],
 		"image": "registry.hub.docker.com/selenium/node-chrome:3.141.59-uranium",
 		"name": "SeleniumNodeChrome"
 	}
EOF
)
 
CONTAINER_MAVEN=$(cat <<EOF
	{
		"dnsSearchDomains": [],
		"logConfiguration": {
			"logDriver": "awslogs",
			"options": {
				"awslogs-group": "/ecs/${BRANCH}",
				"awslogs-region": "eu-west-2",
				"awslogs-stream-prefix": "ecs"
			}
		},
		"entryPoint": [],
		"portMappings": [],
		"command": [],
		"cpu": 0,
		"image": "${ECR_ACCOUNT}/d8maven",
		"name": "Maven",
		"essential": true,
		"mountPoints": [{
			"sourceVolume": "automated_tests",
			"containerPath": "/var/automated_tests"
		}]
	}
EOF
)
 
 
CONTAINERS_AUTOMATIC=$(cat <<EOF
[
${CONTAINER_CODEBASE},
${CONTAINER_SELENIUM_HUB},
${CONTAINER_SELENIUM_FIREFOX},
${CONTAINER_MAVEN},
${CONTAINER_SELENIUM_CHROME}
]
EOF
)
 
 
aws ecs register-task-definition \
  --family "AUTOMATION-${BRANCH}" \
  --volumes "name=automated_tests,host={}" \
  --task-role-arn "$TASK_ARN" \
  --execution-role-arn "${EXECUTION_ARN}" \
  --network-mode "awsvpc" \
  --container-definitions "${CONTAINERS_AUTOMATIC}" \
  --cpu "4096" \
  --memory "30720" \
  --requires-compatibilities "FARGATE"
Running the Fargate Task
Run Task
Running Task
All containers running
All containers stopped

Once the script is executed, the task will be defined and ready to run. This can be run by creating a new shell script (or extending the one above) or by using the console. Since this blog is becoming too code focused, I'll document the running of tasks programmatically in another blog later. For now you should be able to see your task in the console and run it. See screenshot 1 above.

Screenshot 2 show that the task has reached RUNNING state. If you are quick you can click on the task identifier and be taken to the more detailed screen shown in image 3 which shows all the containers that are running. 

Once one of the Docker images completes its activity, a signal will be sent to all the other containers to stop. In the task described in this blog, the first task to complete is the running of the automated tests. Once the tests complete, then all other containers will stop. See image 4. 

Did the Tests Actually Succeed?
CloudWatch success

A valid question! The output of the Maven container is written to a CloudWatch log stream which can be parsed and echoed back to the screen, or sent to a Slack channel (more on this in a later blog). For now I have created a sequence of commands using pipes which includes the open source awslogs utility.

$ aws logs describe-log-streams --log-group-name "/ecs/selenium-testing" --log-stream-name-prefix "ecs/Maven" | jq '.logStreams|=sort_by(-.creationTime)|.logStreams[0].logeamName' -r | xargs -I{} awslogs get "/ecs/selenium-testing" {} -s1d | cut -d ' ' -f 1,2 --complement
mvn
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running mytests.RunCucumberTest
Dec 10, 2019 3:43:46 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Dec 10, 2019 3:43:47 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Dec 10, 2019 3:43:49 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Dec 10, 2019 3:43:50 PM org.openqa.selenium.remote.ProtocolHandshake createSession
INFO: Detected dialect: W3C
Feature: Are we on the Home Page?
  Scenario: On the home page         # mytests/home_page.feature:4
    When User is on the Home Page    # Stepdefs.user_is_on_Home_Page()
    Then Message Displayed Memcached # Stepdefs.message_displayed_memcached()
    When User is on the Home Page    # Stepdefs.user_is_on_Home_Page()
    Then Message Displayed GD        # Stepdefs.gd()
1 Scenarios (1 passed)
4 Steps (4 passed)
0m6.835s
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.055 sec
Results :
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
Troubleshooting Selenium Grid
Custom Rule
Selenium Grid

Getting the Selenium stack working can be fiddly, although the CloudWatch logs are a very good way of diagnosing problems. If you want to satisy yourself that Selenium Hub and its two nodes (Firefox and Chrome) are actually there and available, there is a way but you'll have to work fast on the console since the Fargate tasks completes and shuts down once the tests are complete. 

When the Fargate task transitions from PENDING to RUNNING, click on the task identifier. Copy the public IP address into your clipboard - you'll need it shortly. Now click on the ENI (Elastic Network Interface) link which will launch a new tab. Then click on the security group. Now you need to create a new rule. By default port 4444 is blocked on the public IP address - and to see Selenium working I need that to be open. 

Edit the rules, create a new custom TCP/IP rule, add 4444 as the port and select anywhere for the CIDR, then save. See screenshot 1 above. 

You now instantly have an open port 4444. Open a new browser tab, then paste the IP address you copied a few steps ago, then append /grid/console to it and enter. You should see your Grid + Node configuration as per the second screenshot above.