AWS Fargate Static Analysis with PHP

Submitted by nigel on Monday 13th January 2020

In my earlier blog on AWS Fargate Automated Testing I spoke about creating an AWS Fargate task definition which would include a web app's codebase and the Cucumber / Gherkin / maven / Java runtime stack. This tutorial is an opportunity to look at the static analysis stage of the pipeline where code is inspected for vulnerabilities, coding standards, compilation errors, copy and paste detection amongst others. Since my background is in Drupal and PHP, my tests will be focused around the PHP language and will use the excellent Static Analysis Tools for PHP repo

Looking at the jakzal/phpqa repo, it is clear there is a great deal of tooling available. The purpose of this tutorial is take a small sample of these tools and create a Fargate task to run them against my codebase. My somewhat arbitrary selection is:

  • phpcpd - Copy/paste detection
  • phpcs - Codesniffer that detects coding standards violations
  • phplint - PHP syntax error checker. 

Ok - let's get started. 

Create the Dockerfile and the Entrypoint
As I have previously mentioned, Fargate is a black box and it doesn't lend itself to command line invocation of the tooling. Typically locally on a laptop we would run the jakzal/phpqa Docker image like:
$  docker run -it --rm -v $(PWD):/project -w /project  jakzal/phpqa phpcpd web/core/modules/views/src
$  docker run -it --rm -v $(PWD):/project -w /project  jakzal/phpqa phpcs web/core/modules/views/src
$  docker run -it --rm -v $(PWD):/project -w /project  jakzal/phpqa phplint web/core/modules/views/src
This could obviously be placed in a shell script. Now in Fargate we don't have access to the command line and we have to create a Fargate definition up front before that can be provisioned and then run. So it's a little bit more involved.

The simplest approach is to create a Dockerfile based on the jakzal/phpqa repo, and an extrypoint script that will run each one of the tools I have identified.

The Dockerfile is defined below. Note that I have selected PHP version 7.2. This is the closest supported version of PHP to what I'm running on my production box. Think I need to upgrade that at some point..
FROM jakzal/phpqa:1.28.1-php7.2
ADD entry.sh /entry.sh
RUN /bin/bash -c 'chmod +x /entry.sh'
ENTRYPOINT ["/entry.sh"]
Next up is the entry.sh file. First thing to note is I have added the set +e flag, which is default anyway. This is included to ensure that even if I get some pernickety error messages the script won't terminate immediately, and it acts as an aide memoire that we don't want error termination so don't accidentally copy paste in a competing set -e flag.

There is then a check whether the shared ECS volume has been loaded - this is the responsibility of the codebase container, but because containers can start at varying times, I have to ensure I don't run the PHP analysis before the volume is ready.

The path to the source code to perform the analysis on is defined as /var/static_tests/web/core/modules/views/src. This has two components. The first part /var/static_tests/ is the mount point which will correspond to my codebase top level directory. The second component, web/core/modules/views/src points to the core contributed Drupal Views module. This is purely there as a reference to some arbitrary code. I am using the codebase to my own Badzilla which is a site build with no custom code and thus I have none of my own code to use in this sample.
#!/usr/bin/env sh
 
set +e
 
while [ ! -d /var/static_tests/web ]
do
  sleep 5
  echo "Waiting for volume to be mounted and files copied into position"
done
 
phpcpd /var/static_tests/web/core/modules/views/src
phpcs /var/static_tests/web/core/modules/views/src
phplint /var/static_tests/web/core/modules/views/src
Building the Docker image locally
It isn't mandatory to test everything works before constructing the Fargate task definition, but it makes sense to me to test my work locally first. If the Fargate task subsequently fails then I know the problem area will be localised to the AWS eco-system, and probably to Fargate. So I always build locally and run locally first. To build I cd into my Dockerfile / entry.sh entry and issue:
$ docker build -f Dockerfile -t static-fargate  .
To run it's
$ docker run -it --rm -v $(PWD):/var/static_tests  static-fargate 
You can see above I am establishing the Docker volume which will be shared with my codebase.

The Docker Hub repo for this is at https://hub.docker.com/repository/docker/sanddevil/static-fargate and the GitHub repo for the code is at https://github.com/sanddevil/devops-static-php

Define the Fargate task
With everything working locally it's time to build the Fargate task. This is very similar to the script I created in my earlier automated testing blog. There are a few differences obviously. My container stack comprises of a combined PHP + Apache + my own codebase image, and the PHP static analysis jakzal/phpqa obviously. I use the shell script heredoc convention to define each container. You should note I have now expanded my codebase container to include two shared volumes - one for the automated tests (not discussed here but discussed previously) and the static tests.
#!/bin/bash
 
 
BRANCH="master"
REPO="php7.0-apache/${BRANCH}"
 
# Repo account and repo
ECR_ACCOUNT="xxxxxxxxxxx.dkr.ecr.eu-west-2.amazonaws.com"
ECR_REPO="${ECR_ACCOUNT}/${REPO,,}"
ECS_CLUSTER="FargatePrototype"
 
# Get the Task Role ARN for executing the task
TASK_ARN=`aws iam get-role --role-name ecsTaskExecutionRole | jq .Role.Arn -r`
 
 
# Execution ARN appears to be the same as the Task ARN
EXECUTION_ARN="${TASK_ARN}"
 
 
# Determine if the log group already exists: NOTE: "aws ecs register-task-definition" will not create the log group for you!
LOG_EXISTS=`aws logs describe-log-groups | jq ".logGroups[] | select(.logGroupName == \"/ecs/"${BRANCH}"\") | .logGroupName" -r | wc -l`
if [[ "${LOG_EXISTS}" -eq 0 ]]; then
	aws logs create-log-group --log-group-name "/ecs/${BRANCH}"
fi
 
CONTAINER_CODEBASE=$(cat <<EOF
	{
		"dnsSearchDomains": [],
		"logConfiguration": {
			"logDriver": "awslogs",
			"options": {
				"awslogs-group": "/ecs/${BRANCH}",
				"awslogs-region": "eu-west-2",
				"awslogs-stream-prefix": "ecs"
			}
		},
		"entryPoint": [
			"/usr/local/bin/entry.sh"
		],
		"portMappings": [{
			"hostPort": 80,
			"protocol": "tcp",
			"containerPort": 80
		}],
		"command": [],
		"cpu": 0,
		"image": "${ECR_REPO}",
		"name": "PHPApacheCodebase",
		"mountPoints": [{
			"sourceVolume": "automated_tests",
			"containerPath": "/var/automated_tests",
			"readOnly": false
		},{
			"sourceVolume": "static_tests",
			"containerPath": "/var/static_tests",
			"readOnly": false
		}]
	}
EOF
)
 
CONTAINER_STATIC_PHP=$(cat <<EOF
	{
		"dnsSearchDomains": [],
		"logConfiguration": {
			"logDriver": "awslogs",
			"options": {
				"awslogs-group": "/ecs/${BRANCH}",
				"awslogs-region": "eu-west-2",
				"awslogs-stream-prefix": "ecs"
			}
		},
		"entryPoint": [],
		"portMappings": [],
		"command": [],
		"cpu": 0,
		"image": "registry.hub.docker.com/sanddevil/static-fargate",
		"name": "StaticPHP",
		"essential": true,
		"mountPoints": [{
			"sourceVolume": "static_tests",
			"containerPath": "/var/static_tests"
		}]
	}
EOF
)
 
 
CONTAINERS_STATIC=$(cat <<EOF
[
${CONTAINER_CODEBASE},
${CONTAINER_STATIC_PHP}
]
EOF
)
 
 
aws ecs register-task-definition \
  --family "STATIC-${BRANCH}" \
  --volumes "name=static_tests,host={}" "name=automated_tests,host={}" \
  --task-role-arn "$TASK_ARN" \
  --execution-role-arn "${EXECUTION_ARN}" \
  --network-mode "awsvpc" \
  --container-definitions "${CONTAINERS_STATIC}" \
  --cpu "256" \
  --memory "512" \
  --requires-compatibilities "FARGATE"
I'm not quite done yet though. Somehow I need to copy my codebase into the shared Docker volume so that the PHP static analysis container can see it. This is done in the entry.sh script of my PHP + Apache + Codebase image (not discussed here) but the entry.sh file is shown below for completeness.
#!/bin/bash
set -e
 
# Copy the automated tests into a shared area accessible to maven
cp -R /var/www/html/test  /var/automated_tests/.
 
# Copy the PHP codebase into a shared area accessible to the PHP static analysis container
cp -R /var/www/html/web  /var/static_tests/.
 
# Run Apache in the foreground to prevent the container from quitting
apache2-foreground
Here you can see the copying I do for both the automated tests and the static analysis.
Running the Fargate task
Task Running
CloudWatch Logs

Normally I would invoke the Fargate task programmatically using AWS CLI but since that leads to a rather dry image-free blog, I will use the console in this instance. My previous blog shows how to navigate around the ECS dashboard to run the task, then you can see the task running in the first screenshot above. It will only run until the entrypoint script terminates as per the rules on how Docker works, so to capture that scrennshot I had to be quick. The second screenshot shows the CloudWatch logs for the task, and you can see there are a few copy/paste errors detected.