G'day:

Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

1. Intro / Nginx (this article)
2. PHP (URL TBC)
3. PHPUnit (URL TBC)
4. Bash (URL TBC)
5. MariaDB (URL TBC)
6. Installing Symfony (URL TBC)
7. Using Symfony (URL TBC)
8. Vue.js (URL TBC)

I've already written everything down to and including "Using Symfony", and will release those over the coming days. Originally I was gonna finish the whole series before releasing anything, because I know what I'm like: there's a chance I'll get bored and not finish the series, so would be a bit rubbish to release the first n parts but not finish it. But then it occurred to me that that approach is not very agile, plus if I actually release what I've done, I'm probably more likely to see it through to completion. Each article is stand-alone anyhow, for the most part. I appreciate my mates Dara McGann and Brian Sadler also giving me a nudge on this.

Intro

This "Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers" exercise should be hilarious (at my expense, I mean), cos I've never touched Vue.js, only know enough about Nginx to get myself into trouble (and sometimes - but not always - back out of trouble again), never touched Symfony really. or PHP8 yet. And still a noob with Docker, and all this requires me to work with docker-compose, which I have not ever touched until about 10min ago. So I'll warn you up front that these articles just logs my journey through all this stuff, and any knowledge I am applying is coming to me via googling and reading stuff on Stack Overflow (etc), about 5min before I put it into practice. Nothing here is an expert's view, and no promises as to whether I'm doing anything correctly. Not sure why yer reading it, actually.

The motivation behind this work is multi-fold:

• my client-side exposure is very out of date. I have not written any front-end production code for… blimey five years probably.
• Accordingly I have not touched any of the new fancy JS frameworks, so figured I should take a look at Vue.js.
• It'll need some sort of back-end. I'd usually use Silex for this, but it's end-of-life now, and the recommendation is to use Symfony instead. I've done the smallest amount on Symfony (supervising other devs whilst they do the work, basically), so this is an excuse to do some actual work with it.
• PHP8's just been released so I'll use that, although I can't see it'll make any difference from 7.x for the purposes of what I'll be doing.
• I've messed around with individual containers a bit in Docker now, but it's all been very ad hoc. This is a good excuse to tie some stuff together with Dockerfiles and docker-compose.yml.

To be completely frank, I am unenthused about this concept of "full stack" development. In my exposure to full-stack devs, what it means is instead of being good at one thing; they're very bloody ordinary at a bunch of things ("jack of all trades; masters of none"). I understand it might be useful in smaller shops, I guess. I've always been in the position that wherever I've worked there's been enough bods to have dedicated people in each role. I'm not saying there're no people who are good at an entire stack, but the bulk of people seem not to be. It strikes me as being the enterprise of people without the discipline to stick to one thing. The flipside of this is that I've been overlooked for two roles recently because they want people who do both client-side and back-end dev, and I will always admit to being weak on the front end. However I'm also pragmatic… if I need to improve my exposure here... so be it.

Right. Off we go. I'm gonna work through this in a stepped fashion; getting each component working (to a "G'day World" degree) before integrating the next component.

Before I start actually, I need to point out I am basing my approach on the excellent article "Symfony 5 development with Docker", by Martin Pham. I just found it on Google, but it's covering much of the same territory I need to cover.

Directory structure

I'm following Martin's lead here for the most part. The key bit is putting all the docker-related files into their own directory in my app directory:

(This was taken after I did the Nginx config, below, but it better shows what goes where if I include the Nginx stuff now).

Oh Martin also has the log directory as logs on the host and log in the containers. That bit me on the bum the first time I tried this (see further down), so I'm sticking with a uniform log throughout.

Nginx

Firstly I'm just going to get Nginx serving gdayWorld.html on localhost:80. I'm not yet going to worry about PHP or Symfony or anything like that.

I've largely copied Martin's docker/nginx/Dockerfile:

FROM nginx:alpine
WORKDIR /usr/share/nginx/
CMD ["nginx"]
EXPOSE 80

The only difference is I've changed the WORKDIR value from /var/www to /usr/share/nginx/ which seems to be where Nginx would naturally put its files. Martin's using /var/www because "the same directory with PHP-FPM since we’re gonna share this with a mounted volume" (that's from his article). However I will specifically not be doing that. I will be keeping the website directory and the application source code separate, as one usually would with a modern PHP app. Only the "entry-point" index.php will be in the web root; the rest will be in the adjacent src directory. In the context of this file, it doesn't matter here, but the same directory is referenced later where it does matter, and I want to keep things consistent. Also this seems to be a reference to the Nginx app directory really, not anything related to the web root, per se?

And again, the docker/nginx/nginx.conf file is lifted from Martin's article:

user  nginx;
worker_processes  4;
daemon off;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    access_log  /var/log/nginx/access.log;
    sendfile        on;
    keepalive_timeout  65;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-available/*.conf;
}

And the separate site config file, docker/nginx/sites/default.conf:

server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    server_name localhost;
root /usr/share/fullstackExercise/public;
    index index.html;

    location / {
         try_files $uri $uri/;
    }

    location ~ /\.ht {
        deny all;
    }
}

One difference here is that I'm setting the web root to be /usr/share/fullstackExercise/public, rather than a subdirectory of the Nginx application directory as Martin had it. /public is where the app's web-browseable files will be home (Symfony's location for these is that directory, relative to the application root).

I don't need the PHP stuff in this file yet, so I've removed it for now. We're only interested in serving HTML at the moment.

Lastly the docker-compose.yml file is cut down from the one in the article to just handle Nginx for now:

version: '3'

services:
    nginx:
        build:
            context: ./nginx
        volumes:
            - ../public:/usr/share/fullstackExercise/public
            - ./nginx/nginx.conf:/etc/nginx/nginx.conf
            - ./nginx/sites/:/etc/nginx/sites-available
            - ./nginx/conf.d/:/etc/nginx/conf.d
            - ../log:/var/log
        ports:
            - "80:80"

As mentioned above, the only significant change here is that that first volume has been changed from being ../src:/var/www (PHP src directory) to be ../public:/usr/share/fullstackExercise/public again, as per above.

Oh! One last file! public/gdayWorld.html:

<!doctype html>

<html lang="en">
<head>
<meta charset="utf-8">

<title>G'day world</title>
</head>

<body>
<h1>G'day world</h1>
<p>G'day world</p>
</body>
</html>

We should now be… good to go. Let's try it…

This is promising. So over to a web browser on my host machine:

Cool!

Umm… OK… that didn't actually go as smooth for me as I make it sound there. The first time around - because I did not notice that discrepancy between log/logs in Martin's article - I ended up referencing a non-existent path in docker-compose.yml, and the docker compose call failed:

That was simply cos I had this in my docker-compose.yml:

- ../logs:/var/log

Instead of this:

- ../log:/var/log

(In the file system, the directory is log). I very quickly spotted this and fixed it, and tried to back-out what I had done:

Try as I might I could not work out what file was missing. Nice one, btw, docker-compose for not actually saying what file-read didn't work. Amateur hour there.

After way too long of reading bug reports on Docker and various Stack Overflow Q&A, I spotted a solution ("Failed to execute script docker-compose"). The person answering just suggested "make sure Docker Desktop is running". That sounded off to me (of course it's running), but a comment on the answer made me look again: "I read this, thought what an answer for dummies, and then realised a few minutes later it was the answer I needed! DOH". I I went "oh all right then", and restarted Docker Desktop, dropped out of my bash instance and into a new one, and now:

Then I could just go docker-compose up --build, and the thing worked properly.

It's perhaps important to note I can replicate this at will, so this is just a workaround for some issue with Docker, but at least it got me moving forward. As I've been working through the rest of the articles in this series, I am getting this issue a lot: at least once a day. So it's not just caused by paths being wrong as per above. It seems to kick off quite often if I have changed docker-compose.yml to add/remove sections of config, or sometimes if I change files via the host machine file system (ie: my Windows desktop environment), rather than via the Ubuntu/WSL file system. I've not been able to tie it down to one particular thing. Something to look out for though.

Now to integrate PHP into the mix, see the next part of this series: Part 2: PHP (URL TBC; I'll link through once I release that article, tomorrow (2020-01-07)).

Righto.

--
Adam

G'day:

Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

1. Intro / Nginx
2. PHP (this article)
3. PHPUnit (URL TBC)
4. Bash (URL TBC)
5. MariaDB (URL TBC)
6. Installing Symfony (URL TBC)
7. Using Symfony (URL TBC)
8. Vue.js (URL TBC)

As indicated: this is the second article in the series, and follows on from Part 1: Intro &amp Nginx. I'ts probably best to go have a breeze through that one first.

PHP

The aim for this step is to stick an index.php file in the public directory, and have Nginx pass it over to PHP for processing. At the same time I'm going to get it to run composer install for good measure, so I'll add composer.json in there too. Oh, and PHP will end up needing to use PDO to talk to the DB, so I'll add that extension in now as well. Conveniently, Martin's article does all this stuff too, so all I need to do is copy & paste & tweak some files. Hopefully.

First up, docker/php-fpm/Dockerfile:

FROM php:fpm-alpine
RUN apk --update --no-cache add git
RUN docker-php-ext-install pdo_mysql
COPY --from=composer /usr/bin/composer /usr/bin/composer
WORKDIR  /usr/share/fullstackExercise/
CMD composer install ; php-fpm
EXPOSE 9000

This is largely the same as Martin's one; I've just got rid of some DB stuff he had in his, and pointed the WORKDIR to where the application code will be. note that /usr/share/fullstackExercise/ is the parent directory that contains the web root (/public), and composer.json, /src and /test etc.

One interesting thing is how composer is… erm… availed to the container: the files are just copied straight from Docker's image, then installed in the container.

Next is a simple config file to configure how Nginx passes PHP stuff onwards (docker/nginx/conf.d/default.conf). This is lock-stock from Martin's acticle, and I am just going "yeah… seems legit…?" (I'm such a pro):

upstream php-upstream {
    server php-fpm:9000;
}

And we need to stick the PHP stuff back into the Nginx site configuration file too now (docker/nginx/sites/default.conf):

server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    server_name localhost;
    root /usr/share/fullstackExercise/public;
    index index.html;

    location / {
        autoindex on;
        try_files $uri $uri/ /index.php$is_args$args;
    }

location ~ \.php$ {
try_files $uri /index.php =404;
fastcgi_pass php-upstream;
fastcgi_index index.php;
fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_read_timeout 600;
include fastcgi_params;
}

    location ~ /\.ht {
        deny all;
    }
}

That's all boilerplate stuff really.

Finally we crank up the php-fpm container via docker/docker-compose.yml:

version: '3'

services:
  nginx:
    build:
      context: ./nginx
    volumes:
      - ../public:/usr/share/fullstackExercise/public
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ../log:/var/log
depends_on:
- php-fpm
    ports:
      - "80:80"
stdin_open: true # docker run -i
tty: true        # docker run -t

php-fpm:
build:
context: ./php-fpm
volumes:
- ..:/usr/share/fullstackExercise
stdin_open: true # docker run -i
tty: true        # docker run -t

The PHP bit is pretty simple here. It just mounts the app directory. The only other new thing in here is that I've set both containers to be able to be shelled into with docker exec.

So. Does any of this stuff work? First I need to shut-down what I've got running:

And then crank it all back up again. This outputs a lot of stuff, most of which I've elided here, and most of what I've kept I've only done so for the sake of completeness, and showing it ran OK. Note that I'm now running docker-compose with the --detach option. This just gives me my prompt back after the process runs:

And, again, the last step is to test that PHP running stuff, just via /public/gdayWorld.php for now:

<?php

$message = "G'day World";
echo $message;

When I hit http://localhost/gdayWorld.php, I get:

Good enough for me. I mean I should perhaps include a file from /src to make sure that PHP can find it, but I will admit this did not occur to me at the time.

In the next article, Part 3: PHPUnit (URL TBC; I'll link through once I release that article, tomorrow (2020-01-08)), I will do as indicated: get PHPUnit working, and backfilling some tests of the code I've written so far.

Righto.

--
Adam

G'day:

Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

1. Intro / Nginx
2. PHP
3. PHPUnit (this article)
4. Bash (URL TBC)
5. MariaDB (URL TBC)
6. Installing Symfony (URL TBC)
7. Using Symfony (URL TBC)
8. Vue.js (URL TBC)

As indicated: this is the third article in the series, and follows on from Part 2: PHP. It's probably best to go have a breeze through the earlier articles first.

PHPUnit

I got slightly ahead of myself and added PHPUnit into composer.json when I was working through the PHP configuration part of this exercise, so it's already installed. Before I run it though, I need a phpunit.xml file, so I'll chuck one of those in:

<?xml version="1.0" encoding="UTF-8"?>
<phpunit
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.5/phpunit.xsd"
        colors="true"
        forceCoversAnnotation="true"
        cacheResult="false"
>
<coverage>
<include>
<directory suffix=".php">src</directory>
</include>
<report>
<html outputDirectory="public/test-coverage-report/" />
</report>
</coverage>
<testsuites>
<testsuite name="Functional tests">
<directory>test/functional/</directory>
</testsuite>
<testsuite name="Unit tests">
<directory>test/unit/</directory>
</testsuite>
</testsuites>
</phpunit>

There's no real surprises here. I've got two separate test suits: one for functional tests in which I'll test those test web-browseable files I created in the previous article; and one for unit tests. To test the code coverage config here I'll need some actual code to test and cover.

I had some drama getting PHPUnit working with code coverage, and that in itself is covered in a separate article, PHPUnit: get code coverage reporting working on PHP 8 / PHPUnit 9.5. The stuff I've written there is very focused on PHPUnit and not so much on the Docker side of things, or the testing in the context of this notional application I'm putting together, hence splitting it out into its own article, and also so I can focus on the Docker side of things here.

First things first, I need some tests! I decided to functional-test the two web browseable files: gdayWorld.html and gdayWorld.php. As a reminder their contents are (respectively):

<!doctype html>

<html lang="en">
<head>
<meta charset="utf-8">

<title>G'day world</title>
</head>

<body>
<h1>G'day world</h1>
<p>G'day world</p>
</body>
</html>

And:

<?php

$message = "G'day World";
echo $message;

So for the tests I'm just gonna make sure I can hit them, and the HTML one has the expected title, heading and content; and the PHP one just has the expected string. I could horse around with PHP's native curl implementation to make these work, but its programming interface written like something out of 1995, so I tend to avoid it where I can. I'm gonna use Guzzle instead. Also, and this is slightly OTT I know, but I like using Symfony's Response constants when checking for HTTP status codes to make the code more clear, so I'm adding in symfony/http-foundation. Lastly as I'll be using PHP's DOM API for testing the HTML, I'm placating a warning in PHPStorm that says "ooh but ext-dom might not be installed!" So I'm forcing that too. Oh and I like keeping my code tidy so I'm also slinging PHPMD and PHPCS in there too. My composer.json file becomes:

{
    "name": "adamcameron/full-stack-exercise",
    "description": "Full Stack Exercise",
    "license": "GPL-3.0-or-later",
    "require-dev": {
        "phpunit/phpunit": "^9.5",
        "guzzlehttp/guzzle": "^7",
        "symfony/http-foundation": "^5.2",
        "ext-dom": "*",
        "phpmd/phpmd": "^2.9",
        "squizlabs/php_codesniffer": "^3.5"
    },
    "autoload": {
        "psr-4": {
            "adamCameron\\fullStackExercise\\": "src/"
        }
    },
    "autoload-dev": {
        "adamCameron\\fullStackExercise\\test\\": "test/"
    }
}

Now I can write some tests. I did all this incrementally, but you know how to do that, so here is the "final" (see below for why I put that in quotes) version of WebServerTest:

namespace adamCameron\fullStackExercise\test\functional\_public; // "public" is reserved

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class WebServerTest extends TestCase
{
    /** @coversNothing */
    public function testGdayWorldHtmlReturnsExpectedContent()
    {
        $expectedContent = "G'day world";


        $client = new Client([
            'base_uri' => 'http://localhost/'
        ]);

        $response = $client->get('gdayWorld.html');

        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $html = $response->getBody();
        $document = new \DOMDocument();
        $document->loadHTML($html);

        $xpathDocument = new \DOMXPath($document);

        $hasTitle = $xpathDocument->query('/html/head/title[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasTitle);

        $hasHeading = $xpathDocument->query('/html/body/h1[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasHeading);

        $hasContent = $xpathDocument->query('/html/body/p[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasContent);
    }
}

And I triumphantly run this:

Doh! But… but… but… I quickly went and hit http://localhost/gdayWorld.html in my browser and it was fine. Then the penny dropped. I'm not running the tests from my host machine. I'm running them from with the PHP container. I've told the host machine about the Nginx container's web server; but I've not told the PHP container about it. Reminder as to what the docker-compose.yml is like at the moment:

version: '3'

services:
  nginx:
    build:
      context: ./nginx
    volumes:
      - ../public:/usr/share/fullstackExercise/public
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ../log:/var/log
    depends_on:
      - php-fpm
    ports:
      - "80:80"
    stdin_open: true # docker run -i
    tty: true        # docker run -t

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
    stdin_open: true # docker run -i
    tty: true        # docker run -t

I read a whole bunch of stuff on networking in Docker. I didn't find the Docker docs very useful at the time, but now I re-read them knowing how I'm supposed to interpret them, they seem clear. Not sure if that's an indictment of me or the docs. Or both. I also looked at a whole bunch of Stack Overflow Q&A and the answers were conflicting and divergent. However after distilling what I could from all these sources, it's really pretty easy. Here's the updated version:

version: '3'

services:
  nginx:
    build:
      context: ./nginx
    volumes:
      - ../public:/usr/share/fullstackExercise/public
      - ./nginx/nginx.conf:/etc/nginx/nginx.conf
      - ./nginx/sites/:/etc/nginx/sites-available
      - ./nginx/conf.d/:/etc/nginx/conf.d
      - ../log:/var/log
    depends_on:
      - php-fpm
    ports:
      - "80:80"
    stdin_open: true # docker run -i
    tty: true        # docker run -t
networks:
- backend

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
    stdin_open: true # docker run -i
    tty: true        # docker run -t
networks:
- backend

networks:
backend:
driver: "bridge"

I just added the networks section, and then told the Nginx and PHP containers to be on that backend network. NB: backend has no significance as a word here, it's just a label, and one used in the docs I was reading. After rebuild, I could now see the Nginx server from the PHP container:

Note that I'm using the Nginx services container name there as the host name, ie:

services:
nginx:

Not that it matters, but that seems a bit manky to me, so I wanted to specify a hostname here. That's just a matter of giving the Nginx container a hostname:

services:
  nginx:
    build:
      context: ./nginx
hostname: webserver.backend
    volumes:
      # etc

Now my functional tests should work, provided I use that new hostname:

Cool!

Oh there was a separate test for gdayWorld.php too:

namespace adamCameron\fullStackExercise\test\functional\_public; // "public" is reserved

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class PhpTest extends TestCase
{
    /** @coversNothing */
    public function testGdayWorldPhpReturnsExpectedContent()
    {
        $client = new Client([
            'base_uri' => 'http://webserver.backend/'
        ]);

        $response = $client->get('gdayWorld.php');

        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $content = $response->getBody()->getContents();

        $this->assertSame("G'day world", $content);
    }
}

I'm glad I wrote these tests, because this one identified a small bug I had introduced into gdayWorld.php. I'd fixed it by the time I catpured that output above, but the first run was less positive:

Yay for testing! I did not contrive this situation as an example of "always test first!", but there it is. This is hugely trivial code, but I still messed it up, and simply eyeballing it did not spot the bug.

Speaking of being observant… you will no-doubt have noticed the warning about code coverage driver above. I still need to install XDebug to make this work. This is a matter of adding this into the Dockerfile:

FROM php:fpm-alpine
RUN apk --update --no-cache add git
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
WORKDIR  /usr/share/fullstackExercise/
CMD composer install ; php-fpm
EXPOSE 9000

Or at least that was the theory:

Sigh. I have encountered this sort of thing before with other containers. Alpine is really really pared down, and doesn't include a bunch of packages that tools might need to run. This is fine for a lot of situations, but it's also a pain in the arse for others. In this case I added in autoconf, but then I needed to install a C compiler too. And then after that I think it wanted something else. Sod that. I just stopped using Alpine and wend back to the Debian version of the container, in the Dockerfile:

FROM FROM php:8.0-fpm
RUN apt-get update --yes && apt-get install git --yes
ENV XDEBUG_MODE=coverage

FROM :fpm-alpine
RUN apk --update --no-cache add git

Note: the apk / apt-get change is just the difference in package manager between Alpine and Debian. I won't show you the installation of this because it's 70kB of telemetry, all of which culminates in:

And now we can run our functional tests, and the code coverage report should create (even if it hasn't got anything in it yet, cos I'm not code-covering those functional tests):

Grrrr… what now?. I reviewed the docs and my phpunit.xml was legit-looking, and validated fine, so I was flumoxed. But then I came across this issue with PHPUnit: Misleading error message when no files present in coverage path. That sums it up. I have this in my phpunit.xml file:

<coverage>
<include>
<directory suffix=".php">src</directory>
</include>
<report>
<html outputDirectory="public/test-coverage-report/" />
</report>
</coverage>

But I don't actually have a src/ directory yet. Once I added that, and also added the public/ directory into that <include> block, I get a report. At the same time, I will add a stub PHP class and test thereof into the src/ directory as well, to better test the reporting:

And the coverage report generates fine too:
above

The test for MyClass's needsTesting method is as simple as you might imagine:

namespace adamCameron\fullStackExercise\test\unit;

use adamCameron\fullStackExercise\MyClass;
use PHPUnit\Framework\TestCase;

/** @coversDefaultClass adamCameron\fullStackExercise\MyClass */
class MyClassTest extends TestCase
{
    private $myClass;

    protected function setUp(): void
    {
        $this->myClass = new MyClass();
    }

    /** @covers ::needsTesting */
    public function testNeedsTesting()
    {

        $needsTesting = $this->myClass->needsTesting();
        $this->assertTrue($needsTesting);
    }
}

OK, so I'm in a good place to be able to TDD some PHP code now. But before I do that, I want to get a MariaDB container added into the mix as well, and write an intergration test for its connectivity. Before I get to that though, as I've been doing the work on this project up until now - especially when troubleshooting the networking and the PHPUnit issue I had - I've tweaked some stuff with my Bash environment, and I wanted to rip that out into a separate wee article. So before we get onto MariaDB, I'll write that up: Part 4: Bash (URL TBC; I'll link through once I release that article, tomorrow (2020-01-09)).

Righto.

--
Adam

G'day:

Intro

Please note that this is a sub-article of a larger body of work that is an exercise in setting up a Vue.js-driven website backed by PHP8 and MariaDB running in Docker containers. All of this is completely new to me, so is a learning exercise, rather than some exposition of my wisdom (which I have none of). I initially intended the whole exercise to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers (this article)
5. MariaDB (URL TBC)
6. Installing Symfony (URL TBC)
7. Using Symfony (URL TBC)
8. Testing a simple web page built with Vue.js (URL TBC)

As indicated: this is the fourth article in the series, and - chronologically - follows on from Part 3: PHPUnit. That said, this article is reasonably stand-alone, so not sure if it's really necessary to read everything else first. The only real "cross over" is that I'm experimenting within Docker containers that I created in the earlier articles. I guess you can refer back to the other articles if anything I say here seems to be making assumptions you can't make.

Also please note that there is very little that is earthshattering here. It's more an exercise of me - as a relative *nix noob - getting stuff working how I've become accustomed to it. I discuss nothing tricky or advanced here.

Today's exercise

Whilst doing all the crap to get Nginx, PHP and PHPUnit working, I was spending an awful lot of time in and out of Bash, running various bits of code and testing stuff and the like. I found there were a few annoying things about running Bash in these containers:

• The Debian distro that the PHP container uses doesn't include ping and other network utils I was needing to use.
• Bash history works for the life of the container, but does not - obviously - live between rebuilds of a container. This was a pain in the butt as I had a bunch of things to run every rebuild to check stuff.
• I needed to add a coupla aliases: ll which everyone has; and one, cls, that I've got that is a bit nicer than just clear

In this article I'm gonna work through implementing those, and a coupla other things.

Adding the networking utils was a one-liner in the docker/php-fpm/Dockerfile:

FROM php:8.0-fpm
RUN apt-get update
RUN apt-get install git --yes
RUN apt-get install net-tools iputils-ping --yes
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
#COPY ./.bashrc /root/.bashrc
ENV XDEBUG_MODE=coverage
WORKDIR  /usr/share/fullstackExercise/
CMD composer install ; php-fpm
EXPOSE 9000

That just installs both net-tools and iputils-ping. The --yes just suppresses confirmation the installation asks, which breaks the Docker build if I don't answer it in a batch fashion: the Docker build process is not interactive.

Sorting out the Bash history was slightly trickier. My initial plan was to just mount a stubbed file in the container (this from docker/docker-compose.yml):

  php-fpm:
    build:
      context: ./php-fpm
    volumes:
      - ..:/usr/share/fullstackExercise
- ./php-fpm/.bash_history:/root/.bash_history
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend

This has some drawbacks. The build broke if ./php-fpm/.bash_history wasn't in the host file system. To guarantee it was there, I needed it in source control. But… by its very nature it's getting changed all the time, which gets annoying when I go to commit stuff. There's ways to work around this in git, but those were causing issues as well.

In the end I decided to take this approach:

    volumes:
      - ..:/usr/share/fullstackExercise
- ./php-fpm/root_home:/root

That's the entire home directory for the root user. By default it has nothing in it, so I don't need to replicate much in the file system in source control: just make the directory exist, and .gitignore.bash_history:

And this just works! Below I show rebuilding my containers, going into bash on the PHP container, showing the empty history, doing some stuff then exiting. Then I repeat the whole operation and you can see the history is sticking across containers now.

Quite pleased with that, I am.

Given how I'm voluming-in the entire home directory, doing the aliases as easy enough, I just needed to drop a .bashrc into that docker/php-fpm/root_home directory. The image doesn't have a .bashrc by default, so this is fine.

alias ll='ls -alF'
alias cls='clear; printf "\033[3J"'

There's a coupla things to note here. Initially by accident I saved this file with CRLF line endings, and so the ll alias didn't work:

That's a trap for young players there (even old burnt-out ones like me, too). It was not immediately obvious what the issue is there, so it took a bit of googling and stack-overflow-ing to find the answer. Something to remember. But once one sorts that out, it works fine:

The cls alias I make just solves something that annoys me about clear is that it doesn't clear the scrollback history when SSHed into the box, so if one mouse-scrolls up, one can still move past where you cleared. This makes things hard to find sometimes, and defeats the purpose of clear IMO. The escape sequence in the alias alias cls='clear; printf "\033[3J"' just clears the scrollback as well as the screen. Or something that amounts to that anyhow. Shrug. I just copy & pasted it once upon a time, and still use it.

That's all I had for this one. Next stop: "Part 5: MariaDB" (URL TBC).

Righto.

--
Adam

G'day:

Please note that I initially intended this to be a part of a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers
5. MariaDB (this article)
6. Installing Symfony
7. Using Symfony (URL TBC)
8. Testing a simple web page built with Vue.js (URL TBC)

As indicated: this is the fifth article in the series, and follows on from Part 4: Tweaks I made to my Bash environment in my Docker containers. It's probably best to go have a breeze through the earlier articles first just to contextualise things. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject.

OK, let's get on with this MariaDB stuff.

Firstly; why am I using MariaDB instead of MySQL? Initially I started pottering around with MySQL on Docker for another piece of work I was doing, and I ran up against a show-stopper that doesn't seem to have a resolution. It's detailed on GitHub at "MySQL docker 5.7.6 and later fails to initialize database", and demonstrated here:

I tried everything suggested in that thread, everything else I could find, and nothing improved the situation. However this entry on that issue page above looked like good advice:

So I just decided to run with MariaDB instead, and that worked perfectly:

Now I don't doubt it's possible to get MySQL working in my environment, but… erm… shrug. I don't care. I just need a DB running. I'm not here to fart-arse around with DBs any more than absolutely necessary.

We should actually now back-up a bit. All that stuff above was done a while ago - although I replicated it just now for the sake of the notes above - and for my current exercise we're getting ahead of ourselves. I'm gonna start this exercise with a failing test (test/integration/DatabaseTest.php):

namespace adamCameron\fullStackExercise\test\integration;

use PHPUnit\Framework\TestCase;
use \PDO;

class DatabaseTest extends TestCase
{
    /** @coversNothing */
    public function testDatabaseVersion()
    {
        $connection = new PDO(
            'mysql:dbname=mysql;host=database.backend',
'root',
'123'
        );

        $statement = $connection->query("show variables where variable_name = 'innodb_version'");
        $statement->execute();

        $version = $statement->fetchAll();

        $this->assertCount(1, $version);
        $this->assertSame('10.5.8', $version[0]['Value']);
    }
}

Note: in a better world, I'd never have my user and password hard-coded there (see further down for where I address this), even in a test. And I'd not be using root (I don't address this one quite yet though). Also checking the version right down to the patch level is egregious, I know. Just checking for 10 would perhaps be better there.

This fails as one would expect:

Once we've installed the MariaDB container, got it up and running and networked it: this test should pass.

I'm back to taking Martin Pham's lead in the Docker config for all this (reminder, from his article "Symfony 5 development with Docker")

Here's the docker/mariadb/Dockerfile:

FROM mariadb:latest
CMD ["mysqld"]
EXPOSE 3306

No surprises there. Next the stuff in docker/docker-compose.yml:

  mariadb:
    build:
      context: ./mariadb
    environment:
      - MYSQL_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
    ports:
      - "3306:3306"
    volumes:
      - ./mariadb/data:/var/lib/mysql
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      backend:
        aliases:
          - database.backend

One new thing for me here is the ${DATABASE_ROOT_PASSWORD}. Looking at Martin's set-up, he's also got a file docker/.env. Seems to me one can sling environment variables in there, and docker-compose picks them up automatically. So I've just got this (docker/.env):

DATABASE_ROOT_PASSWORD=123

I'll come back to this in a bit.

Also note that I'm configuring MariaDB to put its data in a volume back on the host machine. This is so the data persists when I shut the container down.

And now I should be able to start everything up, and it'll just work. Right?

OK that's more promising than I expected. How about that integration test?

Gasp (I actually did gasp a bit). Blimey. It only went and worked first time.

I'm intrigued by this .env stuff. I figured if I can set that root password in the .env file, then I can shift it out of my integration test, and just use the environment variable. So I've updated the php-fpm section of docker-compose.yml to also set than environment variable:

php-fpm:
  build:
    context: ./php-fpm
  environment:
    - DATABASE_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
  # etc

And update the integration test to use that:

$connection = new PDO(
    'mysql:dbname=mysql;host=database.backend',
    'root',
    $_ENV['DATABASE_ROOT_PASSWORD']
);

And test that's all OK:

Cool!

Right so that's all that done: it was very easy (thanks largely to Martin's guidance). Note that once I get underway with the app I'll have a specific database to use, and specific credentials to use against it; at that point I'll stop using the root credentials in that test. But for where we are now - just checking that the DB is up and networked and PHP can see it: the test I'm doing is fine.

That was quite a brief article. In the next one I'll get on to installing Symfony.

Righto.

--
Adam

G'day:

These initial boilerplate paragraphs must be getting annoying by now. Sorry. But anyhow, here we go again. Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers
5. MariaDB
6. Installing Symfony (this article)
7. Using Symfony (URL TBC)
8. Testing a simple web page built with Vue.js (URL TBC)

As indicated: this is the sixth article in the series, and follows on from Part 5: MariaDB. It's probably best to go have a breeze through the earlier articles first. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject.

For the earlier articles I had mostly tried-out what I was intending to do before I started writing about it. This time I am writing this at the same time as doing the work. To contextualise, all I have done at the moment is open up that article of Martin Pham's I've been following along ("Symfony 5 development with Docker"), and also got the Symfony docs up: "Installing & Setting up the Symfony Framework / Creating Symfony Applications".

First things first I want a failing test for my minimum requirements here, which is that when I GET/greeting/[some name here]/, I receive back a JSON response, along these lines: {"name":"[some name here]","greeting":"G'day [some name here]"}. Here's a test that tests this (tests/functional/SymfonyTest.php):

namespace adamCameron\fullStackExercise\test\functional;

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class SymfonyTest extends TestCase
{
    /** @coversNothing */
    public function testGreetingsEndPointReturnsPersonalisedGreeting()
    {
$testName = 'Zachary';
$expectedGreeting = (object) [
'name' => $testName,
'greeting' => "G'day $testName"
];

        $client = new Client([
            'base_uri' => 'http://webserver.backend/'
        ]);

$response = $client->get("greeting/$testName/");
$this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

$contentTypes = $response->getHeader('Content-Type');
$this->assertCount(1, $contentTypes);
$this->assertSame('application/json', $contentTypes[0]);

$responseBody = $response->getBody();
$responseObject = json_decode($responseBody);
$this->assertEquals($expectedGreeting, $responseObject);
    }
}

The code is pretty clear, I just perform the GET using "Zachary" as the name, and in return I verify I get a 200 OK response, it's JSON and it has the expected data structure in the body. Simple.

I run the test to check I get a fail with a 404:

OK what? Why the hell is Guzzle deciding a 404 response - which is completely legitimate - is in some way "exceptional", and warrants an exception being thrown. That's daft. Lemme go RTFM and see if there's a way of me configuring Guzzle to not get ahead of itself and just behave. back in a bit…

… OK, easily solved. There's a config param for the get call that switches that behaviour off. Thanks to the Guzzle docs for making that easy to find ("Request options / http-errors").

$response = $client->get(
    "greetings/$testName/",
    ['http_errors' => false]
);

And now we get a test that fails as we'd expect it to:

We're now OK to work out how to get Symfony installed and running, and make an end point that behaves as we need it to.

Or are we? I think I might have got a bit ahead of myself here. I'm back looking at Martin's article, and he says

Let's proceed to the Symfony installation:

$ symfony new src

[…] open http://localhost, you will see a Symfony 5 welcome screen.

OK so I think our first test should be that that Symfony 5 welcome screen works. I've got no idea what the screen says, so let me google for expectations. Right according to Creating your First Symfony App and Adding Authentication by Olususi Oluyemi (from a random Google Images search for "symfony 5 welcome screen") it should look like this:

I'm gonna write a quick test to check for that, and bypass that other test for now:

/** @coversNothing */
public function testSymfonyWelcomeScreenDisplays()
{

    $client = new Client([
        'base_uri' => 'http://webserver.backend/'
    ]);

    $response = $client->get(
        "/",
        ['http_errors' => false]
    );
    $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

    $html = $response->getBody();
    $document = new \DOMDocument();
    $document->loadHTML($html);

    $xpathDocument = new \DOMXPath($document);

    $hasTitle = $xpathDocument->query('/html/head/title[text() = "Welcome to Symfony!"]');
    $this->assertCount(1, $hasTitle);
}

I was scratching my head guessing what the mark-up around the "Welcome to Symfony x.y.z" text in the body is. It could be <h2> and <h3> tags, or it could just be different font-size stylings. Then I noticed the <title> content, and just decided to check for that. If that string is there: we're good. To start with, we want to be not good, so let's run that test:

Right so both Martin's article and the Symfony docs say I need to install Symfony (via Composer), and create an application with it, so I'll do this now.

The first hiccup is that the Symfony installer assumes that yer Symfony app is the centre of the world, and it needs to be the one that creates yer project directory for you. Great. Thanks for that. There's probably (?) a way of coercing it to understand that it's just a frickin' framework and I just need to add it to my existing app which is already underway. This is not so far--fetched I think, but after 15min of googling I drew a blank, so I decided to take a different approach. I'm gonna blow away my stuff to allow Symfony to play its silly little game, and then reintegrate my stuff back into the stuff Symfony creates. After all it's just some tests and composer.json config at present really.

/me thinks things through… OK that won't work. I can't shift all the files out of the fullstackExercise directory because all the Docker stuff is in there, and I can't bring my containers up if they're not there. Without the containers up, I don't have PHP or Composer so I can't do the Symfony install. To be blunt here: screw you, Symfony (well it's actually Composer that is bleating about the directory not being empty I guess. So screw Composer). With Silex? Just add an entry to composer.json, add a line to index.php and that's it. This is how a microframework should work.

Plan B is to do things the other way around. I need to shift my fullStackExercise project directory to have a different name (fullStackExercise.bak) temporarily so I can allow Symfony to create its own one. To do this I need to rejig my docker/docker-compose.yml file and the docker/php-fpm/Dockerfile to reference fullstackExercise.bak. Note that docker/nginx/sites/default.conf file also references root /usr/share/fullstackExercise/public, so at this point I decided that for now I'll just bring up the PHP container for this, so stripped the Nginx and MariaDB stuff out of docker-compose.yml completely. So these are the file changes:

docker-compose.yml:

version: '3'

services:

  php-fpm:
    build:
      context: ./php-fpm
    environment:
      - DATABASE_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
    volumes:
      - ../..:/usr/share/src # was ..:/usr/share/fullstackExercise
      - ./php-fpm/root_home:/root
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      - backend

networks:
  backend:
    driver: "bridge"

Note that I've shifted the main volume up one level in the host's directory hierarchy. This is because I need to let Symfony recreate that fullStackExercise at that parent level.

php-fpm/Dockerfile:

FROM php:8.0-fpm
RUN apt-get update
RUN apt-get install git --yes
RUN apt-get install net-tools iputils-ping --yes
RUN apt-get install zip unzip --yes
RUN docker-php-ext-install pdo_mysql
RUN pecl install xdebug-3.0.1 && docker-php-ext-enable xdebug
COPY --from=composer /usr/bin/composer /usr/bin/composer
ENV XDEBUG_MODE=coverage
#WORKDIR  /usr/share/fullstackExercise.bak/
WORKDIR  /usr/share/src/
CMD composer install ; php-fpm
EXPOSE 9000

I've made the same change to the WORKDIR here too, to match docker-compose.yml. I've also added in a line to install zip and unzip into the container, because when I was testing this, Composer was bleating about them not being present.

Now in theory if I drop my containers and bring them back up again, I should be able to bash into the PHP container and do the Symfony install:

So far so good. now I just need to move a bunch of stuff back from my app into this one. I'll go ahead and do it, and report back when done, and list what I needed to do…

• .git/, .idea/ (this is PHPStorm's project config), docker/, log/ directories could just be copied straight across.
• As could LICENSE, phpcs.xml, phpmd.xml, phpunit.xml, README.md.
• And src/MyClass.php, public/gdayWorld.html and public/gdayWorld.php.
• The only overlap in .gitignore was the vendor/ directory, otherwise I could just move my bits back in.
• composer.json was fairly straight forward, but I raised an eyebrow that Symfony seems to suggest using App as a top-level namespace:
  

  "autoload": {
      "psr-4": {
          "App\\": "src/",
  

  
  To me, using a namespace "App" is like calling a variable something like indexVariable, or totalVariable or something, or naming a class AnimalClass. Ugh. It gets even worse with the testing namespace: App\\Tests\\. For now I'm just adding my own namespacing in as well. Hopefully I don't have to use Symfony's approach here.
• Symfony uses tests as its test directory; I've always used test, but I'm ambivalent about this so am sticking with Symfony's lead here: I'll change my directory to tests. This will require me changing the namespaces in the test source code files too.
• Because I've changed composer.json by hand, I'll need get rid of composer.lock and vendor/ and do a composer install again.
• Lastly I've reverted the Docker files I'd changed above back to how they were before: removing the references to fullstackExercise.bak, and bringing back the Ngxin and MariaDB containers.

OK having done all that, and now will rebuild my containers. Before starting this paragraph I had done all this and had started to fight with an issue I thought I was having with PHPUnit, so I didn't capture the screen showing it all working. It's all just Docker bumpf, then some Composer bumpf, none of which was very surprising, so I'll spare you the detail. Back to the unit test run:

Doh!

Things get weirder cos I hit the site on my host browser:

And that seems fine. First troubleshooting step: factor-out all the code I'm running, and manually curl it from PHP:

So there's all the content. Fine.

Next I want to check just Guzzle by itself, to demonstrate to myself it is working fine. I hand-cranked some Guzzle calls from outside of my test code:

It's still frickin' 404ing.

I scratched my head a lot and googled why guzzle might 404 on something that everything else 200s on. Then I saw it.

That mark-up that is spewing out in the Guzzle error is not generic 404 response. What it is is the content from the Welcome to Symfony! page. I checked something else in my browser…

That "welcome" page? It has a 404 status. Now I can see what they're doing here, and they even warn about it if one pays attn:

However to me a welcome page - that actually has welcome content on it, therefore is precisely what one would be expecting on that URL- is not a 404 situation. Ah well.

So my test is reporting correctly! (Yay for tests!) It's just that the expectations I gave the test were wrong. I need to expect a 404, not a 200. I'll make that change, and now the tests all pass:

Phew. BTW I had to change my test a bit more than I expected:

/** @coversNothing */
public function testSymfonyWelcomeScreenDisplays()
{

    $client = new Client([
        'base_uri' => 'http://webserver.backend'
    ]);

    $response = $client->get(
        "/",
        ['http_errors' => false]
    );
    // unexpectedly perhaps: the welcome page returns a 404. This is "by design"
    $this->assertEquals(Response::HTTP_NOT_FOUND, $response->getStatusCode());

    $html = $response->getBody();
    $document = $this->loadHtmlWithoutHtml5ErrorReporting($html);

    $xpathDocument = new \DOMXPath($document);

    $hasTitle = $xpathDocument->query('/html/head/title[text() = "Welcome to Symfony!"]');
    $this->assertCount(1, $hasTitle);
}

private function loadHtmlWithoutHtml5ErrorReporting($html) : \DOMDocument
{
    $document = new \DOMDocument();
libxml_use_internal_errors(TRUE);
    $document->loadHTML($html);
libxml_clear_errors();

    return $document;
}

There's a bug in PHP's HTML loading that makes it barf on HTML5 tags (see "PHP DOMDocument errors/warnings on html5-tags", so I needed to put some mitigation in for that. It's not a great solution, but it works.

Now to try to work out how to implement my /greetings/Zachary route…

But not for now. I've just checked the length of this, and not counting all the code and telemetry, this is already close to 1800 words (it'll be over by the time I finish this outro). I think this exercise of getting the Symfony framework code into my project and up and running is sufficient for a stand-alone article. I'll do a separate article for the "and now let's get it to do something except return a confusing (to me. Cringe) 404 response". I'll crack on with that tomorrow, over here: "Part 7: Using Symfony" (URL TBC).

Righto.

-- 
Adam

G'day:

Familiar boilterplate about how this is one in a series follows. Please note that I initially intended this to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers
5. MariaDB
6. Installing Symfony
7. Using Symfony (this article)
8. Testing a simple web page built with Vue.js (URL TBC)

As indicated: this is the seventh article in the series, and follows on from Part 6: Installing Symfony. It's probably best to go have a breeze through the earlier articles first, in particular the immediately preceding one. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject. Also I was only originally "planning" one article on getting the Symfony stuff sorted out, but yesterday's exercise was more involved that I'd've liked, so I stopped once I had Symfony serving up its welcome page, and today focusing on the config / code of getting a route, controller, model, view (actually there's no model or view in this endpoint; it's all just done in the controller) up and running.

Two caveats before I start.

Firstly: this is the first time I've done anything with Symfony other than reviewing the work of other members of my team - the ones doing the actual work. I have some familiarity with Symfony, but it's very very superficial.

Secondly, I will also be up front that I don't like Symfony's approach to MVC frameworking. Symfony is kind of a "lifestyle choice": it's opinionated, it expects you to do things a certain way, and it puts its nose into everything. This is opposed to something like Silex which simply provides the bare bones wiring to handle the ubiquitous web application requirement of guiding processing through routing, controller, model and view; other than that it just gets out of the way and is pretty invisible. I loved Silex, but sadly it's EOL so I need to move on. And, yeah, I'm being curmudgeonly and pre-judgemental as I go, I know. I do know Symfony is immensely popular, and I also know from my exposure to using its various independent libraries that it's been developed in a thoughtful, thorough manner. I expect I'm largely just being change-averse here. However I'm forewarning you now, as this will no-doubt come across in my tone, and my patience when things don't go exactly how I'd like them too (like some sort of spoilt brat). But if you've read this blog before… you probably already expect this.

Now that I have that out of my system (maybe), what's my aim for today? When I started yesterday, the intent of my work was described in this functional test:

/** @coversNothing */
public function testGreetingsEndPointReturnsPersonalisedGreeting()
{
    $testName = 'Zachary';
$expectedGreeting = (object) [
'name' => $testName,
'greeting' => "G'day $testName"
];

    $client = new Client([
        'base_uri' => 'http://webserver.backend/'
    ]);

    $response = $client->get(
"greetings/$testName/",
        ['http_errors' => false]
    );
    $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

$contentTypes = $response->getHeader('Content-Type');
$this->assertCount(1, $contentTypes);
$this->assertSame('application/json', $contentTypes[0]);

    $responseBody = $response->getBody();
    $responseObject = json_decode($responseBody);
$this->assertEquals($expectedGreeting, $responseObject);
}

In humanspeke, what I'm gonna do is:

• create a route /greetings/[some name here]/;
• return an object as JSON;
• that confirms the name sent, plus has a greeting string for that name.

Very "Hello World". I did indicate I was not gonna be pushing the boat out too much here.

To start let's run that test, and watch it fail with a 404…

That one failure is the one we want to see, so that's good: I have a test failing in exactly they way I expect it to be, so now I need to work out how to add a route and all that sort of jazz. Time to RTFM. Back soon.

…

Right so the docs for Symfony have been incredibly helpful so far. That's not sarcasm: they've been really bloody good! I'd been looking at the Installing & Setting up the Symfony Framework page, and one of its last links is to Create your First Page in Symfony. This steps one through setting up a route and the controller that services it. The first example did not have a "runtime" parameter in the URL slug like I need here, but that was covered in another linked-to page Routing (specifically the Route Parameters section of same). That was all the information I seemed to need for my code, so off I went.

Routing is done in config/routes.yaml, and there's an example in there already. So it was easy to stick my new route in:

#index:
#    path: /
#    controller: App\Controller\DefaultController::index

greetings:
  path: /greetings/{name}
  controller: adamCameron\fullStackExercise\Controller\GreetingsController::doGet

Curly braces around the {name} just mean that that part of the URL slug is dynamic, and its value is passed to the controller method. I could (and should!) put validation on this, but that's getting ahead of myself. The current requirement I have set only handles the happy path, so we'll stick to that.

The controller goes in the src/Controller directory. I'm more accustomed to using headlessCamelCase for my namespace parts, but I'll stick with Symfony's precedent here:

namespace adamCameron\fullStackExercise\Controller;

use Symfony\Bundle\FrameworkBundle\Controller\AbstractController;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\HttpFoundation\Response;

class GreetingsController extends AbstractController
{
    public function doGet(string $name) : Response
    {
        $greetingsResponse = [
            'name' => $name,
            'greeting' => "G'day $name"
        ];

return new JsonResponse($greetingsResponse);
    }
}

This shows how the dynamic part of the slug from the route passes through into the controller method. Cool. From there I've gone my own way from the docs there cos Silex uses Symfony's request/response mechanism, so I know I can just return a JsonResponse like that, and it'll handle the 200-OK and the application/json part of the requirement. The docs integrate using Twig here to render some output, but there's no need for that complexity here. I suppose here the creation of $greetingsResponse is my "model", and the decision to just return a JsonResponse is my "view".

Being quite pleased with myself at how simple that was, I ran my test to verify how clever I was:

rrarr! OK, not so clever after all it seems. Hitting the URL in the browser gives me more information:

What is this thing on about? See where it's looking for my controller? It's looking for a class App\Controller\GreetingsController, but that class is not in App/Controller, it's in adamCameron/fullStackExercise/Controller, and that's where the route code says it is. I have re-checked everything, and it's all legit.

Sigh. I could guess what it is. Symfony being a) "clever", and b) "opinionated". If you've just read the previous article, you might recall me raising my eyebrow at this bit in composer.json:

"autoload": {
"psr-4": {
"App\\": "src/",
"adamCameron\\fullStackExercise\\": "src/"
    }
},

At the time I was just observing what a dumb namespace that was, but I'm now guessing that in Symfony's opinion that is the namespace we all should be using. I remembered something about some other autowiring config that Symfony has, and guessed there was something in there that might cater to classes in the App namespace, but not other namespaces. Even if the namespaces are explicit in the code (as they are here). I located the culprit in config/services.yaml:

services:
    # default configuration for services in *this* file
    _defaults:
        autowire: true      # Automatically injects dependencies in your services.
        autoconfigure: true # Automatically registers your services as commands, event subscribers, etc.

    # makes classes in src/ available to be used as services
    # this creates a service per class whose id is the fully-qualified class name
App\:
        resource: '../src/'
        exclude:
            - '../src/DependencyInjection/'
            - '../src/Entity/'
            - '../src/Kernel.php'
            - '../src/Tests/'

    # controllers are imported separately to make sure services can be injected
    # as action arguments even if you don't extend any base controller class
App\Controller\:
        resource: '../src/Controller/'
        tags: ['controller.service_arguments']

    # add more service definitions when explicit configuration is needed
    # please note that last definitions always *replace* previous ones

So there's magic taking place for App; I guess I need to make it magical for my actual namespace instead. I'm never gonna write code with App as its namespace, so I'm just gonna punt that I can change that to adamCameron\fullStackExercise\: and adamCameron\fullStackExercise\Controller\: and it'll all be fine. I think ATM I only need to monkey with the Controller one, but I might as well do both now I guess. So with that change in place, I re-run the test:

root@3fc72bf44d38:/usr/share/fullstackExercise# vendor/bin/phpunit tests/functional/SymfonyTest.php --filter=testGreetingsEndPointReturnsPersonalisedGreeting
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

.                                                                   1 / 1 (100%)

Time: 00:06.440, Memory: 8.00 MB

OK (1 test, 4 assertions)

Generating code coverage report in HTML format ... done [00:01.825]
root@3fc72bf44d38:/usr/share/fullstackExercise#

All right, that was easily located / solved and is not the end of the world. However this from Symfony's error message irks me: "The file was found but the class was not in it, the class name or namespace probably has a typo". No mate. That's not the problem at all. The problem is that despite me giving you the exact fully-qualified namespaced class name in the route config, and the namespace in the class file was correct, and in the right place in the file system… and if you just left things be then you would have found the correct class. But: no. You had to try to be clever, and you failed miserably. I really frickin' hate it when technology goes "nono [patronising grimace] I've got this, I'll do it for you" (I know some devs like this too…), and it gets it wrong. I'm all for Symfony having the helpers there should one want them, but it shouldn't make these assumptions. I guess had it gone "hey aaah… that namespace yer using for yer controller? Ya need to configure that to work mate. Go have a look in config.yaml", then I'd just be going "yeah nice one: thanks for that". But then again most people would probably not have written this ranty paragraph at all, and just moved on with their lives, eh? Hey at least I'm self-aware.

All in all, even with the config/namespace hiccup, getting a simple route working was pretty bloody easy here. And this is a good thing about Symfony.

That was actually pretty short. It was about 500 words and some test-runs longer, as I had encountered a weird issue which I was moaning about. However when doing a final read-through of this before pressing "send", I looked at it some more, and it turns out it was just me being a div, and the problem was firmly of the PEBCAK variety. So… erm… I decided to delete that bit. Ahem.

Next I have to start doing some front-end-ish stuff so I've got an excuse to try out Vue.js. This should be interesting. I've never even looked @ Vue.js before. I bet I'll disagree with some of its opinions eh? Stay tuned…

Righto.

--
Adam

G'day:

Familiar boilerplate about how this is one in a series follows. Please note that I initially intended this entire exercise to be a single article, but by the time I had finished the first two sections, it was way too long for a single read, so I've split it into the following sections, each as their own article:

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers
5. MariaDB
6. Installing Symfony
7. Using Symfony
8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer (this article)
9. Refactoring the simple web page into Vue components (URL TBC)

As indicated: this is the eighth article in the series, and follows on from Part 7: Using Symfony. Unlike the previous articles in this series, it's reasonably stand-alone: it doesn't rely on any of the Nginx / PHP / MariaDB / Symfony stuff I've written about so far. I do continue building on my Docker setup, but in an isolated fashion. So I dunno if you'd specifically benefit from going back and reading the earlier articles in the series to contextualise this article. Also as indicated in earlier articles: I'm a noob with all this stuff so this is basically a log of me working out how to get things working, rather than any sort of informed tutorial on the subject. I've said that previous sentence at the beginning of most of the articles, but it is particularly true of my exposure to Vue.js and everything I touch on in this article. One can hardly even call it "exposure": I've heard of Vue.js, I know it exists, and I have the Vue.js homepage open in the adjacent tab. That is the entirety of my exposure. Prior to this exercise I had not even heard of Puppeteer, and have only messed with Mocha and Chai very very superficially (JavaScript TDD: getting Mocha tests running on Bamboo). But such a distinct lack of knowledge has never stopped me blathering about stuff before, and I'm too old to change my ways now, so here goes.

Firstly: some full disclosure. I am going to write this article in good TDD sequence: tests first, then the web page I'm testing. In reality I did the Vue-powered web page first and then went "erm… ain't no way I'm gonna write this up without also having tests for it". Also the Vue side of things was so simple to get a "dynamic" "G'day World" page working that I hardly had anything to write about. So I'm gonna start where I ought to have, with the question of "how the hell am I going to test this?"

That question might seem really daft. My G'day World webpage looks like this:

Basically there's three testable mark-up elements: a title, heading and a paragraph all of which have "G'day World" in them. My intended Vue.js version of this will differ only in that it has "G'day world via Vue" as its intended content. When testing the flat mark-up version of this, all I need to do is curl the page, and then use a DOM parser to locate the <title>, <h1> and first <p> elements, and check their innerText values, eg:

/** @coversNothing */
public function testGdayWorldHtmlReturnsExpectedContent()
{
    $expectedContent = "G'day world";


    $client = new Client([
        'base_uri' => 'http://webserver.backend/'
    ]);

    $response = $client->get('gdayWorld.html');

    $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

    $html = $response->getBody();
    $document = new \DOMDocument();
    $document->loadHTML($html);

    $xpathDocument = new \DOMXPath($document);

    $hasTitle = $xpathDocument->query('/html/head/title[text() = "' . $expectedContent . '"]');
    $this->assertCount(1, $hasTitle);

    $hasHeading = $xpathDocument->query('/html/body/h1[text() = "' . $expectedContent . '"]');
    $this->assertCount(1, $hasHeading);

    $hasContent = $xpathDocument->query('/html/body/p[text() = "' . $expectedContent . '"]');
    $this->assertCount(1, $hasContent);
}

This approach is no good for the Vue.js-driven version of this, because the $expectedContent won't be present in the mark-up: Vue.js will be swapping it in at runtime, dynamically. All the mark-up will have in it is some placeholder like {{message}}. I can't just test for that, because it does not test that the Vue code is actually running correctly.

My first task is to find out how people are testing dynamically-created mark-up documents these days. I just googled that. Most of the material specific to Vue-managed stuff revolves around testing Vue components. This is where I hope to get to (in the next and maybe final article in this series), but for now my first "G'day world via Vue" exercise will be based on the "Hello world" example in the Vue docs, and all be inline within one HTML document, eg:

<!DOCTYPE html>
<html>
<head>
<title>My first Vue app</title>
<script src="https://unpkg.com/vue"></script>
</head>
<body>
<div id="app">
    {{ message }}
</div>

<script>
    var app = new Vue({
      el: '#app',
      data: {
        message: 'Hello Vue!'
      }
    })
</script>
</body>
</html>

No Vue components yet. After a bit more googling I landed on Puppeteer seeming like the way to go to test this sort of thing. From their home page:

Sounds cool.

The first thing I need to do is to add a Node.js container into my ever-growing family of containers. My initial approach was with this sorta thing in the docker/node/Dockerfile:

FROM node
WORKDIR  /usr/share/fullstackExercise/
RUN npm install puppeteer
RUN npm install mocha
RUN npm install chai
RUN npm install chai-as-promised

This would work, but I recalled something about Node.js having a package.json file, similar to Composer's composer.json (I say similar... Composer is based on NPM after all). Anyhow this seemed like a better way to go so I created one of those (with help from NPM itself, which has a wizard to generate the baseline file, via npm init). I ended up with this:

{
  "name": "fullstackexercise",
  "description": "Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers",
  "version": "2.6.0",
  "main": "index.js",
  "directories": {
    "test": "tests"
  },
  "devDependencies": {
    "chai": "^4.2.0",
    "mocha": "^8.2.1",
    "puppeteer": "^5.5.0",
    "chai-as-promised": "^7.1.1"
  },
  "scripts": {
    "test": "mocha tests/**/*.js"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/adamcameron/fullstackExercise.git"
  },
  "author": "",
  "license": "GPL-3.0-or-later",
  "bugs": {
    "url": "https://github.com/adamcameron/fullstackExercise/issues"
  },
  "homepage": "https://github.com/adamcameron/fullstackExercise#readme"
}

That's a mix of the stuff npm init created, and the equivalent values from composer.json. This means the Dockerfile just becomes:

FROM node
WORKDIR  /usr/share/fullstackExercise/
RUN npm install

And the relevant section from docker-compose.yml:

node:
  build:
    context: ./node
  volumes:
    - ..:/usr/share/fullstackExercise
    - ./node/root_home:/root
  stdin_open: true
  tty: true
  networks:
    - backend

There's nothing unfamilar here. This all installs "fine":

So far so good. I've grabbed some code from the Puppeteer getting started docs, and will give that a whirl:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://example.com');
  await page.screenshot({path: 'example.png'});

  await browser.close();
})();

I don't wanna make screenshots of web pages, but it'll do for a start:

Oh. That's not what I wanted to see. I googled / stack-overflow-ed this, and yeah it seems that Puppeteer makes some assumptions about what libs are installed, and I'll be short some. I saw various different lists of stuff it was missing, and different approaches to remedying it, but for starters I figured I'd just install that lib in my container, and see what happens next. I update docker/node/Dockerfile thus:

FROM node
RUN apt-get update
RUN apt-get install libnss3-dev --yes
WORKDIR  /usr/share/fullstackExercise/
RUN npm install

After that I dropped my containers, rebuilt them, and ran that code again. I'm not expecting it to work yet, I just want to see that Puppeteer stops complaining about that library being missing:

It's a different library this time, so I'm calling this an "interim success". I'll rinse and repeat until I stop getting missing lib errors, and report back in a minute. Or so…

OK, so after some trial and error, I nailed it down to these missing libraries:

FROM node
RUN apt-get update
RUN apt-get install libnss3-dev libatk-bridge2.0-dev libx11-xcb1 libdrm-dev libxkbcommon-dev libgtk-3-dev libasound2-dev --yes
WORKDIR  /usr/share/fullstackExercise/
RUN npm install

All I did to track them down is to copy and paste the reference to the lib in the error and search for them on Ubuntu Packages Search. Sometimes the library name was not an exact match with the reference mentioned in the error. Anyway, this lot works for me. Given all the stuff I had read on this listed different combinations of missing packages, I'm guessing it'll all be platform- and situation-specific, so don't treat that list as canonical. Righto, this is what happens now when I run the code:

OK, what? I googled about and found a work around for this on Stack Overflow: Running headless Chrome / Puppeteer with --no-sandbox. It just showed me that one can pass that arg to Puppeteer, adn that makes the problem go away.

The code in question is now this:

const browser = await puppeteer.launch({args: ['--no-sandbox']});

And when I run this, I sat there looking at this for a while:

Nothing. Then I realised I'm a div cos this code doesn't output anything to stdout, it writes a file. So if I look in the correct place for the output, things are better:

And here it is:

Perfect!

What I want to get out of this though is inspecting the actual document, so I have further updated the code to be this:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch({args: ['--no-sandbox']});
    const page = await browser.newPage();
    await page.goto('https://example.com');
const title = await page.title();
console.info(`The title is: ${title}`);
    await browser.close();
})();

Here I'm just grabbing the title and outputing it. This will be part of my first test, so it's a reasonable next step here. And it works:

I'm now content that Puppeteer works, and I have some code that executes, but I need to get this stuff into some tests now.

I've got Mocha (test framework), Chai (assertion library) and chai-as-promised (promise-oriented assertion library for Chai) installed. I'm not going to do a tutorial on running Mocha tests here, I'm just gonna look at my code. Here's my first test:

let puppeteer = require('puppeteer');

let chai = require("chai");
let chaiAsPromised = require("chai-as-promised");
chai.use(chaiAsPromised);

let should = chai.should();

describe("Baseline test of vue.js working", function () {

this.timeout(5000);

    it("should return the correct page title", async function () {
let browser = await puppeteer.launch( {args: ["--no-sandbox"]});
let page = await browser.newPage();
await page.goto("http://webserver.backend/gdayWorld.html");

await page.title().should.eventually.equal("G'day world");

await page.close();
await browser.close();
    });
});

Notes:

• This top bit is all just pulling in all the libs I need to use in the test.
• I had to put this timeout in, as I was getting a timeout error sometimes (see error below).
• We've already discussed this lot, it's from the Puppeteer example code.
• For now, I am running my tests against the flat-HTML version of the file. I'm doing this because I'm kinda testing my usage of the test framework for now, and I don't have the Vue.js-driven version of the file yet.
• And here's the assertion version of the key line in the sample code. I like this fluent syntax.
• The eventually bit is the key bit from chai-as-promised: it handles all the async / promise carry one, and once it's done, runs the assertion. Nice.
• And some tear down stuff. Note that I will refactor some of this stuff out from being inline in the test in the next iteration. For now I'm largely just copying and pasting the example code into a test, and tweaking it so it'll run.

And run it indeed does!

I like this thing that npm did for me when I created package.json. It asks me how to run my tests, and then writes a test script for me:

"scripts": {
  "test": "mocha tests/**/*.js"
},

This is how I can just go npm test to run my tests. Very mundane to seasoned Node.js users obviously, but I am just a noob.

Oh, that timeout error I was getting:

So that's why I added that timeout. Initially I thought I was doing the async stuff wrong given the hinting it was giving me, but it turned out literally to be the case that the test was taking too long to run. I guess the HTTP request and document render takes time.

Brimming with confidence, I now set out do write my other tests, and at the same time refactor the test to be a bit more sensibly organised:

let puppeteer = require('puppeteer');

let chai = require("chai");
let chaiAsPromised = require("chai-as-promised");
chai.use(chaiAsPromised);
let should = chai.should();

describe("Baseline test of vue.js working", function () {
    let browser;
    let page;

    this.timeout(5000);

    const expectedText = "G'day world";

before (async function () {
browser = await puppeteer.launch( {args: ["--no-sandbox"]});
page = await browser.newPage();

await page.goto("http://webserver.backend/gdayWorld.html");
});

after (async function () {
await page.close();
await browser.close();
});

    it("should return the correct page title", async function () {
await page.title().should.eventually.equal(expectedText);
    });

    it("should return the correct page heading", async function () {
        let headingText = await page.$eval("h1", headingElement => headingElement.innerText);

headingText.should.equal(expectedText);
    });

    it("should return the correct page content", async function () {
        let paragraphContent = await page.$eval("p", paragraphElement => paragraphElement.innerText);

paragraphContent.should.equal(expectedText);
    });
});

• I've shifted all the Puppeteer setup/teardown into the appropriate handler functions, so they're only in there once for all three tests.
• This really makes the tests very simple and focused.
• For the tests needing the content of an element, I'm using this $eval method which passes the element to the callback once it's finally fetched…
• … this way we don't need the async-handling "should.eventually code, and just should is fine.

And when I run this:

Wonderful. Now I'm happy my tests are doing the right thing (on the wrong file), I'm ready to implement my Vue.js version of the page.

The mark-up I have come up with has a slight change from the flat HTML version (public/gdayWorld.htm). The new file is public/gdayWorldViaVue.html:

<!doctype html>

<html lang="en">
<head>
<meta charset="utf-8">

<title id="title">{{ message }}</title>
</head>

<body>
<div id="app">
<h1>{{ message }}</h1>
<p>{{ message }}</p>
</div>
</body>
</html>

The Vue objects need to be hung off an element via an id, hence that <div id="app"> being added to handle the heading and content. But I'm gonna need a second Vue object to deal with the <title>. I presume there's a way around this, but I've not got that far yet.

Before I add the Vue part of the code, I'm going to update my tests to hit this new file, and also expect the updated text values:

const expectedText = "G'day world via Vue";

before (async function () {
    browser = await puppeteer.launch( {args: ["--no-sandbox"]});
    page = await browser.newPage();

    await page.goto("http://webserver.backend/gdayWorldViaVue.html");
});

As Vue.js is not wired in to the new file yet, the tests will fail, because the mark-up just contains the placeholder text ({{ message }}), and Vue has not rendered the dynamic values in their place yet.

This is good news: we have a failing test! Now we can do the work. The Vue.js code to do all the work is very very very simple (public/assets/scripts/gdayWorldViaVue.js):

let appData = {message: "G'day world via Vue"};
new Vue({el: '#title', data: appData});
new Vue({el: '#app', data: appData});

As per above, this hangs a Vue object off each of the #title and #app<div> elements, and binds the appData object with the message in it to them. That's it.

Back in the HTML file I need to load in Vue.js and my own script file:

</div>
<script src="https://cdn.jsdelivr.net/npm/vue@2/dist/vue.js"></script>
<script src="assets/scripts/gdayWorldViaVue.js"></script>
</body>
</html>

Now I can run my tests, and test that Vue.js is indeed loading the message into the title, heading and content elements:

Boom! It all works. That took a while, eh? (albeit longer to write it up and then to read about it, than to do the actual work).

For the sake of completeness, let's have a look at the page rendering in a browser. This is, after all, the "end user" requirement. Not simply that some tests pass ;-)

Perfect.

Now I admit that was an awful lot of work to test a full Vue-driven HTML document, and I won't be writing further tests like this (probably) because I'll be testing at Vue component level for the rest of the work. But I needed to get Node, Mocha and Chai all working, and - even though my initial Vue-driven test page was completely contrived - I still needed to test it.

In the next article I'll break this lot down into components and test those. All the time I will still have this test in place testing the end result expectations, and I think at least in the short term this is handy. But first I am gonna look at my green tests for a while, and have a beer. I'll be back on deck with this lot tomorrow.

Righto.

--
Adam

G'day:
I should be writing the article to finish off that series about Docker / … / Vue.js etc, but whilst I was reading the docs and doing some online course (read: watching some videos) on Vue.js, I decided to TDD one of the exercises they were suggesting. The testing requirements were a bit tougher than I'd previously done with Mocha, and it took me longer to nail the testing of this than I would have liked. I figured it's worth working through it again, and jotting down some notes this time. Also I've read a bunch of blog articles instructing on Vue.js, and none of them say stuff like "implement your expectations as tests before you begin", so this is breaking that mould: if you have a dev task… always start by implementing tests for your expectations.

The exercise is part of the video Build a GitHub User Profile Component at vueschool.io. I've found their videos pretty handy, btw, and have been practising my testing whilst working through them in parallel to watching them.

The exercise is to take some flat example HTML and convert it into a Vue-implemented / data-driven solution. Here's the HTML:

<html>

<head>
<link rel="stylesheet"
        href="https://cdnjs.cloudflare.com/ajax/libs/semantic-ui/2.4.1/semantic.min.css">
</head>

<body>

<div id="app"
       class="ui container">
<h1>GitHub Profiles</h1>
<github-user-card username="hootlex"></github-user-card>

<!-- Template for GitHub card -->
<div class="ui card">
<div class="image">
<img src="https://semantic-ui.com/images/avatar2/large/kristy.png">
</div>
<div class="content">
<a class="header">Kristy</a>
<div class="meta">
<span class="date">Joined in 2013</span>
</div>
<div class="description">
          Kristy is an art director living in New York.
</div>
</div>
<div class="extra content">
<a>
<i class="user icon"></i>
          22 Friends
</a>
</div>
</div>
</div>

<!-- Import Vue.js and axios -->
<script src="https://unpkg.com/vue"></script>
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>

<!-- Your JavaScript Code :) -->
<script>
    new Vue({
      el: '#app'
    })
</script>
</body>

</html>

And that displays this sort of thing:

IE: some details from a Github user profile. NB: no idea whether Kirsty is a real person - I hope not - but it's the exampe vueschool gave. The ultimate object of the exercise is to provide the Github user name to the code and it will display that user's deets.

One could see this as a refactoring exercise. Before we can refactor our code, we need to have some green tests to guard that we don't break anything while we refactor. In the real world we'd already have these, but I don't in this case so I'll need to knock some together now. The initial "green" state of the code is that it displays placeholder information at specific places in the DOM. This is all we will test for to start with.

Having reviewed the mark-up, there's seven things we need to test for, best described by the test definitions themselves:

The tests run, they fail, but I have a list of requirements to fulfil, even if I have not yet defined what it is to fulfil them. I'll start doing that now. For the next iteration I am going to do these things:

• make the actual HTTP request with Puppeteer;
• identify where in the DOM I need to check values;
• for now, test against a bad value. The failure message will help is check if we're checking the DOM correctly;

let puppeteer = require("puppeteer");

let chai = require("chai");
let should = chai.should();

describe.only("Tests of githubProfiles page using placeholder data", function () {

    let browser;
    let page;

    before("Load the page", async function () {
        this.timeout(5000);

browser = await puppeteer.launch( {args: ["--no-sandbox"]});
page = await browser.newPage();

await Promise.all([
page.goto("http://webserver.backend/githubProfiles.html"),
page.waitForNavigation()
]);
    });

    after("Close down the browser", async function () {
        await page.close();
        await browser.close();
    });

    it("should have the expected person's name", async function () {
let name = await page.$eval("#app>.card>.content>.header", headerElement => headerElement.innerText)
name.should.equal("INSERT EXPECTED NAME VALUE HERE");
    });

    it("should have the expected person's github page URL", async function () {
        let linkHref = await page.$eval("#app>.card>.content>a.header", headerElement => headerElement.href);
linkHref.should.equal("INSERT EXPECTED LINKHREF VALUE HERE");
    });

    it("should have the expected person's avatar", async function () {
        let avatar = await page.$eval("#app>.card>.image>img", avatarElement => avatarElement.src);
avatar.should.equal("INSERT EXPECTED AVATAR SRC VALUE HERE");
    });

    it("should have the expected person's joining year", async function () {
        let joiningMessage = await page.$eval("#app>.card>.content>.meta>.date", joiningElement => joiningElement.innerText);
joiningMessage.should.equal("INSERT EXPECTED JOINING MESSAGE VALUE HERE");
    });

    it("should have the expected person's description", async function () {
        let description = await page.$eval("#app>.card>.content>.description", descriptionElement => descriptionElement.innerText);
description.should.equal("INSERT EXPECTED DESCRIPTION VALUE HERE");
    });

    it("should have the expected person's number of friends", async function () {
        let friendsText = await page.$eval("#app>.card>.extra.content>a", extraContentAnchorElement => extraContentAnchorElement.innerText);
friendsText.should.equal("INSERT EXPECTED FRIENDS TEXT VALUE HERE");
    });

    it("should have the expected person's friends URL", async function () {
        let linkHref = await page.$eval("#app>.card>.extra.content>a", extraContentAnchorElement => extraContentAnchorElement.href);
linkHref.should.equal("INSERT EXPECTED FRIENDS LINK HREF VALUE HERE");
    });
});

And the run confirms (mostly) that we're inspecting the correct bit of the DOM. I'll just include the AssertionErrors from each test here, cos the rest of the output is largely the same:

Cool it looks like I've mostly nailed the DOM selectors though: the correct values are being extracted by the tests. There's two that I'm not sure about though: the two link hrefs aren't actually in the mark-up, so their values are just blank. Well: all going well that's the reason, but I need to check that. A third thing to note is that there's a leading space in the friends text. Initially I thought this was a typo in the mark-up, but if we have a look, it's collapsed leading whitespace from the other content in the element I'm checking:

<div class="extra content">
<a>
        <i class="user icon"></i>
        22 Friends
</a>
</div>

That's legit, and we'll need our test to expect that. I've gone ahead and "fixed" the test mark-up to include the anchor tag hrefs (with dummy values):

<div class="content">
<a class="header" href="GITHUB_PAGE_URL">Kristy</a>
<div class="meta">
<span class="date">Joined in 2013</span>
</div>
<div class="description">
        Kristy is an art director living in New York.
</div>
</div>
<div class="extra content">
<a href="GITHUB_FRIENDS_PAGE_URL">

Now to re-run the tests, and hopefully see those dummy values being compared to the test values. Again, I'll just include the AssertionExceptions:

OK, nice one. The tests are all good now: they are testing the correct elements in the DOM (except the whitespace one, I'll get to that), so I'm gonna update the tests to check for the actual placeholder values now. IE: make the tests pass for the boilerplate HTML.

let puppeteer = require("puppeteer");

let chai = require("chai");
chai.use(require("chai-string"));
let should = chai.should();

describe.only("Tests of githubProfiles page using placeholder data", function () {

    let browser;
    let page;

let expectedUserData = {
name : "Kristy",
pageUrl : "http://webserver.backend/GITHUB_PAGE_URL",
avatar : "https://semantic-ui.com/images/avatar2/large/kristy.png",
joinedMessage : "Joined in 2013",
description : "Kristy is an art director living in New York.",
friends : "22 Friends",
friendsPageUrl: "http://webserver.backend/GITHUB_FRIENDS_PAGE_URL"
};

    // ...

    it("should have the expected person's name", async function () {
        let name = await page.$eval("#app>.card>.content>.header", headerElement => headerElement.innerText)
name.should.equal(expectedUserData.name);
    });

    // ...

    it("should have the expected person's number of friends", async function () {
        let friendsText = await page.$eval("#app>.card>.extra.content>a", extraContentAnchorElement => extraContentAnchorElement.innerText);
        friendsText.should.containIgnoreSpaces(expectedUserData.friends);
    });

    // ...
});

I've elided a lot of code there due to it either being unchanged from before, or just the same as the examples I'm showing, and just focused on a coupla changes:

• I've moved the expected values out into one object. This is partly to keep it all in one place, partly because I already know I'll be changing how I populate all that very soon.
• I'm testing the actual values from the mark-up now. So the tests should be green.
• I'm dealing with the extraneous whitespace.

When I run this:

Where are we now then? We have some tests that correctly check the correct part of the DOM of our test page. And all those tests pass. We can now start to do the actual work. The first thing we can do is a slight refactor: we can pull the inline mark-up out into a Vue template, and use the template's data object to furnish the template with the dynamic values. The desired result here is that no adjustments to tests should be necessary. First the mark-up file:

<html>
<head>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/semantic-ui/2.4.1/semantic.min.css">
</head>
<body>
<div id="app"
     class="ui container">
<h1>GitHub Profiles</h1>
<github-user-card username="hootlex"></github-user-card>
    
</div>

<script type="text/x-template" id="github-user-card-template">
<div class="ui card">
<div class="image">
<img :src="avatar">
</div>
<div class="content">
<a class="header" :href="pageUrl">{{name}}</a>
<div class="meta">
<span class="date">Joined in {{joinedYear}}</span>
</div>
<div class="description">
{{description}}
</div>
</div>
<div class="extra content">
<a :href="friendsPageUrl">
<i class="user icon"></i>
{{friends}} Friends
</a>
</div>
</div>
</script>

<script src="https://unpkg.com/vue"></script>
<script src="assets/scripts/githubProfiles.js"></script>
</body>
</html>

I've done a few things here:

• Moved the mark-up for the "Github user card" into its own template.
• And taken out the hard-coded values, which I will relocate into the JS application code (see below).
• And I'm actually calling the JS application code.

That's actually the mark-up side of things complete now. Th JS code to support it is thus:

let githubUserCardComponent = {
    template : "#github-user-card-template",
    data : function () {
        return {
            name : "Kristy",
            pageUrl : "http://webserver.backend/GITHUB_PAGE_URL",
            avatar : "https://semantic-ui.com/images/avatar2/large/kristy.png",
            joinedYear : 2013,
            description : "Kristy is an art director living in New York.",
            friends : 22,
            friendsPageUrl: "http://webserver.backend/GITHUB_FRIENDS_PAGE_URL"
        };
    }
};

new Vue({
    el: '#app',
    components: {
        "github-user-card" : githubUserCardComponent
    }
});

Here we're just setting the data values expected by the template. It's still all hard-coded values for now. To check we've not messed anything up, we re-run the tests:

Cool. That all works.

Now we are going to need to do a code change: we need to source the actual data for the user from Github, via its API. But before we do that, we need to update our tests to expect that. We can't have our tests start to break because we've changed the code. We need to update the tests to expect the changes, and then… well… let them break that way instead ;-)

Now in the normal scheme of things, to keep this purely a functional test, I would mock the data provider for the data so when testing I'd not be getting live Github data, I'd be getting "known" values from a mock, and test that the known values are handled correctly. But I don't yet know how to test Vue components separately, so I don't know how to go about mocking the request to Github. What I'm gonna do is turn this into an integration test, and the test will get the correct data from Github, and then check if the app is getting the same (so accordingly correct) data too. Currently the test has this:

let expectedUserData = {
    name : "Kristy",
    pageUrl : "http://webserver.backend/GITHUB_PAGE_URL",
    avatar : "https://semantic-ui.com/images/avatar2/large/kristy.png",
    joinedMessage : "Joined in 2013",
    description : "Kristy is an art director living in New York.",
    friends : "22 Friends",
    friendsPageUrl: "http://webserver.backend/GITHUB_FRIENDS_PAGE_URL"
};

We're gonna get rid of those stubbed values with "live" values from Github. For reference, this is the JSON returned from the API call we're using:

{
  "login": "adamcameron",
  "id": 2041977,
  "node_id": "MDQ6VXNlcjIwNDE5Nzc=",
  "avatar_url": "https://avatars3.githubusercontent.com/u/2041977?v=4",
  "gravatar_id": "",
  "url": "https://api.github.com/users/adamcameron",
  "html_url": "https://github.com/adamcameron",
  "followers_url": "https://api.github.com/users/adamcameron/followers",
  "following_url": "https://api.github.com/users/adamcameron/following{/other_user}",
  "gists_url": "https://api.github.com/users/adamcameron/gists{/gist_id}",
  "starred_url": "https://api.github.com/users/adamcameron/starred{/owner}{/repo}",
  "subscriptions_url": "https://api.github.com/users/adamcameron/subscriptions",
  "organizations_url": "https://api.github.com/users/adamcameron/orgs",
  "repos_url": "https://api.github.com/users/adamcameron/repos",
  "events_url": "https://api.github.com/users/adamcameron/events{/privacy}",
  "received_events_url": "https://api.github.com/users/adamcameron/received_events",
  "type": "User",
  "site_admin": false,
  "name": "Adam Cameron",
  "company": null,
  "blog": "http://blog.adamcameron.me/",
  "location": "London",
  "email": null,
  "hireable": null,
  "bio": null,
  "twitter_username": "adam_cameron",
  "public_repos": 21,
  "public_gists": 211,
  "followers": 27,
  "following": 2,
  "created_at": "2012-07-25T18:02:54Z",
  "updated_at": "2021-01-16T16:57:34Z"
}

We won't need most of that.

Now I'll go ahead and re-jig my test code to make this call, and use it as the basis for the values to test against:

describe.only("Tests of githubProfiles page using github data", function () {
    let browser;
    let page;
    let expectedUserData;

    before("Load the page", async function () {
        this.timeout(5000);

        await loadTestPage();
        expectedUserData = await loadTestUserFromGithub();
    });

Here I have pushed the assignment of expectedUserData into the before handler, and also extracted the implementation into its own function, given there's a lot more code now:

let loadTestUserFromGithub = async function () {
let githubUserData = await new Promise((resolve, reject) => {
let request = https.get(
"https://api.github.com/users/hootlex",
{
auth: `username: ${process.env.GITHUB_PERSONAL_ACCESS_TOKEN}`,
headers: {'user-agent': 'node.js'}
}, response => {
let rawResponseData = "";

response.on("data", data => {
rawResponseData += data;
}).on("end", () => {
resolve(JSON.parse(rawResponseData));
}).on("error", error => {
reject(error.message);
                });
            }
);
request.end();
    });
return {
 name : githubUserData.name,
pageUrl : githubUserData.html_url,
avatar : githubUserData.avatar_url,
joinedYear : new Date(githubUserData.created_at).getFullYear(),
description : githubUserData.bio ?? "",
friends : githubUserData.followers,
friendsPageUrl: githubUserData.html_url + "?tab=followers"
};
}

That's less complicated than it looks. Using the native Node.js https library requires one makes the request easily enough, but then one needs to receive the response data in chunks and assemble it yerself via the data event handler. And then once it's done one can use it in the end handler. To get it back to the calling code where it can be put to use, one needs to wrap all this in a promise, resolving with the data in the end handler. I really do wish there was a OhForGoodnessSakeStopMessingAroundAndJustGiveMeTheResponse handler, which I imagine is what people would use, 99% of the time. Ah well.

It's also worth noting those config params I'm setting in there:

{
    auth: `username: ${process.env.GITHUB_PERSONAL_ACCESS_TOKEN}`,
    headers: {'user-agent': 'node.js'}
}

The Github API has rate-limiting on it, and one is only allowed 60 requests per hour unless one uses some sort of authentication. One can just us a personal access token to work around this restriction. It's secret information so I don't want it anywhere in my code, so I'm just setting an environment variable on my host machine, and passing that through to the Node.js container. From docker-compose.yml:

  node:
    build:
      context: ./node
    environment:
      - GITHUB_PERSONAL_ACCESS_TOKEN=${GITHUB_PERSONAL_ACCESS_TOKEN}

And I'm passing a user agent there because the Github API requires one to.

That should all now "work", in that the tests will be broken, but it'll be testing the hard-coded Vue template data against the expected correct/live values from Github. AGain, I'll just show the AssertionExceptions here:

Argh. OK so the good news is that most of those are failing in the right way: they are testing the correct placeholder vs live values. But the one testing the joining year and the one testing the number of friends are failing for the wrong reasons:

Firstly the "should have the expected person's joining year" test has two problems:

    // test code
    it("should have the expected person's joining year", async function () {
        let joiningMessage = await page.$eval("#app>.card>.content>.meta>.date", joiningElement => joiningElement.innerText);
        joiningMessage.should.equal(expectedUserData.joinedMessage);
    });

    // test data
        return {
            name : githubUserData.name,
            pageUrl : githubUserData.html_url,
            avatar : githubUserData.avatar_url,
joinedYear : new Date(githubUserData.created_at).getFullYear(),
            description : githubUserData.bio ?? "",
            friends : githubUserData.followers,
            friendsPageUrl: githubUserData.html_url + "?tab=followers"
        };

The test data is joinedYear (just the year), and the test is still comparing to the entire message: joinedMessage. So I'll update the test, thus:

it("should have the expected person's joining year", async function () {
const expectedJoiningMessage = `Joined in ${expectedUserData.joinedYear}`;

    let joiningMessage = await page.$eval("#app>.card>.content>.meta>.date", joiningElement => joiningElement.innerText);
    joiningMessage.should.equal(expectedJoiningMessage);
});

And now this test actually passes because - by coincidence - the hard-coded year in the template matches the live one coming from Github. We can't have that so for now I'm changing the value in the template to be a different year. We can't be changing code if its test doesn't fail until before we make the changes to make the test pass.

The "should have the expected person's number of friends" is failing incorrectly for much the same reason as the previous one. Here's the relevant code:

    //test code
    it("should have the expected person's number of friends", async function () {
        let friendsText = await page.$eval("#app>.card>.extra.content>a", extraContentAnchorElement => extraContentAnchorElement.innerText);
        friendsText.should.containIgnoreSpaces(expectedUserData.friends);
    });

        // test data
        return {
            name : githubUserData.name,
            pageUrl : githubUserData.html_url,
            avatar : githubUserData.avatar_url,
            joinedYear : new Date(githubUserData.created_at).getFullYear(),
            description : githubUserData.bio ?? "",
            friends : githubUserData.followers,
            friendsPageUrl: githubUserData.html_url + "?tab=followers"
        };

Again in the test we're expecting the entire message - eg "" - to be in the value we're comparing, but obviously we're only getting the count back from Github. I'll change that in the same way as the previous one (I'll spare you the code, you get the idea). now those tests are failing in the correct way:

Now we can update the app code to get the correct data.

Firstly we're gonna need to pay attn to the username value passed to the template in the parent mark-up:

<div id="app"
     class="ui container">
<h1>GitHub Profiles</h1>
<github-user-card username="hootlex"></github-user-card>
</div>

This is just a matter of adding a property to the template definition:

let githubUserCardComponent = {
    template : "#github-user-card-template",
props : {
username: {
            type: String,
            required: true
        }
    },

Now we can use this.username in the rest of the template code.

Next: a the moment all our data values are hard-coded sample data:

let githubUserCardComponent = {
    template : "#github-user-card-template",
    data : function () {
        return {
            name : "Kristy",
            pageUrl : "http://webserver.backend/GITHUB_PAGE_URL",
            avatar : "https://semantic-ui.com/images/avatar2/large/kristy.png",
            joinedYear : 2013,
            description : "Kristy is an art director living in New York.",
            friends : 22,
            friendsPageUrl: "http://webserver.backend/GITHUB_FRIENDS_PAGE_URL"
        };
    }
};

We're gonna null-out all those as we don't know what they will be when the template is first loaded. However, when the instance of the component is created, we can dart off to Github and get the relevant values.

let githubUserCardComponent = {
    // ...
    data : function () {
        return {
name : null,
pageUrl : null,
avatar : null,
joinedYear : null,
description : null,
friends : null,
friendsPageUrl: null
        };
    },
created () {
axios.get(
`https://api.github.com/users/${this.username}`,
{
auth: {
username: this.$route.query.GITHUB_PERSONAL_ACCESS_TOKEN
}
}
)
.then(response => {
this.name = response.data.name;
this.pageUrl = response.data.html_url;
this.avatar = response.data.avatar_url;
this.joinedYear = new Date(response.data.created_at).getFullYear();
this.description = response.data.bio;
this.friends = response.data.followers;
this.friendsPageUrl = response.data.html_url + "?tab=followers";
});
}
};

There's a coupla things to note here. Firstly, just like in the test, I am passing a personal access token to the Github API so I don't get throttled by them. And, again, I don't want the actual token value to be in the code, so I am passing it on the URL. To get the value from the URL, I need to use the Vue Router (which I know nothing about other than what I found on Stack Overflow when I needed to get something from the query string). To use this I need to initialise the Vue app with a router:

let router = new VueRouter({
 mode: 'history',
routes: []
});

new Vue({
router,
    el: '#app',
    components: {
        "github-user-card" : githubUserCardComponent
    }
});

Then I can access my query string param as per the code above.

The other bit to look at in the created handler above is this Axios thing. It's just an HTTP lib that the tutorial suggested. It seems pretty slick and easier to use than the HTTPS library I was messing about with in the test code. It's relevant to note that Axios has an implementation for Node.js too, but I didn't know that when I wrote the tests. The code there works, I'll leave it.

Both the Vue Router and Axios libs need to be loaded in the main mark-up file too, obvs:

<script src="https://unpkg.com/vue"></script>
<script src="https://unpkg.com/axios/dist/axios.min.js"></script>
<script src="https://unpkg.com/vue-router"></script>
<script src="assets/scripts/githubProfiles.js"></script>
</body>
</html>

OK so in theory now on the client-side we are loading-in the data from Github, and this should match the same data the tests grabbed, and our tests should be green…

Um… where are the client-side values? I checked in the browser, and it all seemed legit:

What's worse is that about 25% of the time, the tests were actually passing! It took me quite a while to work out what was going on, but finally it turned out that I needed a lesson in how to think asynchronously, and what the various processing stages are of a web page. I'll detail the troubleshooting in another article (maybe), but it boiled down to this line of code:

let loadTestPage = async function () {
    browser = await puppeteer.launch( {args: ["--no-sandbox"]});
    page = await browser.newPage();

    await Promise.all([
        page.goto("http://webserver.backend/githubProfiles.html"),
page.waitForNavigation()
    ]);
}

When a web page loads, it's navigable as soon as the asset files (mark-up, CSS, JS etc) are loaded. But it does not wait for asynchronous data requests to complete before the page is considered ready for navigation. So that promise is fulfilled whilst the call to Github is probably still under way. Sometimes it was completing in time for the test code to check the values; most of the time it had not, so the values weren't there. I did not notice this when I first wrote this code because the call to Github wasn't in there then, so there was no problem. Anyhow, this was easily solved:

page.waitForNavigation({waitUntil: "networkidle0"})

After that the tests still failed sometimes, but this time it was because loading the test page and then the test code hitting Github itself was occasionally taking longer than the 5000ms timeout I currently had, so I upped that to 10000ms:

before("Load the page", async function () {
    this.timeout(10000);

And after those two tweaks:

And I repeated the tests dozens of times, and they always passed now, so I'm happy I've nailed the code there.

I actually continued on from here and updated the code to allow passing an override username value in the URL, but this article is already massive and this is the third day working on it, so I'm gonna end here. This was probably a tedious read, but the exercise for me in TDDing some Vue stuff was absolute gold. I still need to find out how to separate Vue components out into separate files so I can test them individually, and outside the context of a web page that is using them, but that's for another day.

Righto.

--
Adam

G'day:

This is a follow-up from something I touched on yesterday ("Polishing my Vue / Puppeteer / Mocha / Chai testing some more"). In that exercise I was using Puppeteer to load a web page I was testing, and then pulling some DOM element values out and checking they matched expectations. The relevant bits of code are thus:

describe.only("Tests of githubProfiles page using github data", function () {
    let browser;
    let page;
    let expectedUserData;

    before("Load the page and test data", async function () {
await loadTestPage();
        expectedUserData = await loadTestUserFromGithub();
    });

let loadTestPage = async function () {
browser = await puppeteer.launch( {args: ["--no-sandbox"]});
page = await browser.newPage();

await Promise.all([
page.goto("http://webserver.backend/githubProfiles.html"),
page.waitForNavigation()
]);
}

    it("should have the expected person's name", async function () {
let name = await page.$eval("#app>.card>.content>a.header", headerElement => headerElement.innerText);
name.should.equal(expectedUserData.name);
    });

• Load the page with Puppeteer
• Example test checking the page's DOM

This code seemed to be running fine, and the tests were passing. As I was adding more code ot my Vue component on the client end, I suddenly found the tests started to fail. Sometimes. If I ran them ten times, they'd fail maybe three times. At the same time, if I was just hitting the page in the browser, it was working 100% of the time. Odd. I mean clearly I was doing something wrong, and I'm new to all this async code I'm using, so figured I was using values before they were available or something. But it seemed odd that this was only manifesting sometimes. The way the tests were failing was telling though:

The values coming from the DOM were blank. And note that it's not a case of the DOM being wrong, because if that was the case, the tests would barf all the time, with something like this:

The relevant mark-up here is:

<a class="header" :href="pageUrl">{{name}}</a>

So {{name}} isn't getting its value sometimes.

I faffed around for a bit reading up on Vue components, and their lifecycle handlers in case created was not the right place to load the data or something like that, but the code seemed legit.

My JS debugging is not very sophisticated, and it's basically a matter of console.logging stuff and see what happens. I chucked a bunch of log calls in to see what happens:

created () {
console.debug(`before get call [${this.username}]`);
    axios.get(
        `https://api.github.com/users/${this.username}`,
        {
            auth: {
                username: this.$route.query.GITHUB_PERSONAL_ACCESS_TOKEN
            }
        }
    )
    .then(response => {
console.debug(`beginning of then [${response.data.name}]`);
        this.name = response.data.name;
		// [etc...]
console.debug("end of then");
    });
console.debug("after get call");
}

Along with some other ones around the place, these all did what I expected when I hit the page in the browser:

I noted that the then call was being fulfilled afterthe mainline code had finished, but in my test I was waiting for the page to fully load, so I'd catered for this. Repeated from above:

await Promise.all([
    page.goto("http://webserver.backend/githubProfiles.html"),
    page.waitForNavigation()
]);

I ran my tests, and was not seeing anything in the console which momentarily bemused me. But then I was just "errr… duh, Cameron. That stuff is logging in the web page's console. Not Node's console from the test run". I'm really thick sometimes.

This flumoxed me for a bit as I wondered how the hell I was going to get telemetry out of the page that I was calling in the Puppeteer headless browser. Then it occurred to me that I would not be the first person to wonder this, so just RTFMed.

It's really easy! The Puppeteer Page object exposes event listeners one can hook into, and one of the events is console. Perfect. All I needed to do is put this into my test code:

page = await browser.newPage();

page.on("console", (log) => console.debug(`Log from client: [${log.text()}] `));

await Promise.all([
    page.goto("http://webserver.backend/githubProfiles.html"),
    page.waitForNavigation()
]);

Then when I ran my tests, I was console-logging the log entries made in the headless browser as they occurred. What I was seeing is:

Note how the tests get underway before the then call takes place. And shortly after that, the tests start passing because by then the dynamic values have actually been loaded into the DOM. This is my problem! that page.waitForNavigation() is not waiting long enough! My first reaction was to blame Puppeteer, but I quickly realised that's daft and defensive of me, given this is the first time I've messed with this stuff, almost certainly I'm doing something wrong. Then it occurred to me that a page is navigable once the various asset files are loaded, but not necessarily when any code in them has run. Duh. I figured Puppeteer would have thought of this, so there'd be something else I could make it wait for. I googled around and found the docs for page.waitForNavigation, and indeed I needed to be doing this:

page.waitForNavigation({waitUntil: "networkidle0"})

After I did that, I found the tests still failing sometimes, but now due to a time out:

I had the time out set for five seconds, but now the tests are waiting for the client to finish its async call as well, I was just edging over that five second mark sometimes. So I just bumped it to 10 seconds, and thenceforth the tests all passed all the time. I've left the telemetry in for one last successful run here:

OK so that was a bit of a newbie exercise, but I'm a noob so yer gonna get that. It was actually pretty fun working through it though. I'm really liking all this tooling I'm checking out ATM, so yer likely get a few more of these basic articles from me.

Righto.

--
Adam

G'day:

This is an article I am inserting into my ongoing series of setting up a web app using Docker containers, running Nginx, PHP (and Symfony within that), MariaDB, with a Vue.js front end being built in a Node.js container. The other articles can be found here:

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers
5. MariaDB
6. Installing Symfony
7. Using Symfony
8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
9. I mess up how I configure my Docker containers (this article)
10. An article about moving files and changing configuration
11. Setting up a Vue.js project and integrating some existing code into it

A week or so ago I wrote an article "Part 8: Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer". In there I detail how I had configured my Node.js container in Dockerfile and docker-compose.yml, and how it all worked nicely. A few days ago when prepping for the last article of that series ("Refactoring the simple web page into Vue components" - not written yet, so no link yet either, sorry), I discovered my Node.js container was only working by a coincidental quirk of how I built it: and as it stood, the configuration I cited didn't actually work as I described it. I had made a rather school-boy-ish error.

I'm going to explain what I did wrong using a stand-along Node.js container / app, because it'll be a bit more clear if I just focus on the one container, not the array of moving parts I have in that other application. Then I show how I applied a working solution to the that main app.

The requirement I am fulfilling here is to set up a simple Docker container running Node.js, and run some Mocha tests for a code puzzle I have. Normally-speaking one would not have both a Dockerfile and a docker-compose.yml file for this sort of minimal requirement, but the issue I had was with how Dockerfile and docker-compose.yml interact, so I'll include both.

Firstly my Dockerfile:

FROM node
WORKDIR  /usr/share/nodeJs/

This just grabs the Node.js image from Docker, sets the directory I'm gonna be using for my code. nothing exciting. In the docker-compose.yml file I've got this lot:

version: '3'

services:
  node:
    build:
      context: node
    volumes:
      - ..:/usr/share/nodeJs
      - ./node/root_home:/root
    stdin_open: true
    tty: true

This is all straight-forward seeming. So as I can develop my source code files as test as I go, I have mapped my host's nodeJs directory to the container's working directory /usr/share/nodeJs. In addition to that I'm mapping another directory I have in my project as the container's root user's home directory. This is so I can a) avail the container of a .bashrc file I've got some aliases in; b) the .bash_history will be stored on my host machine, so it will persist between rebuilds of the container. This stuff with the root_home stuff is not important to this article.

I can build this container and run it and dive inside:

Here I am doing the following:

• Building the container from Dockerfile and docker-compose.yml config;
• shelling into it;
• Remembering I need to actually install the npm modules;
• Running some unit tests to demonstrate the npm stuff all installed OK, and the code is operational.

Oh, for good measure, npm install is installing this lot (from package.json):

"devDependencies": {
"chai": "^4.2.0",
"chai-as-promised": "^7.1.1",
"chai-datetime": "^1.7.0",
"chai-match": "^1.1.1",
"chai-string": "^1.5.0",
"deep-eql": "^3.0.1",
"mocha": "^8.2.1"
  },

So if everything had not worked A-OK, the tests would not have run, let alone passed.

The process for the app I'm writing in the blog article series is the same as this, just with a bunch more stuff in the docker-compose.yml file. And it all worked the same as above, so I'm happy a Larry, and off I go to write my blog article.

…

…

When I come to start prepping for the next article, I remember I had to do the npm install step manually, and that was wrong, so I went to move it into the Dockerfile. At the same time the next article called for me to install Vue CLI, so I chucked that into the Dockerfile too:

FROM node
WORKDIR /usr/share/nodeJs/
RUN npm install -g @vue/cli
RUN npm install

Because I wanted to test this from scratch, I removed the package-lock.json file and node-modules directory from the previous install, and completely rebuilt the container:

That all seems fine: I can see the two npm install runs happening. So I proceed into the container:

So far… so good. Vue CLI is installed and running. And just to make sure the tests still run:

Ummm… what? No node_modules, and no package-lock.json.

but the Vue CLI stuff is working, and all there OK:

I googled all over the place for someone / something to blame for this. I googled for npm install silectly failing. I removed the Vue CLI stuff and the npm install seemed to run, but did nothing. I noticed the Removing intermediate container df849deb15ad, and surmised Docker was doing the second npm install in a temporary container for some reason, and then dropping it so… no more node_modules or package-lock.json in my working directory. So I googled about the place to find ways to stop bloody Docker putting things into intermediate containers (without thinking to google why it might be doing that in the first place).

And this is something that really annoys me about myself sometimes. I also see it in other devs (and I've mentioned it here before, too). My first thought when something is going wrong is that it's someone else's fault. Someone else has buggered something up in Docker, Node, Vue, etc; and I'm just being caught out by it. This is so hubristic of me given this is my first time working with this stuff, and millions of other people get on just fine with it. My first thought should always be: what have I done wrong?

Then it dawned on me. Well actually someone said something in a Stack Overflow answer which was unrelated to this, but got me thinking. Here is what happens:

We have this in the Dockerfile:

WORKDIR /usr/share/nodeJs/
[...]
RUN npm install

What does this do? It created a directory /usr/share/nodeJs/ and then runs npm install. What's in the directory at that point? F*** all. Certainly not a package.json file. That doesn't come into play until the container is created, and we're mapping volumes into it, when DOcker looks at the docker-compose.yml stuff, which isn't until after the Dockerfile processing is done.

OK so I'll need the package.json file around in the Dockerfile. First I just try to copy it from the root of my app directory (where it currently is), and this yields and error when I try to build the container:

Fair enough. it occurs to me I only need it in the root of the app in the container; as I'm not running any ofthis stuff on my host machine, it does not need to be in the root of the application there at all. It can be anywhere. So I shift it into a config subdirectory, within my docker subdirectory:

This time I can see it installing all the dependencies fine, but it's still in a intermediate container:

Unsurprisingly that being the case, the node_modules directory is not there when I check in the container file system. But also…neither is the package.json file! All that's in there is the contents of … [the penny has just dropped] … my host machine's application directory:

Now. Do you think that might be because that's exactly what I'm telling docker-compose.yml to do? Here:

volumes:
  - ..:/usr/share/nodeJs

I'm tellingdocker-compose to plaster that directory from my host machine over the top of the directory that Dockerfile has been working in.

Sigh.

And this makes total sense too.

Right. I have some reorganising to do. The problem really is that I'm voluming-in (to invent a phrase) the entire app directory into the container's working directory - which I know now will not work - I have to think about what I need in the working directory for the app to run:

• I need the package.json file in there at the time Dockerfile is processed, because that's when npm install happens. Cool I've already done that.
• After I've done a successful npm install I should also grab the package-lock.json file and make sure I also stick it in the working directory in the Dockerfile. But I'll deal with that later after everything is working.
• If we have a look at what else is in that root application directory:
  
  camer@DESKTOP-QV1A45UMINGW64/c/src/nodejs((1.0.4))
  $ ll
  total 125
  drwxr-xr-x 1 camer 197609     0 Feb  6 10:14 docker/
  -rw-r--r-- 1 camer 197609 35149 Jan 29 14:47 LICENSE
  -rw-r--r-- 1 camer 197609   680 Feb  6 10:14 package.json
  -rw-r--r-- 1 camer 197609 79915 Feb  6 10:14 package-lock.json
  -rw-r--r-- 1 camer 197609    30 Jan 29 14:47 README.md
  drwxr-xr-x 1 camer 197609     0 Feb  1 12:37 src/
  drwxr-xr-x 1 camer 197609     0 Feb  1 23:30 test/
  
  camer@DESKTOP-QV1A45UMINGW64/c/src/nodejs((1.0.4))
  
  All we really need in the container is the src/ and test/ directories. We don't need to add the whole directory as a volume: just those two will do. Also I'm thinking ahead of my other app here, which has a container for PHP as well, and in that there's a bunch of stuff that lives in the app root directory for Symfony, PHPUnit, PHPMD, PHPCS etc. Currently messed in along with the Node.js stuff. That's a bit of a mess, and when those files are being used, they're never all in a given container: they're used in different ones. So now that I think about it, it's "wrong" that they are all in the same directory in source control. But I'm getting ahead of myself. Back to this simple Node.js container situation.

So the solution to this seems to be:

FROM node
WORKDIR  /usr/share/nodeJs/
COPY config/* ./
RUN npm install

As I said earlier, I move the config stuff that Dockerfile needs into a subdirectory of the docker config.

Next in the application root directory, I have no container-specific stuff, just the docker, src and test directories; and the source control project bumpf:

In docker-compose.yml we specifically make volumes for just the src/ and test/ directories:

version: '3'

services:
  node:
    build:
      context: node
    volumes:
      - ./node/root_home:/root
      - ../src:/usr/share/nodeJs/src
      - ../test:/usr/share/nodeJs/test
    stdin_open: true
    tty: true

This means everything is in the right place when it's needed, and there's no overlap in any of the file system stuff in Dockerfile and docker-compose.yml. And when I now build the container from scratch again…

… it's all good.

I note with interest that the "Removing intermediate container" thing was a red herring in all this: it happens as a matter of course anyhow. The good news is that everything is there and working fine.

That's about all I have to say on that. I have already made equivalent changes to the codebase for the code for the long-running article series I'm writing. I'll summarise that in the next article.

It's just worth reflecting on this again though:

And this is something that really annoys me about myself sometimes. [...] My first thought when something is going wrong is that it's someone else's fault[...] and I'm just being caught out by it. This is so hubristic of me given this is my first time working with this stuff, and millions of other people get on just fine with it. My first thought should always be: what have I done wrong?

Righto.

--
Adam

G'day:

I hope I correctly set the excitement expectations with the title of this one. It's gonna be dead dull. In the previous article ("I mess up how I configure my Docker containers"), I detailed a fundamental flaw in how I was configuring my Dockerfiles and docker-compose.yml file, which pretty much had a logic-conflict in them. Instead of using the fullstack-exercise codebase I've been working on in this series, I used a cut down one that focused specifically on the issue. In this article I am detailing the file-system and config reorganisation I then performed on the fullstack-exercise codebase to fix the issue. TBH I'd probably not bother reading it if I was you (my fictitious reader), cos it's even more dry than my usual efforts. I'm pretty much only writing it out of a sense of completeistness (!), and also in case someone happens to be reading along with the rest of the series and - if they came to the next article - suddenly thought "hang on all the files have moved around? What subterfuge is this?". So it's a full-disclosure exercise I guess. If you do insist on reading this, read the previous article first though, eh? Good luck.

As per usual: I'll remind you that this is part 10 of an 11(?) part series, with the earlier articles linked below:

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers
5. MariaDB
6. Installing Symfony
7. Using Symfony
8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
9. I mess up how I configure my Docker containers
10. An article about moving files and changing configuration (this article)
11. Setting up a Vue.js project and integrating some existing code into it

The TL;DR of the previous article is kinda:

Don't map volumes in docker-compose.yml over the top of the working directory specified in Dockerfile, if the Dockerfile actually creates stuff you need in that working directory (like a node_modules subdirectory, for example). This is because a volume mapping replaces what's there, it does not merge with it.

Schoolkid dumbarsery from me there.

Now I'm gonna apply the lessons learned there to the main codeabse for this project. This is the directory structure I had previously (on github). Note some stuff not relevant to this exercise has been removed:

The two points that make it most obvious that things are poorly-organised here are:

• the root directory which has a mix of stuff intended for the PHP container and other stuff intended for the Node.js container.
• And, similarly the tests subdirectory has a mix of back-end PHPUnit tests and front-end Mocha tests in the same substructure.

Where there's a mess or a mix of things intended for two different purposes in the same place, it's a flag that something's possibly not right. Now I will be honest and say that this decision originally was purposeful on my part. I'm not distinguishing between the front-end part of the app (Node.js, Vue and Mocha stuff), and the back-end running Symfony and PHP. The front-end stuff is the web site for this app; the back-end will be the web service to support the front-end. They are not two distinct apps in my view (or one way of looking at it). This is not to say they can't still be organised a bit more coherently than I have.

To separate my concerns, I've decided to move all the code-related stuff into one of backend or frontend subdirectories. First the backend directory (see backend on Github):

In the backend subdirectory I have all the PHP / Symfony / PHPUnit stuff, plus a public directory that is purely for the back-end web root. And - below - the Docker Nginx config now has separate websites for back-end and front-end; and in the php-fpm section we now have all the PHP / Symfony config stuff moved out of the application root, and into its own specific root:

The key part of the Nginx configuration changes here is that the two sites now have distinct host names: fullstackexercise.backend (see backend.conf on Github) for the PHP-oriented stuff, and fullstackexercise.frontend (see frontend.config on Github) for the Vue- / Node-based stuff. Each website only serves the type of files appropriate for their purpose.

The Nginx Dockerfile (on Github) has not changed significantly, but now the PHP-FPM one (on Github) copies all the application-root stuff into the working directory, rather than docker-compose.yml file using a volume to do this:

WORKDIR  /usr/share/fullstackExercise/
COPY ./app_root/ /usr/share/fullstackExercise/

It's worth looking at the whole lot of the service definitions for these from docker-compose.yml:

services:
  nginx:
    build:
      context: ./nginx
    volumes:
- ../frontend/public:/usr/share/nginx/html/frontend
- ../backend/public:/usr/share/nginx/html/backend
      - ../log:/var/log
      - ./nginx/root_home:/root
    ports:
      - "80:80"
    stdin_open: true # docker run -i
    tty: true        # docker run -t
    networks:
      backend:
        aliases:
- fullstackexercise.frontend
- fullstackexercise.backend

  php-fpm:
    build:
      context: ./php-fpm
    environment:
      - DATABASE_ROOT_PASSWORD=${DATABASE_ROOT_PASSWORD}
    volumes:
- ../backend/config:/usr/share/fullstackExercise/config
- ../backend/public:/usr/share/fullstackExercise/public
- ../backend/src:/usr/share/fullstackExercise/src
- ../backend/tests:/usr/share/fullstackExercise/tests
      - ./php-fpm/root_home:/root
    stdin_open: true
    tty: true
    networks:
      - backend

For Nginx we are mapping-in two separate volumes into the html directory: as per above, one for the back-end site, one for the front-end site. These are then used as the webroots in the site configuration for each website. We are also setting an alias for each website. This is just so the other containers can access the websites too.

In the PHP block, we now have separate volumes for each of the code directories in the application route (note that the config sub-directory there is Symfony app config, not like the composer.json, phpunit.xml.dist etc stuff that has been copied to the application root by (spoilers) php-fpm/Dockerfile. And, yeah, now the Dockerfile (on Github) for the PHP stuff. The only significant line is this one:

COPY ./app_root/ /usr/share/fullstackExercise/

That copies all the config files the PHP components need to run into the application root. One downside of this is that I can't make on-the-fly changes to things like the PHPUnit config from within PHPStorm, I need to use vi in the container, test it, then copy it back to the host machine. But that stuff changes so seldom it's fine by me.

The changes on the front-end side of things is along the same lines. Here's the file structure (and on Github):

And the relevant bit of the node/Dockerfile (on Github):

WORKDIR  /usr/share/fullstackExercise/
COPY config/* ./

And docker-compose.yml (on Github):

  node:
    build:
      context: ./node
    environment:
      - GITHUB_PERSONAL_ACCESS_TOKEN=${GITHUB_PERSONAL_ACCESS_TOKEN}
    volumes:
- ../frontend/public:/usr/share/fullstackExercise/public
- ../frontend/src:/usr/share/fullstackExercise/src
- ../frontend/test:/usr/share/fullstackExercise/test
- ./node/root_home:/root
    stdin_open: true
    tty: true
    networks:
      backend:
        aliases:
          - vuejs.backend

Here we see how - same as with the PHP stuff - we copy the config files over in Dockerfile, and then map volumes for the code directories in docker-compose.yml.

That's pretty much it really. The good thing with all this is that because I have full test coverage of my code, and some functional and integration tests as well, I have testing for all the config and all the interactions between all the containers, so at any moment in time when I go to refactor something - because all this really is is an exercise in refactoring - at every step I can check that everything still works. Or spend time working out why something didn't work. But that safety net is always there.

OK. I promise the next article is actually gonna get around to looking at Vue.js components, testing thereof, and hopefully draw a line under this series. BTW if you read this article all the way to here, you're bloody weird. But well done ;-)

Righto.

--
Adam

G'day:

Working Code (@WorkingCodePod on Twitter) is a podcast by some friends and industry colleagues: Tim Cunningham, Carol Hamilton, Ben Nadel and Adam Tuttle.

(apologies for swiping your image without permission there, team)

It's an interesting take on a techo podcast, in their own words from their strapline:

Working Code is a technology podcast unlike all others. Instead of diving deep into specific technologies to learn them better, or focusing on soft-skills, this one is like hanging out together at the water cooler or in the hallway at a technical conference. Working Code celebrates the triumphs and fails of working as a developer, and aims to make your career in coding more enjoyable.

I think they achieve this, and it makes for a good listen.

So that's that, I just wanted to say they've done good work, and go listen.

…

Oh, just one more thing.

Yesterday they released their episode "Testing". I have to admit my reaction to a lot of what was said was… "poor", so I pinged my namesake and said "I have some feedback". After a brief discussion on Signal, Adam & I concluded that I might try to do a "reaction blog article" on the topic, and they might see if they can respond to the feedback, if warranted, at a later date. They are recording tonight apparently, and I'm gonna try to get this across to them for their morning coffee.

Firstly as a reminder: I'm pretty keen on testing, and I am also keen on TDD as a development practice. I've written a fair bit on unit testing and TDD both. I'm making a distinction between unit testing and TDD very deliberately. I'll come back to this. But anyway this is why I was very very interested to see what the team had to say about testing. Especially as I already knew one of them doesn't do automated testing (Ben), and another (Carol) I believe has only recently got into it (I think that's what she said, a coupla episodes ago). I did not know Adam or Tim's position on it.

And just before I get under way, I'll stick a coupla Twitter messages here I saw recently. At the time I saw them I was thinking about Ben's claim to a lack of testing, and they struck a particular chord with me.

I do not know Maaret or Mathias, but I think they're on the money here.

OK So I'm now listening to the podcast now. I'm going to pull quotes from it, and comment on them where I think it's "necessary".

Ahem.

Adam @ 11:27:

…up front we should acknowledge you know we're not testing experts. None of us [...] have been to like 'testing college'. […]There's a good chance we're going to get something wrong.

I think, in hindsight, this podcast needed this caveat to be made louder and clearer. To be blunt - and I don't think any would disagree with me here - all four are very far from being testing experts. Indeed one is even a testing naesayer. I think there's some dangerously ill-informed opinions being expressed as the podcast progresses, and as these are all people who are looked-up-to in their community, I think there's a risk people will take onboard what they say as "advice". Even if there's this caveat at the beginning. This might seem like a very picky thing to draw on, but perhaps I should have put it at then end of the article, after there's more context.

Carol @ 12:07:

Somebody find that monster already.

Hi guys.

Ben @ 12:41

I test nothing. And it's not like a philosophical approach to life, it's more just I'm not good at testing

Adam @ 13:09:

Clearly that's working out pretty well for you, you've got a good career going.

I hear this a bit. "I don't test and I get by just fine". This is pretty woolly thinking, and it's false logic people will jump on to justify why they don't do things. And Adam is just perpetuating the myth here. The problem with this rationalisation is demonstrated with an analogy of going on a journey without a map and just wandering aimlessly but you still (largely accidentally) arrive at your intended destination. In contrast had you used a map, you'd've been more efficient with your time and effort, and been able to progress even further on the next leg of your journey sooner. Ben's built a good career for himself. Undoubtedly. Who knows how much better it would be had he… used a map.

Also going back to Ben's comment about kind of explaining away why he doesn't test because he's not good at it. Everyone starts off not being good at testing mate. The rest of us do something about it. This is a disappointing attitude from someone as clued-up as Ben. Also if you don't know about something… don't talk about it mate. Inform yourself first and then talk about it.

Ben @ 13:21:

[…] some additional context. So - one - I work on a very small team. Two: all the people who work on my team are very very familiar with the software. Three: we will never ever hire a new engineer specifically for my team. Cos I work on the legacy codebase. The legacy codebase is in the process of being phased out. […] I am definitely in a context where I don't have to worry about hiring a new person and training the up on a system and then thinking they'll touch something in the code that they don't understand how it works. That's like the farthest possible thing from my day-to-day operations currently.

Um… so? I'm not being glib. You're still writing new logic, or altering existing logic. If you do that, it intrinsically needs testing. I mean you admitted you do manual testing, but it beggars belief that a person in the computer industry will favour manually performing a pre-defined repetitive task as a (prone-to-error) human, rather than automating this. We're in the business of automating repetitive tasks!

I'd also add that this would be a brilliant, low-risk, environment for you to get yourself up to speed with TDD and unit testing, and work towards the point where it's just (brain) muscle memory to work that way. And then you'll be all ready once you progress to more mission-critical / contemporary codebases.

Ben @ 14:42:

I can wrap my head around testing when it comes to testing a data workflow that is completely pure, meaning you have a function or you have a component that has functions and you give it some inputs and it generates some outputs. I can 100% wrap my head around testing that. And sometimes actually when I'm writing code that deals with something like that, even though I'm not writing tests per se, I might write a scratch file that instantiates that component and sends data to it and checks the output just during the development process that I don't have to load-up the whole application.

Ben. That's a unit test. You have written a unit test there. So why don't you put it in a test class instead of a scratch file, and - hey presto - you have a persistent test that will guard against that code's behaviour somehow changing to break those rules later on. You are doing the work here, you're just not doing it in a sensible fashion.

Ben @ 15:18:

Where it breaks down immediately for me is when I have to either a) involve a database, or b) involve a user interface. And I know that ther's all kinds of stuff that the industry has brought to cater to those problems. I've just never taken the time to learn.

There's a bit of a blur here between the previous train of thought which was definitely talking about unit tests of a unit of code, and now we're talking about end-to-end testing. These are two different things. I am not saying Ben doesn't realise this, but they're jammed up next to each other in the podcast so the distinction is not being made. These two kinds of testing are separate ideas, only really coupled by the fact they are both types of testing. Ben's right, the tooling is there, and - in my experience at least with the browser emulation stuff - it's pretty easy and almost fun to use. Ben already tests his stuff manually every time he does a release, so it would seem sensible to me to take the small amount of time it takes to get up to speed with these things, and then instead of testing something manually, take the time to automate the same testing, then it's taken care of thenceforth. It just a matter of being a bit more wise with one's time usage.

Adam @ 16:39

The reason that we don't have a whole lot of automated tests for our CFML code is simply performance. So when we started our product I tried really hard to do TDD. If I was writing a new module or a new section of that module I would work on tests along with that code, and would try to stay ahead of the game there. And what ended up happening was I had for me - let's say - 500 functions that could run, I had 400 tests. And I don't want to point a finger at any particular direction, but when you take the stack as a whole and you say "OK now run my test suite" and it takes ten minutes to run those tests and [my product, the project I was working on] is still in its infancy, and you can see this long road of so much more work that has to be done, and it takes ten minutes to run the tests - you know, early on - there was no way that that was going to be sustainable. So we kind of abandoned hope there. [...] I have, in more recent years, on a more recent stack seen way better performance of tests. [...] So we are starting to get more into automated testing and finding it actually really helpful. [...] I guess what I wanted to say there is that a perfectly valid reason to have fewer or no tests is if it doesn't work well on your platform.

Adam starts off well here, both in what he's saying and his historical efforts with tests, but he then goes on to pretty much blame ColdFusion for not being very good at running tests. This is just untrue, sorry mate. We had thousands upon thousands of tests running on ColdFusion, and they ran in an amount of time best measured in seconds. And when we ported that codebase to PHP, we had a similar number of test cases, and they ran in round about the same amount of time (PHP was faster, that said, but also the tests were better). I think the issue is here - and confirms this about 30sec after the quote above, and again about 15min later when he comes back to this - is that your tests weren't written so well, and they were not focused on the logic (as TDD-oriented tests ought to be), they were basically full integration tests. Full integration tests are excellent, but you don't want your tight red / green / refactor testing cycle to be slowed down by external services like databases. That's the wrong sort of testing there. My reaction to you saying your test runs are slow is not to say "ColdFusion's fault", it's to say "your tests' fault". And that's not a reason to not test. It's a reason to check what you've been doing, and fix it. I'm applying hindsight here for you obviously, but this ain't the conclusion/message you should be delivering here.

Carol @ 20:03:

I also want to say that if you are starting out and you're starting to add test, don't let slowness stop you from doing it.

Spot on. I hope Ben was listening there. When you start to learn something new, it is going to take more time. I think this is sometimes why people conclude that testing takes a lot of time: the people arriving at that conclusion are basing it on their time spent on the learning curve. Accept that things go slow when you are learning, but also accept that things will become second nature. And especially with writing automated tests it's not exactly rocket-science, the initial learning time is not that long. Just… decide to learn how to test stuff, start automating your testing, and stick at it.

Ben @ 21:24:

I was thinking about debugging incidents and getting a page in the middle of the night and having to jump on a call and you seeing the problem, and now you have to do a hotfix, and push a deployment in the middle of the night […]. And imagine having to sit there for 30 minutes for your tests to run just so you can push out a hotfix. Which I thought to myself: that would drive me crazy.

At this point I think Ben is just trying to invent excuses to justify to himself why he's right to eschew testing. I'm reminded of Maaret's Twitter message I included above. The subtext of Ben's here is that if one tests manually, then you're more flexible in what you can choose to re-test when you are hotfixing. Well obviously if you can make that call re manual tests, then you can make the same call with automated tests! So his position here is just specious. Doubly so because automated tests are intrinsically going to be faster than the equivalent manual tests to start with. Another thing I'll note that with this entire analogy: you've already got yourself in a shit situation by needing to hotfix stuff in the middle of the night. Are you really sure you want to be less diligent in how you implement that fix? In my experience that approach can lead to a second / third / fourth hotfix being needed in rapid succession. Hotfix situations are definitely ones of "work smarter, not faster".

Ben @ 21:56:

I'm wondering if there should be a test budget that you can have for your team where you like have "here is the largest amount of time we're willing to let testing block a deployment". And anything above that have to be tests that sit in an optional bucket where it's up to the developer to run them as they see fit, but isn't necessarily tests that would block deployment. I don't know if that's totally crazy.

Adam continues @ 22:36:

You have to figure out which tests are critical path, which ones are "must pass", and these ones are like "low risk areas" […] are the things I would look for to make optional.

Yep, OK there's some sense here, but I can't help thinking that we are talking about testing in this podcast, and we're spending our time inventing situations in which we're not gonna test. It all seems a bit inverted to me. How about instead you just do your testing and then if/when a situation arises you then deal with it. Instead of deciding there will be situations and justify to yourselves why you oughtn't test in the first place.

But I also have to wonder: why the perceived rush here? What's wrong with putting over 30min to test stuff if it "proves" thaty your work has maintained stability, and you'll be less likely to need that midnight hotfix. What percentage of the whole cycle time of feature request to delivery is that 30 minutes? Especially if taking the effort to write the tests in the first place will inately improve the stability of your code in the first place, and then help to keep it stable? It's a false economy.

Tim @ 23:15:

When we have contractors do work for us. I require unit tests. I require so much testing just because it's a way for me to validate the truth of what they're saying they've done. So that everything that we have that's done by third parties is very well tested, and it's fantastic because I have a high level of confidence.

Well: precisely. Why do you not want that same level of confidence in your in-house work? Like you say: confidence is fantastic. Be fantastic, Tim. Also: any leader should eat their own dogfood I think. If there's sense in you making the contractors work like this, clearly you ought to be working that way yourself.

Tim @ 22:36:

Any time I start a new project, if I have a greenfields project, I always start with some level of unit tests, and then I get so involved in the actual architecture of the system that I put it off, and like "well I don't really need a test for this", "I'm not really sure where I'm going with this, so I'm not going to write a test first" because I'm kinda experimenting. Then my experiment becomes reality, then my reality becomes the released version. And then it's like "well what's the point of writing a test now?"

I think we've all been there. I think what Tim needs here is just a bit more self-discipline in identifying what is "architectural spike" and what's "now doing the work". If one is doing TDD, then the spike can be used to identify the test cases (eg "it's going to need to capture their phone number") without necessarily writing the test to prove the phone number has been captured. So you write this:

describe("my new thing", function () {
    it ("needs to capture the phone number", function () {
        // @todo need to test this
    });
});

And then when you detect you are not spiking any more, you write the test, and then introduce the code to make the test pass. I also think Tim is overlooking that the tests are not simply there for that first iteration, they are then there proving that code is stable for the rest of the life of the code. This… builds confidence.

Adam @ 24:22:

That's what testing is all about, right? It's increasing confidence that you can deploy this code and nothing is going to be wrong with it. […] When I think about testing, the pinnacle of testing for me is 100% confidence that I can deploy on my way out the door at 4:55pm on Friday afternoon, with [a high degree of ~] confidence that I am not going to get paged on Saturday at 4am because some of that code that I just deployed… it went "wrong".

Exactly.

Carol @ 25:12:

What difference between the team I'm on and the team you guys have is we have I think it's 15-ish people touching the exact same code daily. So a patch I can put out today may have not even been in the codebase they pulled yesterday when they started working on a bug, or a week ago when they had theirs. So me writing that little extra bit of test gives them some accountability for what I've done, and me some.

Again: exactly.

Ben @ 26:36:

Even if you have a huge test suite, I can't help but think you have to do the manual testing, because what if something critical was missed. [...] I think the exhaustive test suite, what that does is it catches unexpected bugs unrelated. Or things that broke because you didn't expect them to break in a certain way. And I think that's very important.

To Ben's first point, you could just as easily (and arguably more validly) switch that around: a human, doing ad-hoc manual testing is more likely to miss something, because every manual test run is at their whim and subject to their focus and attention at the time. Whereas the automated tests - which let's not forget were written by a diligent human, but right at the time they are most focused on the requirements - are run by the computer and it will do exactly the same job every time. What having the historical corpus of automated tests give you is increased confidence that all that stuff being tested still works the way it is supposed to, so the manual testing - which is always necessary, can be more a case of dotting the Is and crossing the Ts. With no automated tests, the manual tests need to be exhaustive. And the effort needs to be repeated every release (Adam mentions this a few minutes later as well).

To the second point: yeah precisely. Automated tests will pick up regressions. And the effort to do this only needs to be done once (writing the test). Without automated tests, you rely on the manual testing to pick this stuff up, but - being realistic - if your release is focused on PartX of the code, your manual tests are going to focus there, and possibly not bother to re-test PartZ which has just inadvertantly been broken by PartX's work.

Ben also mentions this quote from Rich Hickey "Q: What happened to every bug out there? A: it passed the type checker, and it passed all tests." (I found this reference on Google: Simple Made Easy, it's at about 15:45). It's a nifty quote, but what it's also saying is that there wasn't actually a test for the buggy behaviour. Because if there was one: the test would have caught it. The same could be said more readily of manual-only testing. Obviously nothing is going to be 100%, but automated tests are going to be more reliable at maintaining the same confidence level, and be less effort, than manual-only testing.

Ben @ 28:15:

When people say it increases the velocity of development over time. I have trouble embracing that.

(Ben's also alluding back to a comment he made immediately prior to that, relating to always needing to manually test anyhow). "Over time" is one of the keys here. Once a test is written once, it's there. It sticks around. Every subsequent test round there is no extra effort to test that element of the application (Tim draws attention to this a coupla minutes later too). With manual testing the effort needs to be duplicated every time you test. Surely this is not complicated to understand. Ben's point about "you still need to manually test" misses the point that if there's a foundation of automated tests, your manual testing can become far more perfunctory. Without the tests: the manual testing is monolithic. Every. Single. Time. To be honest though, I don't know why I need to point this out. It's a) obvious; and b) very well-trod ground. There's an entire industry that thinks automated tests are the foundation of testing. And then there's Ben who's "just not sure". This is like someone "just not sure" that the world isn't actually flat. It's no small amount of hubris on his part, if I'm honest. And obviously Ben is not the only person out there in the same situation. But he's the one here on this podcast supposedly discussing testing.

Ben @ 37:08:

One thing that I've never connected with: when I hear people talk about testing, there's this idea of being able to - I think they call them spies? - create these spies where you can see if private methods get called in certain ways. And I always think to myself: "why do you care about your private methods?" That's an implementation detail. That private method may not exist next week. Just care about what your public methods are returning and that should inherently test your private methods. And people have tried to explain it to me why actually sometimes you wanna know, but I've ust never understood it.

Yes good point. I can try to explain. I think there's some nuance missing in your understanding of what's going on, and what we're testing. It starts with your position that testing is only concerning itself with (my wording, paraphrasing you from earlier) "you're interested in what values a public method takes, and what it returns". Not quite. You care about given inputs to a unit, whether the behaviour within the unit correctly provides the expected outputs from the unit. The outputs might not be the return value. Think about a unit that takes a username and password, hashes the password, and saves it to the DB. We then return the new ID of the record. Now… we're less interested in the ID returned by the method, we are concerned that the hashing takes place correctly. There is an output boundary of this unit at the database interface. We don't want our tests to actually hit the database (too slow, as Adam found out), but we mock-out the DB connector or the DAO method being called that takes the value that the model layer has hashed. When then spy on the values passed to the DB boundary, and make sure it's worked OK. Something like this:

describe("my new thing", function () {
    it ("hashes the password", function () {
        testPassword = "letmein"
        expectedHash = "whatevs"

        myDAO = new Mock(DAO)
        myDAO.insertRecord.should.be.passed(anything(), expectedHash)

        myService = new Service(myDAO)

        newId = myService.addUser("LOGIN_ID_NOT_TESTED", testPassword)

        newId.should.be.integer() // not really that useful
    });
});

class Service {

    private dao

    Service(dao) {
        this.dao = dao
    }

    addUser(loginId, password) {
        hashedPassword = excellentHashingFunction(password)

        return this.dao.insertRecord(loginId, hashedPassword)
    }
}

class DAO {
    insertRecord(loginId, password) {
        return db.insertQuery("INSERT INTO users (loginId, password) VALUES (:loginId, :password)", [loginId, password])
    }
}

OK so insertRecord isn't a private method here, but the DAO is just an abstraction from the public interface of the unit anyhow, so it amounts to the same thing, and it makes my example clearer. insertRecord could be a private method of Service.

So the thing is that you are checking boundaries, not specifically method inputs/outputs.

Also, yes, the implementation of DAO might change tomorrow. But if we're doing TDD - and we should be - the tests will be updated at the same time. More often than not though, the implementation isn't as temporary as this line of thought often assumes (for the convenience of the argument, I suspect).

Adam @ 48:41:

The more that I learn how to test well, and the more that I write good tests, the more I become a believer in automated testing (Carol: Amen). […] The more I do it the better I get. And the better I get the more I appreciate what I can get from it.

Indeed.

Tim @ 49:32:

In a business I think that short term testing is a sunk cost maybe, but long term I have seen the benefit of it. Particularly whenever you are adding stuff to a mature system, those tests pay dividends later. They don't pay dividends now […] (well they don't pay as many dividends now) […] but they do pay dividends in the long run.

Also a good quote / mindset. Testing is about the subsequent rounds of development as much as the current one.

Ben @ 50:05:

One thing I've never connected with emotionally, when I hear people talk about testing, is when they refer to tests as providing documentation about how a feature is supposed to work. And as someone who has tried to look at tests to understand why something's not working, I have found that they provide no insight into how the feature is supposed to work. Or I guess I should say specifically they don't provide answers to the question that I have.

Different docs. They don't provide developer docs, but if following BDD practices, they can indicate the expected behaviour of the piece of functionality. Here's the test run from some tests I wrote recently:

When I showed this to the person I was doing the work for, he immediately said "no, that test case is wrong, you have it around the wrong way", and they were right, and I fixed it. That's the documentation "they" are talking about.

Oh, and Carol goes on to confirm this very thing one minute later.

Also bear in mind that just cos a test could be written in such a way as to impart good clear information doesn't mean that all tests do. My experience with looking at open-source project's tests to get any clarity on things (and I include testing frameworks' own tests in this!), I am left knowing less than I did before I looked. It's almost like there's a rule in OSS projects that the code needs to be shite or they won't accept it ;-)

And that's it. It was an interesting podcast, but I really really strongly disagreed with most of what Ben said, and why he said it. It would be done thing if he was held to account (and the others tried this at times), but as it is other than joking that Ben is a nae-sayer, I think there's some dangerous content in here.

Oh, one last thing… in the outro the team suggests some resources for testing. Most of what the suggested seems to be "what to do", not "why you do it". I think the first thing one should do when considering testing is to read Test Driven Development by Kent Beck. Start with that. Oh this reminds me… not actually much discussion on TDD in this episode. TDD is tangential to testing per se, but it's an important topic. Maybe they can do another episode focusing on that.

Righto.

--
Adam

G'day:

Hey I just spotted that my mate Alex Skinner - head honcho of Pixl8, pillar of the CFML/Lucee community in London, and bloody good bloke - is doing a charity drive at the moment ("Crowdfunding to refurbish donor or purchased 2nd hand laptops") to get refurbished laptops into the hands of kids / families who perhaps haven't had all the life opportunities that you and I might have had. This is an especially critical consideration at the moment in the UK where kids can't in general attend school, and need to do all their learning online.

So what Alex is doing is this:

We're raising £5,000 to refurbish donor or purchased 2nd hand laptops. Aim to get 200+ devices to families who have limited or no digital access.

Brilliant. If yer in the UK and reading this: go give them some money if you can spare it. If yer outside the UK, either give them some money anyhow (kids are kids after all: doesn't matter where they live), or perhaps seek out a similar good cause in yer own patch.

To make the link very clear, it's this:

https://www.justgiving.com/crowdfunding/digitalvillage

Please also circulate this to yer other colleagues / friends / etc around the place if you can. Nice one.

Righto.

-- 
Adam

G'day:

OK so maybe this will be the last article in this series. But given what a monster it's become: who knows. As a recap, here are links to the earlier articles:

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers
5. MariaDB
6. Installing Symfony
7. Using Symfony
8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
9. I mess up how I configure my Docker containers
10. An article about moving files and changing configuration
11. Setting up a Vue.js project and integrating some existing code into it (this article)

It's up to you whether you read the rest of that lot, or just skim it, or whatever. Looking at the source code might help. This is where it's at as I am writing this sentence: Fullstack Exercise v 2.10. That has already got some of the code in it that is "new" to this article. Any other code links in here I'll link to the final version of the work. I think so far I've done the Vue.js project install, and reconfigured it and Nginx so that I'm using Nginx as the front-end web server, not the one that ships with the Vue.js project. Other than that the Vue code is just the "hello world" stuff the project starts with.

OK so the object of this exercise is to take my gdayWorldViaVue.html page which is is built with its Vue template embedded in just a JS file. I discuss the creation of this work in the article "Part 8: Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer". Here's the code:

frontend/public/gdayWorldViaVue.html:

<!doctype html>

<html lang="en">
<head>
<meta charset="utf-8">

<title id="title">{{ message }}</title>
</head>

<body>
<div id="app">
<greeting :message="message"></greeting>
</div>

<script src="https://cdn.jsdelivr.net/npm/vue@2/dist/vue.js"></script>
<script src="assets/scripts/gdayWorldViaVue.js"></script>
</body>
</html>

And frontend/public/assets/scripts/gdayWorldViaVue.js:

Vue.component('greeting', {
    props : {
        message : {
            type: String,
            required: true
        }
    },
    template : `
<div>
<h1>{{ message }}</h1>
<p>{{ message }}</p>
</div>
    `
});


let appData = {message: "G'day world via Vue"};
new Vue({el: '#title', data: appData});
new Vue({el: '#app', data: appData});

Oh, an for the sake of completeness, the output:

Ah, and of course there's a test (frontend/test/functional/GdayWorldViaVueTest.js):

let puppeteer = require('puppeteer');

let chai = require("chai");
let chaiAsPromised = require("chai-as-promised");
chai.use(chaiAsPromised);
let should = chai.should();

describe("Baseline test of vue.js working", function () {
    let browser;
    let page;

    const expectedText = "G'day world via Vue";

    before("Load the test document", async function () {
        this.timeout(5000);

        browser = await puppeteer.launch( {args: ["--no-sandbox"]});
        page = await browser.newPage();

        await page.goto("http://fullstackexercise.frontend/gdayWorldViaVue.html");
    });

    after("Close down the browser", async function () {
        await page.close();
        await browser.close();
    });

    it("should return the correct page title", async function () {
        await page.title().should.eventually.equal(expectedText);
    });

    it("should return the correct page heading", async function () {
        let headingText = await page.$eval("h1", headingElement => headingElement.innerText);

        headingText.should.equal(expectedText);
    });

    it("should return the correct page content", async function () {
        let paragraphContent = await page.$eval("p", paragraphElement => paragraphElement.innerText);

        paragraphContent.should.equal(expectedText);
    });
});

Output:

Having those test cases there are gold, because it means all this work is basically a refactoring exercise (we've done the red / green bit in the earlier article), so the object of this exercise is to separate-out the template from the JS file and into a .vue file, and know the work is done because those test cases still pass.

Right, so I have hit the Vue.JS website, and done some reading, and to use .vue files, I need to create a Vue project. First I just want to see what the project-creation does, to check if it'll be easier to integrate my work into the project, or the project into the work: you might remember a similar drama when I was installing Symphony ("Part 6: Installing Symfony"). Once-bitten, twice-shy, I'm just gonna see what Vue thinks it's doing first. I'm gonna create this project in /tmp.

Oh Adam… back-up! First of all I need to install Vue CLI. This is the app that deals with project creation and stuff like that. This is easy, I just integrate it into the node/Dockerfile:

FROM node
RUN apt-get update \
&& apt-get install -y wget gnupg \
&& wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' \
&& apt-get update \
&& apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf libxss1 \
      --no-install-recommends \
&& rm -rf /var/lib/apt/lists/*
WORKDIR  /usr/share/fullstackExercise/
COPY config/* ./
RUN npm install -g @vue/cli
RUN npm install
EXPOSE 8080

Just to be clear, the first npm install there is installing Vue CLI globally, the second one is installing the modules necessary for my app. Incidentally, it was getting this stuff working that was the inspiration for two of my recent articles: "Part 9: I mess up how I configure my Docker containers" and "Part 10: An article about moving files and changing configuration". But it's all working now.

Now I've got Vue CLI installed, I'll install a project:

Lovely. Let's do this run-serve thing:

This won't work because I need to poke a hole through from the container to the host machine on 8081, but I'll quickly do this (I'll spare you the detail). Gimme a minute.

Cool. My first challenge is that I've already got a web server, and I want to use it. I don't want to be using Vue's stub web server. Now for dev there's no real need for this: Vue's web server would be fine, but I've got Nginx there, so I'm going to use it. Also I don't want to use localhost to access it. I figured I needed to configure a proxy_pass "thing" (sorry for technical buzzword there), but I had no idea how to do it. I'm a noob with Nginx. Anyhow, I googled about and found this article "VueJS dev serve with reverse proxy" by someone called Marko Mitranić . I didn't understand all the settings it was suggesting, but I grabbed them and it all works (docker/nginx/sites/frontend.conf):

server {
    listen 80;
    listen [::]:80;

    server_name fullstackexercise.frontend;
    root /usr/share/nginx/html/frontend;
    index index.html;

    location / {
        #autoindex on;
        #try_files $uri $uri/ =404;

# from https://medium.com/homullus/vuejs-dev-serve-with-reverse-proxy-cdc3c9756aeb
proxy_pass  http://vuejs.backend:8080/;
proxy_set_header Host vuejs.backend;
proxy_set_header Origin vuejs.backend;
proxy_hide_header Access-Control-Allow-Origin;
add_header Access-Control-Allow-Origin "http://fullstackexercise.frontend";
    }

# from https://medium.com/homullus/vuejs-dev-serve-with-reverse-proxy-cdc3c9756aeb
location /sockjs-node/ {
proxy_pass http://vuejs.backend:8080;
proxy_redirect off;
proxy_set_header Host vuejs.backend;
proxy_set_header Origin vuejs.backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_hide_header Access-Control-Allow-Origin;
add_header Access-Control-Allow-Origin "http://fullstackexercise.frontend";
}

    location ~ /\.ht {
        deny all;
    }
}

The first block is for the website (on http://fullstackexercise.frontend); the second block is for the listener for changes in the source code. Vue does some magic with web sockets and ping up to the browser to reload any code changes that are back on the server. This is quite cool. Oh I also needed to put some settings in docker/node/config/vue.config.js:

module.exports = {
    devServer: {
        host: "vuejs.backend",
        disableHostCheck: false,
        port: 8080,
watchOptions : {
ignored: /node_modules/,
poll: 1000
}
    }
}

But I'm getting ahead of myself here. The Nginx stuff is for the fullstackExercise websites. I'm still with this one in /tmp/ directory. All I want to check is what it installs, which is this lot:

OK so it seems to be just a bunch of NPM libs and stuff in package.json:

All of which will merge into my existing package.json just fine. And some source code files and assets. I took a punt and just integrated all of that into my app, stuck CMD ["npm", "run", "serve"] in my docker/node/Dockerfile, and rebuilt my containers…

Cool. Plus I re-ran all my own tests on the other code in the app, and everything is still currently green. Also I can change my source code on the server, and almost instantly it's displayed in the browser without a reload:

Right. So now I have to try to integrate this gdayWorldViaVue.html page into my Vue app. It took a while to work out how to do this: I'm sure it's in the Vue.js docs somewhere, but I couldn't find it. Ultimately I found a Q&A on Stack Overflow that explained it: "multiple pages in Vue.js CLI" (and a handy Github repo with example code in it). Basically Vue CLI assumes everyone wants a single-page app, and doesn't really expose how to add additional pages into this SPA. But it's just a matter of defining the pages in vue.config.js (and I have the doc reference now: Configuration Reference > Pages. First I just shifted the project's own index page into a page definition:

module.exports = {
pages : {
index: {
entry: "src/index/main.js",
template: "public/index.html",
filename: "index.html"
}
},
    devServer: {
        host: "vuejs.backend",
        disableHostCheck: false,
        port: 8080,
        watchOptions : {
            ignored: /node_modules/,
            poll: 1000
        }
    }
}

This also necessitated moving main.js, components/HelloWorld.vue and App.vue from the base of the src/ directory structure, into the src/index/ subdirectory, as well as changing a relative reference to ./assets/logo.png to ../assets/logo.png, given it's being referenced from a file in that index/ subdirectory now. I rebuilt the container, and restarted it, and the Hello World page still worked. Now to add the gdayWorldViaVue.html page into it.

This was pretty easy, and was just a matter of moving some stuff around in the file system and within files. Previously we had a single HTML file, gdayWorldViaVue.html, and a single JS file, gdayWorldViaVue.js, as per above in this article. First I implemented the page mapping for this page in vue.config.js, as per the index example above:

        gdayWorld: {
            entry: "src/gdayWorld/main.js",
            template: "public/gdayWorldViaVue.html",
            filename: "gdayWorldViaVue.html"
        }

And then I need to distribute the contents of the existing .html and .js file to their more Vue / page / component-centric equivalents. src/gdayWorld/main.js is identical to the one for the index page:

import { createApp } from 'vue'
import App from './App.vue'

createApp(App).mount('#app')

frontend/public/gdayWorldViaVue.html has had the reference to the JS files in the foot of the <body> removed as the application now handles those, it also has the body of the #app <div/> removed as this is handled by the App.vue file now, and also has placeholder text put in for the page title:

<!doctype html>

<html lang="en">
<head>
<meta charset="utf-8">

<title id="title">PAGE-TITLE</title>
</head>

<body>
<div id="app"></div>
</body>
</html>

I was handling the page title in quite an unorthodox fashion before, and I'm going to fix this now.

frontend/src/gdayWorld/App.vue defines what goes in the #app <div/>, and loads in the GreetingMessage component:

<template>
<greeting-message :message="message"></greeting-message>
</template>

<script>
import GreetingMessage from './components/GreetingMessage.vue'
export default {
    name: 'App',
    components: {
        GreetingMessage
    },
    data () {
        return {
            message: "G'day world via Vue"
        };
    },
    created () {
        // document.title = this.message;
    }
}
</script>

Note it will also sets the document.title value now, from the app data. It's commented-out for now because I want to see the test for this fail initially, so I am sure what we're seeing is the refactored implementation.

And frontend/src/gdayWorld/components/GreetingMessage.vue defines the behaviour of itself. So this file is a stand-alone component now, which is what we were aiming to do all along.

<template>
<div>
<h1>{{ message }}</h1>
<p>{{ message }}</p>
</div>
</template>

<script>
export default {
  name: 'GreetingMessage',
  props : {
    message : {
      type: String,
      required: true
    }
  }
}
</script>

I also deleted frontend/public/assets/scripts/gdayWorldViaVue.js as it's now surplus to requirement. Because we've changed the Vue app's config, and that is copied over in Dockerfile, I need to rebuild the containers to see this "change". Once that's done I re-run my tests:

Woohoo! Two test are passing, and the expected one is failing. If I now uncomment that line above (and as soon as I do that my page in the browser now gets its title too btw), the test is now passing:

Similarly if I go and change the message in frontend/src/gdayWorld/App.vue the tests fail appropriately, so I'm happy the tests are testing the new implementation.

Now I just need to work out how to implement a test of just the GreetingMessage.vue component discretely, as opposed to the way I'm doing it now: curling a page it's within and checking the page's content. This is fine for an end-to-end test, but not so good for a unit test. TBH in this simple case the current approach is actually fine, but I want to know how to test components.

Excuse me whilst I do some reading.

…

[Adam runs npm install of many combinations of NPM libs]

…

[Adam downgrades his version of Vue.js and does the npm install crap some more]

…

OK screw that. Seems to me - on initial examination - that getting all the libs together to make stand-alone components testable is going to take me longer to work out than I have patience for. I'll do it later. So. I'm gonna leave this article here at the "and lo, I have .vue-file-based Vue components working now, and kinda understand how that side of things comes together, but - nor now at least - I'm gonna do my front-end testing via HTTP requests of the whole document, not via each component. For what I need to do right now this is fine anyhow, I think. Sigh.

Righto.

--
Adam

G'day

OKOK, another article in that bloody series I've been doing. Same caveats as usual: go have a breeze through the earlier articles if you feel so inclined. That said this is reasonably stand-alone.

1. Intro / Nginx
2. PHP
3. PHPUnit
4. Tweaks I made to my Bash environment in my Docker containers
5. MariaDB
6. Installing Symfony
7. Using Symfony
8. Testing a simple web page built with Vue.js using Mocha, Chai and Puppeteer
9. I mess up how I configure my Docker containers
10. An article about moving files and changing configuration
11. Setting up a Vue.js project and integrating some existing code into it
12. Unit testing Vue.js components (this article)

This is really very frustrating. You might recall I ended my previous article with this:

Now I just need to work out how to implement a test of just the GreetingMessage.vue component discretely, as opposed to the way I'm doing it now: curling a page it's within and checking the page's content. […]

Excuse me whilst I do some reading.

…

[Adam runs npm install of many combinations of NPM libs]

…

[Adam downgrades his version of Vue.js and does the npm install crap some more]

…

OK screw that. Seems to me - on initial examination - that getting all the libs together to make stand-alone components testable is going to take me longer to work out than I have patience for. I'll do it later. […] Sigh.

I have returned to this situation, and have got everything working fine. Along the way I had an issue with webpack that I eventually worked around, but once I circled back to replicate the work and write this article, I could no longer replicate the issue. Even rolling back to the previous version of the application code and step-by-step repeating the steps to get to where the problem was. This is very frustrating. However other people have had similar issues in the past so I'm going to include the steps to solve the problem here, even if I have to fake the issue to get error messages, etc.

Right so I'm back with the Vue.js docs regarding testing: "Vue.JS > Testing > Component Testing". The docs are incredibly content-lite, and pretty much just fob you off onto other people to work out what to do. Not that cool, and kinda suggests Vue.js considers the notion of unit testing pretty superficially. I did glean I needed to install a coupla Node modules.

First, @testing-library/vue, for which I ran into a glitch immediately:

The current version of @testing-library/vue doesn't work with the current version of Vue.js. Neato. Nice one Vue team. After some googling of "wtf?", I landed on an issue someone else had raised already: "Support for Vue 3 #176": I need to use the next branch (npm install --save-dev @testing-library/vue@next). This worked OK.

The other module I needed was @vue/cli-plugin-unit-mocha. That installed with no problem. This all gives me the ability to run vue-cli-service test:unit, which will run call up Mocha and run some tests. The guidance is to set this up in package.json, thus:

"scripts": {
"test": "mocha test/**/*Test.js",
"serve": "vue-cli-service serve --watch",
"build": "vue-cli-service build",
"lint": "vue-cli-service lint",
"test:unit": "vue-cli-service test:unit test/unit/**/*.spec.js"
  },

Then one can jus run it as npm run test:unit.

I looked at the docs for how to test a component ("Unit Testing Vue Components" - for Vue v2, but there's no equivalent page in the v3 docs), and knocked together an initial first test which would just load the component and do nothing else: just to check everything was running (frontend/test/unit/GreetingMessage.spec.js (final version) on Github):

import GreetingMessage from "../../src/gdayWorld/components/GreetingMessage";

import { shallowMount } from "@vue/test-utils";

import {expect} from "chai";

describe("Tests of GreetingMessage component", () => {
    it("should successfully load the component", () => {
let greetingMessage = shallowMount(GreetingMessage, {propsData: {message: "G'day world"}});

expect(true).to.be.true;
    });
});

Here's where I got to the problem I now can't replicate. When I ran this, I got something like:

There's a JS file that has a shell-script shebang thing at the start of it, and the Babel transpiler doesn't like that. Fair enough, but I really don't understand why it's trying to transpile stuff in the node_modules directory, but at this point in time, I just thought "Hey-ho, it knows what it's doing so I'll take its word for it".

Googling about I found a bunch of other people having a similar issue with Webpack compilation, and the solution seemed to be to use a shebang-loader in the compilation process (see "How to keep my shebang in place using webpack?", "How to Configure Webpack with Shebang Loader to Ignore Hashbang…", "Webpack report an error about Unexpected character '#'"). All the guidance for this solution was oriented aroud sticking stuff in the webpack.config.js file, but of course Vue.js hides that away from you, and you need to do things in a special Vue.js way, but adding stuff with a special syntax to the vue.config.js file. The docs for this are at "Vue.js > Working with Webpack". The docs there showed how to do it using chainWebpack, but I never actually got this approach to actually solve the problem, so I mention this only because it's "something I tried".

From there I starting thinking, "no, seriously why is it trying to transpile stuff in the node_modules directory?" This does not seem right. I changed my googling tactic to try to find out what was going on there, and came across "Webpack not excluding node_modules", and that let me to update my vue.config.js file to actively exclude node_modules (copied from that answer):

var nodeExternals = require('webpack-node-externals');
...
module.exports = {
    ...
    target: 'node', // in order to ignore built-in modules like path, fs, etc. 
    externals: [nodeExternals()], // in order to ignore all modules in node_modules folder 
    ...
};

And that worked. Now when I ran the test I have made progress:

From there I rounded out the tests properly (frontend/test/unit/GreetingMessage.spec.js):

import GreetingMessage from "../../src/gdayWorld/components/GreetingMessage";

import { shallowMount } from "@vue/test-utils";

import {expect} from "chai";

describe("Tests of GreetingMessage component", () => {

    let greetingMessage;
    let expectedText = "TEST_MESSAGE";

    before("Load test GreetingMessage component", () => {
        greetingMessage = shallowMount(GreetingMessage, {propsData: {message: expectedText}});
    });

    it("should return the correct heading", () => {
        let heading = greetingMessage.find("h1");
        expect(heading.exists()).to.be.true;

        let headingText = heading.text();
        expect(headingText).to.equal(expectedText);
    });

    it("should return the correct content", () => {
        let contentParagraph = greetingMessage.find("h1+p");
        expect(contentParagraph.exists()).to.be.true;

        let contentParagraphText = contentParagraph.text();
        expect(contentParagraphText).to.equal(expectedText);
    });
});

Oh! Just a reminder of what the component is (frontend/src/gdayWorld/components/GreetingMessage.vue)! Very simple stuff, as the tests indicate:

<template>
<h1>{{ message }}</h1>
<p>{{ message }}</p>
</template>

<script>
export default {
  name: 'GreetingMessage',
  props : {
    message : {
      type: String,
      required: true
    }
  }
}
</script>

One thing I found was that every time I touched the test file, I was getting this compilation error:

But if I ran it again, the problem went away. Somehow ESLint was getting confused by things; it only lints things when they've changed, and on the second - and subsequent - runs it doesn't run so the problem doesn't occur. More googling, and I found this: "JavaScript Standard Style does not recognize Mocha". The guidance here is to let the linter know I'm running Mocha, with the inference that there will be some global functions it can just assume are legit. This is just an entry in package.json:

"eslintConfig": {
"root": true,
"env": {
"node": true,
"mocha": true
    },
    // etc

Having done that, everything works perfectly, and despite the fact that is a very very simple unit test… I'm quite pleased with myself that I got it running OK.

After sorting it all out, I reverted everything in source control back to how it was before I started the exercise, so as to replicate it and write it up here. This is when I was completely unable to reproduce that shebang issue at all. I cannot - for the life of me - work out why not. Weird. Anyway, I decided to not waste too much time trying to reproduce a bug I had solved, and just decided to mention it here as a possible "thing" that could happen, but otherwise move on with my life.

I have now got to where I wanted to get at the beginning of this series of articles, so I am gonna stop now. My next task with Vue.js testing is to work out how to mock things, because my next task will require me to make remote calls and/or hit a DB. And I don't want to be doing that when I'm running tests. But that was never an aim of this article series, so I'll revert to stand-alone articles now.

Righto.

--
Adam

G'day:

I'm ramping-up to do some more development with this fullstackExercise codebase I've been working on (see "Creating a web site with Vue.js, Nginx, Symfony on PHP8 & MariaDB running in Docker containers", parts 1-12), and as part of the yak-shaving phase of things, I decided to port my PHP tests from PHPUnit to Kahlan. The reason for this is twofold. Firstly, I much prefer the describe / it syntax to writing and running tests over xUnit; and secondly - other than perpetually finding new ways to be frustrated by how PHPUnit handles/documents things - I am not learning a great deal about PHPUnit any more. So Kahlan fits in well with this learning exercise.

Porting the tests is easy, largely cos my tests are currently minimal in number and in complexity. But this PHPUnit class (test/functional/ExampleComTest.php):

namespace adamCameron\kahlanIssue\test\functional;

use GuzzleHttp\Client;
use PHPUnit\Framework\TestCase;
use Symfony\Component\HttpFoundation\Response;

class ExampleComTest extends TestCase
{
    /** @coversNothing */
    public function testExampleDotComReturnsExpectedContent()
    {
        $expectedContent = "Example Domain";


        $client = new Client([
            'base_uri' => 'http://example.com/'
        ]);

        $response = $client->get('index.html');

        $this->assertEquals(Response::HTTP_OK, $response->getStatusCode());

        $html = $response->getBody();
        $document = new \DOMDocument();
        $document->loadHTML($html);

        $xpathDocument = new \DOMXPath($document);

        $hasTitle = $xpathDocument->query('/html/head/title[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasTitle);

        $hasHeading = $xpathDocument->query('/html/body/div/h1[text() = "' . $expectedContent . '"]');
        $this->assertCount(1, $hasHeading);
    }
}

Becomes this Kahlan spec(spec/functional/example.com.spec.php):

namespace adamCameron\fullStackExercise\spec\functional;

use GuzzleHttp\Client;
use Symfony\Component\HttpFoundation\Response;

describe('Tests that example.com can be curled', function () {

    beforeAll(function () {
        $client = new Client([
            'base_uri' => 'http://example.com/'
        ]);

        $this->response = $client->get('index.html');

        $html = $this->response->getBody();
        $document = new \DOMDocument();
        $document->loadHTML($html);
        $this->xpathDocument = new \DOMXPath($document);
    });

    it("should have expected status code", function () {
        expect($this->response->getStatusCode())->toBe(Response::HTTP_OK);
    });

    it("should have expected title", function () {
        $hasTitle = $this->xpathDocument->query('/html/head/title[text() = "Example Domain"]');
        expect($hasTitle)->toHaveLength(1);
    });

    it("should have expected heading", function () {
        $hasHeading = $this->xpathDocument->query('/html/body/div/h1[text() = "Example Domain"]');
        expect($hasHeading)->toHaveLength(1);
    });
});

The problems for me started when I came to run that test in Kahlan. I got this:

Eek, OK that's a compilation error, not a runtime exception, which is odd. I checked the code of CurlMultiHandler.php to see if anything was broken in there, and unsurprisingly it wasn't. I decided to chuck a try / catch around the code in question, and see if the actual issue was something other than being reported. I found it odd that Kahlan was reporting it as an Exception, when do me a code-parsing issue is an out-and-out Error. Two different things. So I wondered if Kahlan was doing something odd. I ran the code again and the error spewed out again. What the hell? Then I looked more closely. The error had changed slightly. Now it says this:

Note before it was complaining about CurlMultiHandler.php. Now it's StreamHandler.php. OK. I'll play yer silly game. I stuck a try / catch around that code, and re-ran. Now it errors with this:

Grrr. OK, one last time with the try / catch:

Um. OK. Why's it working? Next I backed-out all my debug code and ran it again. And the tests worked still. Next I blew-away the entire vendor directory and re-did composer install, and the tests continued to work.

It's worth noting here that I know those are all Guzzle files, but If I run the equivalent code in PHPUnit, there is never any issue, hence me doubting it's anything to do with Guzzle specifically, it's how Kahlan is calling Guzzle. Probably with vendor/kahlan/kahlan/src/Jit/ClassLoader.php, which is the preceding file in the callstack, each time. I have to admit I did not look into that file to see what it is doing. I'll crack on with that shortly.

At this point I was thinking that it might be something to do with me running all this in a Docker container, and decided I should verify this in the native file system. At the same time I decided to create a simple repro case: my initial experiments at this stage had not been with the example spec above, they were with one of the fullstack-exercise test specs, and those required the rest of the Docker containers to be up and running, so too complicated to be portable (see "Short, Self Contained, Correct (Compilable), Example"). Accordingly I created the spec above, in a stand-alone application (as per the kahlan-issue repo on Github).

I sighed and installed PHP 8 and Composer on my PC - something I was hoping to avoid needing to do every again, now that I'm using Docker - and ran the test again. Again PHPUnit ran fine, but Kahlan errored-out. The act of putting debug code into the files erroring was enough to make that file now parse and compile, and the error moved on to the next file. Always the same three files. After touching the files once, the problem stayed away. Blowing away vendor and re-doing composer install did not make the erroring situation recur. However if I stuck the app in a different base directory - eg: I had it set up in C:\src\kahlanIssue initially, but moved it to C:\temp\kahlanIssue - then the error recurred again, until I tweakd the files. I noted that putting just whitespace into the file did not "fix" the issue; it needed to be something that PHP would need to parse. Even if I restarted the PC, once I had "touched" those three files once, even if I reverted the touches, the problem did not recur. That was weird.

I was also able to replicate the issue in all of:

• on Ubuntu within a Docker container (host machine is a Windows PC);
• on Ubuntu via WSL on that same Windows PC;
• natively on that Windows PC;
• natively on Windows on a different PC.

It was only when I downgraded from PHP 8.0.2 to 7.4.15 that I could no longer replicate the issue. So I guess something odd Kahlan is doing to load the PHP files worked fine in previous versions of PHP, but PHP 8 doesn't like it. Or something like that.

I think now I have enough to go on to raise an issue with the Kahlan bods, which I shall do now… done: 370.

All that kinda killed my Sunday afternoon (and now Monday morning writing this article). Hopefully I can get some actual work done this evening though…

// Step through the task queue which may add additional requests.
P\Utils::queue()->run();

// Step through the task queue which may add additional requests.($__KMONKEY__396__?$__KMONKEY__396__:
$__KMONKEY__396::queue())->run();

// Step through the task queue which may add additional requests.
($__KMONKEY__396__?$__KMONKEY__396__:$__KMONKEY__396::queue())->run();

Righto.

--
Adam