Thursday, 28 January 2016

Why testing fails (the short of it)

I have been asked to take the speaking place of a colleague, and talk on this for CEWT. Not the kind of thing you want to admit having first-hand experience in when you work for a company falling into the top 100 of almost every desirable list. I'll share my 2 reasons on the topic "Why testing fails".
This, is the short version, if you want to get all the goodness including the line-up of speakers on the same core topic, book a place at CEWT #2 on 28th February at DisplayLink Cambridge. Contact JamesThomas @qahiccupps .

Rushed Implementations 

“Look before you leap” comes to mind.
  • Features without right hygiene loose in the quality department
  • Feature does not solve customer a problem and becomes harder to test
  • Test not involved early enough

Test Planning 

“Fail to plan, and plan to fail", going around in vicious circles comes to mind.
  • Close-down cycle with no resources planned or budgeted for it
  • Planning impacted by rushed implementation
  • Planning is easier than you think (with good data to support it)

Saturday, 23 January 2016

humble bundle green screen challenge

What's the Humble Green Screen Challenge?
Inspired by FMV games, this event allows you to take a crack at making your own full motion video.

How should you make the videos?
We're providing some sample footage that you can use. All we ask is that you somehow involve that. There aren't any prizes to this challenge, so the rules are pretty darn loose.


YT demo clip:
My demo clip:
A bit like those DVD games where the DVD plays a clip then asks you a question, if you press "left", it plays another clip, if you press "right" it goes a different way. A bit like the make your own story choice skip to page X novels.

Stuff I learned along the way:
How to do Dolby in VideoStudio :
How to get 6 tracks (Dolby 5.1) from a stereo track in Audacity :
The audio results are not great- mostly due to not having any Dolby or surround equipment.

Chroma and background sources:
  • : Alex Free Stock Video Footage - Full HD - Fast Night Street
  • : Alex Free Stock Video Footage - Full HD - Animation - Disco Light 
  • : Alex Free Stock Video Footage - Full HD - Highway - Italy - Monte Carlo - GOPR0255
  • : Ufo Alien Spaceship Fly By - free green screen 
  • : fond vert ovni HD - Greenscreen UFO 1080HD
  • : Free Stock Footage_ Fish Swimming in Ocean Kelp Bed
  • : Galactic Journey in Space - Royalty Free Footage

Scoring time!

How do you rate my clip against some of the other subs?

Wednesday, 20 January 2016

Cambridge Lean coffee | Towers Watson

After not seeing the crowd of happy testers over the hectic Xmas break, a trip to Sawston was a welcome way to kick start 2016 with a drive-by to the south of Cambridge.
The "Testing" started, when I got picked up on my visitor registration badge right after arriving, because I had dated it 2012. Which was a good thing, because if I had dated it 2015, I would have been investigating the occurrence of an "off-by-one" defect.

The "checking vs Testing" did not stop there, but let's crack on.
We covered the topics which I paraphrased badly in order to fit them in a hurry onto the well scoped but limited surface-area of a post-it. We formed 2 groups, so these notes are from Chris Georges' table only.

Why pay to have a tester?

Or rather, at which point do we need a tester? Some companies test in the traditional way - they have automated unit tests and things are just fine for them right now. Some teams if small enough will get by just fine for a while. But without the specialist skills a specialist tester brings, all you have is someone who knows how to check stuff and how to write stuff. A professional tester is a integral part of the team and will be involved with requirements, which design review, and be able to get the correct level of detail in a test plan. You do have a test plan? right?
A professional will have the bandwidth to execute all the testing in the background when the developer is busy trying to fix a large list of bugs 2 days before the release deadline. This might also be called shielding your developers - something your support team might be doing right now already.
Did I mention, testing does not actually stop after the ship-party? Having a person on your team who knows that testing is not 100% about running test cases, but is also about helping you judge risk as well. A dedicated tester allows you to get the right detail level in your QA, because it enables a different perspective.
A good tester is a important part of a team, like a cog in a clock, it's important to make sure it is unique and just the right size for the job.

How do I automate legacy code testing?

It's really hard to do, and I can offer some tips on how to do this using clever instrumentation in ways that does not require code changes all over the place. But the question elicited these responses.
1. Prioritize your testing : P1=urgent , P2=less urgent and so on. This lends structure to what you are doing as well.
2. Be methodical - look at the test script (you have a script, right?) and analyze it for high probability blockers - try to ensure that you run things that can block as early as possible. This lets you push blocker into the developer early and buys DEV more time to resolve a blocker while you test down a different path.
3. Do session based testing. This is going to let you work through a weak test plan and by logging your session you will improve not only future test iteration estimates, and thus be able to time the testing to fit a release closedown. but it will also let you see which sessions and thus which features are the most needing testing based on how many bugs you recorded in a session. Excel is a great tool for recording.
4. Traceability - this is going to come out of the above steps.

Ultimately a deep understanding of what features are dependant on what components of the product will guide you to estimate which areas do not need more re-testing due to simple lower risk. risk is driven almost entirely by code churn. So components with minor change tend to break less- but only because of the interface or environment effects.
My tip on how to avoid re-testing legacy code, is to catalog how the environment impacts the features in the product. If environment plays a big part, study the impact and adjust your plans accordingly.

Specific to automation, instrumentation is an avenue worth exploring as a way to automate testing of legacy code, without touching the code-paths. Maybe I'll write something on this in future.

Which GUI tests should I automate?

Since this is a very common automation question, and the dangers are understood, I'll talk a bit more about ROI.
1. Paring it back , taking a good look at what to not automate by identifying the high priority cover areas
2. What things are hard to test manually, automate those first. What tests deliver most value if automated - things like product install /deploy or launch can be easy to automate and unblock your product development (CI system) quickly.
3. Talk about testing earlier - by forcing devs to thing about testing (manual and automated), you involve dev early, and get them to think more like testers. Make the end-application easier to test also makes end-users lives easier too in many cases.
4. Don't automate unit tests - basically system test, test the behavior, not the code!
5. Don't burn out your testers! Gettign testers to run manual tests all day will drive them a bit nuts, identify tests that drive them nuts and try automate those.
 I used a score-sheet (Excel to the rescue again) to decide when to automate. It looks like this.

We have 3 dummy cases here
Each test case or "TCD" will have a script (paper or electronic).

 Score each question (criteria) from 1-5. anything that scored less than 5 overall is just never automatable, anything getting over 50 might be, and so on. You get the idea. This screenshot omits the "weighting" applied and a few other "gating" criteria which link in with the PDLC used where I work at the moment (Citrix ltd.) . But you get enough of the picture.

Monday, 28 December 2015

Contemplative video games Part II

Back, with the round-up and value part of the summary. In Part 1 I rounded up 12 titles that approximately fit the bill of games that might calm the beast within. They were Dear Esther, Bastion, To the Moon, Trauma, Mind : Path to Thalamus, A new Beginning : Final cut, The Novelist, Year Walk, Proteus, Eidolon, Gone Home, The Graveyard.


 Do the game mechanics make sense, is the keyboard+mouse usable? Are you asked to achieve incredible acrobatics with the mouse to click on a tiny item? Is the game repetitive and non nonsensical at times frustrating in terms of UI? A score of 5 means good and obviously 0 = unusable.

Dear Esther5
To the Moon5
Mind : Path to Thalamus5
A new Beginning : Final cut4
The Novelist5
Year Walk5
Gone Home5
The Graveyard2


Price versus all the rest of the scored factors. Anything over £5 is penalised due to playtime expectations, and anything that has a free complete demo like the browser-based option to play Trauma will get a 5 for excellent value.

Dear Esther£6.994
Bastion£2.74 (demo)5
To the Moon£1.395
Trauma£4.39 (demo)5
Mind : Path to Thalamus£2.993
A new Beginning : Final cut£0.795
The Novelist£5.494
Year Walk£4.792
Gone Home£4.495
The Graveyard£3.99 (demo)2

Artwork : Designed by Freepik -

Youtuber New Year resolutions

So we have a little challenge and a new years resolution declaration all in one.
Roux Harbour
The challenge is to share your new years resolutions for your YouTube channel. I hope as many people possible in the Community will participate.

So, here goes with the resolution making rules:
Record Yourself;
- Introduce Yourself, Your Name, Your Age, Your Nationality
- Introduce Your Channel, What Is It About, Mention If You Have Regular Segments etc.
- Tell Us ONE Resolution That You Have For 2016
- Tell Us What Inspired It/The Thought-process Behind It
- Tell Us If You Think It's Going To Be Hard, Fun, Difficult, Easy etc. To Achieve Your Goal
- Say This Sentence; What Is YOUR 2016 Resolution? I'd Love To Read All About It! Share In The Comments Below!

Hi I'm Conrad aka Zaphodikus, 45, South African, naturalized UK citizen.
My channel primarily covers my adventures in the Augmented reality and outdoors Google game called Ingress, it's very much an amateur learning experiment as channels go. More eclectic, often sporadic the channel keeps me honest, it keeps me playing a game which is much healthier than the Minecraft addiction that preceded it. My goal is achievable, but really audacious, I'm a computer programmer, and we call goals like this a BHAG, Big, Haury, Audacious, basically a mission statement size resolution. So my goal is to get to around 5000 views per month - and eventually $5 estimated earnings. It's a very long walk and I'll need to grow from 30 to around 300 subscribers to get there. Wish me luck.

Sunday, 27 December 2015

Xmas went wrong, a little

It's the day after Xmas, and I realize that with Mom flown off to South Africa on a mercy mission, that Dad is stuck with 2 teenagers. Not exactly a problem untill you discover that Mr Braam Junior is having a hump  the gifts the Father Xmas gave for Xmas were not nearly good enough or just not up to expectations. Whatever, it's dumper time.

Rescue me Zoni

So I wrote a Letter to the Sony Entertainment Network (SEN) , yes God knows why they have 2 products that are essentially the same thing, but Playstation or SEN, it's the same piece of console junk at this point. So here is my letter:

1. I thought my son would start using the console again if I got him a game, our PS3 is almost a dustmagnet otherwise. He likes counterstrike, I suppose that's not a playstation game. But the game I got him sux apparently. I paid and downloaded
Call of Duty 4: Modern Warfare (1)
(£19.99)  downloaded beforehand so we would not have to wait for the download, which was a dumb idea. Don't get things for teenagers that they don't want, it got into a bug sulk over this. Please can you refund transaction  -----------------to my wallet?

2. I decided to make the best of a bad thing and topped up the wallet so he could choose something himself, but it turns out that Destiny: The Taken King (1)
(£19.99) is DLC which was not clear from the store - so now we have 1 game that we cannot play (transaction
-------- ) and one that we will not play. Is there a way to rescue this console. I really hope it's possible with a little help from you guys.

Thanks in advance

And I got a response a few days later: 

Hi Conrad,

Just a quick note to let you know that we’ve refunded your purchase of [Destiny Taken King & Call of Duty 4 Modern Warfare] and so £39.99 has been credited back to your account Wallet as agreed.

Your reference number -----------------------

Drop us a line if you ever need us again.


PlayStation Support


In a footnote to this situation, we also have a PS2, why? Will binning it pollute another small town's water-table with lead or should I recycle it along with the PS3 and buy yet another single-purpose device? Actually that's a lie the PS3 is a excellent blue-ray disc player, which would make sense if the remote control was not shaped like a fist and I actually had any blue-ray disks.

Steaming Steam

I hope the SONY support is better than Steam support  - note the dates and the fact that this Steam client bug below is still reproducible today.

Wednesday, 16 December 2015

New genre : contemplative computer games

This is a brand new genre of games currently dominated by no publisher specifically but inhabited largely by Indie games. It's what I call the contemplative games.

Now I've been gaming for ages, like, when you ask me  "What was your first PC based RPG?" I'll shout out TOEE for example. If you don't know it, you were in nappies at the time. I used to create Doom and Quake LAN parties, and my first youtube was in fact a Lan party promotional clip. Yes, we even used phones, serial cables and co-axial cable for our first gaming Saturday. In those days we always had a BBQ (or braai) to follow. But let's leave the romance behind and get to the core of the contemplative games genre.
These games all have only 2 things in common:
  • Dying is not a bad thing, and there is zero stress
  • You will not get carpal tunnel syndrome
In a few of the games in this category, it is possible to die, while playing Anna for example, if you fall off cliffs or just walk into the water, you get a gentle rebuke and resume a few yards away. (spoiler: Anna requires you to step off the edge sometimes, literally and figuratively) . It's no secret and would not be fun, but in Bastion, you will die too. The narrator tells you off, but in a lazy kind of way, almost like a benevolent grandmother chiding you for stealing a biscuit and letting yourself be caught. Bastion is one of the few games in this category where a lot of crazy clicking is involved. But its in this category for a reason, and that reason is a set of elements the contemplative genre all have to score very highly on to get included. A contemplative game is never a shoot-em-up either, so its never going to deliver 60 hours of playtime, in fact 3 hours is the median. They are thus not a target market for publishing houses, and are thus a little harder to find since they occupy a niche. But it's that niche that I have come to love, which is why I'm sharing it today. If any of these games do interest you, please wait for my next instalment before buying anything, I'm going to tackle Money-Value and line them up in a rough order next week. Broadly I'll be scoring on Artwork, Sound, Plot Playability , Value and stinkers (in next week's post).


To be a truly good contemplative game, there must be great artwork, or great soundtrack, sometimes both. Bastion scores so highly in these two areas that it makes up for the fact that it's a top-down isometric platform-shooter game. So at this point I'm going to go into the list and hint at the scores.
Dear Esther 5/5
Bastion 5/5
To the Moon 5/5
Trauma 4/5
Mind : Path to Thalamus 3/5
A new Beginning : Final cut 3/5
The Novelist 4/5
Year Walk 2/5
Proteus 2/5
Gone Home 2/5
The Graveyard 2/5

If a game you see here got a terrible score for Artwork, run down this page a bit more just in case it has a redeeming feature, no guarantees, since I have included a few stinkers.


Music, sound effect usage and environment immersion through sound.
Dear Esther 3/5
Bastion 5/5
To the Moon 5/5
Trauma 4/5
Mind : Path to Thalamus 2/5
A new Beginning : Final cut 2/5
The Novelist 2/5
Year Walk 2/5
Proteus 2/5
Eidolon 3/5
Gone Home 3/5
The Graveyard 2/5


I'm going to come right out here, this is probably the most subjective scoring area. For example I scored A new Beginning into this category only because the plot was about Green activism and time travel. It's a point and click adventure, and unfortunately you will die and will have to do a few puzzles, but none of  those where you go to location A, collect item, return to B, collect Item, go to C and then back to A.

Dear Esther 4/5
Bastion 5/5
To the Moon 5/5
Trauma 5/5
Mind : Path to Thalamus 3/5
A new Beginning : Final cut 3/5
The Novelist 4/5
Year Walk 2/5
Proteus 2/5
Eidolon 2/5
Gone Home 5/5
The Graveyard 2/5

Next week: Playability and Value

To score in this area, the game must be mechanically good, not having mouse support when its really needed will kill you, not having ergonomic keyboard features when it's needed likewise.

Many games in this category are low value, there are many reasons for this, as I've hinted at last week.