Skip to content

Testing

September 29, 2023
June 17, 2015

split to lang specific files

What is Software Testing? The 10 Most Common Types of Tests Developers Use in Projects

Unit testing - Wikiwand
Test automation - Wikiwand
List of unit testing frameworks - Wikiwand
Unit Testing Succinctly - Envato Tuts+ Code Tutorials
Testing Pipeline 101 For Frontend Testing — Smashing Magazine

Ponicode - Visual Studio Marketplace generate TypeScript, JavaScript, Python unit tests

Unit test (a single function)
Integration tests (multiple classes/units)
Functional tests (user-oriented, high level, full stack)
Smoke tests (quickly determine if the system is "working")

Integration/Functional Test: do the pieces work together as I expected?
Unit Test: given input x, is the output y?
JavaScript Testing: Unit vs Functional vs Integration Tests !important
What are Unit Testing, Integration Testing and Functional Testing? | CodeUtopia
testing - What is Unit test, Integration Test, Smoke test, Regression Test? - Stack Overflow
Better software testing through automation | InfoWorld

bbc/notes-on-perf-testing: An open-source book on system performance testing

Theory

Developer's aspect

TDD, RDD and DDD Test-driven development, REPL-driven development, Documentation-driven development
Tests as documentation | Technology Conversations
Giving Up on TDD - Clean Coder Blog
Code Cop: Y U NO TDD
RIP Test Driven Development - Better Programming - Medium

Test-driven development - Wikiwand
Test-Driven Development (TDD) | Technology Conversations

Behavior-driven development - Wikiwand
Introducing BDD | Dan North & Associates
What’s in a Story? | Dan North & Associates
BDD Assistant: It’s alive and cries for help | Technology Conversations
BDD (Behavior-Driven Development): Missing Piece in the Continuous Integration Puzzle | Technology Conversations

What’s the difference between Unit Testing, TDD and BDD? | CodeUtopia
What is the difference between writing test cases for BDD and TDD? - Programmers Stack Exchange
Relation between BDD and TDD - Programmers Stack Exchange
From BDD to TDD, the pros and cons of various agile techniques | InfoWorld
The Truth about BDD - Clean Coder

The path to production: how and where to segregate test environments - CircleCI
How to test software, part I: mocking, stubbing, and contract testing - CircleCI
How to test software, part II: TDD and BDD - CircleCI

TDD Changed My Life. It’s 7:15 am and customer support is… | by Eric Elliott | JavaScript Scene | Medium
Common Myths and Misconceptions of Test Driven Development - DEV Community 👩‍💻👨‍💻
Six Ways of Improving Behaviour-Driven Development
The WHY behind TDD/BDD and the HOW with RSpec
5 Questions Every Unit Test Must Answer — JavaScript Scene — Medium
5 Common Misconceptions About TDD & Unit Tests — JavaScript Scene — Medium
One weird trick that will change the way you code forever: JavaScript TDD

Unit Test Fetish/Lean Testing

Unit test cons:

Lean Testing or Why Unit Tests are Worse than You Think
Write tests. Not too many. Mostly integration. – kentcdodds
Unit Testing Anti-Patterns, Full List
Kent C. Dodds – Write tests. Not too many. Mostly integration. - YouTube
Unit Test Fetish - 250bpm
Exploding Software-Engineering Myths - Microsoft Research
Why-Most-Unit-Testing-is-Waste.pdf

I Mock Your Mocks | Boot.dev
separating logic into finer units avoids the need to mock

TestPyramid
Unit Testing: The Good, Bad, and Ugly - DZone Performance

QA's aspect

Open Lecture by James Bach on Software Testing - YouTube

如何测试洗牌程序 | 酷 壳 - CoolShell.cn

Klarna Engineering

How removing caching improved mobile performance by 25%
Four reasons developers should write their own load tests
Four load testing mistakes developers love to make

User story

How To Write Well-Formed User Stories

Scenario: title
Given [some context]
And [additional context]
When [event]
Then [outcome]

Gherkin/Cucumber

Gherkin is a human and machine friendly spec for specifying behavior:
Cucumber
Gherkin Syntax : Cucumber
Feature Introduction · cucumber/cucumber Wiki

Behavior-driven development - Wikiwand

Behat documentation
Writing better user stories with Gherkin and Cucumber
BDD 101: Gherkin By Example | Automation Panda
behave/features at master · behave/behave examples

Behavior-Driven Development in Python
Python Testing 101: behave | Automation Panda
What is behavior-driven Python? | Opensource.com

cucumber/gherkin at master · cucumber/cucumber Cross platform parser for the Gherkin language
Related tools : Cucumber
Cucumber Selenium tutorials - YouTube

Custom Formatters · cucumber/cucumber Wiki
cucumber/cucumber-html: Cross platform HTML formatter for all implementations of Cucumber
wswebcreation/multiple-cucumber-html-reporter: Generate beautiful Cucumber HTML reports
brynary/features2cards: Create PDFs from Cucumber features and scenarios for printing

Gauge

Open Source Test Automation Framework | Gauge

Test Runners

高效开发运维

Some test runners take the features and generate test scripts in various languages.

Robot Framework Python
RedwoodHQ | Open Source Test Automation Framework polyglot
Cucumber project polyglot
Cucumber, BDD and 21st Century Hackable Atom Editor - SHASHIKANT JAGTAP``
behave Python
Behat PHP

Automated Testing

Automated Testing: Your End-to-End Ecosystem - DZone - Research Guides

From 0 to 100: How to Get Into Automated Testing as a Manual Tester - DZone Agile
What's the Difference Between Automated Testing and Manual Testing? - DZone Performance

Ad-hoc standard

xUnit - Wikiwand
Test Anything Protocol - Wikiwand TAP is a simple text-based interface between testing modules in a test harness. Many test frameworks can output TAP results.
Main Page - Test Anything Protocol


Shell Script

jimeh/test-runner.sh
jimeh/stub.sh
sstephenson/bats

shunit2 - Google Search
Chaos Engineering 101 — Production Ready — Medium
Chaos Monkey for Fun and Profit — Production Ready — Medium

Chaos Engineering

Principles of Chaos Engineering
chaos.community
Chaos Engineering 101 — Production Ready — Medium
Chaos Monkey for Fun and Profit — Production Ready — Medium
The Netflix Simian Army – Netflix TechBlog – Medium
How To Establish a High Severity Incident Management Program
Chaos Engineering: Chaos Testing Your HTTP Micro-Services

Netflix 混沌工程手册
微服务架构下的质量迷思——混沌工程

SE-Radio-Episode-325: Tammy Butow on Chaos Engineering : Software Engineering Radio

A Powerful Chaos Engineering Platform for Kubernetes | Chaos Mesh®

marmelab/gremlins.js: Monkey testing library for web apps and Node.js

Gremlin: Chaos Engineering Tools to Break Things on Purpose
Overview - Gremlin Help
How Gremlin is making chaos engineering accessible [Interview] | Packt Hub

Fuzzing

Fuzzing | OWASP Foundation

A new chapter for OSS-Fuzz | Google Open Source Blog
Announcing the Atheris Python Fuzzer | Google Open Source Blog
google/atheris

AFLplusplus/AFLplusplus: The fuzzer afl++ is afl with community patches, qemu 5.1 upgrade, collision-free coverage, enhanced laf-intel & redqueen, AFLfast++ power schedules, MOpt mutators, unicorn_mode, and a lot more!

Android

Automating User Interface Testing on Android - Tuts+ Code Tutorial

LAVA
LAVA Manual — LAVA Server 2015.07 documentation

C

Write Async Test Case by gtest & gmock for C++ | Thinking beyond Source Code

UI Automation

ui-automation

HTTP endpoint

SoapUI | Functional Testing for SOAP and REST APIs
mapbox/assert-http: Test helpers for testing a HTTP interface
Gor - test your system with real data

Polly.JS

apiaryio/dredd: Language-agnostic HTTP API Testing Framework
Command-line Interface - Dredd

Getting started with HTTPie for API testing | Opensource.com

Karate

Cucumber-like SDL for HTTP agent and browser automation

Karate | Test Automation Made Simple.
karatelabs/karate: Test Automation Made Simple

Intro to all features of Karate (subtitles / CC available) - YouTube

Mock Server

Reqres - A hosted REST-API ready to respond to your AJAX requests

JavaScript

visionmedia/supertest testing server API with extended superagent
Testable API's with Node.js

Python

Getting Started With Testing in Python – Real Python
4 Techniques for Testing Python Command-Line (CLI) Apps – Real Python
How I'm testing in 2020

pytest: helps you write better programs — pytest documentation
Effective Python Testing With Pytest – Real Python
Testing Your Code with Python's pytest | Linux Journal
hackebrot/pytest-emoji: pytest + emoji ==Python 3.x | Asyncio Testing - YouTube

hackebrot/pytest-tricks: Tips and Tricks for the Python Testing Tool
PyTest Tutorial: What is, Install, Fixture, Assertions
Martin Heinz - Pytest Features, That You Need in Your (Testing) Life
schireson/pytest-mock-resources: Pytest Fixtures that let you actually test against external resource (Postgres, Mongo, Redshift...) dependent code.
mwilliamson/python-precisely: Matcher library for Python

Justin Crown - "WHAT IS THIS MESS?" - Writing tests for pre-existing code bases - PyCon 2018 - YouTube slide

Welcome to Hypothesis! — Hypothesis documentation
Property-Based Testing In Python: Hypothesis is AWESOME - YouTube
Our experience with Hypothesis testing (and why do we love it so much !) - Parsec
Getting Started With Property-Based Testing in Python With Hypothesis and Pytest - Semaphore

doctest — Test interactive Python examples — Python documentation
doctest - Wikiwand
doctest — Testing Through Documentation — PyMOTW 3
Python Testing with doctest - Mouse Vs Python

unittest — Unit testing framework — Python documentation
Python Tutorial: Unit Testing Your Code with the unittest Module - YouTube
Python Unit Testing – Structuring Your Project – Patrick's Software Blog
python - Running unittest with typical test directory structure - Stack Overflow

Note to Users — nose documentation nose extends unittest to make testing easier.

Welcome to the tox automation project — tox documentation
Welcome to Nox — Nox documentation easier to break the abstraction
Thea Flowers - Break the Cycle: Three excellent Python tools to automate repetitive tasks - YouTube

Mock/Stub

Welcome to Faker’s documentation! — Faker documentation
joke2k/faker: Faker is a Python package that generates fake data for you.

chris1610/barnum-proj: Python application for generating pseudo-random data

JavaScript

TODO: Define "Test runner", "Assertion", "Framework"

An Overview of JavaScript Testing in 2017 – powtoon-engineering – Medium !important
Testing Node.js in 2018 – Hacker Noon
Incredibly convenient testing of front-end Javascript with Node.js - Staal Forge
5 Questions Every Unit Test Must Answer — JavaScript Scene — Medium
6. Writing Testable JavaScript - YouTube
Codeship's Philosophical Approach to Frontend Testing - via @codeship | via @codeship

Promises in JavaScript Unit Tests: the Definitive Guide
Testing Node.js with Mocha and Chai - Michael Herman

A Gentle Introduction to Javascript Test Driven Development: Part 1
A Gentle Introduction to Javascript Test Driven Development: Part 2
A Gentle Introduction to Javascript Test Driven Development: Part 3

Frameworks

vite#Vitest

SpeckJS - Comment Driven Development

Jest · 🃏 Delightful JavaScript Testing
Delightful JavaScript Testing with Jest - YouTube
ReactiveConf 2017 - Rogelio Guzman: Jest as a Platform - YouTube

Testing Library | Testing Library

Jasmine

Intern: Software testing for humans video

testdouble/teenytest: A very simple, zero-config test runner for Node.js built-in assert or assertion library

Wallaby - Immediate JavaScript test feedback in your IDE as-you-type

avajs/ava: Node.js test runner that lets you develop with confidence 🚀 run tests in parallel, test ends in "*.test.js"
AVA, low-config testing for JavaScript

mochajs/mocha: ☕️ simple, flexible, fun javascript test framework for node.js & the browser pollutes global namespace, need assertion library

bevry/joe: Joe is an accurate and powerful testing framework that can run on node and in the browser
Welcome! Buster.JS is...
Vows « Asynchronous BDD for Node

Comparison

Essential JavaScript: the top five testing libraries | JavaScript | Creative Bloq

List of unit testing frameworks (JavaScript) - Wikiwand
thegrtman/javascript-test-framework-comparison
coderwall.com : establishing geek cred since 1305712800
JavaScript unit test tools for TDD - Stack Overflow
I tested a React app with Jest, Enzyme, Testing Library and Cypress. Here are the differences. | by Sunil Sandhu | JavaScript in Plain English

lab

hapijs/lab: Node.js test framework (need assertion library, say code, includes coverage)
hapijs/shot: Injects a fake HTTP request/response into your node server logic
hapi — Getting Started With Testing Using Lab and Code
Chapter 7: Testing using lab | Hapi With Typescript | Softcover.io

nlf/lab-babel: A transform to allow testing babel.js transpiled code in lab more effectively

const Lab = require("lab");

// Test files must require the lab module, and export a test script
const lab = (exports.lab = Lab.script());

// shortcuts to functions from lab
const experiment = lab.experiment;
const test = lab.test;

experiment("getting started with hapi testing,", () => {
  test("lab considers this test as TODO and skips it");

  test("always succeeding :)", () => {});
});

Mock/Stub

Sinon.JS provides AJAX mock and spy that work in all test frameworks.
Sinon Tutorial: JavaScript Testing with Mocks, Spies & Stubs
marmelab/FakeRest: Patch fetch/XMLHttpRequest to fake a REST API server in the browser, based on JSON data.

pgte/nock
Intercept HTTP Requests with Node.js nock
REM - REST API

Stop mocking fetch
MSW – Seamless API mocking library for browser and Node | Mock Service Worker

typicode/json-server: Get a full fake REST API with zero coding in less than 30 seconds (seriously)
WireMock - WireMock

Marak/faker.js
dejavu1987/jabber: Simple random word / paragraph / lorem ipsum / dummy text generator.
Chance.js: Utility library to generate anything random for JavaScript
drewbrokke/chance-token-replacer
ngneat/falso: All the Fake Data for All Your Real Needs 🙂
adleroliveira/dreamjs: A lightweight json data generator.
json-schema-faker/json-schema-faker: JSON-Schema + Faker
danibram/mocker-data-generator
aharris88/awesome-ipsum

mfncooper/mockery: Simplifying the use of mocks with Node.js
thlorenz/proxyquire: Proxies nodejs require in order to allow overriding dependencies during testing.
rvagg/node-mkfiletree

Introducing frock: Easy fake services for a microservices environment
urbanairship/frock: A plugin-based tool for running fake HTTP and socket services

LINKIWI/endpoint: Super simple mock API endpoints for static JSON data

Raathigesh/Atmo: Server side powertool for prototyping

React

reactjs#Testing

TAP

mocha -R
lab -r tap

Home - Test Anything Protocol
Understand the Test Anything Protocol | The Effective Perler
sindresorhus/awesome-tap

Writing javascript tests with tape - good coders code, great reuse
Testing JavaScript Modules with Tape
Why I use Tape Instead of Mocha & So Should You — JavaScript Scene — Medium
My node test strategy
tape (testling)
how I write tests for node and the browser

TAP in JavaScript node-tap evolved
substack/tape
scottcorgan/tapes, tape with beforeEach(), afterEach()
spion/blue-tape tape with promise
wavded/babel-tape-runner
Jam3/tap-dev-tool: prettifies TAP in the browser's console

rvagg/bustermove use Buster syntax in tape

Reporters:
scottcorgan/tap-spec
substack/faucet
namuol/tap-difflet
tap-nyan - npm

Assertion

hapijs-code · GitHub
Home - Chai
Chai HTTP - Chai
Automattic/expect.js
shouldjs/should.js
sindresorhus/core-assert: Node.js assert as a standalone module

End-to-end

Google Testing Blog: Just Say No to More End-to-End Tests
End-to-End Testing Web Apps: The Painless Way - mtlynch.io

Late to the Party; End-to-End Testing: Part 1 – codeburst
Late to the Party; End-to-End Testing: Part 2 – codeburst
Late to the Party; End-to-End Testing: Part 3 – codeburst
Late to the Party; End-to-End Testing: Part 5 – codeburst

UI-licious | End-to-end user journey testing for the modern web
[Success] Test: Untitled Test

Karma - Spectacular Test Runner for Javascript
ksunair-introtokarma
douglasduteil/isparta: A code coverage tool for ES6 (babel/6to5)

How to automate REST API end-to-end tests in a CI environment with Postman and Newman

Getting started with Mocha and Chai - Rob Dodson talks internets
ludovicofischer-mocha-chai-browser-demo · GitHub

Cypress

JavaScript End to End Testing Framework | Cypress.io TDD, chai, auto wait async calls
Guides | Cypress Documentation
Table of Contents | Cypress Documentation
cypress-io/testing-workshop-cypress: End-to-end testing workshop with Cypress
cypress-io/cypress-example-kitchensink: This is an example app used to showcase Cypress.io testing.
cypress-io/cypress-example-recipes: Various recipes for testing common scenarios with Cypress

Plugins | Cypress Documentation
bjowes/cypress-ntlm-auth: Windows authentication plugin for Cypress
NoriSte/cypress-wait-until: Add the Cypress waiting power to virtually everything 🎉

Sleeping better at night with cypress.io - Tomasz Łakomy - YouTube
Testing Angular with Cypress and Docker - TestDriven.io
Implementing Your First End-to-End Tests In React Using Cypress
UI Testing a React App Using Cypress - Level Up Coding
Cypress.io — Scaling E2E testing with custom commands | by Nicholas Boll | Medium

Tips and Tricks

8 Tricks I Learned From Cypress

Do Not Get Too Detached page reload
race condition
When Can The Test Start?
When Can the Test Click?
After upgrade to version 3.6.1 tests start failing with " CypressError: Timed out retrying: cy.click() failed because this element is detached from the DOM" · Issue #5743 · cypress-io/cypress
Re-query elements that are found 'detached' from the DOM · Issue #7306 · cypress-io/cypress
Cypress cy.intercept Problems | Better world by better software

Remember that cypress commands are not promises and you can't use async/await in Cypress tests.
Introduction to Cypress | Cypress Documentation
Do not support async/await function in es6 style · Issue #158 · NoriSte/cypress-wait-until
Misleading documentation for then method · Issue #3446 · cypress-io/cypress-documentation
Await-ing Cypress Chains · Issue #1417 · cypress-io/cypress
Cypress.io — Using async and await | by Nicholas Boll | Medium
NicholasBoll/cypress-promise: Allow a Cypress chain to resolve to a promise

// use this wrapper
return new Promise((resolve, reject) => {
  cy.Chainable().then(() => resolve());
});

Coverage

Rethinking JavaScript Test Coverage – Node.js Collection – Medium TL;DR, c8
bcoe/c8: output coverage reports using Node.js' built in coverage

gotwarlost/istanbul
Hard Thresholds on JavaScript Code Coverage
The Hidden Trap of Code Coverage
bcoe/nyc: a code coverage tool that works well with subprocesses.

alex-seville/blanket

Benchmark

substack/node-ben
jeffbski/bench-rest

Browser as a Service

Sauce Labs: Selenium Testing, Mobile Testing, JS Unit Testing
Open Sauce

Cross Browser Testing Tool. 300+ Browsers, Mobile, Real IE.

Virtual Machine (VM), Windows Virtual PC & BrowserStack

Continuous Integration

ci-cd