← back to section

Picture a team that ships an update every week. On Monday they added promo codes, on Wednesday they fixed delivery, on Friday they cut a new build. And before every release someone has to walk again through everything that worked yesterday: put an item in the cart, pay, check that the order shows up in the list. By the third week this stops being work and turns into a ritual — the same screens, the same clicks, the same expected result.

That is exactly where the question comes up: let a machine do it. When a person walks through the scenario, that is manual testing. When the same scenario is written as a program that walks through it on its own, that is automated testing. The difference between them is not speed — it is what each of the two is able to notice. Ahead: what an autotest is made of, why it turns red on a perfectly healthy application, and what a green run actually proves. One example runs through the whole article — checkout in an online shop.

autotest — what was recorded person — what they see Checkout Cart: 1 item Total: 1000 Pay Pay now ✓ cart has 1 item ✓ “Pay” button is there ✗ “Pay” button is there ✓ order status = paid 40 seconds, on every build red run, healthy application 1000 instead of 1,000,banner covers the button —none of this was on the list green = the recorded checks matched, not “it works”

An autotest repeats exactly what was recorded in it — fast, and as many times as you like. A person sees what is not on the list; and when an element moves, it is a healthy application that turns the run red.

What a person actually does when testing by hand

From the outside, manual testing looks like clicking through screens off a list. If that were all it is, machines would have replaced people twenty years ago. In practice a tester does three things at once: invents the check, performs it, and decides whether the result is a bug or intended behavior.

The third one is where the value sits. A person adds an item to the cart, sets the quantity to zero, goes back with the browser button, refreshes the page right on the payment step — half of those steps are on no list at all, they appear along the way because the previous screen behaved oddly. And the verdict "this is a bug" gets applied to things nobody agreed on in advance: the total is correct, but the label has slipped onto the next line; the error message is formally accurate, only a shopper reading it has no idea what to do next.

The limit of this approach is arithmetic. A set of sixty checkout checks is roughly a full working day for one complete pass: every scenario has to be prepared (the right product, the right user, the right stock level), executed, and recorded. And there are three or four builds a week. The problem is not even that it is slow: by the end of such a day attention drops, and what gets missed is usually not the rare scenario but the most familiar one — the one you have walked forty times and already know is fine.

What an autotest is made of

The word "autotest" sounds mysterious until you see what is inside. And inside is a short program with three parts.

First, the steps: open the product page, click "Add to cart", go to payment, click "Pay". Second, the expected result, recorded in advance and stated very precisely: one item in the cart, the total equals 1000, the order status after payment is paid. Third, the comparison: the program performs the steps, takes the actual value and matches it against the recorded one. Matched — "passed"; not matched — "failed", and the report gets a line saying "expected 1000, got 1200".

The next thing that matters is where this program lives. An autotest sits in the repository next to the application code, and it is not launched by a tester from a laptop but by the build server: on every code change, and separately at night across the whole suite. In the morning the team arrives to a finished report — "812 checks passed, 4 failed". That is where "you can run it as often as you like" comes from: there is no human in that loop, a person joins only to work through the failures.

One more thing that changes the picture. An autotest is not necessarily a robot dragging a mouse across the screen. Only a minority of checks go through the interface: most automated checks talk to the server directly with requests (API checks) or verify individual functions inside the code. Those run in seconds instead of minutes and rarely break — they have no buttons and no markup to break. Why there are usually more checks without a screen than with one is covered in test levels.

Why a test turns red while the application is healthy

An autotest does not see the page the way a person does. To click a button it first has to find it — by a marker: the caption "Pay", the element's name in the markup, its position in the page structure. That marker is called a locator.

Then the ordinary happens: a designer renames the button to "Pay now". Nothing changed for the shopper, payment works. But the autotest is looking for "Pay", does not find it and fails. The run is red — the application is healthy. Moving the button into a neighboring block does the same.

The second reason a run goes red is data: the test user is blocked, the product from the scenario is out of stock, an order left over from yesterday's run blocks a new one. The third is timing: the application does not answer instantly, and where the server replied in half a second today it takes three seconds tomorrow — the test did not wait and moved on to a screen that is not there yet. Such a test fails every other run with no connection to the code at all; it is called a flaky test, and it is worse than no test: the team gets used to "that one is always red" and stops reading the report in full.

Hence the point about money. An autotest is written once and repaired constantly, simply because the product keeps changing. So a check that will be done once never pays off, while a check that will be repeated two hundred times pays off quickly. The detailed arithmetic is in how automation works.

What a green run means

A green run says exactly one thing: everything recorded in the expectations matched. Not "the application works", and not "we can ship".

Back to the order. The price rendered as "1000" instead of "1,000", and a promo banner slid over the lower half of the Pay button — on a phone you can no longer hit it with a finger. The autotest found the button by its locator, clicked it directly, got the paid status and honestly reported "passed": neither the thousands separator nor the banner was on the list of expectations, so as far as the test is concerned they do not exist.

There is a worse version of this. The test records that the Pay button is present on the page; it is present, the test is green — and clicking it does nothing: presence was checked, when what mattered was the result of the click. A green run that proves nothing because the wrong property was checked is called a false green.

That is why on a project with autotests the tester has a daily job — triaging the red. A failed test has four possible explanations: a bug in the application; an outdated test; stale data; a flaky test. Until it is triaged, red means nothing, and filing a bug report is premature. And what is not on the list of expectations by definition is found by a person, through exploratory testing, where checks are invented on the fly from what just appeared on the screen.

What to check with what: the working rule

The rule follows from the two previous sections on its own: automate what will repeat many times and changes rarely; check by hand what is new, rare, and judged by eye.

The first half is above all the regression run: going through what already worked to make sure the new change broke nothing. Regression scenarios stay the same for weeks and have to be run on every build — precisely a machine's job. The same goes for calculations (discounts, delivery), sweeping through dozens of data variants, and checks that skip the interface.

The second half stays with the person: the feature built yesterday that will be reworked three more times (the autotest would have to be rewritten along with it), look and usability, and rare scenarios where writing a check costs more than walking it once.

Why people enter the profession through manual testing

Automation is programming, and the temptation to start there is understandable. The obstacle is that the test code is the second task; the first is knowing what to check — which scenarios the product has, where bugs usually hide, what matters and what does not. That understanding comes from working with the product by hand.

There is a more tangible reason too. Without knowing the product, people write autotests for things that never break anyway: does the home page open, is the logo on it. The number of checks grows, the report gets longer, and bugs keep reaching shoppers — just in the places nobody looked at.

Who ends up writing the autotests depends on the team: somewhere it is a dedicated person who does nothing else; somewhere developers write them next to their own code; somewhere it is a tester who started by hand. So "start with manual" is a route, not a ban; the transition itself is covered in where to grow next.

And here is the honest version of the "manual is dying" myth. The share of repetitive clicking really is shrinking — that is exactly what a machine was supposed to take over. What grows is the number of decisions: what to check at all, what expected result to record, what today's red run means. That is a change in the content of the work, not the disappearance of a profession: an autotest checks only what was recorded in advance, and a person is the one who records it.

Where this applies

On a project both approaches run at once, at different times of day. At night the build server goes through the regression run; in the morning the tester triages the failures and separates bugs from outdated tests; during the day they check yesterday's work by hand and walk the product with no plan at all. Knowing what to check with what saves the most effort: not running by hand what should long since have gone to the machine, and not automating one-off work.

Where beginners stumble:

  • Reading a green run as "everything works". Green only means the recorded expectations matched; everything else was not checked at all.
  • Filing a bug for a red test without triaging it. First check whether the test itself is outdated and whether the data went stale — otherwise the task comes straight back and trust in the reports drops.
  • Assuming automation will replace manual testing. What gets replaced is the repetitive part, not the job.
  • Rushing into automation before learning to test. The result is checks on things that never break anyway.
  • Walking by hand what should have been automated — the same long regression run on every build. It wears you out and leads to misses in the most familiar places.

In short

  • An autotest is a set of expectations recorded in advance plus a comparison against the facts; what was not recorded is not checked.
  • It lives in the repository and is launched by the build server — on every change and overnight, with no person involved.
  • Most automated checks skip the interface and talk to the server directly, which is why they are fast and rarely break.
  • A red run does not equal a bug: the test may be outdated, the data stale, or the test simply flaky.
  • A green run does not equal "it works": slipped layout and an unreadable error message are seen only by a person.
  • Automation pays off through the number of repeats: automating a one-off check costs more than walking it.
  • People enter the profession through manual testing because a person decides what to check and what red means.