How to Test a Small Operational Change Before Scaling It

Secondary 1 Mathematics tuition small group classroom at eduKate

eduKate Yishun · Learning and the next step forward

Find a useful starting point

Choose the question closest to your situation, or read the full guide in order.

Browse all chapters · Explore recovery and transformation · Explore the learning library

A small operational change is worth testing when you can describe what will change, what improvement you expect and what you will protect while learning. To test a process improvement, begin with a bounded piece of ordinary work: one preparation session, one document handoff or one clearly defined group of items. Record what happened before deciding whether the change deserves a wider trial.

An operational pilot needs more than a favourable completion count. A faster process may require extra preparation, leave difficult work unfinished or create errors that someone repairs later. Small scale change testing becomes useful when its measures include the intended benefit, the resources used and the unwanted effects that could make the apparent gain misleading.

This guide explains how to plan a reversible process improvement experiment, write a prediction, choose practical measures, set stopping conditions and interpret the result. It uses ordinary workflow education, with no promise that a successful pilot proves causality or establishes business viability. Every organisation, person, task record and numerical outcome in the examples is fictional; the worksheets are original teaching illustrations.

Your 50-second route: If your idea is still vague, start with the question you can actually test. If you already have a change in mind, use the boundaries and measurement definitions. For a complete example, follow the preparation case through the decision. To practise independently, attempt the changed situation before reading the answer.

Contents

Chapters 1–5 · Ask a question small enough to answer
  1. Ask a question small enough to answer
  2. Describe the change so someone else could recognise it
  3. Set boundaries and make restoration practical
  4. Write a prediction before you see the result
  5. Define benefits, side effects and denominators
Chapters 6–10 · Prepare an observation sheet people can use
  1. Prepare an observation sheet people can use
  2. Worked case: a preparation tray for community craft packs
  3. Read the arithmetic without hiding unfinished work
  4. Decide what the result does and does not support
  5. Stop, restore or adapt when the trial becomes unhelpful
Chapters 11–15 · Widen the next test by one meaningful condition
  1. Widen the next test by one meaningful condition
  2. Independent task: a shorter handoff for a display project
  3. Reasoned answer: the faster-looking sheet does not earn expansion
  4. Write the decision another person can pick up
  5. Source notes and the limits of these examples

CHAPTER 1 OF 15

1. Ask a question small enough to answer

Back to contents

“Will this improve the organisation?” is too broad for an afternoon trial. “Will a labelled preparation tray help us finish more complete craft packs during one normal preparation session?” identifies something observable. The second question has a work product, a change and a time boundary. It also gives the team a reason to inspect the packs instead of counting movement around the room.

The test question should arise from a problem already worth investigating. Suppose people repeatedly search for the same instruction sheet. Placing the sheets at the point of use is a plausible change. Suppose instead that almost nobody requests the packs. A faster preparation method would answer a different question from the one the organisation needs to resolve. Use How to Diagnose a Process Bottleneck when the difficulty is locating what restricts the workflow.

A useful question names the uncertainty. You may know that a tray is easier to see, but not whether maintaining it creates more work than it removes. You may know that a shorter form is quicker to complete, but not whether the next person receives enough information. Write that tension into the question. “Can we shorten this handoff while preserving the information needed to act?” is more informative than “Can we make the form shorter?”

Decide what answer would change your next move. If any result will lead to permanent adoption, the exercise is a demonstration rather than an open test. An honest pilot might lead to continuing with the old method, modifying the new one, repeating the trial or trying the idea under another condition. These are different decisions, and each can be useful when supported by the observations.

Keep the question within work you are authorised to organise and within the arrangements already agreed by those involved. The examples here concern materials and routine information, not experiments on people. They do not provide a method for clinical work, personnel assessment, employment decisions or other consequential decisions outside an ordinary workflow exercise.

Before collecting anything, finish this sentence: “After this trial, we should know more about whether changing ___ helps us produce ___ under ___ conditions, while preserving ___.” If the blanks require several unrelated processes, reduce the scope. A bounded answer is useful precisely because you can say where it applies.

Contents · Next chapter

CHAPTER 2 OF 15

2. Describe the change so someone else could recognise it

Back to contents

“Improve organisation” does not identify a testable change. “Place one labelled instruction slot beside each pack type and refill it before assembly starts” does. Someone observing the work can distinguish the proposed method from the previous one. They can also notice whether the method was actually used.

Describe the current method first. Where does the task begin? What does the worker receive? What happens before the product is considered finished? This need not become a complicated process diagram. A paragraph can establish that materials are selected, assembled, checked and placed in an accepted-work tray. Without that description, a new “completion” count may quietly omit the checking stage.

Then identify the smallest meaningful alteration. It might be the placement of one item, the wording of one handoff field or the order of two preparation actions. Small does not mean trivial. Moving information to the moment it is needed can affect a whole session. What matters is whether you can tell which part of the work changed and which uncertainty the test addresses.

Do not bundle unrelated improvements merely because the opportunity exists. If you change the layout, simplify the product, provide additional help and extend the session together, the package might perform well, but the result will say little about the layout alone. A combined change can still be tested when the combination is the actual proposal. Call it a combined change and record every component rather than crediting one favoured element.

Use a brief version description. For example: “Tray version A: two text-labelled slots; the existing instruction sheets remain unchanged; a pack stays in the assembly area until the normal final check is complete.” The description makes the first trial repeatable enough to discuss. If you subsequently add a divider or change a label, that becomes another version with another prediction.

Do not confuse a neatly written method with evidence that it works. A clear description enables a test; it does not replace one. A useful preparation question is whether two people reading the description would set up approximately the same arrangement. If not, clarify the uncertain instruction before interpreting inconsistent use as failure of the idea.

The wider choice of what deserves attention belongs in Business Turnaround and Rebuilding. Here the work is narrower: take one plausible operational idea and make its behaviour observable.

Contents · Previous chapter · Next chapter

CHAPTER 3 OF 15

3. Set boundaries and make restoration practical

Back to contents

A pilot needs an entry boundary, an exit boundary and an exposure limit. The entry boundary says which work enters. The exit boundary says what counts as completed. The exposure limit says how much work or time may use the new method before review. “Try it for a while” leaves all three uncertain.

For a material-preparation exercise, entry might mean thirty allocated pack requests available at the start. Completion might mean a fully assembled pack that has passed the existing final check and reached the accepted-work tray. Exposure might mean one sixty-minute preparation session. The agreed end does not move because the team is close to a pleasing result.

Reversible means that the old method can actually resume. Keeping a photograph of the previous layout may help, but it does not preserve the old labels, materials or instructions. List what restoration requires and keep those things available. If restoring the old arrangement would take the rest of the afternoon, an apparently tiny layout change has a larger practical exposure than its physical size suggests.

Also identify work caught between methods. A partly assembled pack should not disappear from the record when you stop the pilot. Put it in a clearly identified holding area, retain its request reference and use the agreed checking method before it can count as accepted. Restoration includes unfinished work and unresolved errors, not merely moving furniture back.

Distinguish restoring the method from reversing the consequences. You can restore an old form immediately, but a confusing message already passed to another person may still need correction. This is one reason to begin with low-stakes material preparation or a practice document set where the consequences remain contained. A reversible tool does not automatically make every use reversible.

Agree who records a problem, who can call a pause and who restores the arrangement. In a small team these may be the same two people doing the work. They should not need to debate authority while a problem accumulates. The stopping conditions must fit their ordinary responsibilities rather than inventing a new approval hierarchy.

For broader thinking about preserving useful capability during change, see Defence: Protect What Still Works. A well-bounded pilot makes that principle concrete: identify the useful function, preserve access to it and specify how unfinished work will be handled.

Contents · Previous chapter · Next chapter

CHAPTER 4 OF 15

4. Write a prediction before you see the result

Back to contents

A prediction states what you think the changed method will do and why. It is not a promise or an instruction to produce a certain number. Its purpose is to make the original reasoning available for comparison after the test, when memory is tempted to make the idea seem more accurate than it was.

In the pack example, the prediction could be: “Separating instruction sheets by pack type should reduce searching and selection mistakes. With the same two people and a sixty-minute session, we expect at least twenty-seven accepted packs from thirty allocated requests, no more than two packs failing their first check and setup using no more than twenty person-minutes.”

The quantities are local teaching choices, not universal standards. Twenty-seven is useful here because the fictional team has already observed twenty-four accepted packs in its reference session and wants a noticeable operational difference worth investigating. The prediction also acknowledges quality and preparation effort. It does not define improvement entirely through output.

Write down the mechanism in ordinary language. If the proposed benefit is less searching, what observation would support that explanation? A brief note that both workers repeatedly found the correct sheet without leaving the work area is relevant. A favourable completion count alone does not establish that searching changed. Conversely, less searching may occur without more accepted packs if a different step consumes the available time.

Separate the hoped-for result from a stopping limit. A prediction of no more than two first-check errors describes an expected condition. A stop rule might require pausing when a third error appears. These statements serve different purposes: one supports learning, while the other limits how far the trial proceeds under an unwanted condition.

Include a plausible alternative explanation before starting. The team might improve simply because it has recently practised the task. A quieter session might reduce interruptions. A different mix of packs might make the work easier. Recording these possibilities does not make the pilot worthless; it helps prevent a small observation from becoming a larger claim than the evidence can support.

Finally, write what an unhelpful result would mean for the next step. Perhaps the idea should be adjusted if setup takes too long, or abandoned if distinguishing pack types remains confusing. A prediction becomes more useful when it leads to a decision you are willing to change.

Contents · Previous chapter · Next chapter

CHAPTER 5 OF 15

5. Define benefits, side effects and denominators

Back to contents

A measure needs a name, a counting rule and a boundary. “Quality improved” lacks all three. “Twenty-six of twenty-seven packs passed the first check during the pilot session” states what was inspected, how many passed and which group the result describes. It also leaves an important question visible: what happened to the remaining allocated requests?

Use separate counts for accepted output, first-check quality and unfinished work. A pack that fails inspection and is corrected may eventually be accepted. Count it once in accepted output and once among packs that failed their first check. Do not count its second inspection as another produced pack or erase the original error because the repair succeeded.

The denominator must match the question. Accepted packs divided by allocated requests answers how much of the offered work was finished. First-check passes divided by packs inspected answers how many inspected packs were correct immediately. Neither fraction, by itself, tells you how good the unfinished packs would have been. State this limitation rather than calling uninspected work error-free.

For a time-bounded exercise, include preparation, assembly, inspection, repair and closeout. Two people working for sixty minutes provide 120 person-minutes of available time. This is not the same as sixty elapsed minutes, and neither quantity establishes how every minute was used. If you report an allocation of person-minutes to activities, make sure those allocations sum to the stated total.

Choose a few side-effect measures that fit the proposed change. A shorter form might create more clarification messages. A quicker preparation method might increase missing items. A new tray might take longer to set up. Do not add measures simply to create an impressive dashboard; each should protect something the intended benefit might otherwise hide.

Record whether the method was used as described. If one person quietly returns to the old arrangement halfway through, the session did not expose all the work to the same method. That is useful information. Mark the switch, preserve the before-and-after counts if available and avoid reporting the result as a full-session test of the original version.

Finally, distinguish zero from unknown. “No incorrect labels among all twenty-seven inspected packs” has an explicit denominator. “No complaints” might mean nobody had an opportunity to inspect the packs yet. A blank checking field is missing information, not a favourable outcome. Careful counting keeps the result connected to work that somebody actually observed.

Contents · Previous chapter · Next chapter

CHAPTER 6 OF 15

6. Prepare an observation sheet people can use

Back to contents

The observation sheet should make the important distinctions easy to record during work. It should not require reconstructing every movement afterwards. Begin with the session conditions, then give each item or small batch enough space for completion and quality notes. Use the simplest record that answers the test question.

At the top, record the date or session label, method version, people involved, elapsed time, allocated work and opening unfinished work. Add the product mix when task types differ. In the fictional pack trial, the two types are standard and large packs. Their contents differ, so an output total without a type count would conceal information relevant to comparison.

For each request, the sheet can contain a reference, pack type, first-check result, correction if needed and accepted-before-close status. The request reference identifies the work rather than evaluating the person who handled it. Keep any observation about the arrangement focused on the task: “large-pack sheet slot empty twice” is more actionable than “worker seemed slow.”

Use one consistent meaning for each mark. A tick under accepted means the pack passed the normal check and reached the accepted-work tray before the end boundary. It does not mean “almost finished,” “probably correct” or “will be repaired immediately afterwards.” Write the definition on the sheet so it survives the rush of the session.

Record unusual conditions as they occur. If an interruption removes one person for several minutes, note the time and the work affected. If materials were already sorted before the clock started, record that preparation instead of treating it as free. If the observation itself consumes working time, include it within the same time boundary or identify the additional resource clearly.

The team can use a short closeout check: allocated requests equal accepted requests plus unfinished requests, assuming nothing was cancelled or removed. If an item was withdrawn, record that category separately. This simple reconciliation often detects double counting and vanished work before anyone turns the totals into percentages.

A sheet becomes useful through clear definitions and consistent use. Attractive formatting cannot repair an ambiguous exit boundary. Before the live exercise, walk one fictional request through the record and ask where a failed check, a repair and a late completion would go. Resolve those questions while the page is still empty.

Contents · Previous chapter · Next chapter

CHAPTER 7 OF 15

7. Worked case: a preparation tray for community craft packs

Back to contents

The fictional Harbour Paper Club prepares materials for an adult paper-craft session. Each allocated request requires either a standard pack or a large pack. Both need the correct instruction sheet, the correct paper allocation and an identifying label. This is a low-stakes preparation exercise; none of the packs contains safety-critical equipment or personal information.

Nadia and Joel use the existing method during a reference session. They later test a tray with separate text-labelled instruction slots. The same two people work for sixty elapsed minutes in each session. Each session starts with thirty unstarted requests: twenty standard and ten large. There are no new requests during the session, no opening partly assembled packs and no already completed packs to count.

The team uses the same written acceptance rules in both sessions. Every completed pack receives a final check. If a defect is corrected before the session ends and the pack then passes, it counts as accepted. The first failure remains in the quality record. Work accepted after sixty minutes does not count in that session’s output.

Both sessions include setup, assembly, checking, repair, recording and closeout within the 120 available person-minutes. The time allocations below are fictional observed totals, not estimates derived from the output counts. Nobody supplies extra preparation or an uncounted observer. The trial uses an equivalent offered mix, but the people have gained practice by the second session.

Observation Reference method Tray version A
Allocated standard requests 20 20
Allocated large requests 10 10
Accepted standard packs by close 18 19
Accepted large packs by close 6 8
Total accepted packs by close 24 27
Packs receiving a first check 24 27
Packs passing the first check 22 26
Packs failing the first check 2 1
Failed packs corrected and accepted by close 2 1
Unfinished requests at close 6 3
Setup person-minutes 10 18
Assembly, checks, repairs and recording person-minutes 106 98
Closeout person-minutes 4 4
Table: 7. Worked case: a preparation tray for community craft packs

The tray trial follows the prewritten prediction from Chapter 4. Its stop conditions are a third first-check failure, an unresolved request-to-pack identity mismatch, setup exceeding twenty person-minutes or a change in agreed working conditions that makes the comparison unusable. A pause means no new packs enter the tray method while the team identifies and contains the problem.

No stop condition occurs in this fictional session. One pack lacks an instruction sheet at its first check; it is corrected and accepted before close. The record also notes that preparing the labelled slots took longer than the old setup. Both observations matter when deciding what to try next.

Contents · Previous chapter · Next chapter

CHAPTER 8 OF 15

8. Read the arithmetic without hiding unfinished work

Back to contents

The reference session accepts 24 of 30 allocated requests, or 80%. The tray session accepts 27 of 30, or 90%. That is three additional accepted packs and a ten percentage-point increase in the share of allocated requests completed. The relative increase in accepted output is 3 divided by 24, or 12.5%. These are different descriptions of the same observed difference.

Both totals reconcile. Reference work is 24 accepted plus 6 unfinished, giving 30 allocated requests. Tray work is 27 accepted plus 3 unfinished, also giving 30. No extra completion is created when a failed pack is repaired. The two repaired reference packs are already included in the twenty-four accepted; the repaired tray pack is already included in the twenty-seven.

First-check quality uses another denominator. The reference session has 22 first-check passes out of 24 inspected packs, approximately 91.7%. The tray has 26 out of 27, approximately 96.3%. The unfinished work has not received a first check, so neither result establishes quality across all thirty allocated requests. Preserve the unfinished count beside the quality figure.

Look at the offered types as well. Standard-pack completion rises from 18 of 20 to 19 of 20. Large-pack completion rises from 6 of 10 to 8 of 10. The observed difference therefore is not entirely produced by completing more standard packs while leaving every large request behind. However, type counts alone cannot rule out differences in individual task difficulty or other session conditions.

The staffing and time denominators remain explicit: two people, sixty elapsed minutes and 120 available person-minutes in each session. The reference produces 24 divided by 2, or 12 accepted packs per available person-hour. The tray produces 27 divided by 2, or 13.5. These are session-level descriptive ratios, not a claim about either person’s performance or a forecast of a future working day.

Preparation effort increased. Setup uses eighteen rather than ten person-minutes, an increase of eight. The remaining activity time balances: 10 + 106 + 4 = 120 for the reference, and 18 + 98 + 4 = 120 for the tray. The higher accepted output occurs despite the extra setup in this example, but that does not show that setup will be worthwhile for a much smaller batch.

The proper arithmetic conclusion is narrow: under these recorded session conditions, version A produced more accepted packs with fewer observed first-check failures, while requiring additional setup. Interpretation begins with that statement and then asks what could explain it.

Contents · Previous chapter · Next chapter

CHAPTER 9 OF 15

9. Decide what the result does and does not support

Back to contents

The tray met the fictional prediction: at least twenty-seven accepted packs, no more than two first-check failures and setup within twenty person-minutes. That supports considering another bounded test. It does not establish that the tray caused the difference, that every future session will benefit or that the arrangement should immediately replace all other preparation methods.

The comparison consists of one reference session followed by one changed session. Nadia and Joel had additional practice by the second session. Minor differences in interruptions, materials or individual requests may remain. Equal staffing, elapsed time and offered pack types improve the usefulness of the description, but they do not remove every competing explanation.

Keep observation and interpretation separate in the review. “Twenty-seven packs were accepted” is an observation. “The separate slots reduced searching” is a possible explanation supported, at most, by the accompanying task notes. “This will raise output everywhere by 12.5%” is an unsupported forecast. A useful review allows the first statement, examines the second and refuses to make the third.

Ask whether the new method was usable without improvised rescue. If Nadia repeatedly supplied missing information from memory, the tray may appear complete while depending on her experience. The next test should examine that dependence through clearer written instructions or an ordinary agreed handoff. Do not turn the question into a test of someone’s worth or ability.

Also inspect what the preferred measure leaves out. The trial has not measured storage convenience between sessions, use with a different pack type or the effort of maintaining the labels over several weeks. These may be worthwhile future questions, but their absence does not need to be disguised. A pilot is allowed to produce a limited answer.

The decision can therefore read: “Keep version A available for another agreed preparation trial. Preserve the old method. Test whether the setup effort remains worthwhile for a smaller batch, using the same acceptance rules and complete time accounting.” This describes the evidence, the unresolved condition and a practical next move without announcing a universal success.

If the evidence instead points toward a different priority, return to Strategy: Choose the Next Useful Move. Continuing a pilot is one possible decision, not an obligation created by the effort already spent designing it.

Contents · Previous chapter · Next chapter

CHAPTER 10 OF 15

10. Stop, restore or adapt when the trial becomes unhelpful

Back to contents

Stopping a trial is an operational action, not a verdict on the people who proposed it. The purpose is to limit exposure, preserve the work and learn what made the proposed arrangement unsuitable. A useful stopping rule should therefore tell the team what to do next, not merely announce that a threshold has been crossed.

Imagine the tray trial had produced a third first-check failure. The agreed response would be to stop introducing new packs to version A, label the unfinished work and restore the original instruction arrangement. Completed packs would retain their checking records. The team would inspect and repair any affected unfinished packs using the normal acceptance process before counting them as accepted.

The record should state when the stop occurred and how much work had entered the trial by then. Do not merge output produced under the restored method into a full-session total attributed to the tray. You can report the overall session output separately, but the version change must remain visible. Otherwise an effective restoration could be misreported as evidence that the failed method worked.

Some problems call for a pause even before a numerical stop threshold. An unresolved identity mismatch, missing material or unclear instruction can invalidate the next decision. Waiting for a target number of errors would add exposure without answering the test question. The written plan should make these qualitative conditions as visible as the numeric ones.

Adaptation means changing the method in response to something observed. If a label is hard to distinguish, enlarge its wording or change its position. Name the new version and write a revised prediction. Avoid making several quiet repairs mid-session and later describing the original method as successful. The repairs may contain the most useful learning in the trial.

An unsuccessful prediction can still produce a valuable next question. Extra setup might overwhelm any assembly benefit in small batches. Instructions might be clear for standard packs but ambiguous for large packs. These findings point toward conditions to investigate; they do not require an elaborate story about why the original idea was secretly correct.

Close the trial with unfinished work accounted for, materials restored where needed and a short record of what changed. Without that closeout, the organisation can inherit a half-adopted method that nobody clearly selected. Restoration completes the learning exercise by returning ordinary work to a known arrangement.

Contents · Previous chapter · Next chapter

CHAPTER 11 OF 15

11. Widen the next test by one meaningful condition

Back to contents

Scaling is not a single leap from one pleasing session to universal use. A wider test should answer an uncertainty that matters to the intended setting. More items can be useful, but a bigger count under the same unusually favourable conditions may leave the central uncertainty untouched.

For the tray case, a smaller batch is a meaningful next condition because setup increased. The team could test twelve requests instead of thirty while preserving the pack definitions, checking process and time accounting. The question changes from whether the tray can support one full preparation session to whether its preparation burden is worthwhile for a shorter run.

A different ordinary preparation location could be another condition if space is part of the intended use. A different pack mix could matter if large packs require another instruction selection. Choose the condition because it tests a plausible weakness in the proposed arrangement, and describe what stays the same so the result remains interpretable.

Do not widen every dimension at once. A new room, different materials, revised labels and a larger batch would test a different package. Sometimes that package is exactly what the organisation needs to understand. If so, name it honestly, provide an appropriate restoration route and avoid pretending that the earlier tray result answered the new question.

Carry earlier observations forward. The extra setup should remain on the next sheet even if the team is now excited about output. The first-check definition should remain stable unless there is a reason to change it. If a definition changes, explain the break in comparison rather than combining the figures as though they measured identical things.

A sequence of favourable trials can support a more confident local decision when the conditions are relevant and the practical burdens are understood. It still does not make every future result certain. Ordinary use requires someone to maintain the arrangement, notice changed conditions and know how to revisit it when the former assumptions no longer hold.

That later question connects to Rebuild: Create a System That Can Last. The contribution of the pilot is specific: it leaves a tested description, a record of conditions, unresolved questions and a justified next decision that a more durable routine can use.

Contents · Previous chapter · Next chapter

CHAPTER 12 OF 15

12. Independent task: a shorter handoff for a display project

Back to contents

Try this changed situation before reading the answer. The fictional Cedar Display Group prepares written instructions for people arranging a community exhibition. It wants to test a shorter handoff sheet. A handoff counts as accepted only when its recipient checks that the location, item reference and arrangement instruction are present and mutually consistent. The task concerns fictional display materials, with no personal or sensitive information.

The reference session uses the current sheet. Two people have thirty elapsed minutes each, giving sixty available person-minutes in total. Sixteen requests are available at the start, with no new arrivals and no opening partly completed handoffs. Ten are routine displays and six are displays with an additional placement constraint.

The reference produces twelve accepted handoffs by close: nine routine and three constrained. All twelve receive a first check; eleven pass immediately and one is corrected before close. Four requests remain unfinished. The complete sixty person-minutes include setup, writing, recipient checks, the repair, recording and closeout. No outside preparation is supplied.

The shorter-sheet pilot has the same two people and thirty elapsed minutes, the same sixteen-request mix and the same acceptance rules. It produces fourteen accepted handoffs: ten routine and four constrained. All fourteen receive a first check; ten pass immediately and four fail. All four failed handoffs are corrected and accepted before close. Two requests remain unfinished.

However, the pilot also requires an additional helper to prepare the shortened sheets for ten minutes before the session. The helper does no other work on the pilot. The two regular participants still use their complete sixty person-minutes within the session. Thus the extra preparation is additional to, rather than included within, their recorded time.

Before starting, the group predicted at least fourteen accepted handoffs with no more than one first-check failure and no additional preparation help. It agreed to pause the shortened sheet after a second first-check failure. The record shows that the second failure occurred while work continued, and the team did not pause.

Write a short decision using these questions. What are the completion fractions? What are the first-check quality fractions? How much total person-time did each method use? Does the apparent output gain survive that resource comparison? Was the stopping agreement followed? What should happen to the shortened sheet now, and what narrower question could a revised trial answer? Keep the unfinished work visible and avoid blaming the people who discovered the missing information.

Contents · Previous chapter · Next chapter

CHAPTER 13 OF 15

13. Reasoned answer: the faster-looking sheet does not earn expansion

Back to contents

The reference completes 12 of 16 allocated requests, or 75%. The shortened sheet completes 14 of 16, or 87.5%. That is a 12.5 percentage-point increase in the share completed, and two additional accepted handoffs. The completion totals reconcile: 12 + 4 = 16 for the reference and 14 + 2 = 16 for the pilot.

First-check quality moves in the opposite direction. The reference has 11 passes from 12 checked handoffs, approximately 91.7%. The pilot has 10 from 14, approximately 71.4%. All failed handoffs are later repaired, but their repairs do not erase the failures. The accepted-output figures and first-check quality figures answer different questions and should appear together.

The reference uses sixty person-minutes, or one person-hour. Its descriptive output ratio is therefore twelve accepted handoffs per person-hour. The pilot uses sixty regular person-minutes plus ten helper person-minutes, giving seventy person-minutes, or seven-sixths of a person-hour. Fourteen divided by seven-sixths equals twelve accepted handoffs per person-hour as well.

Thus the observed output gain does not represent a gain in accepted handoffs per recorded person-hour. The pilot finishes more within the regular participants’ thirty-minute session, but its total resource boundary also includes extra help before that session. This does not prove that the shortened sheet has no possible benefit. It shows that the selected comparison cannot support a claim of improved labour productivity.

The prediction was not met as a whole. Accepted output reached fourteen, but first-check failures exceeded one and additional preparation help was used. The stopping agreement was also breached when work continued after the second failure. The group should record that departure explicitly rather than treating the four failures as an acceptable result because everything was eventually repaired.

A sensible immediate decision is to return to the original sheet for ordinary work while reviewing which required information the short version omitted or made ambiguous. Retain the pilot records and the corrected handoffs. Do not discard the evidence, repeat the unchanged method until it happens to look favourable or generalise the outcome into a judgement about the participants.

A revised trial might test one carefully rewritten field that preserves the placement constraint instead of shortening the entire sheet. Use an agreed bounded set, include all preparation time, retain the original acceptance criteria and act on the stopping condition. The new question becomes whether that field can be clearer without creating missing information elsewhere. That is a different, more informative test than simply trying to obtain fourteen again.

Contents · Previous chapter · Next chapter

CHAPTER 14 OF 15

14. Write the decision another person can pick up

Back to contents

A pilot should end with a decision record short enough to use and complete enough to prevent exaggeration. Someone who did not attend the trial should be able to understand what changed, which work was included, what was observed and what happens next. The record is a bridge between the session and ordinary practice.

Begin with the question and version description. Include the observation boundary: people, time, offered work, opening unfinished work and relevant task mix. Add any extra preparation or help. These details are not background decoration. They define the conditions under which the recorded result occurred.

Then state the prediction beside the observations. Keep accepted output, first-check quality, unfinished work and the chosen side effects separate. Include the denominators in the text rather than leaving a row of percentages that could be read in several ways. A small table is often enough, provided its categories are defined.

Give the result its appropriate level of certainty. “The tray session accepted twenty-seven of thirty requests under the recorded conditions” is useful. “The tray is proven to make the organisation 12.5% more efficient” is not supported by that exercise. Good decision writing preserves what the trial taught while leaving the next uncertainty visible.

Finish with one of four practical outcomes: restore, adapt, repeat under the same stated conditions or test a wider relevant condition. Name the reason and the person or group already responsible for the next ordinary step. If no further trial is justified, record that too. Finishing the learning exercise does not require keeping the idea alive.

You can use this compact form: “We tested ___ on ___, with ___ total time and ___ resources. We predicted ___. We observed ___ accepted from ___ allocated, ___ first-check passes from ___ inspected and ___. The main uncertainty is ___. We will now ___, while preserving ___.” Fill the blanks with observations and decisions, not promotional language.

For another starting point in the wider system, return to Recovery and Transformation. If the operational problem itself needs reconsidering, use the operational recovery guide. If you need to locate a restriction before choosing a change, use the process bottleneck guide. A small trial contributes most when it answers the right question and leaves a usable record for the next move.

Contents · Previous chapter · Next chapter

CHAPTER 15 OF 15

15. Source notes and the limits of these examples

Back to contents

IHI recommends small initial tests, written predictions, observed results and revisions before testing a change across wider conditions. Institute for Healthcare Improvement: Testing Changes.

IHI separates outcome, process and balancing measures so teams can inspect results, execution and problems created elsewhere. Institute for Healthcare Improvement: Establishing Measures.

These sources discuss improvement in a healthcare context. This article draws only on the stated general learning and measurement distinctions; it does not adapt their clinical examples or offer instructions for healthcare work. The community preparation and display exercises, quantities, thresholds, observation sheets and reasoned decisions above are original fictional teaching material.

The examples demonstrate how to keep claims proportional to a small operational observation. They do not establish a statistically validated effect, a universal batch size, an optimal staffing level or a guaranteed route to organisational success. Their practical output is a clearer test question and a more accountable decision about a limited piece of ordinary work.

Contents · Previous chapter · Return to recovery and transformation