Log in | Register | Contact Us | View Cart


No comments

Bias in Lean Data Collection

There are two ways to look at the term ‘bias’. Let’s start with the technical, statistical way. It is the systematic error component, or the difference between what the observed average is and what the actual average is. That just says, in a fancy way, that your sample or observation matches reality.

In some cases, you recognize that your system has bias. You might identify a problem where a measuring device reads consistently off. Perhaps you have a stop on a cutting device that has shifted, so the ruler adds a quarter inch to each piece that you are cutting. Or you might have a tape measure where the little metal tab is getting loose—and adds an eighth of an inch to every measurement. This is most commonly identified when another instrument that is accurate measures the same item and a discrepancy is noted.

It some cases, if you do not have an independent measurement, you may not realize your system has bias. There is a detailed, complicated mathematical process for calculating out the ‘bias of an estimator’, but its practical use is probably going to be limited to statistics experts supporting major continuous improvement efforts. They use some pretty intense math to determine if a sample contains a true representation of the real population (the group you are sampling from), or if something in the way samples are selected makes some groups more likely to be chosen.

The most common way (i.e. no math way) to address bias in continuous improvement is related to how data is collected. The methods people use to observe, measure, and record data might influence the results, and give an incorrect reading.

Anyone who has ever been pulled over by the police when they thought they were going the speed limit is familiar with systematic error. When I was in high school, a good friend of mine had a Camaro, and outfitted it with oversized back tires. The effect would be that his speedometer would read low, because every turn of his tires was a little bit longer than the speedometer thought it was. The systematic bias in the instrument, however, was known, so he could easily compensate by driving, say, 5 mph under the limit all the time. ‘Kentuck windage’ is another example of how to deal with systematic bias. If a rifle consistently shoots low and left, you can adjust the sights (preferred) or you can simply aim high and to the right.

In manufacturing, the most common type of systematic error will come from a problem with a machine—maybe a fixture gets bumped, and all of your products are being welded in the wrong place. In an office environment, perhaps you have the wrong sales tax entered in your system, and you are consistently overcharging each of your customers. In many industries, calibration is required to confirm that measuring devices are accurate, and many organizations have a Gage R&R process in place as well.

Sampling is common in both quality control, and in continuous improvement efforts. When you set up your sampling plan, you want to make sure it is truly random, so you get a representative view of what you are trying to check. For example, if your company runs two shifts, and you only pull samples from the day shift, you may not really be seeing what your output looks like. If you only sample when your process is ‘caught up’, you will miss possible issues that arise during ‘crunch time’. This is a surprisingly common practice during data collection—people stop recording data when things are frantic, shooting themselves in the foot. Data will look better than it really is.

Sampling on a repetitive pattern (i.e. every 10th one) has two issues. First, people can predict which items will be sampled, and the Hawthorne Effect takes over—for example, people tend to put more emphasis on the products they know will be checked. You also risk having products all sampled from a particular operator. If you have 5, or 10 people, every tenth product could come from the same person, depending on the setup of your production line.

Random selection by people isn’t ever really random. People have natural biases that will make them behave in a particular way—maybe a person is more likely to select crumpled forms than flat forms when doing a sample in an office.

Also avoid doing samples at the same time every day, or the same day of the week. There is a common belief that you should not buy a car produced on a Monday (people are still groggy), or a Friday (they are thinking about the weekend). Maybe your office cafeteria has turkey every Wednesday, and the team is groggy all afternoon. To get a good sample, ‘groggy production’ should not be overly represented in a sample, or results can look substantially better or worse than they really are.

In general data collection, bias occurs frequently as well. It may come in how a person does timing. They may be slower or faster than other people when starting the watch, they may adjust for the unusual issues (like someone coming up to a desk and asking a question or an operator looking for a screwdriver), or they may use different start and stop points each time. Unclear standards may contribute to bias when one person interprets something differently than another. One person may give products from their friend less scrutiny than others. There are lots of ways that this sort of bias creeps in—both intentionally and unintentionally. Review your process and make sure it makes sense. If the numbers do not match what you expected, make sure you understand why.

Examples of bias in data collection:

  1. A sampling process showed great results, but didn’t match what the company was hearing from its customers. Turns out, at the times when the sampling was being done, the last operator on the line was adding an inspection step and was pulling bad products off the line. It was a misguided attempt to make the line appear better.
  2. A paint inspection process in one company was very subjective. One inspector called a particular type of blemish a defect, and another accepted the product.
  3. In timing a process, one observer stopped the clock while the person was waiting for parts to come off of a machine, and another included that time. There are reasons for doing it both ways—just make sure that when you look at data, you understand how it was taken, and what it means.
  4. One person counted the startup waste (i.e. scrapped units) on a machine while another only started counting when the machine has been ‘dialed in’ for the day.


If you like our forms & tools, please help us spread the word about them!

Add a Comment

Share Your Thoughts    |No comments|

Practical Guide Info

Leave a Reply

You must be logged in to post a comment.

Copyright © 2009-2016, Velaction Continuous Improvement, LLC | Legal Information