Bias And Variability Key Points Revisited

To summarize the sections above, variability is a major obstacle in biomarker validation, regardless of the matrix, the type of biomarker, and its use. As noted above, there are two types of variability: intraindividual variability, which is usually related to lab techniques, sample timing, drug effects, and within- subject biological processes, and interindividual variability, resulting from different individual responses involving multiple genetic factors [ 40] . Biological variability may be difficult to assess, but it is important to control

TABLE 2 Fundamental Concerns in Biomarker Validation

Concern

Solutions

Overfitting

False positives and negatives

Increase sample size Statistical approaches Receiver operator

High sensitivity and specificity found but fail on independent validation sets characteristic curves Control for

Bias

Misidentification of differences between samples

Generalizability

Can results be applied to appropriate clinical populations?

confounding factors Representative validation cohorts for this factor. A statistical correlation to a clinical endpoint for a candidate biomarker cannot be determined without assessing biological variability, as the overall noise of a sample is a sum of both analytical and biological variability [6] . A biomarker with wide biological variability or time fluctuations that are difficult to control may be rejected [6]. Diurnal variability may require sample pooling or collection at the same time of day [ 6] . Biomarkers have diverse molecular structures, including possible bound states, which also need to be considered as influences on variability. There are specific considerations for any biomarker validation study. Overfitting, bias, and generalizability are three of the most fundamental concerns pertaining to clinical biomarker validation (Table 2) [41]. With the introduction of high-throughput discovery strategies, overfitting has become a particular fear. These discovery platforms are designed to measure countless analytes, and there is therefore a high risk of false discovery. When a large number of variables are measured on a small number of observations to produce high sensitivity and specificity, the results may not be reproducible on independent validation sets. Some biomarker candidates may be derived simply due to random sample variations, particularly with inadequate sample sizes [26]. A false positive may be thought of as critical part of a disease process, when in fact it is either associated only loosely or coincided randomly with disease diagnosis or progression [3]. As mentioned earlier in the chapter, a biomarker may correlate with a disease statistically but not prove to be useful clinically [8,42]. Increasing sample size and use of receiver operator characteristic curves may help overcome this concern of overfitting.

Bias is another major concern during biomarker validation, as there is often potential for misidentifying the cause of the differences in biomarkers between samples. Confounding variables such as age, race, and gender should be controlled for either through statistical modeling or validation study design to limit the effects of bias. Since validation cohorts require suitable diversity for widespread utility, bias may be difficult to avoid entirely through study design.

Similar to bias, most issues pertaining to the generalizability of a biomarker across clinical populations can be addressed through careful consideration of cohort selection. Cohort factors were discussed briefly above. To increase generalizability, the later phases of validation should include more rigorous testing of potential interfering endogenous components by including more diverse populations with less control of these confounding variables [22]. For example, in later-stage clinical trials there should be less control of diet and sample collection and more concomitant medications and co-morbidities [22]. This will allow a biomarker to be used in more clinically diverse situations.

Project Management Made Easy

Project Management Made Easy

What you need to know about… Project Management Made Easy! Project management consists of more than just a large building project and can encompass small projects as well. No matter what the size of your project, you need to have some sort of project management. How you manage your project has everything to do with its outcome.

Get My Free Ebook


Post a comment