Clients and prospective clients invariably ask about the validity of "I Opt." Since "I Opt" (www.iopt.com) is the only tool in Organizational Development that I know of which has been FULLY validated this question is always welcomed.
My typical response is to send them to www.oeinstitute.org and have them read the full study. Recently, I went back to the study. As I read it, it occurred to me that the study presumes the reader has a knowledge of statistics beyond that which most of my clients possess. This Blog is meant to define the dimensions of validity in a language that anyone can understand.
MOVING FROM SPECULATION TO "FACTS"
In final analysis, validity is an argument about whether your theory can be trusted. Validity dimensions are just points of evidence that you use to support your point of view on a subject. In the case of "I Opt" that subject is a theory of the behavior of groups and individuals.
A theory is just a statement of "what causes what." Theory is what you use to understand life. Theory is the basis on which you diagnose a situation. Theory is the foundation on which you base your recommendations that will change the lives of others. Theories are important.
Like everything else, theories come in gradations. At the bottom is speculation. This is an opinion and everyone has them. Like any other commodity with unlimited supply, they are not worth much. At the top are "facts." These are things whose truth is unquestioned by any reasonable person. Facts are usually in short supply and their price is the work you have to do to establish their truth. Most of the things we think we know exist somewhere between speculation and facts. Validity is a tool we use to move up on the scale toward "fact."
DIMENSIONS OF VALIDITY
CONTENT: This means that the components of the tool you are using (e.g., statements in the "I Opt" Survey) relate the theory in a meaningful way. If you do not have Content Validity you could be trying to measure distance with a thermometer. You will never know. "I Opt" has a Nomological Net. This tracks each "I Opt" response directly back to the underlying theory. The Net gives assurance that the "I Opt" tool is measuring what the theory is talking about.
FACE: The respondent (e.g., person who took the "I Opt" Survey) agrees with the diagnostic. This is often dismissed by the academicians. However, from a practitioner's standpoint, it is the most important measure of validity. If you do not have Face Validity you will be arguing with your client about the "truth" of the diagnostic you are trying to give. It is a great way to lose a client. "I Opt" has 99%+ face validity.
CONSTRUCT: This gives assurance that the measurement is right. If your tool measures literacy, a positive result would indicate that the respondent can read. If your tool says that the person can read and he cannot, you have measured the wrong thing. "I Opt" has a statistical significance in this dimension of p=.0152 . In other words, there is less than a 2% chance that this positive result could be found by chance. Academia is a bit more relaxed and usually takes 5% as their standard.
CONVERGENT: This dimension of validity assures you that things in the real world that should relate to each other do relate. For example, a tool that tries to measure "creativity" should be directly related to another tool that measures the frequency of novel responses. If they do not directly relate, you have probably measured something other than "creativity. " The "I Opt" validity study tested this along four dimensions and each met or exceed the academic p< .05 (less than 5% probability that the results are due to chance) standard.
DISCRIMINANT: This dimension of validity assures you that things in the real world that should not relate to each other do not. For example, the "styles" of people in R&D, Customer Service and IT would be expected to be unrelated. The different demands of these different areas would seem to require different "styles." If it turned out that your tool measured the same style mix in these different areas you probably do not have a style measure that can be used in work allocation. "I Opt" met the discriminant validity test with a significance level of p< .000000000000000000000000000000000001 or that there is less than 1 chance in a gazillion that this result was due to chance.
CONCURRENT: Concurrent validity says that the results (e.g., "I Opt" diagnosis) relate properly to other things outside of the theory being tested. For example, if someone scored high on an "introvert" scale in MBTI, we would not expect him or her to be found frequently dancing on tables with a lampshade on their head. If your tool scored a group as having high introversion, and someone else measured them high on a scale of "table dancing", you probably failed this test. "I Opt" has an inaccuracy rate of ZERO in its alignment with other measures. In other words, it fits with other things that it should fit with.
PREDICTIVE: This validity test says that the future can be predicted in a testable way. In other words, you should be able to foretell the future before that future happens with a high level of accuracy. "I Opt" was measured for six consecutive predictions and the worst that it did was p<.01 (less than 1% probability that the results are due to chance). On other tests "I Opt" registered an INACCURACY rate of ZERO. In other words, what "I Opt" says will happen does happen.
CONCLUSION: This test asks if the results are reasonable in the “real world” (sample size, statistically stable, etc.). In other words, can the results be trusted. For example, if you tested your tool on 30 students you may be able to meet all of the validity measures above. However, would you trust your results if that tool were applied to all of the million employees of WalMart? I would not. Or, you may apply parametric statistics (i.e., tests that use the normal curve) on data that has not been tested for "normality." You could argue the statistics are "robust." The question is if they are robust enough. No one will know. "I Opt" demonstrated its Conclusion Validity by using a large sample (over 14,000 people), enlisting 50 mature experts (VP's, Ph.D.'s, Directors of OD, Consultants and etc. It did not use inexperienced students) and applied rigorous statistical assessments. What this tells you is that your validity tests really mean something in practice.
RELIABILITY: This measures says that if you did it again, you get the same result. This is not a measure of validity as much as a measure of repeatability and consistency. "I Opt" again met and exceeded the academic standard.
SUMMARY
Validating a theory is expensive and time consuming. The investment only makes sense when you are working with a theory that you expect to be employed widely and applied in important areas of life. For example, if you are working on a program that will only be applied in your firm you can probably forgo validity tests. You can tell pretty quickly if it meets your needs and then figure out if it is worth the investment.
However, validation is a baseline requirement if you are offering a product that will:
(1) be applied to people you do not know
(2) in areas that you are unfamiliar and
(3) over timeframes that extend beyond your lifetime
Here you are asking people to put their faith in your theory and use it to alter the lives of other people based on its "truth." Under these conditions the time and expense of validation would seem to be a minimal requirement that should be provided by the builder and demanded by the buyer.
