"I Opt" Pattern Reliability Stress Test

By: Gary J. Salton, Ph.D.
Chief: Research & Development
Professional Communications, Inc.

SUMMARY
This study reports on the reliability of the “I Opt” patterns, predictable behavioral sequences. The study was conducted under “stress” conditions. The results are the “worst case” reliability condition that might be reasonably expected. The results of the evidence-based research show that “I Opt” is able to provide highly reliable pattern results even under adverse conditions.

A summary of this research is available on YouTube. Click the icon on the right to launch the video summary.


STUDY DESIGN
This study uses the same data as did the companion style research (for more detail see Salton, 2011). That study used a classic test-retest design. People retested using the same survey spaced minutes to days apart. The design was a stress test. People participating were trying to change their profile. This created a bias against style reliability. The result was a “worst case” level of reliability. The methodology outlined there applies equally here.

Style reliability considered only one variable—a strategic style. That research found that “I Opt” outperformed traditional instruments (e.g., Myers-Briggs®, DiSC®, 16PF®, FIRO-B®, etc.) by a wide margin. This study considers the reliability of styles working in unison. In other words, it considers the reliability of sequences of style behavior being applied to a particular life issue.


STRATEGIC PATTERNS
Global style level characterizations—single styles—are the limit of traditional instruments. They can talk about the use of different styles but cannot give a probabilistic estimate of the likelihood of their use. The reason is that they rely on rank order measurement. You cannot do arithmetic on rank order measures. You cannot divide “sometimes” by “often.” It does not matter if you assign numerals to them. If you try to do arithmetic you will still be dividing “sometimes” by “often.”

If you cannot do arithmetic you cannot calculate style probabilities. If you cannot calculate a style probability you cannot combine styles into any meaningful measure of joint probability. Joint probability is relevant when you want to figure out the likelihood that two or more particular styles will be applied to a particular issue. Since most of life is filled with these interdependent elections this represents a serious limit to traditional tools.

For example, you may launch a problem solving effort with a new idea. But you cannot just keep piling on new ideas. You are going to have to shift to analysis, assessment or some form of action sooner or later. If a theory cannot address that shift, it is going to miss most of life. This is not a good position for someone in the business of predicting and guiding human behavior.

“I Opt” is unique. It uses exact measurement. You can divide one “I Opt” score by another and get a meaningful result. This means that the four “I Opt” style axes can be expressed as probabilities. This alone puts “I Opt” in a league of its own. You can now speak with probabilistic certainty in place of vague, general narratives. But that is not the end of it.

The theory (i.e., what causes what and why) that underlies “I Opt” specifies the exact relationship between the styles. What this means is that the quadrants between the style axes have a specific behavioral meaning. Since each quadrant has a specific meaning (given by “I Opt” theory), the probabilities of every behavioral sequence can be calculated and predicted. These are called “strategic patterns.” They are called “strategic patterns” because they define a strategy (i.e., “plan, method, or series of maneuvers”, Random House, 2010) used to address life situations.

Graphic 1 illustrates the concept of pattern. It is the actual behavioral profile of the average of the 171 re-testers used in this study. It shows that this average person favors a “Conservator” pattern (30.7%). This is the combination of the Hypothetical Analyzer (disciplined assessment) and Logical Processor (rigorous execution) styles. This might be characterized as “Let’s think through our options and then methodically execute the option we choose” strategy.

Graphic 1
"I OPT" RE-TESTER PATTERNS
(n = 171)

Not every situations will yield to a Conservator strategy. When this happens our average person is likely to revert to a Perfector pattern (26.9%). This is a behavior sequence characterized “Let’s come up with some new ideas and then think them though.” If that does not work, the next likely option is the Performer Pattern (22.6%). This is a “Let’s get it done—right if we can, anyway if we have to” strategy.

The reason we were able to predict the likely outcome for the average re-tester in our sample is that “I Opt” could use arithmetic and had a theory to guide its application. This kind of insight is outside of the capacity of the traditional tools. If you cannot use arithmetic, you cannot calculate the probabilities we used to decipher the likely behavior of our composite re-testers.

This brief background demonstrates that “I Opt” stands as the lone member of a new class of tools. What this means is that there is nothing to which “I Opt” can be reasonably compared. Since there is no point of comparison, the reliability of the pattern measurement must rely on absolute measures. If these measures meet the needs of the issues being addressed “I Opt” pattern reliability can be accepted.


PROFILE RELIABILITY
An “I Opt” profile is a representation of all of the styles and patterns considered simultaneously. It describes entire behavioral map that a person will use to navigate all of life’s situations. Any shift in style strength will redistribute the likelihood of all combination's of style use. In other words, the entire profile will also shift. This means we can measure global changes in behavior by comparing one profile (e.g., test) with another (e.g., retest).

Graphic 2 compares the original test with the retest profile for all 171 retest surveys. It shows a minor net change in the overall profile of the retest group. Some individual test-retest surveys did change. However, there was no overall directional change. Individual changes tended to offset each other. As we will find out later, this is no accident.

Graphic 2
COMPARISON OF ORIGINAL AND RETEST PROFILES
(n = 171)

The “I Opt” profile reliability of groups is stable. Policy decisions involving the prediction of group patterns can be relied upon. “I Opt” technology will provide a firm foundation for large scale initiatives. Proposals concerning mergers, acquisitions, policy initiatives and the like will be grounded on a firm factual base.


INDIVIDUAL PATTERN RELIABILITY
Stable profiles can be composed of offsetting individual changes. This is of little comfort for the practitioner working with individuals and smaller groups. For them individual pattern variability is of prime importance.

Several methods can be used to assess individual variability. One is to focus on the dominant pattern. This method treats a pattern as a category. It ignores the degree of change. The only measure is whether rank order of the dominant pattern has changed. It does not matter if the dominant pattern exceeds the secondary by 1% or 50%. This method is inexact but is easily understood and this quality has much merit in field settings.

Graphic 3 shows the stability of strategic patterns (i.e., behavioral sequences) in the stress test sample. A majority of retests did not change dominant patterns. Fully 66% remained stable even under stress conditions.

Graphic 3
DOMINANT PATTERN DISTRIBUTION AMONG 171 RE-TESTERS

It is worth examining just what the “worst case conditions” of the stress test entail. People often retested within minutes of their original test. Many did so with the conscious intention to change the results. They knew the results of the original test and probably remembered their initial responses. This made “engineering” a change easy. Graphic 4 illustrates one such case.

Graphic 4
INDIVIDUAL CHANGE IN DOMINANT PATTERN
RETEST WITHIN 2.3 HOURS
(139 Minutes)

Graphic 4 clearly shows a manipulative result. A profile change of this character requires a person to choose multiple responses that directly contradict their original position. This kind of change in strategic posture does not happen within 2 hours between test and retest. Other similar results happened in as fast as 5 minutes. And this is not all that was confronted in the stress test.

Many retests involved only slight changes. While small, these were enough to flip the dominant style from one category to another. Graphic 5 illustrates a change in dominant pattern resulting from an individual answering 1 statement differently. It was enough to flip this person from a “Perfector” to a “Changer” pattern.

Graphic 5
INDIVIDUAL CHANGE IN DOMINANT PATTERN
RETEST WITHIN ONE DAY

The kind of change illustrated by Graphic 5 probably has no practical consequence in any kind of organizational diagnosis. In total, 16 of the 58 people who changed their dominant pattern still maintained an overall profile overlap of 66% or more between their test and retest. This profile consistency means that their responses just “wiggled.” The relatively minor nature of this high overlap condition is visually illustrated in Graphic 6.

Graphic 6
INDIVIDUAL RE-TESTER EXAMPLE
TEST-RETEST PROFILE OVERLAP OF ~70%

Graphic 7 shows what happens if both the obvious manipulative and minor changes were discounted from the study. Removing the 23 obviously distorted surveys cause the dominant pattern repeatability rate to jump from 66% to 76%. And there is still stress left among those that remain.

Graphic 7
“I OPT” PATTERN RELIABILITY
WITH 23 UNREPRESENTATIVE SURVEYS REMOVED

Even without a specific comparable to act as a standard, the “I Opt” pattern reliability rate clearly exceeds the generally accepted norms of the field of organizational research. This judgment can be reasonably inferred by comparing pattern reliability to the style reliability found in the companion research (Salton, 2011). This is done in Table 1.

Table 1
STYLE VERSUS PATTERN RELIABILITY

As expected, “I Opt” pattern reliability is somewhat less than “I Opt” style reliability. This is because patterns can be affected by more things. The sum of all patterns must add up to 100%. A change in any style will cause its associated pattern percentage to increase or decrease. This means that all of the remaining patterns must shift if they are to continue to add up to 100%. Since more variables can affect patterns, they are inherently less repeatable than are styles.

Overall, “I Opt” patterns compare very favorably with the reliability measures posted by other generally accepted tools in the field of organizational research. And this is under stressed conditions. In actual practice much higher levels of reliability can be reasonably expected. In sum, “I Opt” pattern reliability is meets or exceeds any standard of acceptability.


DIRECTION OF CHANGE
The design of the stress test offers an opportunity to examine the nature of the pattern changes that occurred. Table 2 shows both the patterns that are being moved from (i.e., test) and those that are being moved to (i.e., retest).

Table 2
DIRECTION OF PATTERN CHANGE
(n = 58 re-testers who changed style)

The Conservator pattern stands out at a 40% change rate (far right column). Other patterns each account for roughly 20% of the 58 pattern changes. This suggests that Conservators are the most motivated to attempt to change their reported “I Opt” pattern.

The natural design of the stress test did not allow for interviewing participants. It may be that Conservators were trying to better align themselves with the more socially attractive pattern. Or their inherently skeptical posture may have created more doubt as to the accuracy of the assessment. But whatever the reason, it is clear that they were certainly trying harder.

The “changed to” percent (bottom row) is notable. It shows that there is no overall direction to the change. This is due to the structure of the “I Opt” survey. The survey gives no clue as to the diagnostic consequences of a particular choice. This is evidenced by the random distribution of “changed to” results. In a random situation each pattern has about an equal probability of occurring. That is exactly what the bottom row shows.

Pattern recognition is a typical strategy for attempting to engineer survey outcomes. Experience has shown that test takers will initially examine a survey in search of patterns. If highly motivated re-testers with full knowledge of initial results (the LPs in Table1) cannot find a pattern, it is extremely unlikely that someone taking an initial survey will find one.

A reasonable conclusion to this section is that professionals using “I Opt” technology can trust the initial diagnosis. People are unlikely to be able to “figure out” the survey. The result is that they tend to give an honest assessment of their status. That is what the original Validity Study (Soltysik, 2000) found a decade ago and that is what this study has confirmed.


TIMING EFFECTS
The structure of the survey prevents the respondent from predicting the direction of any change. However an unpredictable change can be caused just by answering differently. To answer differently the initial response has to be remembered. A short time between test and retest improves the odds that the original responses will be remembered.

If people choosing to retest were honestly reflecting a different state we should expect to find no time dependent difference in outcomes. If people were attempting to manipulate the results there should be a significant difference by time. People retaking the test sooner would remember their initial responses better that those more distant from it. Table 2 shows the results of this test.

Table 3
EFFECT OF RETEST TIMING ON RETEST RESULTS

The fact that surveys taken days apart differed significantly from those taken hours-apart (p<.05) suggests a concerted effort to change results. The shorter the time between surveys, the more divergent were the before and after profiles. This was not honest re-evaluative effort. It is evidence of conscious manipulation. The “stress” embedded in the stress test is real. The re-testers were really trying to change the results.

The implications of this finding are material. Even when trying to change the results a majority of the re-testers were unable to do so. This suggests that “I Opt” is able to deliver accurate results even under the most adverse of circumstances. Scholars and professionals can trust an “I Opt” diagnosis. Such trust is well placed.


SUMMARY
This pattern research has confirmed the unique standing of“I Opt”. Being able to probabilistically predict sequences of behavior is alone enough to distinguish it from traditional tools. This judgment is reinforced by the fact that the dominant pattern reliability was able to outperform the less challenging style reliability posted by traditional tools. And it did this under adverse test conditions. Real world experience is likely to be much better.

The analysis of the direction of change among retesters demonstrated the integrity of “I Opt.” The “I Opt” survey offers no clues on the likely result of a particular choice. This means that the only viable strategy available to the test taker is to follow some semblance of their true posture on choices offered. This means that the scholar or professional administering the “I Opt” survey can trust the results.

The study also demonstrated that re-testers were really trying to bias the outcome. The fact that so few of them succeeded is further evidence of the integrity of the “I Opt” instrumentation. Scholars and professionals can rely on “I Opt” diagnosis even in situations where the respondents may be less than fully cooperative.

Overall, the both stress tests (i.e., style and pattern) have confirmed and extended the results of the original validity study of over a decade ago. “I Opt” offers an evidence-based opportunity to both broaden and deepen our understanding of behavior in real world situations. With that will come an improvement in the human condition.


TRADEMARKS
® IOPT is a registered trademark of Professional Communications Inc.
® MBTI, Myers-Briggs Type Indicator, and Myers-Briggs are registered trademarks of the MBTI Trust, Inc.
® FIRO-B is a registered trademark of CPP, Inc.
® DiSC is a registered trademark of Inscape Publishing, Inc.
® 16PF is a registered trademark of the Institute for Personality and Ability Testing, Inc.


BIBLIOGRAPHY
Random House Dictionary, Random House, Inc., 2010.
Salton, Gary (2011), “IOpt Style Reliability Stress Test”, http:\\garysalton.blogspot.com.
Soltysik, Robert (2000), Validation of Organizational Engineering: Instrumentation and Methodology, Amherst: HRD Press.

"I Opt" Style Reliability Stress Test

By: Gary J. Salton, Ph.D.
Chief: Research & Development
Professional Communications, Inc.

SUMMARY
A natural experiment offered an opportunity to test “I Opt”reliability. It used a “worst case possible” design. The experiment was biased AGAINST "I Opt" reliability. The outcome was compared to industry standards. The standards accepted were the most favorable reported by those that had a vested interest in these traditional tools. The worst possible outcomes of “I Opt” were compared the best reported reliability results of alternative tools. This created a natural “stress” test.

The study found that the worst “I Opt” results exceeded the best results of alternatives. These results give the practitioner and scholar confidence that “I Opt” is a tool that can be relied upon even in difficult field situations. You can access a video summarizing the research by clicking the icon to the right.


A NATURAL EXPERIMENT
A program to increase the visibility of “I Opt”® technology created a natural experiment. Random people were offered a free Advanced Leader, Career or Emotional Impact Management Report. They could take the “I Opt Survey” on-line without user codes or passwords. The use was anonymous (i.e., fake names were an option). The report was automatically generated and sent to any email address designated.

An accompanying email invited people to use it as they wished. They could retake the survey without penalty. Table 1 outlines common reasons for retest.

Table 1
POTENTIAL REASONS FOR RETEST

The reasons cited in Table 1 involve TRYING to change original outcome. Most retest protocols seek to eliminate this possibility. They try to insure that motivations and conditions are constant between test and retest. This creates a bias toward consistency (i.e., reliability). This experiment does just the opposite. It burdens “I Opt” with a bias towards inconsistency (i.e., unreliability).

Thus the structure of the experiment acts as a stress test. It measures “I Opt” reliability under the "worst case" conditions. Passing this stress test offers strong evidence of “I Opt” technologies inherent reliability.

A COMPARATIVE FRAMEWORK
Any test requires a standard of judgment. The natural standard would be the results of the reliability studies of comparable tests. The most stringent would be results published by those with a vested interest in the success of these tests. Accepting this standard takes the “stress” of the stress test up to the another level.

The Center for the Application of Psychological Type (2011) reports that using MBTI® “on retest, people come out with three to four type preferences the same 75-90% of the time.” That means that the best that can be expected under controlled conditions is 90%. In addition, this result applies to only “three to four” of the 16 possible type preferences (i.e., ESTJ, INFP, etc.). That means that an ESTJ could be retested as a STJE and still qualify as successful retake—3 of the 4 stayed the same. A practitioner who had to explain to a client why their dominant style changed might view this as less than a “success.”

The Consulting Psychologist Press, the publisher of MBTI®, does not cite reliability data on their website. However, a book published by the organization does cite results (Harvey, 1996). About 50% of people tested within nine months remain the same overall type, and 36% remain the same type after more than nine months (Wikipeida, 2011). Averaging MBTI results gives a overall standard of about 63% (i.e., average 75%, 90%, 36%, 50%).

Internet research on DiSC® provided no simply described evidence on test-retest reliability. However, Inscape Publishing (the publisher of DiSC) does provide a table of correlation coefficients (Inscape Publishing, 2005). That table reports correlation coefficients of between .89 and .71 depending on the time between retests. The timing was ~1 week (n=142), 5-7 months (n=174) and 10-14 months (n=138).

A correlation coefficient measures the difference between things, not the things themselves. To make it meaningful it has to be converted. Squaring the correlation coefficient (e.g., .89 x.89) does this. The result is called the Coefficient of Determination or r2. (called "r squared" - see Biddle, p.14). Applying this to the highest DiSC correlation yields a r2 of 79%.

The same method can be applied to the lowest DiSC correlation reported. The corresponding r2 would be 50% (i.e., .71 x .71). Averaging all of the correlation coefficients reported by Inscape Publishing (12 in total) yields an overall correlation coefficient of .763. Squaring that to gives a meaningful Coefficient of Determination of about 58%. Overall, DiSC can be expected to retest differently about 42% of the time (100% -58%).

FIRO-B® is published by CPP, Inc. On their website (CPP, 2009) they report a test-retest reliability as “ranging from .71 to .85—for three different samples as reported in the FIRO-B® Technical Guide (Hammer & Schnell, 2000).” Using r2 this translates into an expected test-retest success rate of between 50% and 72%. Averaging these numbers gives an overall retest consistency of 61%.

The Sixteen Personality Factor or 16PF® publishers (IPAT, Inc.) do not cite reliability statistics on their site. However, Cantrell and Mead in the Sage Handbook of Personality Theory and Assessment do quote statistics. These were taken from the 16PF Fifth Edition Technical Manual. The Institute for Personality and Ability Testing—the predecessor of IPAT—published this manual. They report a 2-week test-retest reliability of .8 and .7 over a two-month interval. This translates to r2 percentages of 49% and 64% for an average r2 of about 57%.

There are many more personality tests of this character. However, Table 2 shows a developing a pattern.

Table 2
OVERALL TEST-RETEST RELIABILITY ESTIMATES
REPEATABILITY PERCENT

All of the instruments appear to approximate 60% test-retest repeatability. Since these results were published by organizations with a vested interest, it is a very high standard. It is likely that these organizations have published the most favorable rates available.


DATA SOURCE
Participants accessed the free reports via an internet connection. The internet server used in this experiment recorded the timing, score and email address of users. Table 3 is an outline of origins of the sample from the server data.

Table 3
SAMPLE CHARACTERISTICS

The diversity of origins suggests that this is a fair sample of the universe of potential “I Opt” users. It is unlikely that there is a selection bias that might contaminate the results (e.g., all college students, all members of a single firm, etc.).


RETEST PROFILE
Participants could choose to rerun a report at any the time. The service was entirely automatic. People could retake the survey without any worry about having to defend the retest to an administrator. The response could be anonymous giving a further level of comfort. Users were effectively unconstrained.

Graphic 1
RETEST COMPARED TO TOTAL SURVEYS

Graphic 1 shows the usage. A total of 6,298 reports were run. There were 171 retests—a 2.7% retest rate. This is a very low rate given the multiple possible reasons for retest (see Table 1), the ease of access and the penalty free nature of a retake.

This result confirms the high “I Opt” face validity found in the original validity study (Soltysik, 2000). Face validity is an “unscientific” measure of validity. However, reliability is not a measure of validity. Reliability is a measure of consistency. It is meant to provide assurance that you are not using a “rubber ruler.”

Most people did not choose to retest even though they could do it with ease. This suggests that they found the results consistent with their internal estimates. In other words, the low retest rate can be viewed as evidence of the reliability judgment of the participants as measured by their own internal standards. At 2.7% it is very high.

The diversity of data sources (Table3) indicates that there is little likelihood of an external selection bias (e.g., all college students). However, a question could arise on whether there is a particular “I Opt” style inclined to take a retest? The answer is no.

Graphic 2 shows the profiles of re-testers (n=171) are a mirror image of the general test taker (n=6,298). Statistical tests confirm that there is no significant difference (p<.05) in any “I Opt” dimension. The motivation for retesting does not reside in the “I Opt” style. Thus the chance of auto-correlation confounding the results is minimized.

Graphic 2
STRATEGIC STYLE DISTRIBUTION
RE-TESTER INITIAL SCORES versus ALL PARTICIPANT SCORES
(n = 171 versus n=6,298)

The sample size is large and diverse and is a fair representation of people likely to use “I Opt.” The number of retests (n=171) is enough to give meaningful insights. The “mirror image” profiles between all testers and re-testers mean results are unlikely to be confounded by this dimension of auto-correlation. Finally, self-selected retesting means that all of the possible motives (Table 1) can operate thus maximizing the “stress” in the stress test. The study rests on a firm foundation.


RETEST TIMING
The time between test and retest is relevant to the stress test. Short time periods maximize the chance producing an inconsistent result. Over short time periods people are likely to remember their responses to the original survey. If the motive is to change the result (see Table 1) a short retake cycle makes this much easier. Table 4 shows the retest timing of the experiment.

Table 4
TIMING OF RETEST

Fully two-thirds of people retested almost immediately. This is a strong indication that they wanted to explore variation in the results. This reduces the likelihood of consistency (i.e., reliability). This short-cycle retake further increases the “stress” of the stress test.

RETEST STYLE RESULTS
A person’s dominant style is the practitioners’ most important measure. It is the one that the client is likely to see as characterizing their behavior. The last thing a practitioner wants is to argue with a client over a discrepant result.

Other tools (i.e., MBTI, DiSC, Pf16, etc) do not specify dominant style repeatability rates. Rather, they tended to mix all of the styles (i.e., primary, secondary, peripheral, etc.). This strategy implies that all had equal importance. If the dominant style had fared better it is likely to have been celebrated. It was not. The “I Opt” stress test does not avoid dominant style visibility as is shown in Graphic 3.

Graphic 3
CHANGED VERSUS UNCHANGED
DOMINANT STYLES ON RETEST

In spite of a strong bias against consistency fully 74% of the “I Opt” retest surveys yielded exactly the same dominant style as obtained in the initial test. This substantially exceeds the implied ~60% repeatability of the other “non-stressed” tools.

Graphic 4
CHANGED VERSUS UNCHANGED
“DETERMINED” vs. REGULAR RE-TESTERS

Graphic 4 shows a deeper examination of the 26% that changed styles. It further improves the outcome. Eighteen of 45 people who changed dominant style took the survey 3 or more times (for a combined total 48 surveys). Ultimately, 14 of these 18 “determined” people (77%) finally managed to change their primary style. If these 14 people were removed on the basis of gross distortion the repeatability rate would jump from 74% to 81%.

Table 5
OVERALL TEST-RETEST RELIABILITY ESTIMATES
REPEATABILITY PERCENT

Table 5 shows that whether considered in its raw (74%) or refined (81%) form, “I Opt” clearly passes the stress test. It exceeds the ~60% average repeatability standard. It accomplishes this even with a experimental design heavily biased against it.


SUMMARY
The natural experiment arising from an “I Opt” visibility program provides strong evidence of the inherent reliability of “I Opt” technology. This study confirms and extends the similar findings of the original Validity Study (Soltysik, 2000) of over a decade ago.

The result was that “I Opt” substantially exceeded the average reported reliability of traditional tools used in the field under heavily “stressed” conditions. It is reasonable to judge “I Opt” technology to be the most reliable tool available in the field. If there is an equal or superior, it has yet to make itself visible.

TRADEMARKS
® IOPT is a registered trademark of Professional Communications Inc.
® MBTI, Myers-Briggs Type Indicator, and Myers-Briggs are registered trademarks of the MBTI Trust, Inc.
® FIRO-B is a registered trademark of CPP, Inc.
® DiSC is a registered trademark of Inscape Publishing, Inc.
® 16PF is a registered trademark of the Institute for Personality and Ability Testing, Inc.

BIBLIOGRAPHY
Biddle, Daniel (Publication date not provided). Retrieved from http://www.biddle.com/documents/bcg_comp_chapter2.pdf, January 1, 2011.

Cattell, Heather and Mead, Alan (2000).“The Sixteen Personality Factor Questionnaire (16PF)” in The SAGE Handbook of Personality Theory and Assessment: Personality Theories and Models (Volume 1). Retrieved from http://www.gl.iit.edu/reserves/docs/psy504f.pdf, January 1, 2011.

Center for the Application of Psychological Type, 2010. “The Reliability and Validity of the Myers-Briggs Type Indicator® Instrument” Retrieved from http://www.capt.org/mbti-assessment/reliability-validity.htm, January 1, 2011.

Conn, S.R. and Rieke, M.L. (1994) The 16PF Fifth Edition Technical Manual. Champaign, IL: Institute for Personality and Ability Testing.

CPP (2009). Retrieved from https://www.cpp.com/products/firo-b/firob_info.aspx, January 1, 2011

Harvey, R J (1996). Reliability and Validity, in MBTI Applications A.L. Hammer, Editor. Consulting Psychologists Press: Palo Alto, CA. p. 5- 29.

Inscape Publishing (2005). DiSC Validation Research Report. Inscape Publishing, Minneapolis, MN. Retrieved from http://www.discprofile.com/downloads/DISC/ResearchDiSC_ValidationResearchReport.pdf January 4, 2011.

Schnell, E. R., & Hammer, A. (1993). Introduction to the FIRO-B in organizations. Palo Alto, CA: Consulting Psychologists Press, Inc.

Wikipeida (2011). “Myers-Briggs Type Indicator.” Retrieved from http://en.wikipedia.org/wiki/Myers-Briggs_Type_Indicator#cite_note-39, January 4, 2011.

Soltysik, Robert (2000), Validation of Organizational Engineering: Instrumentation and Methodology, Amherst: HRD Press.

The Nursing MS Degree in Management

By: Gary J. Salton, Ph.D.
Chief: Research & Development
Professional Communications, Inc.


INTRODUCTION
This research blog investigates whether students attending Second Career and traditional Master of Science programs are equal sources of nurse management talent. The research finds that program participants are different and will appear so to observers. But they are virtually identical in their ability to provide “management ready” talent.

The research then compares nursing with people pursuing a master’s degree in other professions. It finds that nursing MS programs provide less than half as much managerial perspective to the talent pool than do other professions.

Finally, a Migration Strategy of offsetting the shortage of nursing is offered. The strategy can be applied to any nurse (AA, BS or MS) and provides a non-threatening, measured option for both the nurse and the medical institution. This strategy is more fully specified in an Addendum to this research blog.


DIFFERENT KINDS OF MS GRADUATES
There are two major programs producing nurses with MS degrees. The traditional program admits nurses who have completed undergraduate nursing programs. The Second Career MS program admits students with who completed their undergraduate degree in other fields.

Data is available from 29 students completing Second Career Master of Science (MS) degrees and 81 students in a traditional MS program at a major research university. Graphic 1 shows that the students in the two programs are statistically different along two dimensions.

Graphic 1
SECOND CAREER AND TRADITIONAL
MASTER OF SCIENCE PROGRAMS

Traditional students put more reliance on the idea-oriented RI strategy. The Second Career students put greater emphasis on the disciplined action of the LP style. However, this is not the relevant test for the issue at hand. That issue is how well the two group profiles match the needs of nursing management.

Graphic 2 shows only one statistically significant difference between Second Career students and existing management. Second Career students tend to use the innovative RI strategic style less than does existing management.

Graphic 2
SECOND CAREER AND TRADITIONAL MASTER OF SCIENCE
STUDENTS vs. ESTABLISHED NURSING MANAGEMENT

The difference in innovation based RI is large enough for both educators and employers to notice it. However, no single strategic style determines overall managerial “fit.” That requires considering all of a person’s strategic styles simultaneously.

To test the overall “fit” a composite profile was constructed by averaging the “I Opt” scores from all hospital management levels (from CNO to Assistant Nurse Manager). Student profiles falling within 30% of this standard were deemed to share management’s information processing perspective. They are likely to approach issues in about the same manner as existing management. Effectively, they can be seen as “management ready.”

Graphic 3 shows the MS Program participants who lie within 30% of the management standard. The circles (i.e., centroids) are Cartesian averages. They locate a point of central tendency along all four of the “I Opt” styles simultaneously. Blue circles are the Second Career students, the yellow are the traditional. The red circle is the composite management centroid.

Graphic 3
SECOND CAREER AND TRADITIONAL
MASTER OF SCIENCE STUDENTS SCREENED
BY 30% MANAGERIAL CANDIDATE CONVENTION

The two types of MS programs appear to be functionally equivalent. Variation in some styles is compensated for by differences in others. The dispersion of both student groups is roughly equal.

Table 1
PROPORTION MS STUDENTS WHO
DEVIATE 30% OR LESS FROM THE
EXISTING MANAGEMENT PERSPECTIVE

Table 1 below reinforces equivalence. It shows that both programs are virtually identical in the depth of talent they provide. About 17% of the people in both programs have an “I Opt” profile that “fits” with the existing management. For managerial assessment purposes, the two programs can be treated as a single entity.



ADEQUACY OF THE “MANAGEMENT READY” NURSING POOL
Having two MS programs able to supply management talent is to be welcomed by the profession. However, the adequacy of the absolute size of the management pool merits investigation.

One method of testing adequacy is to compare nursing MS students with master degree candidates in other professions. A non-nursing average management standard was constructed using 4,945 executives from all industries and areas. The positions sampled were from General Manager through supervisor. The “I Opt” profiles of these executives were averaged to arrive at a non-nursing management standard.

A total of 611 masters’ candidates in disciplines such as engineering, business, computer science and manufacturing science from five universities provided a non-nursing sample. These students will typically fall under the supervision of the management identified as the standard. Students falling within a 30% range of the non-nursing “all management” standard are shown in Table 2.

Table 2
NON-NURSING AND NURSING MASTERS CANDIDATES
SCREENED BY 30% MANAGERIAL CANDIDATE RULE

The results are striking. Nursing has less than half of the depth of “management ready” masters’ candidates. One cause might be a difference in the standard being used. In other words, nursing might have a management standard (represented by the centroid of the average manager) more challenging than that of the other professions. Graphic 4 addresses this possibility.

Graphic 4
NURSE AND NON-NURSE ENTRY LEVEL SUPERVISORS INFORMATION PROCESSING PROFILES

Statistical tests confirm the obvious. There is no statistically significant difference between the two management groups. In information processing terms, nursing management could move to industry and nobody is likely to notice the difference—and vice versa

If the profiles of nurse/non-nurse management are the same and the methodology is the same, the character of people being attracted to nursing MS programs must be different. This is exactly the case. But the difference is not obvious. It requires the exact measurement capabilities of “I Opt” technology to lay the reason bear.

Graphic 5 shows that there are statistically significant differences between nursing and non-nursing masters’ candidates. The nurses are more idea-oriented (RI) and fall a bit short in their inclination toward analysis and assessment (HA). However the size of the differences do not appear to be enough to account for nursing having 50% fewer “management ready” candidates.

Graphic 5
NURSE AND NON-NURSE
MASTER OF SCIENCE STUDENTS

If averages cannot account for the divergence the answer must be in the distribution of students. This is exactly the case. Graphic 6 shows the centroid distribution of both nursing and non-nursing masters’ students. The non-nurse portion of the graphic uses a 110 person random sample drawn the 611-person non-nursing students. This makes the non-nursing group visually comparable to the 110 nurse MS population. There is no need to make mental adjustments for different size samples.

A quadrant by quadrant comparison reveals that the nurses are more widely scattered than their non-nursing counterparts. Nursing is apparently more hospitable to and thus attracts a wider range of perspectives than do other professions. The compassion that drives many nurses is more widely spread that are the mathematical capabilities of engineers or the logic of the computer scientists. This is as it should be in a healing profession.

Graphic 6
NURSE AND NON-NURSE MASTERS CANDIDATE
"MANAGEMENT READY" DISTRIBUTION

The effect of the dispersion of MS nurses is seen in the magnification. The circle in the center shows the number of people falling within 30% of the respective management standard (i.e., the green and red circles). Even though the sample size is the same, there are twice as many yellow circles among the non-nursing professions. The position of the management centroids differs slightly. But the wider ranging “I Opt” profiles among the nurses’ accounts for most of the dispersion.

IMPLICATIONS
Table 3 compares nurses in the MS programs with general staff nurses (including both graduate and non-graduate nurses).

Table3
NURSING MS CANDIDATES Vs STAFF NURSES
SCREENED BY 30% MANAGERIAL CANDIDATE RULE

The MS offers a small increase in the pool of “management ready” talent. But the MS degree does not serve as a strong management filter. Since the nursing MS is targeted primarily at providing talent for the various nursing specialties, this is not an unexpected result.

However, nursing management is itself a specialty. Earlier studies (Staff Nursing Paradox and The Nurse Management Staircase) have shown that it demands a unique perspective. That perspective carries with it skills that are not widely shared. The exercise of these skills (or absence of them) effects such important areas as nurse retention, quality, efficiency and effectiveness. This is not a minor matter.

Simply attaching more standard “management” courses to the nursing curriculum is unlikely to have any effect in adding to the nurse management pool. Adding familiar course content focused on techniques, processes or organizational theory is unlikely to have an effect.

One reason is that the problem is not technical knowledge, it is in management perspective. Trying to address hospital level problems with the detailed orientation of a staff nurse is predestined to failure. Equally, trying to deal with the mechanics of a ward using the expansive hospital level thinking is likely to create a degree of very visible chaos. It does not matter how well the techniques used to apply these misaligned perspectives are executed.

Leadership training is also unlikely to remedy the condition identified here. The managerial perspective revealed by these nursing studies is not confined to leadership. It applies how problems are defined, the meaning of terms (e.g., “fast” means different things to different styles) and the “right” way to address an issue. All of these things and more are precedent to leadership. They define the direction that leadership will take. Adding skills on how to execute that direction will do nothing to address the fundamental issue of what that direction should be.

This research blog indicates that the dearth of management talent in nursing is going to persist. Nursing schools are unlikely to fill the gap. It is doubtful that students better aligned with a management perspective could be attracted in any appreciable numbers. A program to show nurses how to prepare themselves could help (see Migration Strategy below) but its effects in appreciably increasing the management talent pool will take many years to realize. Medical institutions will probably have to rely on themselves to grow the talent that they need.


THE MIGRATION STRATEGY
The interests of hospitals are probably best served by helping existing nurses who want to enter management to realize their aspirations. Standard management programs can teach them techniques and processes. What is needed is a method of aligning their information processing perspective with that of management. This does not happen automatically.

Unlike psychological states, “I Opt” information processing profiles can be changed. However, change cannot be imposed. This is because change is not confined to work. It affects an entire life and a personal commitment is needed to effect that kind of change. It is also not fast. Profile shifts typically take at least 18 months. A nursing management program aimed at aligning profiles will be neither inexpensive nor fast. But it can be done so that produces positive, cost reducing returns to the hospital along the way.

The basic idea is to provide the nurse candidate with specific tools to offset the vulnerabilities inherent in whatever profile that she holds. Then structure an environment so that she can use the tool repeatedly. As it is used, performance is improved. Another process then takes hold to yield lasting benefit to all involved. That process is that success breeds success.

The tool is merely a temporary aid. As it is used a nurse becomes increasingly familiar with the behavioral option it promotes. In practicing she is actually practicing the use of an alternative strategic style(s). With success the behavior becomes embedded in her repertoire of automatic responses—her strategic profile. Effectively, her profile is migrating from one state into another. This new state is preparing her to assume managerial responsibilities.

The migration strategy is a measured approach. There is no sudden shift in overall behavior. The nurse gets to work the new approach into her life pattern—at work, home and other venues in which she participates. Co-workers get the opportunity to adjust their expectations. The pressure on the nurse candidate to maintain past behavior patterns is reduced. The hospital gets steadily improving management performance.

Migration strategy process is simple. (1) Identify specific behavioral vulnerabilities using “I Opt” technology (2) design methods to offset them one at a time (3) practice (4) Once command is gained, return to item #1 for another vulnerability and restart the process. Since each nurse has a unique profile, the migration strategy is tailored to the specific needs of each nurse. The results can reasonably be expected to be more powerful than any “one size fits all” solution.

Space limitations prevent a fuller specification of the Migration Strategy here. An “I Opt” Engineering research blog Addendum is available for those interested in more detail


SUMMARY
This research has demonstrated that traditional and second career nursing MS programs are equivalent in their ability to produce managerial talent. Their common level exceeds that available from the general nursing staff but only by a small amount. Advanced nursing education does not appear to be geared to fill the nursing management gap.

The study also shows that the nursing Master’s program also falls far short of the results posted by other disciplines and areas. These other areas produce twice as many “management ready” graduates than does nursing. Evidence shows that this is not the result of the demands of nursing management. It is due to the nature of the nurses themselves.

This study traced the nurse management shortfall to the wide dispersion of “I Opt” profiles among nurses. This is likely that this is due to the nursing MS degree serving primarily as a tool for entering nursing specialty areas rather than as a vehicle for promotion to managerial ranks. This means that it is unlikely that traditional management development programs will address the issue identified. The issue lies at the very way the average nurse perceives the world, not how they go about executing a course though it.

Finally, an outline of a Migration Strategy for developing a managerial perspective was offered. It proposes a staged, systematic migration that will equip nurses to handle the kind of issues encountered at the various management levels. The process is outlined in this research blog and more fully specified in its Addendum.

Engineering Leadership

By: Gary J. Salton, Ph.D., Chief R&D and CEO
Professional Communications, Inc.

SUMMARY
This research blog shows that all levels of engineering share a consistency in their information processing (i.e., “I Opt” strategic style) approach. The research also discovered statistically significant style differences based on rank. The blog gives direction on how engineering development can leverage these differences into more effective leadership at an earlier point.


THE SAMPLE
The “I Opt” scores for 456 professional engineers were drawn from the “I Opt” database. These were divided into 3 categories. The Vice President category consisted of 44 people from 24 different firms. These people carried the title of EVP, Sr.VP, VP or Chief Engineer.

The Manager category consisted of 102 people taken from 41 different firms. They typically carried the title of Manager or Director. They were drawn from a variety of engineering areas. The subfields include mechanical, electrical, hydraulic, chemical, civil, process and a host of others.

The Professional category consisted of 310 people from 62 different firms. Titles vary widely. They include prefixes such as senior, research, design, project, process, industrial and many other kinds of engineering.

Table 1 summarizes the source data. The size, diversity and multiple firms involved suggests that sample is a reasonable estimate of the engineering as a whole. It probably can be trusted as representative of the field.

Table 1
PROFESSIONAL ENGINEER SAMPLE

SHORT-TERM DECISION MAKING
People have information processing preferences. They choose different inputs, favor different outputs and have different strategies to get from input to output. No one is completely objective in choosing the processing method they use. They normally have a standard starting point and move forward from there. If we had to decide on how to decide every decision, nothing would get done. Adopting a conventional starting point is a necessary part of life.

“I Opt” refers to basic information processing approaches as strategic styles. These are the short-term decision strategies. People use a particular starting point because it works in the environment within which they live. The discipline of engineering has a consistency that creates something of a common environment. This should evidence itself in a common approach. And it does.

THE ENGINEERING PREFERENCE
Engineers work on things with a long gestation period. There is time to think. The things they work on can involve large capital investments. There is a lot of incentive for getting it right the first time. In addition, people actually use and not just think about the products of engineering. Do it wrong and health or safety can be affected. This creates moral as well as legal imperatives. Engineering is a world intolerant of error.

Engineers respond to these common conditions in a similar way. It does not matter if they are VPs or practicing professionals. Their most frequent starting point is the Hypothetical Analyzer or HA strategic style. The HA style is characterized by exhaustive research, complete evaluation and a through review of all options. This is the dominant starting point for engineers.

The engineering setting also creates approaches which are to be avoided. Engineers put least reliance on the spontaneous Reactive Stimulator strategic style. The RS option uses expedient means to quickly resolve issues. It is favored where speed matters and/or where the penalty for failure is low. Neither of these conditions are typical in engineering. We would expect this option to be least favored among engineers. And it is. It is at the bottom of the list for everyone from VP to engineering intern.

The distribution of the HA and RS strategic styles are shown in Table 1. Each style registers at a consistent level across all ranks. Statistical tests reveal that there is no significant different between VPs, managers or professional engineers for either the HA or RS style. This consistency can be seen as a product of the common environment that all engineers share.

Graphic 1
DISTRIBUTION OF ENGINEER’S RS and HA
STRATEGIC STYLE COMMITMENT

THE ENGINEERING DIFFERENCES
Rank does have environmental effects. The decision horizon extends farther into the future with higher rank. More time is spent on unusual issues. Executives “do” less and direct more. These changes should be reflected in the engineer’s profile as rank increases. And they do.

The differences can be found in the relative strengths of the two remaining strategic styles—the idea-oriented Relational Innovator (RI) and the disciplined action of the Logical Processor (LP) styles. Graphic 2 shows a statistically significant difference by rank. Yellow coloring highlights this difference.

Graphic 2
DISTRIBUTION OF ENGINEER’S RI and LP
STRATEGIC STYLE COMMITMENT

A loss in Logical Processor (LP) commitment is the most pronounced strategic style effect of moving up in the hierarchy. The LP style is detail sensitive, focused on certainty and concerned with perfect execution.

The LP style declines as engineers move to higher levels. Problems become ever more obscure as rank increases. For example, detail availability decreases with lengthening horizons. Ambiguity rises and this lessens the value of proven methods. Certainty of outcome becomes more questionable with the volatility inherent in a longer perspective. The LP strategy that is optimal in an operating echelon can become dysfunctional at the highest ranks.

The other change happens with the Relational Innovator (RI) style. Between professional and managerial levels there is no change at all. The same strategy that worked at the professional level will probably work at the managerial level. New skills or competencies may have to be learned. However, the basic way the world is perceived and interpreted remains the same.

Moving from manager to VP is another story. The focus changes from what is or can be to what might be. Possibility becomes more important than probability. Variables unconnected to the current situation take center stage. New and unexpected solutions emerge from the morass of factors churning in the VPs world.


QUELLING THE LP
Organizational development faces challenges in engineering. Engineers seeking advancement need support in reducing their reliance on the disciplined LP style. This sounds easy. It is not.

The first step is to explain “why” a change is needed. The LP strategy is skeptical. A logical, consistent and verifiable explanation is needed. “I Opt” technology is an ideal starting point. Showing the effects of shifting time horizons, increased ambiguity and competing values typical of larger scale projects are usually enough to get the initial buy-in. After hearing the explanation engineers will grant that there are changes that have to be accommodated.

A second step puts an emphasis on “how” to make the needed changes. Routine suggestions such as to “think broadly” or “take a chance” will not work. What is needed is a formula not a specific prescription. Engineering covers many areas and it is unlikely that a single global approach will be viable in all of them. However, there are options.

One option is to leverage the engineers strong analytical HA tendencies. For example, a procedure can be setup where the first step is always a challenge. Does the use of detailed, exacting and highly certain methods make sense for this issue? If left on automatic, you may find that strong LPs will be spending more in time and money than the thing they are trying to perfect than it is worth. This initial challenge sets the stage to offset this tendency.

The second step is to have the engineers themselves design another alternative. This will typically involve breaking an issue into its components. For example, is speed important? If so, the little used expedient RS strategy might work. Or, how valuable is precision? If it makes little difference the exacting LP methods might be relaxed.

This is a natural strategy for engineers. This is what they do. All you are doing is redirecting it to the choice of a strategy rather than the actual resolution of an issue. As a bonus, the result will be a plan that is appropriate for the particular kind of engineering being done.


ENHANCING THE RI
The Relational Innovator (RI) style takes on value when moving from the manager to corporate officer. This means that development efforts can be confined to the relatively few engineers who are candidates for the highest level offices. Fewer people mean that more individual attention is possible.

The process of enhancing RI capability first involves explaining exactly what is needed and why it is desirable. Again, “I Opt” technology is ideal for this purpose. It can explain the why and how of the RI style in a logical, rational and totally transparent way.

For example, the ideas created by the RI are not just creative. They are totally unique. They have a breakthrough character. They cannot be derived from the analysis or study of the issue. That approach will create ideas but they will tend to be enhancements to existing technology.

The way the RI achieves a “breakthrough” level of creativity is by working with things totally unrelated to the issue in question. Anything will work. For example, Einstein speculated about the speed of light and in the process “saw” the connection between time and space. No space, no time.

No amount of study, analysis or assessment of then available thought could have led Einstein to his space-time fabric. It requires seeing the connection in unconnected things.

Teaching engineers to enhance their RI strategy involves a little counter-intuitive thinking. Logical connections are discarded, detail is ignored and the engineering tools hard won in years of education are abandoned. This is how you get “fresh eyes.”

Your engineers must return to the fundamentals of their education. They need to focus on how systems and theories themselves are created. The concept of causal relations—x causes y—is basic. Sequence in time continues to be vital—cause precedes result. Connectivity is important—change x and y changes every time. The principles that underlie any theory about anything are the tools of the RI.

The basic process is simple. Use ANYTHING that comes to mind (i.e., the variables). Last night’s dinner, the color of the sky today, the sound of tires as you drove to work—anything. With no natural common denominator, your engineers will be forced to look for one. The “common denominators” that are discovered are the ground from which breakthrough ideas will spring.

The search for denominators is best done using pictures rather than symbols. The reason is that pictures contain as much information as you want to see. Focus on any part of a picture that you can hold in your mind and a thousand aspects (i.e., variables) will appear. In effect, pictures allow you to quickly run a dragnet over a host of possibilities.

One outcome of the “dragnet” may be more pictures. Thinking of the sound of tires might give rise to images of rhythmic waves on oceans. The wave images can cause your engineers to think of aerodynamic turbulence. The pothole that interrupted the sound of the tires might generate an idea on how to modify an equation that will resolve an issue at hand.

This kind of thinking can be a challenge for engineers. Their world is one of exact connections and precise symbols (mathematical or symbolic). It can take a while for engineers to appreciate the value of the unpredictable RI “dragnet” strategy.


RESEARCH SUMMARY
This research has shown that developing engineers for management is a two stage process. The first stage involves controlling LP tendencies. This development stage applies to all levels.

The second stage applies to engineers ready to move from managerial to corporate officer levels. It involves enhancing the engineer’s RI capacities. This is challenging. But everyone has some RI capacity in their behavioral inventory to begin with. It is always a matter of building on what exists. It never involves creating the capacity from scratch.

Identifying the strategic style changes needed as engineers rise in the corporate hierarchy has satisfied the purpose of this research blog. It is beyond the scope of the blog to fully outline how to adjust the LP and RI capacities. However, it is hoped that enough insight was given to launch development efforts in the right direction.