In a recently published study, my co-authors and I compare the predictive validity of single-item and multiple-item measures utilized in Just-in-Time (JIT) research. The study examines if single-item measures could be used for some of the JIT practices, especially if the object of inquiry is concrete singular and if the attribute to be researched is concrete. Arguments are developed for the concrete nature of the JIT practice of “set-up time reduction” and we examine the ability of a single-item measure of this variable to predict the criterion variable (delivery performance). In addition, the study also examines the efficacy of using multiple-item measures for variables that are abstract in nature, and thereby attempts to develop a continuum of JIT constructs ranging from concrete to abstract. The results obtained by analyzing two sets of survey data show that multiple-item measures are not necessarily more valid than single-item measures for all constructs. The findings provide evidence that multiple-item measures and single-item measures for scale development should be contingent upon the nature of constructs. For concrete constructs, single-item measures are as valid as multi-item measures. Meanwhile, for abstract constructs it is important to ensure that multiple items are considered to capture the multi-dimensional nature of these constructs. Results also reveal that JIT practices display significant differences in terms of abstract/concrete perceptions. The paper presents theoretical and practical implications of the findings, and offers directions for future research.
Data was collected by means of a survey administered to part-time MBA students of a large public university located in U.S. The students work in a wide range of industries including banking, manufacturing, IT, and healthcare. Since JIT practices are applicable in both manufacturing and service industries, two versions of the questionnaire were administered; one for respondents with experience in manufacturing sector and the other for respondents with experience in the services sector. The survey respondents represent very large organizations with, on average, more than 10,000 employees in both the manufacturing and service organizations. Respondents from manufacturing organizations reported that, on average, there were 923.82 people in their department. Respondents from service organizations reported that, on average, there were 506.10 in their department. The respondents represent several diverse industries including automotive, oil, telecommunications, food manufacturing, chemicals, healthcare and pharmaceuticals in the manufacturing group, and banking, financial services and software in the service group.
The respondents work at the positions such as project manager, quality engineer, controller, analyst, operations manager, vice president, senior consultant, and business specialist. The tenure of the respondents with their current employers range from 6 months to 20 years with an average of 5.23 years for the manufacturing group, while this time range from 1.5 months to 34 years with an average of 4.86 years. These figures indicate that the respondents spent sufficient time at their positions to be able to comment on the JIT related processes and practices. The survey items were adapted from previous research. The respondents were asked to rate the extent to which they used various JIT practices. A seven-point Likert scale where 1 indicates strong disagreement and 7 indicates strong agreement was used. A ‘not applicable’ (N/A) option was provided, as well as performance measures. We received 260 responses during this first round of data collection. In line with literature, the constructs considered in hypotheses 1 to 3 are operationalized as follows:
Single-item conceptualization of set-up time reduction (STR(1)):
Manufacturing industry: We have low set-up times of equipment in our plant.
Service industry: We have low setup times of equipment in our service processes.
Multiple-item conceptualization of set-up time reduction (STR(4)): 4 item scale
Manufacturing industry: 1. We are aggressively working to lower set-up times in our plant; 2. We have converted most of the set-up times to external times while the machine is running; 3. We have low set-up times of equipment in our plant; 4. Our crews practice set-ups to reduce the time required.
Service industry: 1. We are aggressively working to lower setup times in our service processes; 2. We have converted most of the setup times to external times such that the setups can be performed while the service is being offered; 3. We have low setup times of equipment in our service processes; 4. Our crews practice setups to reduce the time required.
Single-item conceptualization of delivery performance (DP(1)):
Manufacturing and service industries: How does your organization rank relative to your competitors in terms of the delivery performance?
Multiple-item conceptualization of delivery performance (DP(3)): 3-item scale
Manufacturing and service industries: 1, How does your organization rank relative to your competitors in terms of the delivery performance?; 2. How does your organization rank relative to your competitors in terms of fast delivery?; 3. How does your organization rank relative to your competitors in terms of lead time (order to delivery)?
A second wave of data was collected to test Hypothesis 4. An online survey was sent to 2500 practitioners to measure perceived concreteness and abstractness of JIT constructs.
Specifically, in the survey we first provide the definitions of “concrete” and “abstract” constructs as follows:
Concrete: A JIT practice is "concrete" if a single item/question is sufficient to capture its core underlying essence. There is very high level of agreement as to what the practice refers to.
Abstract: A JIT practice is "abstract" if multiple items/questions are needed to capture its core underlying essence. The practice has several dimensions or characteristics.
Then the respondents rate the JIT practices within a continuum with “Definitely Concrete” and “Definitely Abstract” representing the two ends of the continuum. We received 310 complete responses. These respondents represent a wide variety of titles ranging from Master Production Scheduler to Vice President of Operations. All respondents have at the least an undergraduate degree in business and an average work experience of 21 years providing reasonable evidence that the respondents are aware of the practices underlying the JIT domain. For both waves of surveys we conduct several t-tests to test for non-response bias. We assume that the responses of the later waves of respondents are representative of the non-respondents. The early wave and the late wave respondents were compared using “Tenure with the company” for manufacturing and service groups to test whether non-response was a problem in the sample. As a result, we did not observe any significant differences between the waves of responses.
As an outcome of the study we present the following continuum of scales that ranges from highly concrete to highly abstract:
|
STR: Set-up time reduction |
RMS: Repetitive master schedule |
|
SLS: Small lot sizes |
PM: Preventive maintenance |
|
JIT: JIT delivery from suppliers |
EL: Equipment layout |
|
SQL: Supplier quality level |
PDS: Product design simplicity |
|
MFW: Multifunction workers |
KANBAN: Kanban |
|
SGPS: Small-group problem solving |
PSS: Pull system |
|
TRA: Training |
MRP: MRP adaptation to JIT |
|
DSA: Daily Schedule adherence |
ACC: Accounting adaptation |
Source: Nair, A., Ataseven, C., Habermann, M., Dreyfus, D. 2016 Toward a Continuum of Measurement Scales in Just-In-Time (JIT) Research – An Examination of the Predictive Validity of Single-Item and Multiple-Item Measures. Operations Management Research, 9(1-2), pp.35-48.