Scale modification: alternative approaches and their consequences

Adam Finn and Ujwal Kayande

Marketing scales are often modified after development, either being adapted for use in a particular applied/theory-testing context or refined to improve their psychometric properties for a particular application. For example, the published SERVQUAL scale has been modified in at least thirty-eight academic studies and likely many more times in practitioners’ studies. The implicit assumption underlying scale modification is that the modified scale will, in some way, be better for the particular application than the original scale. However, little attention has been paid to the consequences of how modifications are made, the criteria that are used to assess how well a modified scale performs, and whether scale modification is in fact worthwhile. To investigate these issues, we review approaches to scale modification, select an influential marketing scale and use context, and then examine the effects of the different approaches to scale modification on which items are included in the refined scale. We then cross-validate the psychometric performance of the resulting scales using criteria that reflect the multiple managerial purposes to which a scale can be applied.

Retailers use marketing scales such as SERVQUAL for multiple managerial purposes. For example, a retailer may be interested in segmenting the market based on customers’ perceptions of the service quality provided by retailers. To do this, the instrument that measures service quality perceptions (for e.g., SERVQUAL or some modification of it) must adequately scale respondents. On the other hand, the retailer may be interested in benchmarking itself against competitors on the quality of its service. Thus, the retailer needs to compare the quality of service of the competing retailers. This managerial purpose requires the measurement instrument to adequately scale retailers, which now become the object of measurement. Most scale modification approaches used in marketing are drawn from individual-differences psychology. They are capable of improving scale performance when it is used to scale respondents, but may not improve scale performance when it is used to scale retailers, an equally important managerial objective. Drawing on recent arguments for the greater importance of content validity (Rossiter 2002), we also contend that adapting a scale will have a far greater impact on the performance of the scale than simply refining it using current scale modification approaches.

To this end, we conduct a two-phased empirical study. In the first phase, we collect data on service quality perceptions of multiple retailers using an augmented version of the SERVQUAL scale. We then modified the scale using several common approaches. Subsequently, in the second phase, we cross validate the performance of the modified scales in a second set of perception data. We examine scale performance for multiple managerial purposes, including the scaling of respondents and the scaling of retailers (reflecting segmentation and benchmarking respectively). The results of the studies strongly support our contentions. First, the most advanced scale modification approach, aggressive confirmatory factor analysis, performs best when scaling respondents. However, it is not the best modification approach when scaling retailers. In fact, we show that a random selection of items can perform just as well when scaling retailers. Second, scale adaptation, reflecting a modification in concept validity, has a far greater impact on scale performance than scale refinement, supporting our second contention.

Our findings suggest serious problems in assuming multi-purpose adequacy of scale modification approaches. We clearly demonstrate that scale modification without taking into account the different purposes of measurement can be misleading. In the absence of formal scale modification approaches for scaling objects other than respondents, we suggest that researchers focus on adapting scales on the basis of content validity claims rather than modify scales using extant scale refinement methods.