Measuring the business value of IT: simplify with caution

 

Demonstrating performance and productivity gains to organizations from investments in information technology (IT) continues to be challenging. The notions of the productivity paradox (the main idea in Erik Brynjolfsson ‘s famous 1993 CACM Magazine article discussing the paradox between technological advances and the relatively slow growth of productivity) and IT Doesn’t Matter (Nicholas Carr’s widely-discussed Harvard Business Review article from 2003), have received widespread attention over the last few decades.

In sharp contrast, based on findings from 303 studies published during 1990-2013, we find clear evidence of the business value of IT (BVIT). Our data span several decades that represent various stages of IT progress including the mainframe computing era (1965-1982), the personal computing era (1983-1994), the network computing era (1995-2002), and the mobile computing era (2003-2013).

First, we make an important distinction between organizational and inter-organizational IT. Organizational IT may be deployed for many different purposes across departments, facilities, or regions and may make it difficult for organizations to track the impacts of the investment. By contrast, inter-organisational IT is dedicated to external linkages, usually with important partners, and it may be easier for organizations to directly link performance improvements to the IT investment.

Our key findings may be broadly grouped into two categories and include the following:

• Importance of other aspects

  • Investments in IT alone are not sufficient to realize its business value; indeed, investment metrics may not be able to indicate the actual ways in which IT was used or deployed for organizational operations.
  • Investments are able to demonstrate BVIT when IT adoption and use is also considered together.
  • Business value is considerably weaker when investments alone are considered for information technology used in an organization’s internal activities.
  • Non-financial metrics are able to better demonstrate business value especially for IT that is aligned with business strategies.

• Changes over time

  • BVIT shows an increasing trend through the decades despite variations in metrics.
  • The variables used to measure BVIT have continued to evolve over time, from profitability measures such as return-on-assets (ROA) and return-on-equity (ROE) to other performance measures.
  • The variables used to measure IT have also continued to evolve over time, from pure financial measures such as IT capital and budget to applied measures such as IT use and capability.
  • IT has continued to provide business value to organizations in both the developed and developing regions of the world. In fact, the business value in developing regions seems to be increasing at a faster rate relative to developed regions.

Our results suggest that the challenges in demonstrating the business value of IT during the early years were driven in part by the nature of the metrics used. For any organization, investments merely signal the allocation of resources for IT but do not reveal details about the efficiency and effectiveness of how the organization acquires, deploys, or uses the technology. Metrics used in the later years, such as IT capability and alignment, offer greater opportunities to clearly discern business value. Moreover, organizations may use IT for different purposes and may generate value in a variety of aspects such as by increasing sales output, capacity utilization, and net profit, or by lowering labor hours, cycle time, and coordination costs. These measures are not all applicable to all organizations and are not simultaneously examined in prior studies.

For technology used inside an organization, BVIT is lower when the measure focuses on investment. But the reverse is true for inter-organisational IT, as BVIT is higher when the measure focuses on investment.

Business value is more easily demonstrable for IT that is aligned with business strategies. BVIT, in this case, reaches significantly higher levels than when that alignment is not present. Since senior managers are more likely to decide about technology that influences or aligns with business strategies, the impacts of aligned IT may be more visible and tracked better by organizations.

In conclusion, our results lead us to offer the following suggestions to business and practitioners as they engage with IT:

• Practitioners may find it more beneficial to take a multi-dimensional view of business value. Profitability measures should not hold exclusivity but alternative metrics consistent with the organizational operations should be developed.
• Similarly, a multi-dimensional view of IT may be appropriate. Financial measures alone do not provide a complete picture; rather, organizations can identify the ways in which IT may be effectively acquired, deployed, and used.
• In identifying measures for both IT and business value, a few, relevant, and high-profile key metrics should be used for evaluation instead of developing several metrics that cannot be measured or interpreted effectively.
• When evaluating BVIT, both the organization’s and IT’s idiosyncratic characteristics, applications, and uses should be considered in devising and measuring IT and business value.

In summary, measuring IT impact is complex, and organizations should not oversimplify by using single performance metrics or single IT metrics or excluding the context.