T. Sammut Bonnici,
University of Malta, Malta

J. Sarkis
Graduate School of Management, Clark University, USA

Introduction
The case study chronicles the implementation of a performance measurement system (PMS) for SkyTel, a paging company in its start-up phase in Malta. The objective of the exercise was to observe the effects of measurement on the company and identify its advantages and disadvantages. The paper describes how lead and lag indicators were selected for the operations monitoring process. A departmental, market driven, activity-based, approach was used, extracting operational variables from the companyís functional units. The measures were to be directly tied to what the market (customer) viewed as important, with a strong emphasis on the processes (activities) adding value to the customer.

Performance Measurement Requirements
The following perspectives where taken into account in the design phase of the measurement system:
1. Performance measurements should permeate the organizationís hierarchy. That is, a linkage between operational measures and strategic objectives should be made and integrated. Cross and Lynch (1991) propose such a structure of measures that permeate through the organization's hierarchy in order to integrate performance.
2. Measurements should link across the organization. In order to be effective the measurement system would have to recognize the merits of the service profit chain, whereby, customer relations, quality service, and employee satisfaction have a direct effect on profits (Sasser, Heskett, Jones, Schlesinger, 1994; Rubin, 1995).  This requirement has also been noted by Crosbyís third point of implementing a total quality management system.
3. The monitoring system should monitor the company from a financial performance perspective, customer knowledge and internal business processes. It would have to balance objective and subjective factors. Both tangible and intangible measures need to be integrated. The PMS would have to channel the energies, abilities, and specific knowledge held by people towards achieving long-term goals (Norton and Kaplan, 1996).
4. During the design stage companies need to consider the multi-dimensional nature PMSs, in terms of the balance between external and internal factors, possible conflicts between performance measures, and the link between what is being measured and corporate strategy. PMS complexity may be minimized by employing structured methodologies. Research shows that firms that use a structured approach find it significantly easier to a) decide what should be measured b) decide how to measure it c) collect the appropriate data and, d) eliminate conflict in the measurement system. (Neely, et al., 1996)
5. Avoid looking at the firm from a mechanistic viewpoint. An organization needs to be viewed as a living organism, which needs to be sustained, motivated and provided the opportunity to learn and improve. Measures should be balanced by reporting on success and failure, and focus on the future and the past (Senge, 1990). This issue points to the need for dynamic characteristics of any measurement system (see Dixon, et al., (1992); Adams, et al. (1995)). An evolutionary approach, rather than a revolutionary approach fits well within this dynamic organizational perspective, and is usually preferable to most organizations (Blossom and Bradley, 1998).
6. The measurement system should act as a tool to encourage people to be creative, to improve skills, to derive satisfaction, and to produce better results for the company. It should not serve as a blame tool, but as a tool for improvement.
7. Marketing acumen is central to the paging company. The system should monitor 1) how the company seeks to be a master of a market, for which it seeks customer-satisfying products and services, 2) assess how a company is retaining and developing its expertise of a certain technology, and 3) how markets are being effectively sought.
8. Deming (1986) warns against the ëparalysis by analysisí. The reporting system would have to be easy to understand and quick to read, and information limited to what is really needed to make operational decisions to optimise market performance.

Organizational Background
SkyTel, in its start-up phase, required a monitoring tool to assess its progress in its first months of operation and organizational life cycle. The company could assess its competence by benchmarking against international standards in the paging industry. A measurement system for producing benchmarking metrics and gauging activities of vital operational functions was needed.

Early on the company was aware of the strict and volatile competitive environment it faced. The paging service to be introduced in the region after mobile telephony a reverse introduction of what happened in the United States and other European countries. Managers were concerned with the company's ability to penetrate a difficult telecommunications market. They required a feedback system to highlight progress and to indicate fresh opportunities to gain market share.

Metric Identification Process
Activities to be measured were selected by their organizational role. The PMS was to represent a balance between what managers think they need, what managers really need, and the economic feasibility. Managers were interviewed to discover their concern with operational processes and information needs. Special attention was given to upper managementís information desires and concerns. The development of SkyTelís PMS was achieved through a systematic approach and framework that secured commitment about how to turn the company and the departmentís strategies into operational measures.

Identification of Corporate Objectives
The initial stage was to identify the primary corporate objectives. The company had been set up a few months earlier. Its primary goals where identified and rated in the following order of priority:

Departmental Evaluations
The selected company activities can be categorized into four departments: Marketing, Customer Care, Dispatch Bureau, and Operations. A series of workshops was devised with the key management decision-makers of each department. The objective was to identify performance measures for each department, based in the functions and the process flow of activities within the departments. Four sessions were held with each group to address the following subjects:

PHASE 1 - INTRODUCTORY SESSION ON PERFORMANCE MEASUREMENT SYSTEMS
Commitment and trust from management were achieved through discussion of measurement practices across different industries. Managers started to identify how they could benefit from the system in terms of better evaluation of resources and to focus more clearly on objectives for which they were held accountable.

PHASE 2 - CHARTING OF ACTIVITY BASED PROCESS FLOWS
Lists of activities, flow charts of work processes and job descriptions were configured and drawn up to describe what goes on within each department. The visualization of the work processes per department had a tangential effect on management. It helped them focus on how the departments were evolving at a time when the company was in its infancy. Managers used the exercise to assess their functional operations and to help it evolve faster.

PHASE 3 - LINK ACTIVITIES WITH OTHER DEPARTMENTS
The activities charted in the previous session were examined for overlap of functions between departments. This approach helped managers define accountability for performance more accurately. Furthermore, the departments obtained a clearer picture of how the functions interface with each other, and where co-operation must be strengthened to ensure performance targets were reached.

PHASE 4 - IDENTIFY THE KEY PERFORMANCE MEASURES
Once the departmental activities where charted the metrics worth measuring were identified. The criteria for selecting the measures were based on the following rationale:

-   The measure selected was for a value-adding activity that was of relevance to the customer. In this way value-adding activities were encouraged and rewarded. The net effect was for people in the organization to reduce non-value adding work in order to increase value-adding operations. Examples of such measures would be: sales patterns, handling of customer queries, and the efficiency of repair services.

-   Key activities, which were not directly value adding, would be included if they were drivers of value adding activities, for example: commissions earned per salesperson and the number of pagers ordered per sales contract. Such measures would act motivate performance and efficiency in the sales force.

During the system building process, comprehensive documentation was generated containing flowcharts of the departmentsí activities. Figure 1 provides an outline of the contents of this documentation, as well as the overall measurement development process for the marketing department as a sample. A summary of the performance measures adopted within each department are shown in Table 1.

Performance Data Collection and Processing
A standard form was constructed with each department manager. The form contained a list of the relevant performance metrics. At the end of each week the department would complete the forms and dispatch them for processing. The data was translated into tables and charts. The graphs issued were found to be effective tools to monitor the variations in performance. The reports were supplied on a weekly basis. Given that the company was in its development phases, data needed to be supplied frequently, to troubleshoot problems as they arose. Even though they were valuable to management, the forms required repeated updating. This updating was completed on a monthly basis and reflected the changing market environment and the expansion of the companyís operation.

Monitoring and Feedback of the Performance Measurement System
An important dimension for any PMS is that it will be dynamic and timely. That is why a critical element of any development process is the feedback loop to and from the developers, managers, and users of the system. To accomplish this phase in the development process observations are made, followed by the implementation of improvements and alterations.

Observation of Effect on the Organization
It is important that a PMS is closely linked to the actual performance of an organization, one of the common sense and fundamental purposes of PMSs that is typically overlooked (Blossom and Bradley, 1998). As part of this effort the monitoring included an evaluation of how the data was used by top management in strategic decision making. From an operational perspective, the extent to which the reports were understood, and how easily they could be translated into actions, were also measured.

During the implementation phase of the PMS, managers were assisted in understanding the data and developing action plans. When a slow-down in performance was detected, an inter-departmental task force would be assembled to address the problem. A strategic plan of action would be devised to improve performance. In most cases the corrective plans included the implementation of new operational procedures, to improve performance in the short term, and avoid a re-occurrence of negative performance variance in the long run.

An important and final element of the monitoring and feedback phase was to determine how well the PMS and its purpose have diffused throughout the organization. Specifically, recommendations to assist employee knowledge and acceptance of PMS philosophies were provided. In this stage several discussions were held with the management team to establish how the reports could chart the path towards improving the companyís performance.

Findings and Results from the Performance Measurement System
Positive Factors of the Performance Measurement System
1. The reports and measures supplied were concise, accurate and informative. The elements chosen for measurement were appropriate and highly relevant to performance.
2. The company could see itself as it really was, free of assumptions on performance of staff and market penetration.
3. The charting of customersí opinions and levels of satisfaction highlighted the companyís strengths and areas for improvement.
4. The organization was measured from all angles via the monitoring of market growth trends, channel analysis, market shares, sales trends, and external demand.
5. All targets were concise and focused.
6. All measures and targets were tied to a given department, its subsections and its staff members.
7. Staff were aware of their targets to satisfy market requirements.

Limitations of the Performance Measurement System
The negative results of the PMS were also obvious in this case. The following issues were targeted for improving the system. These observations are lessons that provide insight into what managers should attempt to avoid in a market driven PMS.

1. There was a conflict of ëfití between the highly structured PMS and the young, rapidly evolving company. From a subjective point of view the mechanistic characteristics of the reports and performance measures seemed to detract and constrain managementís creativity. Creativity may be necessary for effective growth in a difficult and competitive market.

2. Even though the results of the PMSs indicated what the weaknesses were, the market and organizational environment did not facilitate the action necessary for improvement. Cultural and organizational constraints limited the effectiveness of the PMS.

3. Personnel found it difficult to relate the marketing oriented targets to what they had to do in their daily routines. Even though there was some effort to diffuse the measures vertically through the organization, this effort was difficult and not completely successful.

4. The corporate culture was starting to focus on operational problems. Thus, the charting of problems on a regular basis could have been de-motivating. Success is sometimes viewed as something employees are rewarded while failure is penalized.

5. Management and personnel required further training to learn how to handle PMS information. If the reports were used ineffectively, a blame culture could emerge within the organization. Once again, the cultural goal of improvement, and not blame-laying, is a difficult barrier to overcome when it comes to the actual utilization of performance measurement outputs.

6. The performance measurement outputs (reports and measures) needed to show more effectively, how the work that was accomplished in the organization affected the customer, or of what value they were to the customer. Customer opinions and feelings were recognized, but the impacts of the actions on the customer were not easily determined.

7. The system lacked the ability to help the organization help itself. It told the company where it was, and where it should be but did not motivate action or create a dynamic environment conducive of change. Clearly, this is a large goal for any system, but provides the lesson that any strategic project in an organization will require a substantial cultural evolution as well.

Conclusion
The challenges that arose with the implementation of performance measurement concerned the introduction of a relatively rigid measurement process, within a vibrant evolving company, which was in its start-up phase. Corporate knowledge, structure, and skills were in the process of development. The learning curve within the organization seemed to require a natural period to reach maturity, which could not be accelerated by the performance data reports. If one looks at the issue from Arie de Geusís (1997) perspective of corporations as ëlivingí companies, the organization needs to evolve in response to its surroundings. The organizationís response to the performance reports would come at a second stage, when the identity and culture of the company become established.

The PMS was effective in terms of design influence on customer oriented operations procedures The main lesson learned is that the process could neither be forced into the structure nor rushed to completion. The company would have been identified as being in its adolescent phase in terms of the ëLearning and Growthí perspectives (Kaplan and Norton, 1996). A natural progression of the operation would be required until it could use the full potential of the more complex PMS. One issue that arose is the need to investigate the "life-cycle" characteristics of an organization and how to model appropriate PMSs for different life cycle stages.

The case study highlights an area of research of performance measurement that requires additional investigation namely: how to integrate a performance system (which is mechanistic and numbers oriented) with a corporation (which can be viewed as a living and evolving entity). The results could provide valuable insight on how to create and manage viable measurement systems.

References
Adams, Scott, Sarkis, Joseph, Liles, Don, (1995), "The Development of Strategic Performance Metrics," Engineering Management Journal, Vol 7, No. 1, p 24-32, 1995.
Blossom, A.P., and Bradley, J.R., (1998), "Nine Mistakes Managers Make in Using Performance Measures to Motivate Employees," Working Paper, S.C. Johnson Graduate School of Management, Cornell University, Ithaca, USA.
Crosby, P.B. (1984), Quality without Tears, McGraw-Hill, New York.
De Geus, A. (1997), The Living Company, Harvard Business School Press, Boston.
Deming, W.E. (1986), Out of the Crisis, MIT, Cambridge, Massachusetts.
Heskett, L., Jones, T., Loveman,. G., Sasser, E., Schlesinger L. (1994), "Putting the Service-Profit Chain to Work," Harvard Business Review, Vol. 72, No. 2, p 164-165.
Kaplan, R., Norton, D. (1996), The Balance Score Card, Harvard Business School Press, Boston.
Lynch, R.L. and Cross, K.F., Measure Up - The Essential Guide to Measuring Business Performance, Mandarin, London, 1991.
Neely A., Mills J., Platts K., Gregory M., Richards H., (1996), "Performance Measurement System Design: Should Process Based Approaches be adopted?", International Journal of Production Economics, p 423- 431.
Nanni, A.J.; Dixon, J.R. and Vollmann, T.E., (1992), "Integrated Performance Measurement- Management Accounting to Support the New Manufacturing Realities," Journal of Management Accounting Research, p 4.
Senge, Peter M., (1990), The Fifth Discipline New York: Doubleday.
Rubin, M., "Reinventing Customer Management: Lessons From The Best Of The Best," Arthur D Little Prism, Fourth Quarter 1995, p 3.