WEDNet Indonesia

Training | Consultant | Software |

Designing Performance Measures and Metrics

How do organizations know when they are meeting strategic objectives? Driving the right behaviors in their organizations? Beating or lagging behind the competition? Identifying warning factors of negative change? Organizations find answers in well-designed and carefully implemented measures. But what does that really entail?

Those familiar with performance measurement design, implementation, and analysis know the challenges of establishing effective performance measures; but the successful organizations attest to the rewards—namely, a stronger bottom line. Measuring results allows an organization to determine what is successful and what is not. By identifying success, organizations can reward, build support, and learn from best practices.

Cindy Hubert, an executive director at the American Productivity & Quality Center (APQC), recently discussed best practices that organizations can use in defining measures to drive performance excellence. "Measures can enable an organization to focus attention on desired behavior and results and tie process improvements to key performance indicators," said Hubert, "as well as link strategy and tactics to assessment and baseline information, indicators of desired behavior and outcomes, and feedback."

Establishing a Framework
Measurement begins with establishing a framework. A measurement framework is critical to linking organizational objectives to the business unit and individual levels by ensuring everyone understands not only how roles align with organizational objectives, but also how each unit and individual contributes to the outcomes. The end result is a scorecard that provides strategic framework, alignment, balanced measures, cascaded scorecards, critical success factors that yield strong measures, and meaningful aggregation.

Key measurement frameworks include balanced scorecards, family of measures, and Input-Output Measurement Framework. Each of these key measurement frameworks, whether used individually or in support of another, provides structure for organizational measurement. These frameworks enable an organization to realize crucial benefits by unifying its focus through: communication using agreed-upon and consistent definitions; an aligned set of performance targets using validated, normalized data; and a collective, diagnostic tool to identify areas for improvement and set priorities.

Ideally, measures should be reflected in a balanced, cascading scorecard. A balanced scorecard helps to align measures with key strategies, enable progress tracking, assign accountability, capture gains already made, measure future improvements, and connect current strategic and tactical improvement activities. Organizations can achieve this "balance" by establishing measures in four quadrants (which can be weighted differently based on organization priorities) that reflect key objectives.

1. Customers--measures performance against expectations (e.g., satisfaction, retention, acquisition, and profitability).
2. Financial--measures economic consequences of actions already taken (e.g., income, return on equity, return on investment, growth, and cash flow).
3. Internal--measures effectiveness, adaptability, and efficiency of internal processes. Such measures may identify a need for new processes.
4. Innovation and learning--measures employee skills, information exchange, and organizational procedure.

The family of measures framework, Hubert advised, focuses on a cluster of measures that should track at least four of the following process variables: productivity, quantity, quality, timeliness, cycle time, resource utilization, or costs. For each characteristic, condition, or variable (i.e., a critical success factor), a process measure can be identified as a reference standard for quantitative comparison. Two examples follow for the customer complaint handling process.

Category: Cost
Critical success factor: Complaint handled efficiently
Process measure: Cost per complaint, percentage of total budget, etc.

Category: Quality
Critical success factor: Courteous service
Process measure: Operator responsiveness, minutes on hold, etc.

Input-Outcome Measurement Framework is yet another means of presenting a snapshot of an organization's performance. This method focuses on the core processes of an organization. It defines core processes that convert inputs to outputs by aligning the key activities with business outcomes. For example, in a sales process, the framework converts budgeted cost categories such as labor (input) into negotiating and closing sales (activities), which leads to closed sales (output) and increased revenue (outcome).

Types of performance measures to consider include baseline (starting point), trending (process performance), control (inside/outside predetermined boundaries), diagnostic (problem identification), and planning (prediction/future planning).

Designing Measures
"The key to developing effective measures is to identify those that will directly help achieve the desired results and then deliver them to the right people at the right time."

So what should organizations measure? "Organizations must begin by selecting measures that align with strategic objectives, demonstrate results, and focus on outcomes," said Hubert. They must work to produce measures that:

are meaningful;
respond to multiple organizational priorities;
encourage operational improvements;
provide a complete, accurate, and believable picture of performance; and
blend leading and lagging indicators.

In the design process, organizations must strive to examine business strategies and objectives, determine critical success factors, and describe measures. For critical success factors, Hubert advised to ask: What would you see if the goal were achieved? and What important outcomes would be realized? In describing measures, she recommended determining if a measure can be tracked, if the measure directly links to a critical success factor, and if the outcome of a measure can be influenced.
Potential measures should be assessed for:

purpose--is worth collecting and answers a question to support decision making,
validity--measures what it claims to measure,
precision--returns consistent value with each measurement,
accuracy--matches the true value of the attribute, and
cost effectiveness--is not too costly to track and report.

Measures may come from, for instance, activity-based costing (e.g., hours or cost), surveys (e.g., customer, supplier, or internal), and databases (internal or external).
Measures are often collected and reported in "dashboards" or "scorecards," which are common terms for measurement snapshots. As with a game scorecard or a car dashboard, a measurement snapshot presents a quick overview of key operating details such as key drivers, enablers, and results.

Hubert cautioned to be aware of the following potential challenges:

Complexity—too many metrics, excessive detail, or burdensome data capture—can make metrics too difficult to use.
Metrics aimed at short-term performance often have unintended long-term consequences, because employees tend to do well on what is measured rather than what is not.
Know the difference between in-process and end-process metrics. In-process metrics are used to help understand what is working. End-process metrics measure process effectiveness.
Quantitative metrics often miss important subjective elements (i.e., qualitative factors).

To help avoid pitfalls and build more effective measures, Hubert advised to develop measures that drive behavior and measure real work outputs and accomplishments. It is also helpful to ensure usefulness and relevance by tying a specific performance measurement to a specific user by name or position. Finally, she advised to develop measurement collection tools that provide adequate warnings of negative change.

Comparing Measures to Metrics
Once the measurement framework is established, organizations can use specific metrics to drive progress and results. "Metrics are normalized, objective, and quantitative measures," said Hubert. "They are used to gauge operational performance or resource allocation."

Metrics are quantitative key performance indicators, which are essential to understanding operational health. Key performance indicators result from operational objectives, are based on outcomes, and are central to measuring impact on key stakeholders (i.e., stockholders, customers, and employees). Key performance indicators also include supporting detailed indicators that disaggregate the parts and become key in statistical testing. Qualitative drivers, such as management practices and systems, are a necessary component to key performance indicators; their relationships must be understood.

Normalization is required to put data on a common basis (e.g., per unit); this mitigates issues of organizational scale and demographic factors and establishes a common denominator for benchmarking standard units across organizations. Metrics do not include subjective ratings due to unreliability and potential bias.

Hubert explained the most common metrics categories.

Cost effectiveness--indicates how well operating costs are managed. Key performance indicators usually include cost per unit, cost as a percentage of revenue, cost as a percentage of total budget, and actual cost compared to budgeted cost. Supporting indicators usually include cost components as a percentage of total and disaggregated cost per unit.
Staff productivity--indicates output for each full-time equivalent (FTE) employee. Key performance indicators usually include units of output (e.g., invoices and purchase orders) per FTE and workload (e.g., customers and general ledger accounts) per FTE. Supporting indicators usually focus on factors that influence staff productivity, such as hours of training per FTE and employee tenure.
Process efficiency--indicates how well procedures and systems are supporting the operation. Key performance indicators may include error rate and forecast accuracy rate. Supporting indicators may focus on factors that influence process efficiency, such as system downtime rate and the degree of process automation.
Cycle time--indicates the duration to complete a task. These key performance indicators are measures in units of time and may include processing time and time to resolve customer inquiry. Supporting indicators usually focus on factors that influence cycle time, such as the frequency of system breakdowns.

Examples of customer service/call centers metrics follow.

Cost effectiveness--cost per call and cost per reported complaint
Staff productivity--calls per representative and resolved complaints per FTE
Process efficiency--first-call resolution rate and total resolution rate
Cycle time--average time to answer and average time to resolve complaint

"It is important to understand," Hubert said, "that customer and employee satisfaction measures are not metrics, because they provide subjective evaluations and perceptions rather than objective data." However, these measures provide organizations with important insights that can be used to direct traditional metrics benchmarking. For example, if a customer satisfaction rating is low in process cycle time, the benchmarking should focus on this area.

Designing Metrics
At a more tactical level, organizations must ensure metrics can and will be implemented. This requires defining the use of metrics, the message to each audience, and narrative framework and metrics (i.e., how to present the desired message and what quantitative and qualitative metrics will be used).

Metrics allow organizations to understand operational performance, which can be tracked over time, relative to external benchmarks (e.g., industry average or top performers) as well as internal ones. Designing a tool of key performance drivers helps organizations not only better manage internal processes, but also identify key external practices that can be adopted to improve performance. By looking beyond the numbers at the qualitative drivers, management can also reveal the factors that most influence favorable performance. Thus, by choosing metrics wisely, organizations can focus.

A top-down and bottom-up interactive process extracts the information that matters most to the organization. This process involves identifying stakeholders, decision makers, information needed, the potential impact at each level, and data sources.