Customer Satisfaction Pilot Studies and Analysis

State-to-State Comparisons

Customer satisfaction information is available by State (i.e., score data for State A, State C, State D, State E, and State F). A comparison of State responses answers the question, is there a difference in customer satisfaction between States? Is one State's satisfaction score statistically and practically significantly higher (or lower) than the other States? This comparison is calculated by taking all the ASCI scores and performing a statistical test called an ANOVA . 9

Strengths

A comparison of customer satisfaction by State provides a clear indication of the perception of service by different groups. The results of this comparison can be presented without using statistical jargon or a large amount of numbers so that it is easily understood (e.g., State A's customers were significantly more satisfied with the service they received than States B's and C's customers).

This comparison can indicate possible areas of concern in service delivery in a specific State and contribute to a continuous improvement effort for access and equity of service. It may also identify States that have a particularly strong approach to serving customers and are sources of "best practices" information.

Weaknesses

Some comparisons, although they might yield a statistically significant result, may not be appropriate. Two States may have such different populations and differ to such a degree in the types of services needed that a comparison of the two may not produce useful information. Therefore, such comparisons are most understandable when accompanied by information about program design, populations served, and economic conditions.

The nature of the comparison and the type of analysis must be clearly explained so that the audience understands the limitations of the comparison.

Primary Audience

A state-to-state comparison of customer satisfaction is particularly useful for State level program planners and policy makers. It is often a good marketing tool with stakeholders.

When a comparison was performed of the ACSISAT scores of the six States, several significant or practical differences were found between some of the States. In the following matrix (Table 5), each State is presented with those States with which it had a practical difference in ACSISAT scores. That is, the first column lists each State. The second column presents those States with a significantly different score than the State in the first column. Those States that did not have significantly different scores from the State in question are not presented. The third column presents the respective ACSISAT scores for each State.

Table 5: Inter-State Comparison on Participant ACSISAT Scores

Inter-State Comparison
State Significantly different from the following states:
ACSISAT Score
State A
 
73.21
 
State D
67.00
State B
 
73.23
 
State D
67.00
State C
 
71.88
 
No State
 
State D
 
67.00
 
State A
73.21
 
State B
73.23
 
State E
72.08
 
State F
74.37
State E
 
72.08
 
State D
67.00
State F
 
74.37
 
State D
67.00

Table 5 Comments

This analysis indicates that State D's ASCI scores are significantly lower than those of four other States. What is different about State D's service and customers? What can the other States learn from State D? Is there some characteristic of its service and customers that lowers their scores? Is that aspect within the control of the One-Stop Centers? If additional analysis were to indicate that some external factor such as major work dislocations negatively impacted the ACSISAT scores for State D, any reporting of these results should be footnoted with that information so that no unjustified conclusions are drawn from the data. As stated above, it is important when doing comparisons that the comparison is apples to apples and not apples to oranges (or cheese).

Comparisons Over Time>>

9A statistical test to determine the significance of differences between averages and the likelihood that these results could be expected consistently.

 
 
Home | About | Resources | Services | Forums