DRAFT AMBULANCE SERVICE PERFORMANCE STANDARDS
KERN COUNTY EMS DEPARTMENT
JAN DE LEEUW
Department of Statistics, University of California, Los Angeles, CA
Date: November 15, 2006. REPLY BY KCEMS DEPT.: November 21, 2006
I consulted both Draft Version 5 , and Draft Feedback/Comments Version 2
This note comments briefly on Section IX of these standards, regulating response time performance
The standard for compliance is set at 90%, i.e. 90% or more of the responses have to be within the speciified time limits. There is no argument supporting this percentage, which seems arbitrary. How does it compare with other counties?
Note that KCFD in its comments suggests 93%.
REPLY BY KCEMS DEPT.: A compliance rate of 90 percent is the standard used in most EMS systems throughout the nation. Most of the counties in California have a 90 percent standard. There are a couple of California counties that use a 95 percent standard, but in those counties there is greater latitude in granting exceptions for late calls. Kern County has used a 90 percent standard for the past 20 years, and there is no evidence to suggest we establish a different one.
Required response times differ per zone (Metro/Urban/Suburban/Rural/Wilderness).
It is unclear from the standards document how zones are defined, but the whole concept seems somewhat counter-intuitive.
Any location in the EOA can be described by its distance to the nearest ambulance station (or the nearest three ambulance stations). Response times are more naturally defined in terms of these geographical coordinates.
REPLY BY KCEMS DEPT.: Time zones were established years ago based on distance from the ambulance station and proximity to population centers. We have proposed changes to the time zone geography to address population growth over the past 20 years and historic call volume.
If the provider does not meet the standards in a particular month, nothing seems to happen. If the provider does not meet the standards in a particular year, the EMS department may modify the zoning for a particular area (after consulting with the provider). This does not look like enforcement of compliance at all.
One would, at the very least, have to tie costs of the permits or the levels of county tax to performance.
It says that "The ambulance provider will take precautions to assure that no zone within the EOA is chronically underserved." But this does not define "chronically" and it does not specify any form of sanction.
REPLY BY KCEMS DEPT.: Response time enforcement occurs at three levels, as follows:
Except from the Draft Ambulance Service Performance Standards, Section IX, D. 1. states, “Compliance is achieved when 90 percent or more of calls for each priority by zone, in each Exclusive Operating Area (EOA) meets the specified response time criteria over a month.. . .”
Except from the Draft Ambulance Service Performance Standards, Section IX, G. 2. states, “. . . Any priority code, by zone, resulting in less than the 90 percent response time performance is non-compliant with the Standards.”
Except from the Ambulance Service Contract, Section 3, Breach, states, “3.1.4. Failure of PROVIDER to meet the zone response time standards specified in the performance standards for three consecutive months in the same zone, or four months in any consecutive 12-month period in the same zone.” (is sufficient grounds for declaring breach of contract).
Except from the Ambulance Ordinance, Section 8.12.170 Complaint and investigation procedures., states:
D. Infractions are violations of the provisions of this chapter. An egregious violation that is of a nature so serious as to pose an imminent threat to public health and safety is a misdemeanor.
E. An infraction shall include, but is not limited to:
g. Failure to meet the zone response time standards for three consecutive months in the same zone, or four months in any consecutive 12-month period in the same zone
As you can see, there is ample authority to enforce the standards, and the standards are very rigorous.
Leeuw makes some kind of reference to cost of permit to levels of county tax to performance. There is not sufficient information to respond to this concern. In other words, I don’t know what he’s talking about.
The word “chronically” has been removed from the proposed performance standards.
Response time is time between "call" and "arrival at scene." There is no problem with reliably establishing "call" time, since it will be in the dispatch records. But "arrival at scene" time is more difficult to measure reliably.
For various reasons, there may be a time interval between "wheels stopped" and "contact with the patient," and it is obvious that "contact with the patient" is the more important time point.
Also, "arrive at the scene" time is established when the dispatcher is notified by ambulance crew. There should be information in the document about the validity and reliability of the "arrive at the scene" time measurement.
REPLY BY KCEMS DEPT.: The standard response time measurement used to measure ambulance performance is arrival at scene. Occasionally, clinical studies are performed to measure the time it takes to be at the patient’s side. However, this is not the standard being proposed to the Board of Supervisors that we use to measure ambulance performance on a daily basis.
Conceivably, one measurement could be substituted for the other, but the result would be that the time zone geography would need to be changed to account for the new way of measuring response times. The change would not result in better performance; just a different measurement.
Ambulance companies are required to honestly report their “arrived at scene” times to the Department. Failure to report accurate information is grounds for declaring breach of the contract. Most often, a fire department will respond to an emergency call with an ambulance. If the fire department observes the ambulance company stating they are on-scene when in fact they have not yet arrived will result in a complaint to the EMS Department. Similarly, if a hospital is reporting an ambulance company is late for transfers, but the ambulance company is reporting they arrived on time, the EMS Department will receive a complaint. Such trends will prompt the Department to conduct an audit of ambulance companies dispatch log and compare it to the GPS vehicle locator data, and any other information sources available to us to verify compliance.
If more than one ambulance is required, the response of the first arriving ambulance determines compliance. Although this is probably a rare event, it does imply some bias in the measurements – it would make more sense to use the response time of the last arriving
ambulance. For instance, if two ambulances are needed and the second one does not arrive at all, that could not reasonably be called compliance.
REPLY BY KCEMS DEPT.: The response time requirements are based on the need for one ambulance per incident. On occasion, more than one ambulance is needed. However, we cannot feasibly build an EMS system that is capable of responding multiple ambulances to every call within the required response time – such a proposal is unrealistic.
In instances in the past, when multiple ambulances are needed, the ambulance providers have dispatched them immediately, and the units have arrived just as soon as they could get there. This has NOT been a problem in the past, and I cannot imagine it ever being an issue. If multiple resources are needed, they are sent immediately – period. Holding an ambulance provider contractually responsible for having multiple ambulance on-scene within the same response time as the first ambulance is completely unrealistic in a County our size.
There are clearly valid reasons to grant exceptions to the response time standards. Nevertheless EMS has to be very careful here. For instance, if an exception is requested based on "weather and roadway conditions", the Department has to make sure these conditions actually were severe enough to warrant the exception.
This has to take into account, for example, that rural areas already have considerably longer response times. And that rural areas demand appropriately customized ambulances and equipment.
If suitable infrastructure is available, then "weather and roadway conditions" should not be used as a reason for an exception (for the same reason as equipment failures and traffic congestion cannot be used, see IX F3).
REPLY BY KCEMS DEPT.: Response time requirements are based on good weather conditions. If the roads are covered with snow or ice, regardless of whether the ambulance is equipped with automatic chains, the response will be slower than under normal conditions. This is an unavoidable consequence of poor weather. For example, if under good road conditions, it takes about 25 minutes for the ambulance to respond from Frazier Park to Pine Mountain Club, then during the winter, response times will be slower. If Mil Portero is iced over, it is not physically possible for the ambulance to arrive within the same time frame as if the road is in good condition.
Numerous calls for 4-wheel drive vehicles have been made by the community. Response by Frazier Park unit of Hall Ambulance using automatic chains last winter showed malfunction, prolonging response time. There seems to be confusion as to whether the contractor which holds the exclusive operating franchise in an area should be held accountable to actually serve that area adequately, no matter what its altitude and weather challenges.
The "overload score" for an hour is defined as the mean plus 1.5 times the standard deviation for priority 15 calls dispatched at that hour for the past 20 weeks.
So if there have been 10 calls on average at 4 p.m. over the last 20 weeks, with standard deviation 4, then 10 + 1.5 × 4 = 16 is the overload score.
Thus a provider handling more than 16 calls is in overload and can get a response time exception. In the case of an approximate normal distribution of calls, this will mean there is overload 7% of the time.
If the load over 20 weeks is approximately constant, then the standard deviation is almost zero, and the provider is in overload any time the call volume is above the mean (which will happen about 50% of the time).
In order to form an opinion about the reasonableness of this rule, one would have to have an idea about the actual value of the mean and standard deviation.
Clearly, if the standard deviation is really small, the rule is not reasonable, because it provides an exception in a situations which is not at all rare.
REPLY BY KCEMS DEPT.: The overload score will be used to establish a reasonable “level of effort” for the ambulance provider. If the formula results in a score of 2, this means that the ambulance provider can reasonably expect to have two calls dispatched within that hour. If a third call is dispatched within that hour, and the ambulance provider can verify that he was already working on two calls during that time, then the ambulance provider’s late response on the third call will not be counted against him. In other words, it was not reasonable to expect that third call (it was an unexpected spike in demand), therefore, the time compliance is waived for that third call. The overload score is currently being used in Arlington, TX; Ft. Worth, TX; Monterey County, CA; and Scottsdale, AZ.
100 Response Rule:
Compliance on a monthly basis in any priority category is only required if there are more than 100 responses in that priority category. Otherwise the last 100 responses will be used to establish compliance.
This means, for instance, that wilderness areas may not have ac cumulated 100 responses total, so there are no compliance requirements.
It also means that if service deteriorates in low density areas, then it will take relatively long until the provider is out of compliance.
Again, to evaluate the reasonableness of this rule, one would need to know the number of responses in the various zones over time.
REPLY BY KCEMS DEPT.: The purpose of the 100 response rule is to establish a reasonable history of calls in areas where there is typically fewer calls. The ambulance provider is in breach of contract if response time performance standards lag for three consecutive months in the same zone, or four months in any consecutive 12-month period in the same zone.
If there are two calls in the wilderness zone in January, and one call is on-time and the other is late, the ambulance provider has a response time compliance of 50 percent. If this were to persist for four months within the year, the ambulance provider would be in breach of the contract. Given the infrequency of the responses, it is unrealistic to hold the ambulance provider to such a high standard – no one would be able to meet this standard. Therefore, the 100 response rule provides a reasonable population of calls in which to build a compliance base.
In order to find out if the 90% limit, the overload rule, and the 100 response rule are appropriate, or are appropriate for any given area, one would need to have detailed monthly information about call volume and response times from EMS.
In other words, one would need to have the data mentioned in Section X on Records and Reports. I don’t know if these data are publicly available in a suitable electronic format.
REPLY BY KCEMS DEPT.: Not all of the data being requested in Section X is available to the Department today. The proposed reporting requirements will over time greatly improve the data available to the Department. The annual reports listed in Section XII of the Ambulance Performance Standards will be available for public review, and such transparency of information should allow any interested member of the public to monitor the performance of the EMS system.
Department of Statistics, University of California, Los Angeles, CA 900951554 Email address, Jan de Leeuw: firstname.lastname@example.org URL, Jan de Leeuw: http://gifi.stat.ucla.edu
Data derived from http://www.co.kern.ca.us/ems/news.asp