How are CSA scores calculated?
Updated: Jun 22, 2021
In another post, we explored 5 surefire ways to improve your CSA/SMS scores. But before you work to improve your scores, you should first understand how they are calculated, at least in a broad sense. In this article, we'll tackle the SMS scoring methodology, but generally speaking:
The FMCSA calculates a motor carrier's CSA scores--also known as Safety Measurement System or SMS scores--with data gathered through roadside inspections, crashes, and investigations occurring within the past 24 months. Violations discovered through inspections and/or investigations are assigned to one of seven categories known as BASICs, and then weighted based, in part, on their severity and how recently they occurred. Each carrier's performance is then evaluated against similarly-situated motor carriers in each BASIC, and the carrier is assigned a percentile score in each BASIC.
What is CSA?
Since 2010, the FMCSA has used its Compliance, Safety, Accountability (CSA) system to prioritize federally-regulated motor carriers for enforcement action. The Safety Measurement System (SMS) is one major component of CSA, which ranks carriers against their peers in seven categories known as the BASICs.
A carrier’s CSA/SMS scores are primarily a function of the number, type, and severity of regulatory violations uncovered during roadside inspections and/or compliance reviews. Carriers that perform worse than a certain percentage (e.g., 65%) of their peers in the BASIC categories are placed higher on FMCSA’s prioritization list and are more likely to receive warning letters, fines, and compliance reviews. For this reason—and because poor SMS scores can also lead to increased insurance premiums and loss of business—it is imperative that carriers understand how their scores are calculated and consistently monitor and work to improve them.
What factors play into a carrier's CSA scores?
CSA scores are calculated according to the FMCSA's SMS Methodology, which is publicly available to download at this link. But fair warning...the methodology is 188 pages of dense reading and is not for the faint of heart! Fortunately, the scoring methodology can be understood in the following broader strokes:
Carriers are assigned scores in 7 BASICS. The FMCSA assesses motor carrier on-road performance and compliance by grouping investigation and inspection results into 7 categories known as the Behavior Analysis & Safety Improvement Categories (BASICs). Carriers are scored in each of the 7 BASICs, which results in what we know as CSA or SMS scores. The 7 BASICs are as follows:
Unsafe Driving: The Unsafe Driving BASIC pertains to the operation of commercial motor vehicles (CMVs) by drivers in a dangerous or careless manner. Examples include speeding, reckless driving, improper lane change, and inattention.
Crash Indicator: the Crash Indicator BASIC considers histories or patterns of high crash involvement, such as frequency and severity. It is based on information from State-reported crashes that meet reportable crash standards (e.g., fatalities, injuries requiring medical attention away from the scene, disabling damage requiring a vehicle to be towed).
HOS Compliance: The HOS Compliance BASIC includes violations of the regulations pertaining to records of duty status (RODS) and hours-of-service limits. Examples include a driver operating more hours than allowed under HOS regulations and falsification of RODS.
Vehicle Maintenance: The Vehicle Maintenance BASIC pertains to regulations regarding properly maintaining a CMV and preventing shifting loads, spilled or dropped cargo, and overloading of a CMV. Proper maintenance includes, among other things, ensuring that lamps and reflectors are working, and tires are not worn. Examples include operating an out-of-service vehicle, improper loading/securement, or operating a vehicle with inoperative brakes, lights, and/or other mechanical defects, and failure to make required repairs.
Controlled Substances/Alcohol: The Controlled Substances/Alcohol BASIC pertains to the operation of CMVs by drivers who are impaired due to alcohol, illegal drugs, and the misuse of prescription or over-the-counter medications. Examples include operating under the influence of drugs or alcohol.
Hazardous Materials Compliance: The HM Compliance BASIC addresses the requirements in Part 397 of the FMCSRs as well as those in the Hazardous Materials Regulations (HMRs). Examples include failing to mark, label, or placard in accordance with the regulations and not properly securing a package containing HM.
Driver Fitness: The Driver Fitness BASIC concerns the operation of CMVs by drivers who are unfit to operate a CMV due to a lack of training, experience, or medical qualifications. Examples include failure to have a valid and appropriate commercial driver’s license (CDL) and being medically unqualified to operate a CMV.
Every conceivable safety-related violation is assigned to one of the BASICs. As you might imagine, there are hundreds if not thousands of ways a motor carrier or its drivers might run afoul of the FMCSRs or HMRs--everything from a burned-out headlight all the way up to operating under the influence. For purposes of scoring carriers in each of the 7 BASICs, the FMCSA has assigned each and every conceivable violation to one of those BASICs. As a general rule, these are safety-related violations only, meaning that registration-type violations, for example, won't impact a carrier's scores. Each violation assignment is listed in a chart in the FMCSA's SMS Methodology.
Each violation carries a severity weight. As shown in the chart above, each violation is assigned a "severity weight," which reflects the FMCSA's assessment of its association with crash occurrence and crash consequences. Severity weights are on a scale from 1 to 10 for each BASIC, where 1 represents the lowest crash risk and 10 represents the highest crash risk relative to the other violations in the BASIC. Additionally, violations in certain BASICs that result in the driver or vehicle being placed out-of-service will carry an extra severity weight (e.g., violation's normal severity weight plus 2 additional points for an out-of-service violation).
Absolute BASIC measures are a function of violation severity and time.
Although each BASIC is scored slightly differently, generally speaking, each carrier is assigned an "absolute measure" score in each BASIC each month, based on the violations it incurred in those BASICs over the past 24 months. Generally speaking, the absolute measure score is a function of each violation's severity and time weight. In terms of time, each violation is assigned a weight based on how recently it occurred. Violations occurring in the past 6 months carry a time weight of 3. Those occurring between 6 and 12 months ago carry a time weight of 2. And those occurring between 12 and 24 months ago carry a time weight of 1.
A carrier's absolute measure score in the Vehicle Maintenance BASIC, for example, is calculated by multiplying each violation's severity weight by its time weight to arrive at a total weight, adding the total weights for all violations, and then dividing the total weight of all violations by the total time weight for all violations, as shown in the sample below. Some of the other BASICs factor in additional information besides severity and time, such as the average number of power units that the carrier has operated over a certain time frame.
Importantly, a carrier's absolute measure score is not, itself, what we know as a CSA or SMS score. The absolute measure score has to be compared against other motor carriers with similar operating characteristics to arrive at a percentile score, which is generally understood as the carrier's CSA score.
Consider this analogy: When my 2 year old son goes in for his annual checkup, the nurse measures his weight and height and compares those measures against his peers. The report we receive lists his actual weight (e.g., 26 lbs.) and height (e.g., 33"), and also his percentile for weight (e.g., 60%) and height (e.g., 53%). These percentiles tell us how he is growing relative to his peers, while his actual weight and height just tell us how he has progressed individually from the last report. In this sense, a carrier's absolute measure scores are akin to my son's actual height and weight, whereas its CSA scores are akin to his percentile measures.
Carriers are ranked against their peers in each BASIC each month. As noted, a carrier's absolute measure scores in a given month must be compared against those of its peers in order to arrive at the percentile CSA scores. Obviously, it would be inherently unfair to compare the scores of a large motor carrier with thousands of vehicles and corresponding violations against those of a single-truck motor carrier that has virtually no data in the system. For this reason, the SMS systems normalizes data and places carriers in "safety event groups" based primarily on the number of inspections they have incurred that have resulted in violations in a particular BASIC. For example, let's say Carrier A and Carrier B have both incurred 25 roadside inspections in the past 24 months that resulted in violations in the Vehicle Maintenance BASIC. In all likelihood, those carriers would fall within the same safety event group for this BASIC and their scores would be compared against one another (and others in the same group) to generate a percentile score.
CSA percentile scales are on a scale of 0 to 100, with 100 assigned to the worst performing carrier in the safety event group and 0 assigned to the best performing. Put differently, a carrier that has a score of 95% in the Vehicle Maintenance BASIC on a given month is performing worse than 95% of its peers in that category. Percentile scores are updated each month. The system also has built-in data sufficiency thresholds, so that carriers with very few inspections with violations in a particular category will not be scored in that category unless and until they incur a sufficient number of inspections with violations (e.g., 5 inspection with violations over a 24-month period) to meaningfully analyze and rank their performance.
Percentile scores are used to prioritize carriers for intervention. The primary purpose of the CSA and SMS systems is to prioritize carriers for enforcement action. With this in mind, the FMCSA has set "intervention thresholds" for each BASIC. These thresholds are expressed in percentile form, and if a carrier's CSA score exceeds an intervention threshold in a particular BASIC on a given month, the carrier's score will show a warning triangle (see sample above), and the carrier will be at an increased risk for intervention (e.g., warning letter, safety audit, increased number of inspections, civil penalties). The intervention thresholds differ for each BASIC and are based upon whether the carrier hauls hazmat or passengers. The following chart lists the current thresholds.
How do CSA scores improve?
If your scores are above the intervention thresholds, you're probably asking how to improve them. As noted, in a prior article we covered 5 ways to improve your SMS scores, but generally speaking, there are a couple of main ways that scores improve. The first and most common way is with the passage of time. As addressed above, individual violations weigh less on a carrier's scores as time passes and not at all after 24 months. So, as time passes, assuming you incur no more (or less weighty) violations in a BASIC, your scores will organically start to improve. This can obviously be more difficult than it sounds because it requires ensuring that drivers are properly trained and equipped so that they don't incur any additional (or at least fewer) violations in the problematic BASIC on a go-forward basis.
The second main way that CSA scores improve is through successful appeals of erroneous violations through the FMCSA's DataQs system. The DataQs system is the exclusive means by which motor carriers can challenge these erroneous violations that weigh on its scores. DataQs appeals are fairly informal, but they do require carriers to submit details and evidence to support their contention that a particular violation was incorrectly assessed. And, at least in my experience, if the appeal comes down to your word against the officer's, your chances of success are very slim. Nevertheless, if you prevail on a DataQs challenge, the associated violation will either be removed or weigh less heavily on your scores the next month.
Conclusion
The FMCSA's methodology for calculating motor carriers' CSA scores is fairly complex, but from a high level, the scores are really just a function of the number, severity, and timing of violations incurred by a carrier in roadside inspections or safety audits. Because the FMCSA uses those scores to prioritize carriers for enforcement action, it's important that you understand how your scores are calculated and what steps you need to take to improve them on a go-forward basis.
For deep-dive video lessons on improving CSA scores and other safety metrics, be sure to check out our DOT Compliance: Enforcement Master Class offered through Trucksafe Academy. And if you need help evaluating your scores or developing a plan to improve them, please feel free to contact us!