Evaluating sUAS Drone Pilot Skills

NIST has developed a turnkey set of sUAS Standard Test Methods (USTMs) that are available free of charge.

More than 1,700 public safety agencies have acquired small unmanned aircraft systems (sUAS), about one-third of them fire-service oriented. Although commonly called drones, sUAS are specifically defined as unmanned aircraft weighing less than 55 pounds. Fire departments are using sUAS to gain additional information at structure and wildland fires scenes and at natural and man-made disasters. sUAS tasks include providing real-time video for fire scene management, penetrating hot zones such as hazmat spill areas and potential boiling liquid/expanding vapor explosion evacuation zones, post-disaster damage assessment, and post-response assessment and evaluation for training. The relatively low cost of sUAS makes the technology affordable for most fire service agencies.

Part 107 Regulations

The Federal Aviation Administration (FAA) enacted 14 CFR Part 107, the “Small Unmanned Aircraft System Rule,” in 2016. Part 107 regulations address sUAS regulatory issues including maximum altitude; maximum velocity; cloud clearances; minimum visibility; and, most applicable to this discussion, remote pilot certification. Part 107 created a new FAA pilot certification, the remote pilot certificate (RPC). Applicants for an RPC must pass a 60-question FAA examination that covers 12 aeronautical subject matter areas, apply for the certificate in person at an FAA Flight Standards District Office, or apply using the FAA’s online Integrated Airman Certificate and Rating Application and be vetted by the Transportation Security Administration. The FAA Remote Pilot Examination covers a broad spectrum of topics including airspace, aeronautical charts, meteorology, aeronautical decision making, and the specifics of Part 107 regulations. Interestingly, an individual can obtain an FAA RPC without ever having flown a sUAS. The lack of an FAA remote pilot practical skills exam leaves public safety agencies on their own for establishing standards for remote pilot minimum skills. Agencies that fail to establish flight skills standards risk exposure to additional liability in their sUAS flight operations.

Pilot Certification Standards

Fifteen years ago, Adam Jacoff, an engineer and project manager at the National Institute of Standards and Technology (NIST), established a team to develop standard test methods for drones and is now leading an international effort to develop standard test methods for sUAS. The initial suites for Maneuvering and Payload Functionality can be used to quantitatively evaluate various system capabilities and remote pilot proficiency. They are being standardized through the ASTM International Standards Committee on Homeland Security Applications Response Robots (ASTM E54.09). They are also referenced as job performance requirements in National Fire Protection Association 2400, Standard for Small Unmanned Aircraft Systems Used For Public Safety Operations, and the ASTM Standard Guide for Training for Remote Pilot in Command of Unmanned Aircraft Systems Endorsement (ASTM F38.03 F3266-18). The U.S. Department of Homeland Security’s Science and Technology Directorate has been supporting the development of these tests.

RELATED FIREFIGHTER TRAINING

UAVs in Public Safety: The Next Steps in Technology

Command, Coordination, and Drones Key at Bowling Alley Fire

The Expanding World of Unmanned Aerial Vehicles in the Fire Service

NIST has developed a turnkey set of sUAS Standard Test Methods (USTMs) that are available free of charge for adoption and use by agencies that wish to either internally credential sUAS pilots or serve as a national credentialing resource. The Airborne Public Safety Accreditation Commission is strongly considering adopting the NIST USTMs as a portion of their sUAS Unit Accreditation Program, as is the Civil Air Patrol (an auxiliary of the United States Air Force), which has more than 1,200 volunteer remote pilots operating more than 2,300 sUAS nationwide.

Test Lane Protocols

The NIST USTMs encompass four “test lane” protocols: Basic Proficiency Evaluation for Remote Pilots (Part 107 Qualification), Open Test Lane, Obstructed Test Lane, and Confined Test Lane. The test methods may be used to evaluate sUAS airframes and sensor systems and remote pilot evaluation and credentialing. They are easily performed using test apparatus that can be assembled inexpensively with materials readily available at any large hardware store. NIST has created a comprehensive user guide, scoring sheets, and apparatus target printable documents.

NIST: Standard Test Methods for Small Unmanned Aircraft Systems

The Basic Proficiency Evaluation for Remote Pilots (BPERP) is the entry-level test method. It is designed to complement the FAA’s Part 107 RPC by providing an inexpensive, easily duplicable mechanism for assessing remote pilot flying skills. The BPERP can be administered in 10 minutes using three bucket stands, a 50-foot tape measure, and a stopwatch. The BPERP requires a compact test area of 50 feet × 20 feet, so it can easily be administered indoors or outdoors (Figure 1).

Figure 1. Evaluation Setup

Source: National Institute of Standards and Technology.

The BPERP requires the remote pilot to conduct three takeoffs and landings from a 12-inch radius circle; climb to specified altitudes of 10 feet and 20 feet above ground level; conduct yawing turns; and conduct forward, reverse, and traverse flight maneuvers. The goal is to capture still images of 36 targets that are placed within two-gallon buckets that are fastened to three bucket test stands constructed from 2- × 4-inch and 4- × 4-inch lumber. The bucket stands are easy to assemble and can be transported in a couple of nylon golf club bags or simply stacked and placed in a vehicle (Figure 2).

Figure 2. Evaluation Setup Components

Source: National Institute of Standards and Technology.

The test consists of one maneuvering phase and two traverse flight phases. Pilots earn one point for each accurately captured target image, two points for an accurate first landing, and one point each for accurate second and third landings. Scoring sheets are available from NIST; agencies set their own benchmark scores for passing the test.

Ben Miller, director of the Colorado Center of Excellence for Advanced Technology Aerial Firefighting (CoE), has followed NIST’s USTM project from its inception: “The method supports acquisition decisions as well as employment considerations. The NIST sUAS Standard Test Methods produce data that can be used to answer the questions of what system do I buy and what system do I use for which mission?”

Miller believes that the NIST USTMs have great applicability to public safety sUAS operations. “The Colorado Department of Public Safety has adopted the NIST sUAS Standard Test Methods within our UAS certification process. The CoE provides this certification process to stakeholder public safety agencies within Colorado. To date, 16 agencies and 42 sUAS operators have gone through the process.”

Adoption of NIST USTMs provides public safety agencies with a mechanism for evaluating sUAS airframes and sensor systems and raises the bar on remote pilot credentialing. The combination of requiring remote pilots to obtain an FAA RPC and achieve a minimum passing score on the NIST sUAS BPERP will provide fire agencies with a solid foundation and show due diligence in their efforts to ensure safety of their sUAS operations.

Author’s note: Throughout the next year, the Airborne Public Safety Association (APSA) will present several NIST sUAS Standard Test Methods Train-the-Trainer Workshops throughout the nation. These three-day courses are appropriate for experienced sUAS pilots who serve as trainers, supervisors, and managers within sUAS units. Check the APSA Web site, www.publicsafetyaviation.org, for dates and locations of upcoming workshops. Agencies wishing to co-host a workshop should contact me at alan.frazier@georgetown.edu.


Alan Frazier is a senior fellow at Georgetown University. He is assigned to the National Institute of Standards and Technology, where he serves on a team developing standard test methods for small unmanned aircraft systems.

No posts to display