Testing medium-cost UAVs for cartographic and topographic mapping

DJI Phantom 4

As UAV technology is rapidly developing, the question of the economic feasibility of using certain UAVs for certain tasks often arises. We decided to test several drones and cameras of the popular DJI manufacturer in the middle and lower price segments for the accuracy of the data obtained for large-scale mapping in difficult terrain.

The survey was carried out at the testing ground of the Kharkiv National Agrarian University (Dokuchaevske village of the Kharkiv region) with a total area of ​​about 5.5 hectares. The territory of the grounds has a distinctly typical relief pattern for the forest-steppe Ukraine – a creek.The elevation difference between the lowest and highest points is 23,872m. Test flights were conducted under similar weather conditions – clear, windless, sunny weather. The surveys were made at the time of the “high” sun to minimize the possible shadows, and capture the most natural color scheme.

рис.1

The test grounds with GCPs

 

The approach

14 ground control points (the “GCP”) were fixed at the site before the start of the test flights, they were used to assess the accuracy of the results. The GCPs were fixed permanently on the ground with markers (white plastic discs), which could be clearly distinguished among the vegetation and soil cover. The coordinates of the reference points were determined twice (for accuracy control) using the electronic station “Leica TCR 405”. The differences between the first and second survey of the coordinates of the control points were: plain coordinates (X, Y) +/- 2-5 mm, elevation (H) +/- 5-7 mm, which is a permissible deviation given the terrain.

IMG_20170524_100848

 

For testing, six quadcopters and eight cameras were used. We used “Pix4D Capture” and “DJI Ground Station Pro” applications for displaying telemetry, these are the most stable and have positive feedback from users, compared to dozens of existing ones. As far as we know, and it is practically not proved, that any combination of mobile gadgets with a UAV does not influence the final result. Certain issues still arise, but as a rule, they manifest themselves immediately and can have some effect on the convenience and speed of organization and performance of work. For the most part, these are related to connection loss, software compatibility, and so on, and can be fixed mainly by a few clicks in device settings or after software updates.

 

Characteristics of the UAVs and cameras used

UAV Weight, gr Max. speed,

m/s

Camera Resolution, Mpx
DJI Mavic Air 430 18 FC220 12
DJI  Inspire-1 2845 10 X3 FC350 12
X5 FC550 16
DJI Phantom 2 Vision+ 1242 15 FC200 14
DJI Phantom 3 Advanced 1280 16 FC300C 13
DJI Phantom 4 Pro 1388 20 FC6310 20
Custom «Lady Bug» 3000 10 Canon Power Shot S100 12
Hawkeye Firefly 6C 12

 

For testing, we used several missions (survey routes) of automated flight at 25, 50 and 100 m elevations with 80% overlapping of images. During the tests, 31 flight missions were completed, 5 of which were not included in the processing due to various issues caused both by technical and human factors. The main reasons for the issues are as follows: pilot errors during flight designing; certain inaccuracies in base maps in mobile applications; difficulty to define landmarks in terrain and base maps; uniform color patterns (especially in the summer season), which, in clear sunny weather, is disorienting for the pilot when setting up the flight route; the absence of a “buffer area” around the study territory. As a result of these shortcomings, the optimal overlapping of images in the peripheral zones of the study area was not ensured. Technical failures also occur, that lead to the need for an additional mission, but the frequency of technical issues is usually an order of magnitude less than the human error.

IMG_20170526_112626

 

Results

As a result of the flight tasks, the construction of digital models of relief and orthophoto maps was carried out using special software. The created DTMs were divided into 2 groups: 1) models constructed according to coordinates obtained from onboard GPS; 2) models built using GCPs. For the second group of models, georeferencing was carried out using up to 6 GCP, numbers: 1, 2, 3, 4, 5, 14.

When postprocessing aerial imagery (and I would call it “drone imagery”), given the above reasons, a number of DTMs were discarded. The total number of models that were included in the analysis: 14 – without georeferencing to the GCPs, per coordinates of the UAV taken by onboard GPS (not using RTK); 2) 13 – georeferenced to GCPs. The difference in the number between the first and second groups is due to the fact that some of the above-mentioned points of reference were not present in some routes.

2016-06-23_222447

The tables below summarize the results of the processing of the obtained models and present the acceptable scale of plotting of cartographic and topographic materials based on the received digital models.

Based on the results of planar and elevation mean-square errors, and based on the fact that the accuracy of the plans is determined by the accuracy of its scale, and the accuracy of the construction of topographical plans is determined also by the nature of the relief (angle of inclination) and the height of the section of the relief – the conventionally indicated recommended scale is divided into 2 groups: planar (X, Y); and topographic (H). The “-” marks the results that have an error that significantly exceeds the permissible values ​​for a given scale in the conditions of the terrain of the testing area.

 

Obtained results for the models constructed from onboard GPS coordinates of the UAV

UAV Camera Scale
planar elevation
Altitude 25 m
DJI  Inspire-1 X3 FC350 1:25000 1:500
DJI Phantom 3 Advanced FC300C 1:5000 1:2000
Custom «Lady Bug» Canon Power Shot S100 1:25000
Altitude 50 m
DJI Mavic FC220 1:50000
DJI  Inspire-1 X3 FC350 1:25000
DJI  Inspire-1 X5 FC550 1:10000 1:5000
DJI Phantom 2 Vision+ FC200 1:25000 1:500
DJI Phantom 3 Advanced FC300C 1:5000 1:500
DJI Phantom 4 Pro FC6310 1:25000 1:2000
Custom «Lady Bug» Canon Power Shot S100 1:5000
Altitude 100 m
DJI  Inspire-1 X3 FC350 1:25000 1:500
DJI  Inspire-1 X5 FC550 1:25000 1:2000
DJI  Inspire-1 (повторність) X5 FC550 1:25000 1:2000
DJI Phantom 4 Pro FC6310 1:25000 1:2000

 

Obtained results for the models constructed using  to GCPs

UAV Camera Scale
planar elevation
Altitude 25 m
DJI  Inspire-1 X3 FC350 1:25000
DJI Phantom 3 Advanced FC300C 1:2000
Custom «Lady Bug» FIREFLY6S 1:10000
Custom «Lady Bug» Canon Power Shot S100 1:5000
Altitude 50 m
DJI Mavic FC220
DJI  Inspire-1 X3 FC350 1:25000
DJI  Inspire-1 X5 FC550 1:10000
DJI Phantom 2 Vision+ FC200 1:2000 1:500
DJI Phantom 4 Pro FC6310 1:50000
Custom «Lady Bug» Firefly 6C 1:1000
Custom «Lady Bug» Canon Power Shot S100 1:2000
Altitude 100 m
DJI  Inspire-1 X5 FC550 1:25000
DJI Phantom 4 Pro FC6310 1:10000

 

Unfortunately, we were not able to conduct repetitive flights for most drones/cameras, but we can draw some conclusions from the results. These have some differences from the studies of our colleagues, in this case, towards the reduction of accuracy. In our opinion, this reduction in the accuracy of the results can partly be explained by the lack of using RTK-positioning, the more complex relief of the research site, and the lack of correction and distribution systems for corrections to GPS data – WAAS (wide-area augmentation system) and EGNOS (European geostationary navigation overlay service) that are inherent in North America and Western Europe respectively.

As for the results for the models constructed onboard coordinates of UAV, it is interesting to note that the accuracy of the high-altitude survey was higher than planned. It is worth noting the good results of the DJI Phantom drones, which represent a number of the most affordable ones among those we used in the tests. DJI Phantom 3 Advanced is the most impressive. Unfortunately, the economic and time factors have prevented the repeated flights at those heights and routes that were discarded. It would be very interesting to get a complete picture of each of the heights of the flight for each UAV, which showed a satisfactory result.

Regarding the analysis of the results for the second group of models constructed with GCP coordinates, this is not all so clear. Only one drone achieved the required accuracy for high-altitude surveying – it was the most affordable DJI Phantom 2 Vision+. 

As for the accuracy of obtaining planar coordinates, from the table, for the most part, we see an aggregation of allowable scales, which indicates an increase in the accuracy of the data obtained. Therefore, the use of GCP is a justifiable measure, but there remains an open question in their number, density, and positioning. Analyzing international experience, we have only certain recommendations that are generally not methodical. Each scientist, based on the results of their research, has a certain point of view justified for a particular territory.

We will try to research it further, given the gained experience, hopefully, the research will be timely and rich in information.

 

UAV mapping: international reviews

How accurate is your drone survey? Everything you need to know 

The Resultant Positional Accuracy for the Orthophotos Obtained with Unmanned Aerial Vehicles (UAVs) 

Precision Mapping with drones without the need of Ground Control Points

Accuracy assessment on low altitude UAV-borne photogrammetry outputs influenced by ground control point at different altitude