Abstract

The ability to precisely classify different types of terrain is extremely important for Unmanned Aerial Vehicles (UAVs). There are multiple situations in which terrain classification is fundamental for achieving a UAV's mission success, such as emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation. Previous research works describe different terrain classification approaches mainly using static features from RGB images taken onboard UAVs. In these works, the terrain is classified from each image taken as a whole, not divided into blocks; this approach has an obvious drawback when applied to images with multiple terrain types. This paper proposes a robust computer vision system to classify terrain types using three main algorithms, which extract features from UAV's downwash effect: Static textures- Gray-Level Co-Occurrence Matrix (GLCM), Gray-Level Run Length Matrix (GLRLM) and Dynamic textures- Optical Flow method. This system has been fully implemented using the OpenCV library, and the GLCM algorithm has also been partially specified in a Hardware Description Language (VHDL) and implemented in a Field Programmable Gate Array (FPGA)-based platform. In addition to these feature extraction algorithms, a neural network was designed with the aim of classifying the terrain into one of four classes. Lastly, in order to store and access all the classified terrain information, a dynamic map, with this information was generated. The system was validated using videos acquired onboard a UAV with an RGB camera.

Original languageEnglish
Article number2501
JournalRemote Sensing
Volume11
Issue number21
DOIs
Publication statusPublished - 1 Nov 2019

Fingerprint

imagery
matrix
texture
computer vision
vehicle
research work
hardware
navigation
decision making

Keywords

  • Downwash effect
  • FPGA
  • GLCM
  • GLRLM
  • image processing
  • Optical flow
  • Terrain classification
  • Texture
  • UAV

Cite this

@article{e70ef85025e54738a1ead73bd9179790,
title = "Static and dynamic algorithms for Terrain classification in UAV Aerial Imagery",
abstract = "The ability to precisely classify different types of terrain is extremely important for Unmanned Aerial Vehicles (UAVs). There are multiple situations in which terrain classification is fundamental for achieving a UAV's mission success, such as emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation. Previous research works describe different terrain classification approaches mainly using static features from RGB images taken onboard UAVs. In these works, the terrain is classified from each image taken as a whole, not divided into blocks; this approach has an obvious drawback when applied to images with multiple terrain types. This paper proposes a robust computer vision system to classify terrain types using three main algorithms, which extract features from UAV's downwash effect: Static textures- Gray-Level Co-Occurrence Matrix (GLCM), Gray-Level Run Length Matrix (GLRLM) and Dynamic textures- Optical Flow method. This system has been fully implemented using the OpenCV library, and the GLCM algorithm has also been partially specified in a Hardware Description Language (VHDL) and implemented in a Field Programmable Gate Array (FPGA)-based platform. In addition to these feature extraction algorithms, a neural network was designed with the aim of classifying the terrain into one of four classes. Lastly, in order to store and access all the classified terrain information, a dynamic map, with this information was generated. The system was validated using videos acquired onboard a UAV with an RGB camera.",
keywords = "Downwash effect, FPGA, GLCM, GLRLM, image processing, Optical flow, Terrain classification, Texture, UAV",
author = "Matos-Carvalho, {J. P.} and Filipe Moutinho and Salvado, {Ana Beatriz} and Tiago Carrasqueira and Rog{\'e}rio Campos-Rebelo and D{\'a}rio Pedro and Campos, {Luis Miguel Braga da Costa} and Fonseca, {Jos{\'e} M.} and Andr{\'e} Mora",
note = "This work was not possible without the support and commitment of PDMFC Research group. This work was also supported by Portuguese Agency {"}Funda??o para a Ci?ncia e a Tecnologia{"} (FCT), in the framework of projects PEST (UID/EEA/00066/2019) and IPSTERS (DSAIPA/AI/0100/2018). This work is supported by the European Regional Development Fund (FEDER), through the Regional Operational Programme of Lisbon (POR LISBOA 2020) and the Competitiveness and Internationalization Operational Programme (COMPETE 2020) of the Portugal 2020 framework [Project 5G with Nr. 024539 (POCI-01-0247-FEDER-024539)]. This project has also received funding from the ECSEL Joint Undertaking (JU) under grant agreement No 783221. The JU receives support from the European Union's Horizon 2020 research and innovation programme and Austria, Belgium, Czech Republic, Finland, Germany, Greece, Italy, Latvia, Norway, Poland, Portugal, Spain, Sweden.",
year = "2019",
month = "11",
day = "1",
doi = "10.3390/rs11212501",
language = "English",
volume = "11",
journal = "Remote Sensing",
issn = "2072-4292",
publisher = "Molecular Diversity Preservation International (MDPI)",
number = "21",

}

TY - JOUR

T1 - Static and dynamic algorithms for Terrain classification in UAV Aerial Imagery

AU - Matos-Carvalho, J. P.

AU - Moutinho, Filipe

AU - Salvado, Ana Beatriz

AU - Carrasqueira, Tiago

AU - Campos-Rebelo, Rogério

AU - Pedro, Dário

AU - Campos, Luis Miguel Braga da Costa

AU - Fonseca, José M.

AU - Mora, André

N1 - This work was not possible without the support and commitment of PDMFC Research group. This work was also supported by Portuguese Agency "Funda??o para a Ci?ncia e a Tecnologia" (FCT), in the framework of projects PEST (UID/EEA/00066/2019) and IPSTERS (DSAIPA/AI/0100/2018). This work is supported by the European Regional Development Fund (FEDER), through the Regional Operational Programme of Lisbon (POR LISBOA 2020) and the Competitiveness and Internationalization Operational Programme (COMPETE 2020) of the Portugal 2020 framework [Project 5G with Nr. 024539 (POCI-01-0247-FEDER-024539)]. This project has also received funding from the ECSEL Joint Undertaking (JU) under grant agreement No 783221. The JU receives support from the European Union's Horizon 2020 research and innovation programme and Austria, Belgium, Czech Republic, Finland, Germany, Greece, Italy, Latvia, Norway, Poland, Portugal, Spain, Sweden.

PY - 2019/11/1

Y1 - 2019/11/1

N2 - The ability to precisely classify different types of terrain is extremely important for Unmanned Aerial Vehicles (UAVs). There are multiple situations in which terrain classification is fundamental for achieving a UAV's mission success, such as emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation. Previous research works describe different terrain classification approaches mainly using static features from RGB images taken onboard UAVs. In these works, the terrain is classified from each image taken as a whole, not divided into blocks; this approach has an obvious drawback when applied to images with multiple terrain types. This paper proposes a robust computer vision system to classify terrain types using three main algorithms, which extract features from UAV's downwash effect: Static textures- Gray-Level Co-Occurrence Matrix (GLCM), Gray-Level Run Length Matrix (GLRLM) and Dynamic textures- Optical Flow method. This system has been fully implemented using the OpenCV library, and the GLCM algorithm has also been partially specified in a Hardware Description Language (VHDL) and implemented in a Field Programmable Gate Array (FPGA)-based platform. In addition to these feature extraction algorithms, a neural network was designed with the aim of classifying the terrain into one of four classes. Lastly, in order to store and access all the classified terrain information, a dynamic map, with this information was generated. The system was validated using videos acquired onboard a UAV with an RGB camera.

AB - The ability to precisely classify different types of terrain is extremely important for Unmanned Aerial Vehicles (UAVs). There are multiple situations in which terrain classification is fundamental for achieving a UAV's mission success, such as emergency landing, aerial mapping, decision making, and cooperation between UAVs in autonomous navigation. Previous research works describe different terrain classification approaches mainly using static features from RGB images taken onboard UAVs. In these works, the terrain is classified from each image taken as a whole, not divided into blocks; this approach has an obvious drawback when applied to images with multiple terrain types. This paper proposes a robust computer vision system to classify terrain types using three main algorithms, which extract features from UAV's downwash effect: Static textures- Gray-Level Co-Occurrence Matrix (GLCM), Gray-Level Run Length Matrix (GLRLM) and Dynamic textures- Optical Flow method. This system has been fully implemented using the OpenCV library, and the GLCM algorithm has also been partially specified in a Hardware Description Language (VHDL) and implemented in a Field Programmable Gate Array (FPGA)-based platform. In addition to these feature extraction algorithms, a neural network was designed with the aim of classifying the terrain into one of four classes. Lastly, in order to store and access all the classified terrain information, a dynamic map, with this information was generated. The system was validated using videos acquired onboard a UAV with an RGB camera.

KW - Downwash effect

KW - FPGA

KW - GLCM

KW - GLRLM

KW - image processing

KW - Optical flow

KW - Terrain classification

KW - Texture

KW - UAV

UR - http://www.scopus.com/inward/record.url?scp=85074663080&partnerID=8YFLogxK

U2 - 10.3390/rs11212501

DO - 10.3390/rs11212501

M3 - Article

VL - 11

JO - Remote Sensing

JF - Remote Sensing

SN - 2072-4292

IS - 21

M1 - 2501

ER -