Terrain Classification Using Vision Transformer for Autonomous Off-road Vehicle Navigation Systems

The use of autonomous vehicles in off-road environments is growing. The primary challenge of these systems is to navigate diverse and unpredictable terrain, which significantly impacts vehicle performance. This research develops a terrain classification system for off-road autonomous vehicles using the Vision Transformer (ViT) architecture. Considering the challenges of unstructured environments such as texture diversity, lighting variations, and surface roughness, the proposed model is designed based on ViT-base-patch16-224 and trained on a dataset consisting of 1757 images of 6 terrain classes (soil, rocky, grassy, muddy, gravelly, and hard). The results show that the model achieves an overall accuracy of 97% on the test data and an average F1-score of 0.90. The analysis of the confusion matrix and ROC curves indicates the model's high ability in class discrimination. However, challenges were observed in recognizing the soil class (25% error) due to the visual similarity with muddy terrain. Comparing class-by-class metrics, the model performs better in grass (F1=0.96) and gravel (F1=0.94) than in mud (F1=0.82). This study demonstrates that the ViT attention-based architecture, despite data limitations, possesses a high capability in extracting high-level features from complex images and can be utilized as part of the perceptual system of autonomous vehicles in unstructured environments. The findings suggest that adjusting class-by-class thresholds and increasing training data can improve the model's performance, particularly for complex classes.
Introduction
The application of off-road autonomous vehicles is rapidly expanding across diverse sectors such as agriculture, mining, military operations, forestry, environmental monitoring, and planetary exploration. Unlike urban autonomous vehicles that operate within structured environments defined by clear road markings and infrastructure, off-road vehicles must navigate unpredictable and unstructured terrains. The operational efficiency and safety of these vehicles are heavily dependent on their ability to perceive and interpret the ground they traverse. Consequently, developing robust terrain classification systems is essential for identifying traversable paths and potential hazards. While technical literature often uses terms like terrain awareness, scene understanding, and traversability estimation, the core objective remains the same: utilizing sensory data—such as images, LiDAR point clouds, and radar signals—to interpret the surrounding environment. Historically, Convolutional Neural Networks (CNNs) have dominated machine vision tasks; however, the introduction of the Vision Transformer (ViT) has marked a paradigm shift in image processing. By leveraging self-attention mechanisms, ViTs can capture long-range dependencies and complex spatial relationships within images, a capability that is critical for analyzing intricate off-road environments. A significant challenge in deploying these models, however, is their substantial demand for training data. This research aims to evaluate the efficacy of the ViT architecture in a constrained data environment, specifically without employing data augmentation techniques, to classify various off-road terrains.
 
Materials and Methods
In this study, a terrain classification system was developed based on the ViT-base-patch16-224 architecture. The dataset utilized comprised 1,757 images curated from the Roboflow platform, categorized into six primary classes: Soil, Rocky terrain, Grass, Muddy terrain, Gravel terrain, and Rough terrain. The data preparation process involved removing irrelevant images, manual verification of labels, and balancing the number of samples across classes to prevent bias. To rigorously assess the model's generalizability, a completely independent and unseen test dataset consisting of 360 images (60 images per class) was established, ensuring that none of these images were involved in the training or validation phases.
Image preprocessing was executed using the ViTImageProcessor module, which included resizing images to 224×224 pixels, normalization, and conversion into PyTorch tensors. The base ViT model, featuring 12 transformer layers and 12 attention heads, was initialized with weights pre-trained on the ImageNet-21k dataset, and the final layer was modified to accommodate the six target classes. Training was conducted in the PyTorch environment using an NVIDIA 1660-Ti GPU. The training parameters were set to 30 epochs, a batch size of 32, a learning rate of 0.0001, and the AdamW optimizer. Cross-entropy loss was employed as the evaluation metric.
Results and Discussion
Experimental results demonstrated that the proposed model achieved a remarkable overall accuracy of 97% on the independent test dataset, despite the limited data volume. The weighted average F1-score was recorded at 0.90, indicating satisfactory model performance. Analysis of the training curves revealed that the model converged rapidly, with the loss value dropping below 0.01 after just a few epochs, suggesting an absence of overfitting. However, minor fluctuations in validation accuracy were observed, attributed to the model's sensitivity to the specific composition of data batches given the limited dataset size.
A detailed class-by-class performance analysis revealed that the model excelled in identifying Grass (F1=0.96) and Gravel (F1=0.94) terrains. This success is likely due to the distinct visual features characterizing these classes. Conversely, the most significant challenge was observed in distinguishing between the "Soil" and "Muddy" classes. The confusion matrix explicitly highlighted that 25% of Soil samples were misclassified as Muddy. Qualitative analysis indicated that visual similarities in color and texture under specific lighting conditions were the primary cause of this systematic error. It appears the model relied heavily on low-level features and failed to extract higher-level attributes such as moisture content or subtle 3D structural differences. The Receiver Operating Characteristic (ROC) curve, with Area Under the Curve (AUC) values close to 1 (ranging from 0.99 to 1.00), confirmed that the model possesses high discriminative power, suggesting that the observed errors could potentially be mitigated through optimized classification thresholding.
Conclusion
This study confirms that the Vision Transformer (ViT) architecture holds significant potential for feature extraction from complex terrain imagery and can classify various surface types with high accuracy. However, visual similarities between specific classes (e.g., soil and mud) present a challenge that requires targeted solutions. To address these limitations, it is recommended to employ targeted data augmentation techniques (such as moisture simulation and artificial lighting), utilize hybrid architectures (combining CNNs and ViTs) for finer feature extraction, and implement class-specific threshold tuning. Furthermore, integrating multi-modal data, such as thermal or depth imagery, could enhance the distinction between visually similar surfaces. This research provides a framework for debugging deep learning model performance in unstructured environments and paves the way for developing more reliable perception systems for autonomous off-road vehicles.