GmClass, using the force feedback in the robot-granules interaction. Specifically, we transform the force sequences into the frequency domain and integrate them with high-dimensional textual information into a two-branch architecture for multimodal supervised contrastive learning (MSCL). This method achieves an 84.10% classification accuracy, surpassing traditional supervised learning by 10% and outperforming supervised contrastive learning by more than 40%, demonstrating the positive impact of adding text modality on classification, and when applied to a larger dataset, it attains an even higher 85.28% accuracy, further validating its effectiveness. Also, we demonstrate the performance of our approach in handling unseen particles and the generalization capability for varying data collection parameters.">
A Joint Learning of Force Feedback of Robotic Manipulation and Textual Cues for Granular Materials Classification (original) (raw)