Cited By
View all- Xu XZhang JLi Z(2023)Learning from hybrid labels with partial labels via hybrid-grained contrast regularizationApplied Soft Computing10.1016/j.asoc.2023.110533144:COnline publication date: 1-Sep-2023
Weakly Supervised Fine-grained Recognition based on Combined Learning for Small Data and Coarse Label
Authors: 
Anqi Hu, Zhengxing Sun, Qian LiAuthors Info & Claims
Anqi Hu, Zhengxing Sun, Qian LiAuthors Info & ClaimsPages 194 - 201
Abstract
Learning with weak supervision already becomes one of the research trends in fine-grained image recognition. These methods aim to learn feature representation in the case of less manual cost or expert knowledge. Most existing weakly supervised methods are based on incomplete annotation or inexact annotation, which is difficult to perform well limited by supervision information. Therefore, using these two kind of annotations for training at the same time could mine more relevance while the annotating burden will not increase much. In this paper, we propose a combined learning framework by coarse-grained large data and fine-grained small data for weakly supervised fine-grained recognition. Combined learning contains two significant modules: 1) a discriminant module, which maintains the structure information consistent between coarse label and fine label by attention map and part sampling, 2) a cluster division strategy, which mines the detail differences between fine categories by feature subtraction. Experiment results show that our method outperforms weakly supervised methods and achieves the performance close to fully supervised methods in CUB-200-2011 and Stanford Cars datasets.
Supplementary Material
Presentation video introduces the research of ?Weakly Supervised Fine-grained Recognition Based on Combined Learning for Small Data and Coarse Label?. We demonstrate the motivation, existing methods, approach and experiments in this video. Motivation is caused by the high annotation cost of fine-grained labels. We divide the existing methods in to supervision, weak supervision, and discuss the limitations of them. The proposed approach consists of two branches: one network branch is used to preserve the structure information and another branch focus on mining local detail for fine-grained recognition. Experiments show the ability of our method in SOTA comparison, ablation study and image retrieval task.
- Download
- 763.82 MB
References
[1]
Matthieu Guillaumin, Jakob Verbeek, and Cordelia Schmid. 2010. Multimodal semi-supervised learning for image classification. In 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). San Francisco, CA, 902--909.
[2]
Xin Li and Yuhong Guo. 2013. Adaptive Active Learning for Image Classification. In 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Portland, OR, 859--866.
[3]
Rodrigo Benenson, Stefan Popov, and Vittorio Ferrari. 2019. Large-Scale Interactive Object Segmentation With Human Annotators. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, 11692--11701.
[4]
Jinhui Tang, Haojie Li, Guo-Jun Qi, and Tat-Seng Chua. 2010. Image Annotation by Graph-Based Inference With Integrated Multiple/Single Instance Representations. IEEE Transactions on Multimedia 12, 2 (Feb, 2010), 131--141.
[5]
Yixin Chen, and James Z. Wang. 2004. Image Categorization by Learning and Reasoning with Regions. Springer US 14, 6 (2004), 99--121.
[6]
Jie Lei, Zhenyu Guo, and Yang Wang. 2017. Weakly Supervised Image Classification with Coarse and Fine Labels. In 2017 14th Conference on Computer and Robot Vision (CRV). Edmonton, AB, Canada, 240--247.
[7]
Qihan Jiao, Zhi Liu, Linwei Ye, and Yang Wang. 2019. Weakly labeled fine-grained classification with hierarchy relationship of fine and coarse labels. Journal of Visual Communication and Image Representation 63 (2019), 102584--102594.
[8]
Shaoli Huang, Zhe Xu, Dacheng Tao, and Ya Zhang. 2016. Part-Stacked CNN for Fine-Grained Visual Categorization. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, 1173--1182.
[9]
Di Lin, Xiaoyong Shen, Cewu Lu, and Jiaya Jia. 2015. Deep LAC: Deep Localization, Alignment and Classification for Fine-grained Recognition. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, 1666--1674.
[10]
Weifeng Ge, Xiangru Lin, and Yizhou Yu. 2019. Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification From the Bottom Up. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, 3029--3038.
[11]
Zhe Xu, Dacheng Tao, Shaoli Huang, and Ya Zhang. 2017. Friend or Foe: Fine-Grained Categorization With Weak Supervision. IEEE Transactions on Image Processing 26, 1 (Jan, 2017), 135--146.
[12]
Tsung-Yu Lin, Aruni RoyChowdhury, and Subhransu Maji. 2018. Bilinear Convolutional Neural Networks for Fine-Grained Visual Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 6 (Jun, 2018), 1309--1322.
[13]
Yue Chen, Yalong Bai, Wei Zhang, and Tao Mei. 2019. Destruction and Construction Learning for Fine-grained Image Recognition. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, 5152--5161.
[14]
Jianlong Fu, Heliang Zheng, and Tao Mei. 2017. Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-grained Image Recognition. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, 4476--4484.
[15]
Heliang Zheng, Jianlong Fu, Tao Mei, and Jiebo Luo. 2017. Learning Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition. In 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy, 5219--5227.
[16]
Zhi-Hua Zhou. 2018. A brief introduction to weakly supervised learning. National Science Review 5, 1 (Jan, 2018), 44--53.
[17]
Cheng Cui, Zhi Ye, Yangxi Li, and Xinjian Li, et al. 2020. Semi-Supervised Recognition under a Noisy and Fine-grained Dataset. arXiv preprint arXiv:2006.10702. arXiv:https://arxiv.org/abs/2006.10702.
[18]
Xiu-Shen Wei, Peng Wang, Lingqiao Liu, Chunhua Shen, and Jianxin Wu. 2019. Piecewise classifier mappings: Learning fine-grained learners for novel categories with few examples. IEEE Transactions on Image Processing 28, 12 (Dec, 2019), 6116--6125.
[19]
Huaxi Huang, Junjie Zhang, Jian Zhang, Jingsong Xu, and Qiang Wu. 2020. Low-Rank Pairwise Alignment Bilinear Network For Few-Shot Fine-Grained Image Classification. IEEE Transactions on Multimedia 23 (Jun, 2020), 1666--1680.
[20]
Xiaoxu Li, Jijie Wu, Zhuo Sun, Zhanyu Ma, and Jie Cao. 2020. BSNet: Bi-Similarity Network for Few-shot Fine-grained Image Classification. IEEE Transactions on Image Processing 30 (Dec, 2020), 1318--1331.
[21]
Karen Simonyan, and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014). arXiv preprint arXiv:1409.1556. arXiv:https://arxiv.org/abs/1409.1556.
[22]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, 770--778.
[23]
Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, and Jiebo Luo. 2019. Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, 5007--5016.
[24]
Amit Alfassy, Leonid Karlinsky, Amit Aides and Joseph Shtok, et al. 2019. LaSO: Label-Set Operations networks for multi-label few-shot learning. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, 6541--6550.
[25]
Peter Welinder, and Steve Branson. 2010. Caltech-UCSD Birds 200. California Institute of Technology, CNS-TR-2010-001 (2010).
[26]
Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3D object representations for fine-grained categorization. In 2013 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). Sydney, NSW, Australia, 554--561.
[27]
Peng Chen, Peng Li, Qing Li, and Dezheng Zhang. 2019. Semi-Supervised Fine-Grained Image Categorization Using Transfer Learning With Hierarchical Multi-Scale Adversarial Networks. IEEE Access 7, 99 (Aug, 2019), 118650--118668.
[28]
Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo, and Tao Mei. 2020. Learning Rich Part Hierarchies With Progressive Attention Networks for Fine-Grained Image Recognition. IEEE Transactions on Image Processing 29, 99 (Jun, 2020), 476--488.
[29]
Ming Sun, Yuchen Yuan, Feng Zhou, and Errui Ding. 2018. Multi-Attention Multi-Class Constraint for Fine-grained Image Recognition. In Computer Vision - ECCV 2018 - 15th European Conference (ECCV). Munich, Germany, 834--850.
[30]
Xiu-Shen Wei, Jian-Hao Luo, Jianxin Wu, and Zhi-Hua Zhou. 2017. Selective Convolutional Descriptor Aggregation for Fine-Grained Image Retrieval. IEEE Transactions on Image Processing 26, 6 (Jun, 2017), 2868--2881.
Index Terms
- Weakly Supervised Fine-grained Recognition based on Combined Learning for Small Data and Coarse Label
Recommendations
Supervised spectral feature learning for fine-grained classification in small data set
AbstractFine-grained image classification is a challenging task due to the small inter-class variance, the large intra-class difference, and the small training data. Traditional methods typically rely on large-scale training samples with annotated part ...
Highlights- Fully mine and exploit the discriminative potentials of region correlations for fine-grained image classification in weak supervision.
- Spectral graph captures the internal region structure to ensure the image’s comparison in a natural ...
Weakly Supervised Random Forest for Multi-Label Image Clustering and Segmentation
ICMR '15: Proceedings of the 5th ACM on International Conference on Multimedia RetrievalClustering is a useful statistical tool in data mining and computer vision. Supervised information is introduced to improve the clustering performance. However, labeling each piece of data accurately is extremely expensive when the amount of data is ...
Weakly Supervised Object Localization with Noisy-Label Learning
Pattern Recognition and Computer VisionAbstractA novel perspective for Weakly Supervised object localization is proposed in this paper. Most recent pseudo-label-based methods only consider how to get better pseudo-labels and do not consider how to apply these imperfect labels properly. We ...
Comments
Information & Contributors
Information
Published In
June 2022
714 pages
ISBN:9781450392389
DOI:10.1145/3512527
- General Chairs:
- Vincent Oria,
- Maria Luisa Sapino,
- Shin'ichi Satoh,
- Brigitte Kerhervé,
- Program Chairs:
- Wen-Huang Cheng,
- Ichiro Ide,
- Vivek Singh
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 June 2022
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- The National High Technology Research and Development Program of China
- The Program for New Century Excellent Talents in University of China
- The China Postdoctoral Science Foundation
- The Science and technology program of Jiangsu Province
- The National Natural Science Foundation of China
- Innovation Fund of State Key Laboratory for Novel Software Technology
Conference
ICMR '22
Sponsor:
Acceptance Rates
Overall Acceptance Rate 254 of 830 submissions, 31%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 129Total Downloads
- Downloads (Last 12 months)40
- Downloads (Last 6 weeks)2
Reflects downloads up to 05 Nov 2024
Other Metrics
Citations
Cited By
View all- Xu XZhang JLi Z(2023)Learning from hybrid labels with partial labels via hybrid-grained contrast regularizationApplied Soft Computing10.1016/j.asoc.2023.110533144:COnline publication date: 1-Sep-2023
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in