publications | Tingting Liang

2022

BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework

Tingting Liang, Hongwei Xie, Kaicheng Yu, and 6 more authors

In Neural Information Processing Systems (NeurIPS), 2022

Abs Bib PDF Blog Code

Fusing the camera and LiDAR information has become a de-facto standard for 3D object detection tasks. Current methods rely on point clouds from the LiDAR sensor as queries to leverage the feature from the image space. However, people discovered that this underlying assumption makes the current fusion framework infeasible to produce any prediction when there is a LiDAR malfunction, regardless of minor or major. This fundamentally limits the deployment capability to realistic autonomous driving scenarios. In contrast, we propose a surprisingly simple yet novel fusion framework, dubbed BEVFusion, whose camera stream does not depend on the input of LiDAR data, thus addressing the downside of previous methods. We empirically show that our framework surpasses the state-of-the-art methods under the normal training settings. Under the robustness training settings that simulate various LiDAR malfunctions, our framework significantly surpasses the state-of-the-art methods by 15.7% to 28.9% mAP. To the best of our knowledge, we are the first to handle realistic LiDAR malfunction and can be deployed to realistic scenarios without any post-processing procedure.
@inproceedings{liang2022bevfusion, title = {{BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework}}, author = {Liang, Tingting and Xie, Hongwei and Yu, Kaicheng and Xia, Zhongyu and Lin, Zhiwei and Wang, Yongtao and Tang, Tao and Wang, Bing and Tang, Zhi}, booktitle = {{Neural Information Processing Systems (NeurIPS)}}, year = {2022}, }
CBNet: A Composite Backbone Network Architecture for Object Detection

Ting-Ting Liang, Xiaojie Chu, Yudong Liu, and 5 more authors

IEEE Transactions on Image Processing (TIP), 2022

Abs Bib PDF Blog Code

Modern top-performing object detectors depend heavily on backbone networks, whose advances bring consistent performance gains through exploring more effective network structures. In this paper, we propose a novel and flexible backbone framework, namely , to construct high-performance detectors using existing open-source pre-trained backbones under the pre-training fine-tuning paradigm. In particular, architecture groups multiple identical backbones, which are connected through composite connections. Specifically, it integrates the high- and low-level features of multiple identical backbone networks and gradually expands the receptive field to more effectively perform object detection. We also propose a better training strategy with auxiliary supervision for CBNet-based detectors. CBNet has strong generalization capabilities for different backbones and head designs of the detector architecture. Without additional pre-training of the composite backbone, CBNet can be adapted to various backbones (i.e., CNN-based vs. Transformer-based) and head designs of most mainstream detectors (i.e., one-stage vs. two-stage, anchor-based vs. anchor-free-based). Experiments provide strong evidence that, compared with simply increasing the depth and width of the network, CBNet introduces a more efficient, effective, and resource-friendly way to build high-performance backbone networks. Particularly, our CB-Swin-L achieves 59.4% box AP and 51.6% mask AP on COCO test-dev under the single-model and single-scale testing protocol, which are significantly better than the state-of-the-art results (i.e., 57.7% box AP and 50.2% mask AP) achieved by Swin-L, while reducing the training time by 6x. With multi-scale testing, we push the current best single model result to a new record of 60.1% box AP and 52.3% mask AP without using extra training data.
@article{DBLP:journals/tip/LiangCLWTCCL22, author = {Liang, Ting{-}Ting and Chu, Xiaojie and Liu, Yudong and Wang, Yongtao and Tang, Zhi and Chu, Wei and Chen, Jingdong and Ling, Haibin}, title = {CBNet: {A} Composite Backbone Network Architecture for Object Detection}, journal = {{IEEE Transactions on Image Processing (TIP)}}, volume = {31}, pages = {6893--6906}, year = {2022}, url = {https://doi.org/10.1109/TIP.2022.3216771}, doi = {10.1109/TIP.2022.3216771}, timestamp = {Mon, 05 Dec 2022 13:33:25 +0100}, biburl = {https://dblp.org/rec/journals/tip/LiangCLWTCCL22.bib}, bibsource = {dblp computer science bibliography, https://dblp.org}, }

Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection

Kaicheng Yu, Tao Tang, Hongwei Xie, and 10 more authors

arXiv preprint: 2205.14951, 2022

@article{DBLP:journals/corr/abs-2205-14951,
  author = {Yu, Kaicheng and Tang, Tao and Xie, Hongwei and Lin, Zhiwei and Wu, Zhongwei and Xia, Zhongyu and Liang, Tingting and Sun, Haiyang and Deng, Jiong and Hao, Dayang and Wang, Yongtao and Liang, Xiaodan and Wang, Bing},
  title = {Benchmarking the Robustness of LiDAR-Camera Fusion for 3D Object Detection},
  journal = {arXiv preprint: 2205.14951},
  year = {2022},
  url = {https://doi.org/10.48550/arXiv.2205.14951},
  doi = {10.48550/arXiv.2205.14951},
  timestamp = {Wed, 01 Jun 2022 13:56:25 +0200},
  biburl = {https://dblp.org/rec/journals/corr/abs-2205-14951.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org},
}

FlowNAS: Neural Architecture Search for Optical Flow Estimation

Zhiwei Lin, Tingting Liang, Taihong Xiao, and 3 more authors

arXiv preprint: 2207.01271, 2022

Bib PDF Code

@article{DBLP:journals/corr/abs-2207-01271,
  author = {Lin, Zhiwei and Liang, Tingting and Xiao, Taihong and Wang, Yongtao and Tang, Zhi and Yang, Ming{-}Hsuan},
  title = {FlowNAS: Neural Architecture Search for Optical Flow Estimation},
  journal = {arXiv preprint: 2207.01271},
  year = {2022},
  url = {https://doi.org/10.48550/arXiv.2207.01271},
  doi = {10.48550/arXiv.2207.01271},
  timestamp = {Wed, 06 Jul 2022 15:50:43 +0200},
  biburl = {https://dblp.org/rec/journals/corr/abs-2207-01271.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org},
}

2021

OPANAS: One-Shot Path Aggregation Network Architecture Search for Object Detection

Tingting Liang, Yongtao Wang, Zhi Tang, and 2 more authors

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021

Abs Bib PDF Blog Code

Recently, neural architecture search (NAS) has been exploited to design feature pyramid networks (FPNs) and achieved promising results for visual object detection. Encouraged by the success, we propose One-Shot Path Aggregation Network Architecture Search (OPANAS) algorithm, which significantly improves both searching efficiency and detection accuracy. Specifically, we first introduce six heterogeneous information paths to build our search space, namely top-down, bottom-up, fusing-splitting, scale-equalizing, skip-connect and none. Second, we propose a novel search space of FPNs, in which each FPN candidate is represented by a densely-connected directed acyclic graph (each node is a feature pyramid and each edge is one of the six heterogeneous information paths). Third, we propose an efficient one-shot search method to find the optimal path aggregation architecture, that is, we first train a super-net and then find the optimal candidate with an evolutionary algorithm. Experimental results demonstrate the efficacy of the proposed OPANAS for object detection: (1) OPANAS is more efficient than state-of-the-art methods (i.e., NAS-FPN and Auto-FPN), at significantly smaller searching cost (i.e., only 4 GPU days on MS-COCO); (2) the optimal architecture found by OPANAS significantly improves main-stream detectors including RetinaNet, Faster R-CNN and Cascade R-CNN, by 2.3 3.2 % mAP comparing to their FPN counterparts; and (3) a new state-of-the-art accuracy-speed trade-off (52.2 % mAP at 7.6 FPS) at smaller training costs than comparable state-of-the-arts.
@inproceedings{DBLP:conf/cvpr/LiangWTHL21, author = {Liang, Tingting and Wang, Yongtao and Tang, Zhi and Hu, Guosheng and Ling, Haibin}, title = {{OPANAS:} One-Shot Path Aggregation Network Architecture Search for Object Detection}, booktitle = {{IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}}, pages = {10195--10203}, publisher = {Computer Vision Foundation / {IEEE}}, year = {2021}, doi = {10.1109/CVPR46437.2021.01006}, timestamp = {Mon, 18 Jul 2022 16:47:41 +0200}, biburl = {https://dblp.org/rec/conf/cvpr/LiangWTHL21.bib}, bibsource = {dblp computer science bibliography, https://dblp.org}, }

2020

CBNet: A Novel Composite Backbone Network Architecture for Object Detection

Yudong Liu, Yongtao Wang, Siwei Wang, and 4 more authors

In Association for the Advancement of Artificial Intelligence (AAAI), 2020

Bib PDF Code

@inproceedings{DBLP:conf/aaai/LiuWWLZTL20,
  author = {Liu, Yudong and Wang, Yongtao and Wang, Siwei and Liang, Tingting and Zhao, Qijie and Tang, Zhi and Ling, Haibin},
  title = {CBNet: {A} Novel Composite Backbone Network Architecture for Object
                 Detection},
  booktitle = {{Association for the Advancement of Artificial Intelligence (AAAI)}},
  pages = {11653--11660},
  publisher = {{AAAI} Press},
  year = {2020},
  url = {https://ojs.aaai.org/index.php/AAAI/article/view/6834},
  timestamp = {Mon, 07 Mar 2022 16:57:58 +0100},
  biburl = {https://dblp.org/rec/conf/aaai/LiuWWLZTL20.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org},
}

2019

Pyramidal region context module for semantic segmentation

Tingting Liang, Qijie Zhao, Zhuoying Wang, and 4 more authors

In Proceedings of the ACM Turing Celebration Conference - China, ACM TUR-C 2019, Chengdu, China, May 17-19, 2019, 2019

Bib PDF

@inproceedings{DBLP:conf/acmturc/LiangZWSZWT19,
  author = {Liang, Tingting and Zhao, Qijie and Wang, Zhuoying and Shan, Kaiyu and Zhang, Huan and Wang, Yongtao and Tang, Zhi},
  title = {Pyramidal region context module for semantic segmentation},
  booktitle = {Proceedings of the {ACM} Turing Celebration Conference - China, {ACM}
                 {TUR-C} 2019, Chengdu, China, May 17-19, 2019},
  pages = {102:1--102:6},
  publisher = {{ACM}},
  year = {2019},
  url = {https://doi.org/10.1145/3321408.3322635},
  doi = {10.1145/3321408.3322635},
  timestamp = {Thu, 11 Mar 2021 17:04:51 +0100},
  biburl = {https://dblp.org/rec/conf/acmturc/LiangZWSZWT19.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org},
}

2018

APNet: Semantic Segmentation for Pelvic MR Image

Ting-Ting Liang, Mengyan Sun, Liangcai Gao, and 2 more authors

In Pattern Recognition and Computer Vision - First Chinese Conference, PRCV 2018, Guangzhou, China, November 23-26, 2018, Proceedings, Part II, 2018

Bib PDF

@inproceedings{DBLP:conf/prcv/LiangSGLT18,
  author = {Liang, Ting{-}Ting and Sun, Mengyan and Gao, Liangcai and Lu, Jing{-}Jing and Tsutsui, Satoshi},
  editor = {Lai, Jian{-}Huang and Liu, Cheng{-}Lin and Chen, Xilin and Zhou, Jie and Tan, Tieniu and Zheng, Nanning and Zha, Hongbin},
  title = {APNet: Semantic Segmentation for Pelvic {MR} Image},
  booktitle = {Pattern Recognition and Computer Vision - First Chinese Conference,
                 {PRCV} 2018, Guangzhou, China, November 23-26, 2018, Proceedings,
                 Part {II}},
  series = {Lecture Notes in Computer Science},
  volume = {11257},
  pages = {259--272},
  publisher = {Springer},
  year = {2018},
  url = {https://doi.org/10.1007/978-3-030-03335-4\_23},
  doi = {10.1007/978-3-030-03335-4\_23},
  timestamp = {Fri, 03 Dec 2021 12:20:53 +0100},
  biburl = {https://dblp.org/rec/conf/prcv/LiangSGLT18.bib},
  bibsource = {dblp computer science bibliography, https://dblp.org},
}