0%

(CVPR 2017) Annotating object instances with a polygon-rnn

Keyword [Semi-Automatic] [Polygon-RNN]

Castrejon L, Kundu K, Urtasun R, et al. Annotating object instances with a polygon-rnn[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 5230-5238.



1. Overview


1.1. Motivation

  • Most current method treat object segmentation as a pixel-labeling problem

In this paper, it cast this segmentation task as polygon prediction

  • proposed Polygon-RNN architecture for semi-automatic annotation
  • speed up the annotation process by 4.7 times in Cityscapes dataset

1.2. Polygon-RNN



1.2.1. Input

  • image crop
  • vertices sequence


1.2.2. Feature Extractor

  • modified VGG
  • (boundary) low-level about the edge and corner
  • (see object) high-level about the semantic information
  • exploit bilinear interpolation or max-pooling before concat

1.2.3. RNN

  • ConvLSTM. preserve spatial information; reduce parameters compared to FC-RNN


1.3.1. Semi-automatic Annotation

  • GrabCut. exploit annotation
  • GrabCut + CNN
    Most define a graphical model at the pixel-level which are hard to incorporate shape prior.

    1.3.2. Annotation Tool

    1.3.3. Instance Segmentation

  • pixel-level explicit box or patch
  • produce polygon

1.4. Training Detail

  • cross-entry at each time step of RNN
  • feed t-1, t-2 gt to prediction t step
  • for the first vertex prediction. train another CNN using multi-task loss
  • 250 ms/img about inference time
  • set chessboard distance threshhold T. If distance large than T, simulated human correction

1.5. Dataset

  • Cityscapes
  • KITTI

1.6. Data Process

  • perform polygon simplification with zero error in the quantized grid. eliminate vertices which are in a line or fall into same grid

1.7. Data Augmentation

  • random flip
  • enlarged box 10%~20%
  • random select starting vertex



2. Experiments


2.1. Metrics

  • IoU
  • number of clicks

2.2. Step Limitation



  • set max step to 70
  • instance-wise. treat the entire instance as an example
  • component-wise. treat each component as a single example

2.3. Result