Tensorflow Object Detection API - TFRecord creation

Understand how to create TFRecords is an important point when you use Tensorflow Object Detection API. In my Github repo (https://github.com/koheikawata/objectdetectiontest), you can find what directory structure looks like.

``` . └── research ├── object_detection | ├── legacy | | ├── train.py | | └── eval.py | └── test1 | ├── eval_images | | └── export-image-0-b'IMG_0000'.png | ├── checkpoints | | └── model.ckpt-4495.data-00000-of-00001 | | └── model.ckpt-4495.index | | └── model.ckpt-4495.meta | ├── output | | └── saved_model | | └── frozen_inference_graph.pb | ├── Annotations | | └── ***.xml | ├── JPEGImages | | └── ***.jpg | ├── flask_pics | ├── pascal_label_map.pbtxt | ├── train.record | └── val.record ├── faster_rcnn_resnet101_coco_2018_01_28 | └── model.ckpt ├── faster_rcnn_resnet101_coco_test1.config ├── create_tf_record_test1.py ├── object_detection_app_test1.py ├── object_detection_inference_test1.py ├── object_detection_app_test1_docker.py └── Dockerfile ```

1. DEFINE parameters

``` flags = tf.app.flags flags.DEFINE_integer('num_shards', 10, 'Number of TFRecord shards') flags.DEFINE_float('ratio_train', 0.8, 'Ratio of train and val') flags.DEFINE_string('label_map_path', 'object_detection/test1/pascal_label_map.pbtxt', 'Path to label map proto') flags.DEFINE_string('image_dir', 'object_detection/test1/JPEGImages', 'Path to jpg image directory') flags.DEFINE_string('annotation_dir', 'object_detection/test1/Annotations', 'Path to annotation xml directory') flags.DEFINE_string('train_output_path', 'object_detection/test1/train.record', 'Path to train output TFRecords') flags.DEFINE_string('val_output_path', 'object_detection/test1/val.record', 'Path to val output TFRecords') FLAGS = flags.FLAGS ```

num_shards: how you divide TFRecord files. In this code, I do not use this and TFRecord is formatted to a single file.
ratio_train: ratio of training examples from xml/jpg files. 0.8 means 80% for training and 20% for evaluation.
label_map_path: path to .pbtxt file, which defines class of objects like below.

``` item { id: 1 name: 'Ignition coil' } item { id: 2 name: 'Oil level gauge' } item { id: 3 name: 'Air Cleaner' } item { . . . ```

image_dir: path to jpg image file directory. The folder name is JPEGImages. The extension of jpeg file should be .jpg.
annotation_dir: path to xml file directory which contains the annotation information of jpg images. The file name of xml files and jpg image files should be 1:1 relations. The folder name is Annotations. xml files should be formatted as Pascal VOC like below.

``` <annotation verified="yes"> <folder>Annotation</folder> <filename>IMG_00001</filename> <path>C:\work\Capture_output\JPEGImages\IMG_00001.jpg</path> <source> <database>Unknown</database> </source> <size> <width>892</width> <height>595</height> <depth>3</depth> </size> <segmented>0</segmented> ＜object＞ <name>IgnitionCoil</name> <pose>Unspecified</pose> <truncated>0</truncated> <difficult>0</difficult> <bndbox> <xmin>522</xmin> <ymin>307</ymin> <xmax>622</xmax> <ymax>407</ymax> </bndbox> ＜/object＞ ＜object＞ <name>OilLevelGauge</name> <pose>Unspecified</pose> <truncated>0</truncated> <difficult>0</difficult> <bndbox> <xmin>1</xmin> <ymin>382</ymin> <xmax>121</xmax> <ymax>512</ymax> </bndbox> ＜/object＞ ＜object＞ <name>AirCleaner</name> <pose>Unspecified</pose> <truncated>0</truncated> <difficult>0</difficult> <bndbox> <xmin>503</xmin> <ymin>356</ymin> <xmax>553</xmax> <ymax>406</ymax> </bndbox> ＜/object＞ </annotation> ```

train_output_path: path to TFRecord file for training. The file name is train.record
val_output_path: path to TFRecord file for evaluation. The file name is val.record

2. Main function
tf.train.Example is the key component to create tf_examples. In the main function, it searches *.xml files in "Annotations" folder and create examples_list. By random function, files are shuffled and separated into train examples and evaluation examples based on ratio_train.

``` label_map_dict = label_map_util.get_label_map_dict(os.path.join(os.getcwd(), FLAGS.label_map_path)) image_dir = os.path.join(os.getcwd(), FLAGS.image_dir) annotation_dir = os.path.join(os.getcwd(), FLAGS.annotation_dir) examples_list = [] xml_filelist = os.listdir(annotation_dir) for i in xml_filelist: if(i[-4:] == '.xml'): xml_filename, ext = os.path.splitext(i) examples_list.append(xml_filename) random.seed() random.shuffle(examples_list) num_examples = len(examples_list) num_train = int(FLAGS.ratio_train * num_examples) train_examples = examples_list[:num_train] val_examples = examples_list[num_train:] train_output_path = os.path.join(os.getcwd(), FLAGS.train_output_path) val_output_path = os.path.join(os.getcwd(), FLAGS.val_output_path) create_tf_record(train_output_path, FLAGS.num_shards, label_map_dict, image_dir, annotation_dir, train_examples) create_tf_record(val_output_path, FLAGS.num_shards, label_map_dict, image_dir, annotation_dir, val_examples) ```

3. create_tf_record function

``` writer = tf.python_io.TFRecordWriter(output_filename) for idx, example in enumerate(examples): xml_path = os.path.join(annotation_dir, example + '.xml') with tf.gfile.GFile(xml_path, 'r') as fid: xml_str = fid.read() xml = etree.fromstring(xml_str) data = dataset_util.recursive_parse_xml_to_dict(xml)['annotation'] tf_example = dict_to_tf_example(data, label_map_dict, image_dir) writer.write(tf_example.SerializeToString()) writer.close() ```

create_tf_record function mainly generates "data" dictionary from *.xml files. Below is examples that show how it divides examples into train and evaluation.

``` examples_list = ['0001', '0002', '0003', '0004', '0005', '0006', '0007', '0008', '0009', '0010'] ```

``` random.seed() random.shuffle(examples_list) num_examples = len(examples_list) num_train = int(0.8 * num_examples) train_examples = examples_list[:num_train] val_examples = examples_list[num_train:] ```

``` train_examples = ['0008', '0007', '0006', '0010', '0001', '0002', '0003', '0004'] ```

``` val_examples = ['0009', '0004'] ```

``` annotation_dir = os.path.join(os.getcwd(), 'object_detection/test1/Annotations') example = 'IMG_00000' xml_path = os.path.join(annotation_dir, example + '.xml') with tf.gfile.GFile(xml_path, 'r') as fid: xml_str = fid.read() xml = etree.fromstring(xml_str) data = dataset_util.recursive_parse_xml_to_dict(xml)['annotation'] ```

``` data = {'folder': 'Annotation', 'filename': 'IMG_00000', 'path': 'C:\\work\\Capture_output\\JPEGImages\\IMG_00000.jpg', 'source': {'database': 'Unknown'}, 'size': {'width': '892', 'height': '595', 'depth': '3'}, 'segmented': '0', 'object': [{'name': 'IgnitionCoil', 'pose': 'Unspecified', 'truncated': '0', 'difficult': '0', 'bndbox': {'xmin': '522', 'ymin': '307', 'xmax': '622', 'ymax': '407'}}, {'name': 'OilLevelGauge', 'pose': 'Unspecified', 'truncated': '0', 'difficult': '0', 'bndbox': {'xmin': '1', 'ymin': '382', 'xmax': '121', 'ymax': '512'}}, {'name': 'AirCleaner', 'pose': 'Unspecified', 'truncated': '0', 'difficult': '0', 'bndbox': {'xmin': '503', 'ymin': '356', 'xmax': '553', 'ymax': '406'}}]} ```