Usando modelo quantizado ssd mobilenet v1 no exemplo Android do Tensoflow Lite para Object Detection

colocando modelo quantizado ssd mobilenet v1 no Android
1- App
https://github.com/tensorflow/examples/tree/master/lite/examples/object_detection
Baixe no seu computador, abra com o Android Studio, e teste no seu celular

2- Download do modelo (ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18):
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md
Especificamente
http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.tar.gz


3- Convertendo
funcionou (4 outputs), QUANTIZED_UINT8 (tflite de 6,7MB): :
toco --graph_def_file=C:\tmp\my_object_detection\ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18-ModelZoo\tflite_graph.pb --output_file=C:\tmp\my_object_detection\ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18-ModelZoo\tflite\ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tflite --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 --inference_type=QUANTIZED_UINT8 --mean_values=128 --std_dev_values=128 --change_concat_input_ranges=false --allow_custom_ops

2-Labels (opcional, pois o labelmap.txt original serve para este modelo, mas vou colocar a instrução mesmo assim)
https://github.com/nightrome/cocostuff/blob/master/labels.txt
troque ": " por ";" mude a extensão para .csv, abra no Excel, delete a primeira coluna.
salve como txt.
substitue a primeira linha de "unlabeled" por "???" (sem aspas)
delete todas linhas depois da 91, O arquivo deve ter 91 linhas e a última label é toothbrush. (São 90 objetos fora a primeira linha que tem ???)
Salve o arquivo como: labels_map_ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.txt

3-Copie para:
C:\object_detection\app\src\main\assets
(este diretório só vai existir depois de abrir o Android Studio e sincronizar tudo)

4-Edite DetectorActivity.java para usar o modelo e as labels.

private static final String TF_OD_API_MODEL_FILE = "ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tflite";

Se o arquivo com labels foi trocado use isso, senão use labelmap.txt original.
private static final String TF_OD_API_LABELS_FILE = "file:///android_asset/labels_map_ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.txt";

Para este modelo use o mesmo do exemplo para estas variáveis:
private static final int TF_OD_API_INPUT_SIZE = 300;
private static final boolean TF_OD_API_IS_QUANTIZED = true;

Transfer Learning (re-trainnig a deep learning model)

Retrain a deep learning model ssd_mobilenet_v1_0.75_depth_quantized_coco using a custom dataset.

Post based in: https://github.com/EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10#2-set-up-tensorflow-directory-and-anaconda-virtual-environment


Instructions using Windows 10 and TensorFlow 1.13

Preparation

First, install TensorFlow according to these instructions.

Install Object Detection, instructions here.

Donwload de models/master and copy to your machine, see in https://github.com/tensorflow/models/releases the release that you download and rename models-master to tensorflow plus version

For instance, when I write this post de release was 1.11 (it is not the TensorFlow version), so I rename models-master to:
c:\tensorflow1.11  (I will assume this name until the end of this post)

Protobuf

Download the last version of Protocol Buffers:
https://developers.google.com/protocol-buffers/docs/downloads

In my case the current version is 3.7.1, I download and copy to my machine in:
C:\Program Files (x86)\protoc-3.7.1.

From: C:\tensorflow1.11\research

Execute the command (point to the protoc.exe in your machine):
"C:\Program Files (x86)\protoc-3.7.1\bin\protoc" --python_out=. .\object_detection\protos\anchor_generator.proto .\object_detection\protos\argmax_matcher.proto .\object_detection\protos\bipartite_matcher.proto .\object_detection\protos\box_coder.proto .\object_detection\protos\box_predictor.proto .\object_detection\protos\calibration.proto .\object_detection\protos\eval.proto .\object_detection\protos\faster_rcnn.proto .\object_detection\protos\faster_rcnn_box_coder.proto .\object_detection\protos\grid_anchor_generator.proto .\object_detection\protos\hyperparams.proto .\object_detection\protos\image_resizer.proto .\object_detection\protos\input_reader.proto .\object_detection\protos\losses.proto .\object_detection\protos\matcher.proto .\object_detection\protos\mean_stddev_box_coder.proto .\object_detection\protos\model.proto .\object_detection\protos\optimizer.proto .\object_detection\protos\pipeline.proto .\object_detection\protos\post_processing.proto .\object_detection\protos\preprocessor.proto .\object_detection\protos\region_similarity_calculator.proto .\object_detection\protos\square_box_coder.proto .\object_detection\protos\ssd.proto .\object_detection\protos\ssd_anchor_generator.proto .\object_detection\protos\string_int_label_map.proto .\object_detection\protos\train.proto .\object_detection\protos\keypoint_box_coder.proto .\object_detection\protos\multiscale_anchor_generator.proto .\object_detection\protos\graph_rewriter.proto


It will generate .py files.

Environment Variables

Create a variable named PYTHONPATH
and add
C:\tensorflow1.11\models
C:\tensorflow1.11\models\research
C:\tensorflow1.11\models\research\slim

also add in PATH:
C:\tensorflow1.11\models
C:\tensorflow1.11\models\research
C:\tensorflow1.11\models\research\slim

Obs: Point to your directories, the name used here correspond to my installation.

Setup 

From: C:\tensorflow1.11\research

python setup.py build

python setup.py install

Preparing the object_detection directory

Model

Download and extract the pre-trained model:
ssd_mobilenet_v1_0.75_depth_quantized_coco
More models here.

Copy the directory ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18 to:

C:\tensorflow1.11\research\object_detection

Training directory

Create an empty training directory:

C:\tensorflow1.11\research\object_detection\training

Copy 2 files, inside this directory:

labelmap.pbtxt (see item 5a)

Copy:
C:\tensorflow1\models\research\object_detection\inference_graph_ssdlite_mobilenet_v2_coco\pipeline.config

To:
 C:\tensorflow1\models\research\object_detection\training

Rename to:
ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.config

And edit to your directories:

Edit ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.config
in: train_config

batch_size: 1
fine_tune_checkpoint: "C:/tensorflow1.11/research/object_detection/ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18/model.ckpt"
from_detection_checkpoint: true
eplicas_to_aggregate: 1


train_input_reader
  label_map_path: "C:/tensorflow1.11/research/object_detection/training/labelmap.pbtxt"
  input_path: "C:/tensorflow1.11/research/object_detection/train.record"


eval_config:
  num_examples: 76     

eval_input_reader
  label_map_path: "C:/tensorflow1.11/research/object_detection/training/labelmap.pbtxt"
     input_path: "C:/tensorflow1.11/research/object_detection/test.record"

Dataset

Copy:
test.record
train.record


to:
 C:\tensorflow1.11\models\research\object_detection

Retraining

From:
C:\tensorflow1.11\research

Execute
python C:\tensorflow1.11\research\object_detection\legacy\train.py --logtostderr --train_dir=object_detection\training --pipeline_config_path=object_detection\training\ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.config


After, open another prompt, activate your Tensorflow environment and from:
C:\tensorflow1.11\research

execute:
tensorboard --logdir=training

Export Inference Graph

Create a directory:
inference_graph
inside:
C:\tensorflow1.11\research\object_detection


python C:\tensorflow1.11\research\object_detection\export_inference_graph.py --input_type image_tensor --pipeline_config_path C:\tensorflow1.11\research\object_detection\training\pipeline.config --trained_checkpoint_prefix training/model.ckpt-XXXX --output_directory C:\tensorflow1.11\research\object_detection\inference_graph

Where XXXX must be the highest number in the model.ckpt files generated.

This command will generate the: frozen_inference_graph.pb and model.ckpt files.

For TensorFlow Lite

Export TensorFlow Lite SSD Graph

Create a directory:
tflite
inside:
C:\tensorflow1.11\research\object_detection\inference_graph


python C:\tensorflow1.11\research\object_detection\export_tflite_ssd_graph.py --pipeline_config_path=C:\tensorflow1.11\research\object_detection\training\ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.config --trained_checkpoint_prefix=C:\tensorflow1.11\research\object_detection\inference_graph\model.ckpt --output_directory=C:\tensorflow1.11\research\object_detection\inference_graph\tflite --add_postprocessing_op=true

This command will generate the: tflite_graph.pb and tflite_graph.pbtxt



toco --graph_def_file=C:\tensorflow1.11\research\object_detection\inference_graph\tflite\tflite_graph.pb --output_file=C:\tensorflow1.11\research\object_detection\inference_graph\tflite\ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tflite --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 --inference_type=QUANTIZED_UINT8 --mean_values=128 --std_dev_values=128 --change_concat_input_ranges=false --allow_custom_ops

This command will generate the: ssd_mobilenet_v1_quantized_300x300_coco14_sync_2018_07_18.tflite

Errors

c_api.TF_GetCode(self.status.status))

tensorflow.python.framework.errors_impl.InvalidArgumentError: NewRandomAccessFile failed to Create/Open: C:     ensorfloesearch\object_detection        raining\labelmap.pbtxt : A sintaxe do nome do arquivo, do nome do diret\udcf3rio ou do r\udcf3tulo do volume est\udce1 incorreta.

; Unknown error

In .config file write paths with / not \ , example:

C:/tensorflow1.11/research/object_detection/training/labelmap.pbtxt

not:

C:\tensorflow1\models\research\object_detection\training\labelmap.pbtxt

ValueError: not enough values to unpack (expected 7, got 0)

https://stackoverflow.com/questions/48795950/tensorflowobject-detection-valueerror-when-try-to-train-with-num-clones-2

Edit ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.config
in: train_config
change: replicas_to_aggregate: 8
to:replicas_to_aggregate: 1




WARNING:root:Variable [MobilenetV1/Conv2d_0/BatchNorm/beta] is not available in checkpoint
https://github.com/tensorflow/models/issues/4862


Edit ssd_mobilenet_v1_0.75_depth_quantized_300x300_coco14_sync_2018_07_18.config
in: train_config

add: from_detection_checkpoint: true

More information:

Configuring the Object Detection Training Pipeline