2020-06-06 01:49:27 +02:00
|
|
|
---
|
2023-04-04 00:12:06 +02:00
|
|
|
layout: forward
|
|
|
|
target: https://developers.google.com/mediapipe/solutions/vision/object_detector/
|
2020-06-06 01:49:27 +02:00
|
|
|
title: Object Detection
|
2023-04-04 02:41:28 +02:00
|
|
|
parent: MediaPipe Legacy Solutions
|
2021-06-03 22:13:30 +02:00
|
|
|
nav_order: 9
|
2020-06-06 01:49:27 +02:00
|
|
|
---
|
|
|
|
|
|
|
|
# MediaPipe Object Detection
|
|
|
|
{: .no_toc }
|
|
|
|
|
2020-12-10 04:13:05 +01:00
|
|
|
<details close markdown="block">
|
|
|
|
<summary>
|
|
|
|
Table of contents
|
|
|
|
</summary>
|
|
|
|
{: .text-delta }
|
2020-06-06 01:49:27 +02:00
|
|
|
1. TOC
|
|
|
|
{:toc}
|
2020-12-10 04:13:05 +01:00
|
|
|
</details>
|
2020-06-06 01:49:27 +02:00
|
|
|
---
|
|
|
|
|
2023-03-01 18:19:12 +01:00
|
|
|
**Attention:** *Thank you for your interest in MediaPipe Solutions.
|
2023-04-04 00:12:06 +02:00
|
|
|
As of March 1, 2023, this solution was upgraded to a new MediaPipe
|
|
|
|
Solution. For more information, see the
|
|
|
|
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/vision/object_detector/)
|
2023-03-01 18:19:12 +01:00
|
|
|
site.*
|
|
|
|
|
|
|
|
----
|
|
|
|
|
2022-09-06 23:29:51 +02:00
|
|
|
![object_detection_android_gpu.gif](https://mediapipe.dev/images/mobile/object_detection_android_gpu.gif)
|
2020-06-06 01:49:27 +02:00
|
|
|
|
|
|
|
## Example Apps
|
|
|
|
|
|
|
|
Note: To visualize a graph, copy the graph and paste it into
|
|
|
|
[MediaPipe Visualizer](https://viz.mediapipe.dev/). For more information on how
|
|
|
|
to visualize its associated subgraphs, please see
|
2020-07-30 02:33:39 +02:00
|
|
|
[visualizer documentation](../tools/visualizer.md).
|
2020-06-06 01:49:27 +02:00
|
|
|
|
|
|
|
### Mobile
|
|
|
|
|
|
|
|
Please first see general instructions for
|
2020-12-10 04:13:05 +01:00
|
|
|
[Android](../getting_started/android.md) and [iOS](../getting_started/ios.md) on
|
|
|
|
how to build MediaPipe examples.
|
2020-06-06 01:49:27 +02:00
|
|
|
|
|
|
|
#### GPU Pipeline
|
|
|
|
|
|
|
|
* Graph:
|
|
|
|
[`mediapipe/graphs/object_detection/object_detection_mobile_gpu.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection/object_detection_mobile_gpu.pbtxt)
|
|
|
|
* Android target:
|
|
|
|
[(or download prebuilt ARM64 APK)](https://drive.google.com/open?id=1di2ywCA_acf3y5rIcJHngWHAUNsUHAGz)
|
|
|
|
[`mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu:objectdetectiongpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectiongpu/BUILD)
|
|
|
|
* iOS target:
|
|
|
|
[`mediapipe/examples/ios/objectdetectiongpu:ObjectDetectionGpuApp`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/objectdetectiongpu/BUILD)
|
|
|
|
|
|
|
|
#### CPU Pipeline
|
|
|
|
|
|
|
|
This is very similar to the [GPU pipeline](#gpu-pipeline) except that at the
|
|
|
|
beginning and the end of the pipeline it performs GPU-to-CPU and CPU-to-GPU
|
|
|
|
image transfer respectively. As a result, the rest of graph, which shares the
|
|
|
|
same configuration as the GPU pipeline, runs entirely on CPU.
|
|
|
|
|
|
|
|
* Graph:
|
|
|
|
[`mediapipe/graphs/object_detection/object_detection_mobile_cpu.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection/object_detection_mobile_cpu.pbtxt))
|
|
|
|
* Android target:
|
|
|
|
[(or download prebuilt ARM64 APK)](https://drive.google.com/open?id=1eRBK6V5Qd1LCRwexitR2OXgrBBXbOfZ5)
|
|
|
|
[`mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectioncpu:objectdetectioncpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/android/src/java/com/google/mediapipe/apps/objectdetectioncpu/BUILD)
|
|
|
|
* iOS target:
|
|
|
|
[`mediapipe/examples/ios/objectdetectioncpu:ObjectDetectionCpuApp`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/ios/objectdetectioncpu/BUILD)
|
|
|
|
|
|
|
|
### Desktop
|
|
|
|
|
|
|
|
#### Live Camera Input
|
|
|
|
|
2020-12-10 04:13:05 +01:00
|
|
|
Please first see general instructions for [desktop](../getting_started/cpp.md)
|
|
|
|
on how to build MediaPipe examples.
|
2020-06-06 01:49:27 +02:00
|
|
|
|
|
|
|
* Graph:
|
|
|
|
[`mediapipe/graphs/object_detection/object_detection_desktop_live.pbtxt`](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection/object_detection_desktop_live.pbtxt)
|
|
|
|
* Target:
|
|
|
|
[`mediapipe/examples/desktop/object_detection:object_detection_cpu`](https://github.com/google/mediapipe/tree/master/mediapipe/examples/desktop/object_detection/BUILD)
|
|
|
|
|
|
|
|
#### Video File Input
|
|
|
|
|
|
|
|
* With a TFLite Model
|
|
|
|
|
|
|
|
This uses the same
|
2022-09-06 23:29:51 +02:00
|
|
|
[TFLite model](https://storage.googleapis.com/mediapipe-assets/ssdlite_object_detection.tflite)
|
2020-06-06 01:49:27 +02:00
|
|
|
(see also
|
|
|
|
[model info](https://github.com/google/mediapipe/tree/master/mediapipe/models/object_detection_saved_model/README.md))
|
|
|
|
as in [Live Camera Input](#live-camera-input) above. The pipeline is
|
|
|
|
implemented in this
|
|
|
|
[graph](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection/object_detection_desktop_tflite_graph.pbtxt),
|
|
|
|
which differs from the live-camera-input CPU-based pipeline
|
|
|
|
[graph](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection/object_detection_mobile_cpu.pbtxt)
|
|
|
|
simply by the additional `OpenCvVideoDecoderCalculator` and
|
|
|
|
`OpenCvVideoEncoderCalculator` at the beginning and the end of the graph
|
|
|
|
respectively.
|
|
|
|
|
|
|
|
To build the application, run:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/object_detection:object_detection_tflite
|
|
|
|
```
|
|
|
|
|
|
|
|
To run the application, replace `<input video path>` and `<output video
|
|
|
|
path>` in the command below with your own paths:
|
|
|
|
|
|
|
|
Tip: You can find a test video available in
|
|
|
|
`mediapipe/examples/desktop/object_detection`.
|
|
|
|
|
|
|
|
```
|
|
|
|
GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/object_detection/object_detection_tflite \
|
2020-08-05 07:34:32 +02:00
|
|
|
--calculator_graph_config_file=mediapipe/graphs/object_detection/object_detection_desktop_tflite_graph.pbtxt \
|
|
|
|
--input_side_packets=input_video_path=<input video path>,output_video_path=<output video path>
|
2020-06-06 01:49:27 +02:00
|
|
|
```
|
|
|
|
|
|
|
|
* With a TensorFlow Model
|
|
|
|
|
|
|
|
This uses the
|
2023-03-17 06:47:34 +01:00
|
|
|
[TensorFlow model](https://storage.googleapis.com/mediapipe-assets/object_detection_saved_model/archive.zip)
|
2020-06-06 01:49:27 +02:00
|
|
|
( see also
|
2023-03-17 06:47:34 +01:00
|
|
|
[model info](https://storage.googleapis.com/mediapipe-assets/object_detection_saved_model/README.md)),
|
2020-06-06 01:49:27 +02:00
|
|
|
and the pipeline is implemented in this
|
|
|
|
[graph](https://github.com/google/mediapipe/tree/master/mediapipe/graphs/object_detection/object_detection_mobile_cpu.pbtxt).
|
|
|
|
|
|
|
|
Note: The following runs TensorFlow inference on CPU. If you would like to
|
|
|
|
run inference on GPU (Linux only), please follow
|
2022-03-21 20:07:37 +01:00
|
|
|
[TensorFlow CUDA Support and Setup on Linux Desktop](../getting_started/gpu_support.md#tensorflow-cuda-support-and-setup-on-linux-desktop)
|
2020-06-06 01:49:27 +02:00
|
|
|
instead.
|
|
|
|
|
|
|
|
To build the TensorFlow CPU inference example on desktop, run:
|
|
|
|
|
|
|
|
Note: This command also builds TensorFlow targets from scratch, and it may
|
|
|
|
take a long time (e.g., up to 30 mins) for the first time.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 --define no_aws_support=true --linkopt=-s \
|
|
|
|
mediapipe/examples/desktop/object_detection:object_detection_tensorflow
|
|
|
|
```
|
|
|
|
|
|
|
|
To run the application, replace `<input video path>` and `<output video
|
|
|
|
path>` in the command below with your own paths:
|
|
|
|
|
|
|
|
Tip: You can find a test video available in
|
|
|
|
`mediapipe/examples/desktop/object_detection`.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
GLOG_logtostderr=1 bazel-bin/mediapipe/examples/desktop/object_detection/object_detection_tflite \
|
2020-08-05 07:34:32 +02:00
|
|
|
--calculator_graph_config_file=mediapipe/graphs/object_detection/object_detection_desktop_tensorflow_graph.pbtxt \
|
|
|
|
--input_side_packets=input_video_path=<input video path>,output_video_path=<output video path>
|
2020-06-06 01:49:27 +02:00
|
|
|
```
|
|
|
|
|
|
|
|
### Coral
|
|
|
|
|
|
|
|
Please refer to
|
|
|
|
[these instructions](https://github.com/google/mediapipe/tree/master/mediapipe/examples/coral/README.md)
|
|
|
|
to cross-compile and run MediaPipe examples on the
|
|
|
|
[Coral Dev Board](https://coral.ai/products/dev-board).
|
|
|
|
|
|
|
|
## Resources
|
|
|
|
|
2020-09-16 03:31:50 +02:00
|
|
|
* [Models and model cards](./models.md#object_detection)
|