Merge pull request #4235 from priankakariatyml:ios-object-detection-containers

PiperOrigin-RevId: 521553151
This commit is contained in:
jqtang 2023-04-03 15:12:06 -07:00
commit 3c05df9c46
85 changed files with 1815 additions and 376 deletions

View File

@ -6,6 +6,20 @@ nav_order: 1
![MediaPipe](https://mediapipe.dev/images/mediapipe_small.png) ![MediaPipe](https://mediapipe.dev/images/mediapipe_small.png)
----
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
*This notice and web page will be removed on June 1, 2023.*
----
<br><br><br><br><br><br><br><br><br><br>
<br><br><br><br><br><br><br><br><br><br>
<br><br><br><br><br><br><br><br><br><br>
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## Live ML anywhere ## Live ML anywhere
@ -21,15 +35,6 @@ ML solutions for live and streaming media.
---- ----
**Attention:** *Thanks for your interest in MediaPipe! We are moving to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation
site for MediaPipe starting April 3, 2023.*
*This notice and web page will be removed on April 3, 2023.*
----
## ML solutions in MediaPipe ## ML solutions in MediaPipe
Face Detection | Face Mesh | Iris | Hands | Pose | Holistic Face Detection | Face Mesh | Iris | Hands | Pose | Holistic

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/framework/framework_concepts/graphs_cpp
title: Building Graphs in C++ title: Building Graphs in C++
parent: Graphs parent: Graphs
nav_order: 1 nav_order: 1
@ -12,6 +13,12 @@ nav_order: 1
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
C++ graph builder is a powerful tool for: C++ graph builder is a powerful tool for:
* Building complex graphs * Building complex graphs

View File

@ -13,6 +13,12 @@ nav_order: 1
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
Each calculator is a node of a graph. We describe how to create a new Each calculator is a node of a graph. We describe how to create a new
calculator, how to initialize a calculator, how to perform its calculations, calculator, how to initialize a calculator, how to perform its calculations,
input and output streams, timestamps, and options. Each node in the graph is input and output streams, timestamps, and options. Each node in the graph is

View File

@ -14,6 +14,12 @@ has_toc: false
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## The basics ## The basics
### Packet ### Packet

View File

@ -13,6 +13,12 @@ nav_order: 5
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Overview ## Overview
MediaPipe supports calculator nodes for GPU compute and rendering, and allows combining multiple GPU nodes, as well as mixing them with CPU based calculator nodes. There exist several GPU APIs on mobile platforms (eg, OpenGL ES, Metal and Vulkan). MediaPipe does not attempt to offer a single cross-API GPU abstraction. Individual nodes can be written using different APIs, allowing them to take advantage of platform specific features when needed. MediaPipe supports calculator nodes for GPU compute and rendering, and allows combining multiple GPU nodes, as well as mixing them with CPU based calculator nodes. There exist several GPU APIs on mobile platforms (eg, OpenGL ES, Metal and Vulkan). MediaPipe does not attempt to offer a single cross-API GPU abstraction. Individual nodes can be written using different APIs, allowing them to take advantage of platform specific features when needed.

View File

@ -13,6 +13,12 @@ nav_order: 2
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Graph ## Graph
A `CalculatorGraphConfig` proto specifies the topology and functionality of a A `CalculatorGraphConfig` proto specifies the topology and functionality of a

View File

@ -13,6 +13,12 @@ nav_order: 3
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
Calculators communicate by sending and receiving packets. Typically a single Calculators communicate by sending and receiving packets. Typically a single
packet is sent along each input stream at each input timestamp. A packet can packet is sent along each input stream at each input timestamp. A packet can
contain any kind of data, such as a single frame of video or a single integer contain any kind of data, such as a single frame of video or a single integer

View File

@ -13,6 +13,12 @@ nav_order: 6
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Real-time timestamps ## Real-time timestamps
MediaPipe calculator graphs are often used to process streams of video or audio MediaPipe calculator graphs are often used to process streams of video or audio

View File

@ -13,6 +13,12 @@ nav_order: 4
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Scheduling mechanics ## Scheduling mechanics
Data processing in a MediaPipe graph occurs inside processing nodes defined as Data processing in a MediaPipe graph occurs inside processing nodes defined as

View File

@ -15,6 +15,12 @@ nav_order: 1
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
Please follow instructions below to build Android example apps in the supported Please follow instructions below to build Android example apps in the supported
MediaPipe [solutions](../solutions/solutions.md). To learn more about these MediaPipe [solutions](../solutions/solutions.md). To learn more about these
example apps, start from [Hello World! on Android](./hello_world_android.md). example apps, start from [Hello World! on Android](./hello_world_android.md).

View File

@ -14,6 +14,12 @@ nav_order: 3
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
***Experimental Only*** ***Experimental Only***
The MediaPipe Android Archive (AAR) library is a convenient way to use MediaPipe The MediaPipe Android Archive (AAR) library is a convenient way to use MediaPipe

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/
title: MediaPipe Android Solutions title: MediaPipe Android Solutions
parent: MediaPipe on Android parent: MediaPipe on Android
grand_parent: Getting Started grand_parent: Getting Started
@ -13,14 +14,9 @@ nav_order: 2
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We are moving to **Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe) [https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation as the primary developer documentation site for MediaPipe as of April 3, 2023.*
site for MediaPipe starting April 3, 2023. This content will not be moved to
the new site, but will remain available in the source code repository on an
as-is basis.*
*This notice and web page will be removed on April 3, 2023.*
---- ----

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/
title: Building MediaPipe Examples title: Building MediaPipe Examples
parent: Getting Started parent: Getting Started
nav_exclude: true nav_exclude: true
@ -12,14 +13,9 @@ nav_exclude: true
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We are moving to **Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe) [https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation as the primary developer documentation site for MediaPipe as of April 3, 2023.*
site for MediaPipe starting April 3, 2023. This content will not be moved to
the new site, but will remain available in the source code repository on an
as-is basis.*
*This notice and web page will be removed on April 3, 2023.*
---- ----

View File

@ -15,6 +15,12 @@ nav_order: 5
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
Please follow instructions below to build C++ command-line example apps in the Please follow instructions below to build C++ command-line example apps in the
supported MediaPipe [solutions](../solutions/solutions.md). To learn more about supported MediaPipe [solutions](../solutions/solutions.md). To learn more about
these example apps, start from [Hello World! in C++](./hello_world_cpp.md). these example apps, start from [Hello World! in C++](./hello_world_cpp.md).

View File

@ -13,6 +13,12 @@ nav_order: 9
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
### How to convert ImageFrames and GpuBuffers ### How to convert ImageFrames and GpuBuffers
The Calculators [`ImageFrameToGpuBufferCalculator`] and The Calculators [`ImageFrameToGpuBufferCalculator`] and

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/
title: Getting Started title: Getting Started
nav_order: 2 nav_order: 2
has_children: true has_children: true
@ -12,13 +13,8 @@ has_children: true
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We are moving to **Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe) [https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation as the primary developer documentation site for MediaPipe as of April 3, 2023.*
site for MediaPipe starting April 3, 2023. This content will not be moved to
the new site, but will remain available in the source code repository on an
as-is basis.*
*This notice and web page will be removed on April 3, 2023.*
---- ----

View File

@ -13,6 +13,12 @@ nav_order: 7
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## OpenGL ES Support ## OpenGL ES Support
MediaPipe supports OpenGL ES up to version 3.2 on Android/Linux and up to ES 3.0 MediaPipe supports OpenGL ES up to version 3.2 on Android/Linux and up to ES 3.0

View File

@ -14,6 +14,12 @@ nav_order: 1
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Introduction ## Introduction
This codelab uses MediaPipe on an Android device. This codelab uses MediaPipe on an Android device.

View File

@ -14,6 +14,12 @@ nav_order: 1
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
1. Ensure you have a working version of MediaPipe. See 1. Ensure you have a working version of MediaPipe. See
[installation instructions](./install.md). [installation instructions](./install.md).

View File

@ -14,6 +14,12 @@ nav_order: 1
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Introduction ## Introduction
This codelab uses MediaPipe on an iOS device. This codelab uses MediaPipe on an iOS device.

View File

@ -13,6 +13,12 @@ nav_order: 8
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Technical questions ## Technical questions
For help with technical or algorithmic questions, visit For help with technical or algorithmic questions, visit

View File

@ -13,6 +13,12 @@ nav_order: 6
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
Note: To interoperate with OpenCV, OpenCV 3.x to 4.1 are preferred. OpenCV Note: To interoperate with OpenCV, OpenCV 3.x to 4.1 are preferred. OpenCV
2.x currently works but interoperability support may be deprecated in the 2.x currently works but interoperability support may be deprecated in the
future. future.

View File

@ -15,6 +15,12 @@ nav_order: 2
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
Please follow instructions below to build iOS example apps in the supported Please follow instructions below to build iOS example apps in the supported
MediaPipe [solutions](../solutions/solutions.md). To learn more about these MediaPipe [solutions](../solutions/solutions.md). To learn more about these
example apps, start from, start from example apps, start from, start from

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/
title: MediaPipe in JavaScript title: MediaPipe in JavaScript
parent: Getting Started parent: Getting Started
nav_order: 4 nav_order: 4
@ -14,12 +15,7 @@ nav_order: 4
**Attention:** *Thanks for your interest in MediaPipe! We are moving to **Attention:** *Thanks for your interest in MediaPipe! We are moving to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe) [https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation as the primary developer documentation site for MediaPipe starting April 3, 2023.*
site for MediaPipe starting April 3, 2023. This content will not be moved to
the new site, but will remain available in the source code repository on an
as-is basis.*
*This notice and web page will be removed on April 3, 2023.*
---- ----

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/
title: MediaPipe in Python title: MediaPipe in Python
parent: Getting Started parent: Getting Started
has_children: true has_children: true
@ -14,6 +15,12 @@ nav_order: 3
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Ready-to-use Python Solutions ## Ready-to-use Python Solutions
MediaPipe offers ready-to-use yet customizable Python solutions as a prebuilt MediaPipe offers ready-to-use yet customizable Python solutions as a prebuilt

View File

@ -12,6 +12,11 @@ nav_order: 1
1. TOC 1. TOC
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
The MediaPipe Python framework grants direct access to the core components of The MediaPipe Python framework grants direct access to the core components of
the MediaPipe C++ framework such as Timestamp, Packet, and CalculatorGraph, the MediaPipe C++ framework such as Timestamp, Packet, and CalculatorGraph,

View File

@ -13,6 +13,12 @@ nav_order: 10
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
## Missing Python binary path ## Missing Python binary path
The error message: The error message:

View File

@ -6,6 +6,20 @@ nav_order: 1
![MediaPipe](https://mediapipe.dev/images/mediapipe_small.png) ![MediaPipe](https://mediapipe.dev/images/mediapipe_small.png)
----
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
*This notice and web page will be removed on June 1, 2023.*
----
<br><br><br><br><br><br><br><br><br><br>
<br><br><br><br><br><br><br><br><br><br>
<br><br><br><br><br><br><br><br><br><br>
-------------------------------------------------------------------------------- --------------------------------------------------------------------------------
## Live ML anywhere ## Live ML anywhere
@ -21,15 +35,6 @@ ML solutions for live and streaming media.
---- ----
**Attention:** *Thanks for your interest in MediaPipe! We are moving to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation
site for MediaPipe starting April 3, 2023.*
*This notice and web page will be removed on April 3, 2023.*
----
## ML solutions in MediaPipe ## ML solutions in MediaPipe
Face Detection | Face Mesh | Iris | Hands | Pose | Holistic Face Detection | Face Mesh | Iris | Hands | Pose | Holistic

View File

@ -1,3 +1,3 @@
MediaPipe MediaPipe
===================================== =====================================
Please see https://docs.mediapipe.dev. Please see https://developers.google.com/mediapipe/

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/guide#legacy
title: AutoFlip (Saliency-aware Video Cropping) title: AutoFlip (Saliency-aware Video Cropping)
parent: Solutions parent: Solutions
nav_order: 14 nav_order: 14
@ -20,12 +21,10 @@ nav_order: 14
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
We have ended support for this MediaPipe Legacy Solution as of March 1, 2023. We have ended support for this MediaPipe Legacy Solution as of March 1, 2023.
For more information, see the new For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/guide#legacy
title: Box Tracking title: Box Tracking
parent: Solutions parent: Solutions
nav_order: 10 nav_order: 10
@ -20,12 +21,10 @@ nav_order: 10
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
We have ended support for this MediaPipe Legacy Solution as of March 1, 2023. We have ended support for this MediaPipe Legacy Solution as of March 1, 2023.
For more information, see the new For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/face_detector/
title: Face Detection title: Face Detection
parent: Solutions parent: Solutions
nav_order: 1 nav_order: 1
@ -20,12 +21,10 @@ nav_order: 1
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/face_landmarker/
title: Face Mesh title: Face Mesh
parent: Solutions parent: Solutions
nav_order: 2 nav_order: 2
@ -20,12 +21,10 @@ nav_order: 2
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/image_segmenter/
title: Hair Segmentation title: Hair Segmentation
parent: Solutions parent: Solutions
nav_order: 8 nav_order: 8
@ -19,13 +20,11 @@ nav_order: 8
--- ---
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of April 4, 2023, this solution was upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/vision/image_segmenter/)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
![hair_segmentation_android_gpu_gif](https://mediapipe.dev/images/mobile/hair_segmentation_android_gpu.gif) ![hair_segmentation_android_gpu_gif](https://mediapipe.dev/images/mobile/hair_segmentation_android_gpu.gif)

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/hand_landmarker
title: Hands title: Hands
parent: Solutions parent: Solutions
nav_order: 4 nav_order: 4
@ -19,13 +20,11 @@ nav_order: 4
--- ---
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of March 1, 2023, this solution was upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/vision/hand_landmarker)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://github.com/google/mediapipe/blob/master/docs/solutions/holistic.md
title: Holistic title: Holistic
parent: Solutions parent: Solutions
nav_order: 6 nav_order: 6
@ -20,12 +21,10 @@ nav_order: 6
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/guide#legacy
title: Instant Motion Tracking title: Instant Motion Tracking
parent: Solutions parent: Solutions
nav_order: 11 nav_order: 11
@ -20,12 +21,10 @@ nav_order: 11
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
We have ended support for this MediaPipe Legacy Solution as of March 1, 2023. We have ended support for this MediaPipe Legacy Solution as of March 1, 2023.
For more information, see the new For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/face_landmarker/
title: Iris title: Iris
parent: Solutions parent: Solutions
nav_order: 3 nav_order: 3
@ -20,12 +21,10 @@ nav_order: 3
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/guide#legacy
title: KNIFT (Template-based Feature Matching) title: KNIFT (Template-based Feature Matching)
parent: Solutions parent: Solutions
nav_order: 13 nav_order: 13
@ -20,12 +21,10 @@ nav_order: 13
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
We have ended support for this MediaPipe Legacy Solution as of March 1, 2023. We have ended support for this MediaPipe Legacy Solution as of March 1, 2023.
For more information, see the new For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/guide#legacy
title: Dataset Preparation with MediaSequence title: Dataset Preparation with MediaSequence
parent: Solutions parent: Solutions
nav_order: 15 nav_order: 15
@ -24,8 +25,6 @@ For more information, see the new
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/guide#legacy
title: Models and Model Cards title: Models and Model Cards
parent: Solutions parent: Solutions
nav_order: 30 nav_order: 30
@ -22,8 +23,6 @@ MediaPipe Legacy Solutions will continue to be provided on an as-is basis.
We encourage you to check out the new MediaPipe Solutions at: We encourage you to check out the new MediaPipe Solutions at:
[https://developers.google.com/mediapipe/solutions](https://developers.google.com/mediapipe/solutions)* [https://developers.google.com/mediapipe/solutions](https://developers.google.com/mediapipe/solutions)*
*This notice and web page will be removed on April 3, 2023.*
---- ----
### [Face Detection](https://google.github.io/mediapipe/solutions/face_detection) ### [Face Detection](https://google.github.io/mediapipe/solutions/face_detection)

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/object_detector/
title: Object Detection title: Object Detection
parent: Solutions parent: Solutions
nav_order: 9 nav_order: 9
@ -19,13 +20,11 @@ nav_order: 9
--- ---
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of March 1, 2023, this solution was upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/vision/object_detector/)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
![object_detection_android_gpu.gif](https://mediapipe.dev/images/mobile/object_detection_android_gpu.gif) ![object_detection_android_gpu.gif](https://mediapipe.dev/images/mobile/object_detection_android_gpu.gif)

View File

@ -1,4 +1,31 @@
## TensorFlow/TFLite Object Detection Model ---
layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/object_detector
title: Object Detection
parent: Solutions
nav_order: 9
---
# MediaPipe Object Detection
{: .no_toc }
<details close markdown="block">
<summary>
Table of contents
</summary>
{: .text-delta }
1. TOC
{:toc}
</details>
---
**Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution was upgraded to a new MediaPipe
Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/vision/object_detector)
site.*
----
### TensorFlow model ### TensorFlow model

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/guide#legacy
title: Objectron (3D Object Detection) title: Objectron (3D Object Detection)
parent: Solutions parent: Solutions
nav_order: 12 nav_order: 12
@ -20,12 +21,10 @@ nav_order: 12
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
We have ended support for this MediaPipe Legacy Solution as of March 1, 2023. We have ended support for this MediaPipe Legacy Solution as of March 1, 2023.
For more information, see the new For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/pose_landmarker/
title: Pose title: Pose
parent: Solutions parent: Solutions
has_children: true has_children: true
@ -22,12 +23,10 @@ nav_order: 5
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/vision/pose_landmarker/)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/pose_landmarker/
title: Pose Classification title: Pose Classification
parent: Pose parent: Pose
grand_parent: Solutions grand_parent: Solutions
@ -21,12 +22,10 @@ nav_order: 1
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/vision/pose_landmarker/)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/vision/image_segmenter/
title: Selfie Segmentation title: Selfie Segmentation
parent: Solutions parent: Solutions
nav_order: 7 nav_order: 7
@ -19,13 +20,11 @@ nav_order: 7
--- ---
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
As of March 1, 2023, this solution is planned to be upgraded to a new MediaPipe As of April 4, 2023, this solution was upgraded to a new MediaPipe
Solution. For more information, see the new Solution. For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/vision/image_segmenter/)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
## Overview ## Overview

View File

@ -1,12 +1,12 @@
--- ---
layout: default layout: default
title: Solutions title: MediaPipe Legacy Solutions
nav_order: 3 nav_order: 3
has_children: true has_children: true
has_toc: false has_toc: false
--- ---
# Solutions # MediaPipe Legacy Solutions
{: .no_toc } {: .no_toc }
1. TOC 1. TOC
@ -29,6 +29,12 @@ Solutions at:
---- ----
<br><br><br><br><br><br><br><br><br><br>
<br><br><br><br><br><br><br><br><br><br>
<br><br><br><br><br><br><br><br><br><br>
----
MediaPipe offers open source cross-platform, customizable ML solutions for live MediaPipe offers open source cross-platform, customizable ML solutions for live
and streaming media. and streaming media.

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/solutions/guide#legacy
title: YouTube-8M Feature Extraction and Model Inference title: YouTube-8M Feature Extraction and Model Inference
parent: Solutions parent: Solutions
nav_order: 16 nav_order: 16
@ -20,12 +21,10 @@ nav_order: 16
**Attention:** *Thank you for your interest in MediaPipe Solutions. **Attention:** *Thank you for your interest in MediaPipe Solutions.
We have ended support for this MediaPipe Legacy Solution as of March 1, 2023. We have ended support for this MediaPipe Legacy Solution as of March 1, 2023.
For more information, see the new For more information, see the
[MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy) [MediaPipe Solutions](https://developers.google.com/mediapipe/solutions/guide#legacy)
site.* site.*
*This notice and web page will be removed on April 3, 2023.*
---- ----
MediaPipe is a useful and general framework for media processing that can assist MediaPipe is a useful and general framework for media processing that can assist

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/
title: Performance Benchmarking title: Performance Benchmarking
parent: Tools parent: Tools
nav_order: 3 nav_order: 3
@ -12,6 +13,12 @@ nav_order: 3
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
---
*Coming soon.* *Coming soon.*
Future mediapipe releases will include tools for visualizing and analysing the Future mediapipe releases will include tools for visualizing and analysing the

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/
title: Tools title: Tools
nav_order: 4 nav_order: 4
has_children: true has_children: true
@ -11,3 +12,9 @@ has_children: true
1. TOC 1. TOC
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----

View File

@ -1,5 +1,6 @@
--- ---
layout: default layout: forward
target: https://developers.google.com/mediapipe/
title: Tracing and Profiling title: Tracing and Profiling
parent: Tools parent: Tools
nav_order: 2 nav_order: 2
@ -12,6 +13,12 @@ nav_order: 2
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
----
The MediaPipe framework includes a built-in tracer and profiler. The tracer The MediaPipe framework includes a built-in tracer and profiler. The tracer
records various timing events related to packet processing, including the start records various timing events related to packet processing, including the start
and end time of each Calculator::Process call. The tracer writes trace log files and end time of each Calculator::Process call. The tracer writes trace log files

View File

@ -13,6 +13,12 @@ nav_order: 1
{:toc} {:toc}
--- ---
**Attention:** *Thanks for your interest in MediaPipe! We have moved to
[https://developers.google.com/mediapipe](https://developers.google.com/mediapipe)
as the primary developer documentation site for MediaPipe as of April 3, 2023.*
---
To help users understand the structure of their calculator graphs and to To help users understand the structure of their calculator graphs and to
understand the overall behavior of their machine learning inference pipelines, understand the overall behavior of their machine learning inference pipelines,
we have built the [MediaPipe Visualizer](https://viz.mediapipe.dev/) we have built the [MediaPipe Visualizer](https://viz.mediapipe.dev/)

View File

@ -59,3 +59,44 @@ cc_library(
"@com_google_absl//absl/status:statusor", "@com_google_absl//absl/status:statusor",
], ],
) )
cc_library(
name = "pose_landmarker_graph",
srcs = ["pose_landmarker_graph.cc"],
deps = [
":pose_landmarks_detector_graph",
"//mediapipe/calculators/core:clip_vector_size_calculator",
"//mediapipe/calculators/core:clip_vector_size_calculator_cc_proto",
"//mediapipe/calculators/core:gate_calculator",
"//mediapipe/calculators/core:gate_calculator_cc_proto",
"//mediapipe/calculators/core:pass_through_calculator",
"//mediapipe/calculators/core:previous_loopback_calculator",
"//mediapipe/calculators/image:image_properties_calculator",
"//mediapipe/calculators/util:association_calculator_cc_proto",
"//mediapipe/calculators/util:association_norm_rect_calculator",
"//mediapipe/calculators/util:collection_has_min_size_calculator",
"//mediapipe/calculators/util:collection_has_min_size_calculator_cc_proto",
"//mediapipe/framework/api2:builder",
"//mediapipe/framework/api2:port",
"//mediapipe/framework/formats:detection_cc_proto",
"//mediapipe/framework/formats:image",
"//mediapipe/framework/formats:landmark_cc_proto",
"//mediapipe/framework/formats:rect_cc_proto",
"//mediapipe/framework/formats:tensor",
"//mediapipe/framework/port:status",
"//mediapipe/tasks/cc:common",
"//mediapipe/tasks/cc/components/utils:gate",
"//mediapipe/tasks/cc/core:model_asset_bundle_resources",
"//mediapipe/tasks/cc/core:model_resources_cache",
"//mediapipe/tasks/cc/core:model_task_graph",
"//mediapipe/tasks/cc/core:utils",
"//mediapipe/tasks/cc/metadata/utils:zip_utils",
"//mediapipe/tasks/cc/vision/pose_detector:pose_detector_graph",
"//mediapipe/tasks/cc/vision/pose_detector/proto:pose_detector_graph_options_cc_proto",
"//mediapipe/tasks/cc/vision/pose_landmarker/proto:pose_landmarker_graph_options_cc_proto",
"//mediapipe/tasks/cc/vision/pose_landmarker/proto:pose_landmarks_detector_graph_options_cc_proto",
"//mediapipe/util:graph_builder_utils",
"@com_google_absl//absl/strings:str_format",
],
alwayslink = 1,
)

View File

@ -0,0 +1,384 @@
/* Copyright 2023 The MediaPipe Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include <memory>
#include <type_traits>
#include <utility>
#include <vector>
#include "absl/strings/str_format.h"
#include "mediapipe/calculators/core/clip_vector_size_calculator.pb.h"
#include "mediapipe/calculators/core/gate_calculator.pb.h"
#include "mediapipe/calculators/util/association_calculator.pb.h"
#include "mediapipe/calculators/util/collection_has_min_size_calculator.pb.h"
#include "mediapipe/framework/api2/builder.h"
#include "mediapipe/framework/api2/port.h"
#include "mediapipe/framework/formats/detection.pb.h"
#include "mediapipe/framework/formats/image.h"
#include "mediapipe/framework/formats/landmark.pb.h"
#include "mediapipe/framework/formats/rect.pb.h"
#include "mediapipe/framework/formats/tensor.h"
#include "mediapipe/framework/port/status_macros.h"
#include "mediapipe/tasks/cc/common.h"
#include "mediapipe/tasks/cc/components/utils/gate.h"
#include "mediapipe/tasks/cc/core/model_asset_bundle_resources.h"
#include "mediapipe/tasks/cc/core/model_resources_cache.h"
#include "mediapipe/tasks/cc/core/model_task_graph.h"
#include "mediapipe/tasks/cc/core/utils.h"
#include "mediapipe/tasks/cc/metadata/utils/zip_utils.h"
#include "mediapipe/tasks/cc/vision/pose_detector/proto/pose_detector_graph_options.pb.h"
#include "mediapipe/tasks/cc/vision/pose_landmarker/proto/pose_landmarker_graph_options.pb.h"
#include "mediapipe/tasks/cc/vision/pose_landmarker/proto/pose_landmarks_detector_graph_options.pb.h"
#include "mediapipe/util/graph_builder_utils.h"
namespace mediapipe {
namespace tasks {
namespace vision {
namespace pose_landmarker {
namespace {
using ::mediapipe::NormalizedRect;
using ::mediapipe::api2::Input;
using ::mediapipe::api2::Output;
using ::mediapipe::api2::builder::Graph;
using ::mediapipe::api2::builder::SidePacket;
using ::mediapipe::api2::builder::Source;
using ::mediapipe::tasks::components::utils::DisallowIf;
using ::mediapipe::tasks::core::ModelAssetBundleResources;
using ::mediapipe::tasks::metadata::SetExternalFile;
using ::mediapipe::tasks::vision::pose_detector::proto::
PoseDetectorGraphOptions;
using ::mediapipe::tasks::vision::pose_landmarker::proto::
PoseLandmarkerGraphOptions;
using ::mediapipe::tasks::vision::pose_landmarker::proto::
PoseLandmarksDetectorGraphOptions;
constexpr char kImageTag[] = "IMAGE";
constexpr char kNormRectTag[] = "NORM_RECT";
constexpr char kNormLandmarksTag[] = "NORM_LANDMARKS";
constexpr char kWorldLandmarksTag[] = "WORLD_LANDMARKS";
constexpr char kAuxiliaryLandmarksTag[] = "AUXILIARY_LANDMARKS";
constexpr char kPoseRectsNextFrameTag[] = "POSE_RECTS_NEXT_FRAME";
constexpr char kExpandedPoseRectsTag[] = "EXPANDED_POSE_RECTS";
constexpr char kDetectionsTag[] = "DETECTIONS";
constexpr char kLoopTag[] = "LOOP";
constexpr char kPrevLoopTag[] = "PREV_LOOP";
constexpr char kMainTag[] = "MAIN";
constexpr char kIterableTag[] = "ITERABLE";
constexpr char kSegmentationMaskTag[] = "SEGMENTATION_MASK";
constexpr char kPoseDetectorTFLiteName[] = "pose_detector.tflite";
constexpr char kPoseLandmarksDetectorTFLiteName[] =
"pose_landmarks_detector.tflite";
struct PoseLandmarkerOutputs {
Source<std::vector<NormalizedLandmarkList>> landmark_lists;
Source<std::vector<LandmarkList>> world_landmark_lists;
Source<std::vector<NormalizedLandmarkList>> auxiliary_landmark_lists;
Source<std::vector<NormalizedRect>> pose_rects_next_frame;
Source<std::vector<Detection>> pose_detections;
Source<std::vector<Image>> segmentation_masks;
Source<Image> image;
};
// Sets the base options in the sub tasks.
absl::Status SetSubTaskBaseOptions(const ModelAssetBundleResources& resources,
PoseLandmarkerGraphOptions* options,
bool is_copy) {
auto* pose_detector_graph_options =
options->mutable_pose_detector_graph_options();
if (!pose_detector_graph_options->base_options().has_model_asset()) {
ASSIGN_OR_RETURN(const auto pose_detector_file,
resources.GetFile(kPoseDetectorTFLiteName));
SetExternalFile(pose_detector_file,
pose_detector_graph_options->mutable_base_options()
->mutable_model_asset(),
is_copy);
}
pose_detector_graph_options->mutable_base_options()
->mutable_acceleration()
->CopyFrom(options->base_options().acceleration());
pose_detector_graph_options->mutable_base_options()->set_use_stream_mode(
options->base_options().use_stream_mode());
auto* pose_landmarks_detector_graph_options =
options->mutable_pose_landmarks_detector_graph_options();
if (!pose_landmarks_detector_graph_options->base_options()
.has_model_asset()) {
ASSIGN_OR_RETURN(const auto pose_landmarks_detector_file,
resources.GetFile(kPoseLandmarksDetectorTFLiteName));
SetExternalFile(
pose_landmarks_detector_file,
pose_landmarks_detector_graph_options->mutable_base_options()
->mutable_model_asset(),
is_copy);
}
pose_landmarks_detector_graph_options->mutable_base_options()
->mutable_acceleration()
->CopyFrom(options->base_options().acceleration());
pose_landmarks_detector_graph_options->mutable_base_options()
->set_use_stream_mode(options->base_options().use_stream_mode());
return absl::OkStatus();
}
} // namespace
// A "mediapipe.tasks.vision.pose_landmarker.PoseLandmarkerGraph" performs pose
// landmarks detection. The PoseLandmarkerGraph consists of two subgraphs:
// PoseDetectorGraph, MultiplePoseLandmarksDetectorGraph
//
// MultiplePoseLandmarksDetectorGraph detects landmarks from bounding boxes
// produced by PoseDetectorGraph. PoseLandmarkerGraph tracks the landmarks over
// time, and skips the PoseDetectorGraph. If the tracking is lost or the
// detected poses are less than configured max number poses, PoseDetectorGraph
// would be triggered to detect poses.
//
//
// Inputs:
// IMAGE - Image
// Image to perform pose landmarks detection on.
// NORM_RECT - NormalizedRect @Optional
// Describes image rotation and region of image to perform landmarks
// detection on. If not provided, whole image is used for pose landmarks
// detection.
//
//
// Outputs:
// NORM_LANDMARKS: - std::vector<NormalizedLandmarkList>
// Vector of detected pose landmarks.
// WORLD_LANDMARKS: std::vector<LandmarkList>
// Vector of detected world pose landmarks.
// AUXILIARY_LANDMARKS: - std::vector<NormalizedLandmarkList>
// Vector of detected auxiliary landmarks.
// POSE_RECTS_NEXT_FRAME - std::vector<NormalizedRect>
// Vector of the expanded rects enclosing the whole pose RoI for landmark
// detection on the next frame.
// POSE_RECTS - std::vector<NormalizedRect>
// Detected pose bounding boxes in normalized coordinates from pose
// detection.
// SEGMENTATION_MASK - std::vector<Image>
// Segmentation masks.
// IMAGE - Image
// The input image that the pose landmarker runs on and has the pixel data
// stored on the target storage (CPU vs GPU).
// All returned coordinates are in the unrotated and uncropped input image
// coordinates system.
//
// Example:
// node {
// calculator: "mediapipe.tasks.vision.pose_landmarker.PoseLandmarkerGraph"
// input_stream: "IMAGE:image_in"
// input_stream: "NORM_RECT:norm_rect"
// output_stream: "NORM_LANDMARKS:pose_landmarks"
// output_stream: "LANDMARKS:world_landmarks"
// output_stream: "NORM_LANDMAKRS:auxiliary_landmarks"
// output_stream: "POSE_RECTS_NEXT_FRAME:pose_rects_next_frame"
// output_stream: "POSE_RECTS:pose_rects"
// output_stream: "SEGMENTATION_MASK:segmentation_masks"
// output_stream: "IMAGE:image_out"
// options {
// [mediapipe.tasks.vision.pose_landmarker.proto.PoseLandmarkerGraphOptions.ext]
// {
// base_options {
// model_asset {
// file_name: "pose_landmarker.task"
// }
// }
// pose_detector_graph_options {
// min_detection_confidence: 0.5
// num_poses: 2
// }
// pose_landmarks_detector_graph_options {
// min_detection_confidence: 0.5
// }
// }
// }
// }
class PoseLandmarkerGraph : public core::ModelTaskGraph {
public:
absl::StatusOr<CalculatorGraphConfig> GetConfig(
SubgraphContext* sc) override {
Graph graph;
if (sc->Options<PoseLandmarkerGraphOptions>()
.base_options()
.has_model_asset()) {
ASSIGN_OR_RETURN(
const auto* model_asset_bundle_resources,
CreateModelAssetBundleResources<PoseLandmarkerGraphOptions>(sc));
// Copies the file content instead of passing the pointer of file in
// memory if the subgraph model resource service is not available.
MP_RETURN_IF_ERROR(SetSubTaskBaseOptions(
*model_asset_bundle_resources,
sc->MutableOptions<PoseLandmarkerGraphOptions>(),
!sc->Service(::mediapipe::tasks::core::kModelResourcesCacheService)
.IsAvailable()));
}
ASSIGN_OR_RETURN(
auto outs,
BuildPoseLandmarkerGraph(
*sc->MutableOptions<PoseLandmarkerGraphOptions>(),
graph[Input<Image>(kImageTag)],
graph[Input<NormalizedRect>::Optional(kNormRectTag)], graph));
outs.landmark_lists >>
graph[Output<std::vector<NormalizedLandmarkList>>(kNormLandmarksTag)];
outs.world_landmark_lists >>
graph[Output<std::vector<LandmarkList>>(kWorldLandmarksTag)];
outs.auxiliary_landmark_lists >>
graph[Output<std::vector<NormalizedLandmarkList>>(
kAuxiliaryLandmarksTag)];
outs.pose_rects_next_frame >>
graph[Output<std::vector<NormalizedRect>>(kPoseRectsNextFrameTag)];
outs.segmentation_masks >>
graph[Output<std::vector<Image>>(kSegmentationMaskTag)];
outs.pose_detections >>
graph[Output<std::vector<Detection>>(kDetectionsTag)];
outs.image >> graph[Output<Image>(kImageTag)];
// TODO remove when support is fixed.
// As mediapipe GraphBuilder currently doesn't support configuring
// InputStreamInfo, modifying the CalculatorGraphConfig proto directly.
CalculatorGraphConfig config = graph.GetConfig();
for (int i = 0; i < config.node_size(); ++i) {
if (config.node(i).calculator() == "PreviousLoopbackCalculator") {
auto* info = config.mutable_node(i)->add_input_stream_info();
info->set_tag_index(kLoopTag);
info->set_back_edge(true);
break;
}
}
return config;
}
private:
// Adds a mediapipe pose landmarker graph into the provided builder::Graph
// instance.
//
// tasks_options: the mediapipe tasks module PoseLandmarkerGraphOptions.
// image_in: (mediapipe::Image) stream to run pose landmark detection on.
// graph: the mediapipe graph instance to be updated.
absl::StatusOr<PoseLandmarkerOutputs> BuildPoseLandmarkerGraph(
PoseLandmarkerGraphOptions& tasks_options, Source<Image> image_in,
Source<NormalizedRect> norm_rect_in, Graph& graph) {
const int max_num_poses =
tasks_options.pose_detector_graph_options().num_poses();
auto& pose_detector =
graph.AddNode("mediapipe.tasks.vision.pose_detector.PoseDetectorGraph");
pose_detector.GetOptions<PoseDetectorGraphOptions>().Swap(
tasks_options.mutable_pose_detector_graph_options());
auto& clip_pose_rects =
graph.AddNode("ClipNormalizedRectVectorSizeCalculator");
clip_pose_rects.GetOptions<ClipVectorSizeCalculatorOptions>()
.set_max_vec_size(max_num_poses);
auto clipped_pose_rects = clip_pose_rects.Out("");
auto& pose_landmarks_detector_graph = graph.AddNode(
"mediapipe.tasks.vision.pose_landmarker."
"MultiplePoseLandmarksDetectorGraph");
pose_landmarks_detector_graph
.GetOptions<PoseLandmarksDetectorGraphOptions>()
.Swap(tasks_options.mutable_pose_landmarks_detector_graph_options());
image_in >> pose_landmarks_detector_graph.In(kImageTag);
clipped_pose_rects >> pose_landmarks_detector_graph.In(kNormRectTag);
// TODO: Add landmarks smoothing calculators to
// PoseLandmarkerGraph
auto landmarks = pose_landmarks_detector_graph.Out("LANDMARKS")
.Cast<std::vector<NormalizedLandmarkList>>();
auto world_landmarks = pose_landmarks_detector_graph.Out(kWorldLandmarksTag)
.Cast<std::vector<LandmarkList>>();
auto aux_landmarks =
pose_landmarks_detector_graph.Out(kAuxiliaryLandmarksTag)
.Cast<std::vector<NormalizedLandmarkList>>();
auto pose_rects_for_next_frame =
pose_landmarks_detector_graph.Out(kPoseRectsNextFrameTag)
.Cast<std::vector<NormalizedRect>>();
auto segmentation_masks =
pose_landmarks_detector_graph.Out(kSegmentationMaskTag)
.Cast<std::vector<Image>>();
if (tasks_options.base_options().use_stream_mode()) {
auto& previous_loopback = graph.AddNode("PreviousLoopbackCalculator");
image_in >> previous_loopback.In(kMainTag);
auto prev_pose_rects_from_landmarks =
previous_loopback[Output<std::vector<NormalizedRect>>(kPrevLoopTag)];
auto& min_size_node =
graph.AddNode("NormalizedRectVectorHasMinSizeCalculator");
prev_pose_rects_from_landmarks >> min_size_node.In(kIterableTag);
min_size_node.GetOptions<CollectionHasMinSizeCalculatorOptions>()
.set_min_size(max_num_poses);
auto has_enough_poses = min_size_node.Out("").Cast<bool>();
// While in stream mode, skip pose detector graph when we successfully
// track the poses from the last frame.
auto image_for_pose_detector =
DisallowIf(image_in, has_enough_poses, graph);
auto norm_rect_in_for_pose_detector =
DisallowIf(norm_rect_in, has_enough_poses, graph);
image_for_pose_detector >> pose_detector.In(kImageTag);
norm_rect_in_for_pose_detector >> pose_detector.In(kNormRectTag);
auto expanded_pose_rects_from_pose_detector =
pose_detector.Out(kExpandedPoseRectsTag);
auto& pose_association = graph.AddNode("AssociationNormRectCalculator");
pose_association.GetOptions<mediapipe::AssociationCalculatorOptions>()
.set_min_similarity_threshold(
tasks_options.min_tracking_confidence());
prev_pose_rects_from_landmarks >>
pose_association[Input<std::vector<NormalizedRect>>::Multiple("")][0];
expanded_pose_rects_from_pose_detector >>
pose_association[Input<std::vector<NormalizedRect>>::Multiple("")][1];
auto pose_rects = pose_association.Out("");
pose_rects >> clip_pose_rects.In("");
// Back edge.
pose_rects_for_next_frame >> previous_loopback.In(kLoopTag);
} else {
// While not in stream mode, the input images are not guaranteed to be in
// series, and we don't want to enable the tracking and rect associations
// between input images. Always use the pose detector graph.
image_in >> pose_detector.In(kImageTag);
norm_rect_in >> pose_detector.In(kNormRectTag);
auto pose_rects = pose_detector.Out(kExpandedPoseRectsTag);
pose_rects >> clip_pose_rects.In("");
}
// TODO: Replace PassThroughCalculator with a calculator that
// converts the pixel data to be stored on the target storage (CPU vs GPU).
auto& pass_through = graph.AddNode("PassThroughCalculator");
image_in >> pass_through.In("");
return {{
/* landmark_lists= */ landmarks,
/* world_landmarks= */ world_landmarks,
/* aux_landmarks= */ aux_landmarks,
/* pose_rects_next_frame= */ pose_rects_for_next_frame,
/* pose_detections */
pose_detector.Out(kDetectionsTag).Cast<std::vector<Detection>>(),
/* segmentation_masks= */ segmentation_masks,
/* image= */
pass_through[Output<Image>("")],
}};
}
};
REGISTER_MEDIAPIPE_GRAPH(
::mediapipe::tasks::vision::pose_landmarker::PoseLandmarkerGraph);
} // namespace pose_landmarker
} // namespace vision
} // namespace tasks
} // namespace mediapipe

View File

@ -0,0 +1,190 @@
/* Copyright 2023 The MediaPipe Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
#include <optional>
#include "absl/flags/flag.h"
#include "absl/status/statusor.h"
#include "absl/strings/str_format.h"
#include "absl/strings/string_view.h"
#include "mediapipe/framework/api2/builder.h"
#include "mediapipe/framework/api2/port.h"
#include "mediapipe/framework/calculator_framework.h"
#include "mediapipe/framework/deps/file_path.h"
#include "mediapipe/framework/formats/image.h"
#include "mediapipe/framework/formats/landmark.pb.h"
#include "mediapipe/framework/formats/rect.pb.h"
#include "mediapipe/framework/packet.h"
#include "mediapipe/framework/port/file_helpers.h"
#include "mediapipe/framework/port/gmock.h"
#include "mediapipe/framework/port/gtest.h"
#include "mediapipe/tasks/cc/core/mediapipe_builtin_op_resolver.h"
#include "mediapipe/tasks/cc/core/proto/base_options.pb.h"
#include "mediapipe/tasks/cc/core/proto/external_file.pb.h"
#include "mediapipe/tasks/cc/core/task_runner.h"
#include "mediapipe/tasks/cc/vision/pose_detector/proto/pose_detector_graph_options.pb.h"
#include "mediapipe/tasks/cc/vision/pose_landmarker/proto/pose_landmarker_graph_options.pb.h"
#include "mediapipe/tasks/cc/vision/pose_landmarker/proto/pose_landmarks_detector_graph_options.pb.h"
#include "mediapipe/tasks/cc/vision/utils/image_utils.h"
namespace mediapipe {
namespace tasks {
namespace vision {
namespace pose_landmarker {
namespace {
using ::file::Defaults;
using ::file::GetTextProto;
using ::mediapipe::api2::Input;
using ::mediapipe::api2::Output;
using ::mediapipe::api2::builder::Graph;
using ::mediapipe::api2::builder::Source;
using ::mediapipe::file::JoinPath;
using ::mediapipe::tasks::core::TaskRunner;
using ::mediapipe::tasks::vision::DecodeImageFromFile;
using ::mediapipe::tasks::vision::pose_landmarker::proto::
PoseLandmarkerGraphOptions;
using ::testing::EqualsProto;
using ::testing::Pointwise;
using ::testing::TestParamInfo;
using ::testing::TestWithParam;
using ::testing::Values;
using ::testing::proto::Approximately;
using ::testing::proto::Partially;
constexpr char kTestDataDirectory[] = "/mediapipe/tasks/testdata/vision/";
constexpr char kPoseLandmarkerModelBundleName[] = "pose_landmarker.task";
constexpr char kPoseImageName[] = "pose.jpg";
constexpr char kExpectedPoseLandmarksName[] =
"expected_pose_landmarks.prototxt";
constexpr char kImageTag[] = "IMAGE";
constexpr char kImageName[] = "image";
constexpr char kNormRectTag[] = "NORM_RECT";
constexpr char kNormRectName[] = "norm_rect";
constexpr char kNormLandmarksTag[] = "NORM_LANDMARKS";
constexpr char kNormLandmarksName[] = "norm_landmarks";
constexpr float kLiteModelFractionDiff = 0.05; // percentage
template <typename ProtoT>
ProtoT GetExpectedProto(absl::string_view filename) {
ProtoT expected_proto;
MP_EXPECT_OK(GetTextProto(file::JoinPath("./", kTestDataDirectory, filename),
&expected_proto, Defaults()));
return expected_proto;
}
// Struct holding the parameters for parameterized PoseLandmarkerGraphTest
// class.
struct PoseLandmarkerGraphTestParams {
// The name of this test, for convenience when displaying test results.
std::string test_name;
// The filename of the model to test.
std::string input_model_name;
// The filename of the test image.
std::string test_image_name;
// The expected output landmarks positions.
std::optional<std::vector<NormalizedLandmarkList>> expected_landmarks_list;
// The max value difference between expected_positions and detected positions.
float landmarks_diff_threshold;
};
// Helper function to create a PoseLandmarkerGraph TaskRunner.
absl::StatusOr<std::unique_ptr<TaskRunner>> CreatePoseLandmarkerGraphTaskRunner(
absl::string_view model_name) {
Graph graph;
auto& pose_landmarker = graph.AddNode(
"mediapipe.tasks.vision.pose_landmarker."
"PoseLandmarkerGraph");
auto* options = &pose_landmarker.GetOptions<PoseLandmarkerGraphOptions>();
options->mutable_base_options()->mutable_model_asset()->set_file_name(
JoinPath("./", kTestDataDirectory, model_name));
options->mutable_pose_detector_graph_options()->set_num_poses(1);
options->mutable_base_options()->set_use_stream_mode(true);
graph[Input<Image>(kImageTag)].SetName(kImageName) >>
pose_landmarker.In(kImageTag);
graph[Input<NormalizedRect>(kNormRectTag)].SetName(kNormRectName) >>
pose_landmarker.In(kNormRectTag);
pose_landmarker.Out(kNormLandmarksTag).SetName(kNormLandmarksName) >>
graph[Output<std::vector<NormalizedLandmarkList>>(kNormLandmarksTag)];
return TaskRunner::Create(
graph.GetConfig(),
absl::make_unique<tasks::core::MediaPipeBuiltinOpResolver>());
}
// Helper function to construct NormalizeRect proto.
NormalizedRect MakeNormRect(float x_center, float y_center, float width,
float height, float rotation) {
NormalizedRect pose_rect;
pose_rect.set_x_center(x_center);
pose_rect.set_y_center(y_center);
pose_rect.set_width(width);
pose_rect.set_height(height);
pose_rect.set_rotation(rotation);
return pose_rect;
}
class PoseLandmarkerGraphTest
: public testing::TestWithParam<PoseLandmarkerGraphTestParams> {};
TEST_P(PoseLandmarkerGraphTest, Succeeds) {
MP_ASSERT_OK_AND_ASSIGN(
Image image, DecodeImageFromFile(JoinPath("./", kTestDataDirectory,
GetParam().test_image_name)));
MP_ASSERT_OK_AND_ASSIGN(auto task_runner, CreatePoseLandmarkerGraphTaskRunner(
GetParam().input_model_name));
auto output_packets = task_runner->Process(
{{kImageName, MakePacket<Image>(std::move(image))},
{kNormRectName,
MakePacket<NormalizedRect>(MakeNormRect(0.5, 0.5, 1.0, 1.0, 0))}});
MP_ASSERT_OK(output_packets);
if (GetParam().expected_landmarks_list) {
const std::vector<NormalizedLandmarkList>& landmarks_lists =
(*output_packets)[kNormLandmarksName]
.Get<std::vector<NormalizedLandmarkList>>();
EXPECT_THAT(landmarks_lists,
Pointwise(Approximately(Partially(EqualsProto()),
GetParam().landmarks_diff_threshold),
*GetParam().expected_landmarks_list));
}
}
INSTANTIATE_TEST_SUITE_P(
PoseLandmarkerGraphTests, PoseLandmarkerGraphTest,
Values(PoseLandmarkerGraphTestParams{
/* test_name= */ "PoseLandmarkerLite",
/* input_model_name= */ kPoseLandmarkerModelBundleName,
/* test_image_name= */ kPoseImageName,
/* expected_landmarks_list= */
{{GetExpectedProto<NormalizedLandmarkList>(
kExpectedPoseLandmarksName)}},
/* landmarks_diff_threshold= */ kLiteModelFractionDiff}),
[](const TestParamInfo<PoseLandmarkerGraphTest::ParamType>& info) {
return info.param.test_name;
});
} // namespace
} // namespace pose_landmarker
} // namespace vision
} // namespace tasks
} // namespace mediapipe

View File

@ -29,3 +29,15 @@ mediapipe_proto_library(
"//mediapipe/tasks/cc/core/proto:base_options_proto", "//mediapipe/tasks/cc/core/proto:base_options_proto",
], ],
) )
mediapipe_proto_library(
name = "pose_landmarker_graph_options_proto",
srcs = ["pose_landmarker_graph_options.proto"],
deps = [
":pose_landmarks_detector_graph_options_proto",
"//mediapipe/framework:calculator_options_proto",
"//mediapipe/framework:calculator_proto",
"//mediapipe/tasks/cc/core/proto:base_options_proto",
"//mediapipe/tasks/cc/vision/pose_detector/proto:pose_detector_graph_options_proto",
],
)

View File

@ -0,0 +1,48 @@
/* Copyright 2023 The MediaPipe Authors. All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/
syntax = "proto2";
package mediapipe.tasks.vision.pose_landmarker.proto;
import "mediapipe/framework/calculator.proto";
import "mediapipe/framework/calculator_options.proto";
import "mediapipe/tasks/cc/core/proto/base_options.proto";
import "mediapipe/tasks/cc/vision/pose_detector/proto/pose_detector_graph_options.proto";
import "mediapipe/tasks/cc/vision/pose_landmarker/proto/pose_landmarks_detector_graph_options.proto";
option java_package = "com.google.mediapipe.tasks.vision.poselandmarker.proto";
option java_outer_classname = "PoseLandmarkerGraphOptionsProto";
message PoseLandmarkerGraphOptions {
extend mediapipe.CalculatorOptions {
optional PoseLandmarkerGraphOptions ext = 516587230;
}
// Base options for configuring Task library, such as specifying the TfLite
// model file with metadata, accelerator options, etc.
optional core.proto.BaseOptions base_options = 1;
// Options for pose detector graph.
optional pose_detector.proto.PoseDetectorGraphOptions
pose_detector_graph_options = 2;
// Options for pose landmarks detector graph.
optional PoseLandmarksDetectorGraphOptions
pose_landmarks_detector_graph_options = 3;
// Minimum confidence for pose landmarks tracking to be considered
// successfully.
optional float min_tracking_confidence = 4 [default = 0.5];
}

View File

@ -24,7 +24,6 @@ objc_library(
"//mediapipe/tasks/cc:common", "//mediapipe/tasks/cc:common",
"//mediapipe/tasks/ios/common:MPPCommon", "//mediapipe/tasks/ios/common:MPPCommon",
"@com_google_absl//absl/status", "@com_google_absl//absl/status",
"@com_google_absl//absl/strings",
"@com_google_absl//absl/strings:cord", "@com_google_absl//absl/strings:cord",
], ],
) )

View File

@ -44,3 +44,13 @@ objc_library(
hdrs = ["sources/MPPEmbeddingResult.h"], hdrs = ["sources/MPPEmbeddingResult.h"],
deps = [":MPPEmbedding"], deps = [":MPPEmbedding"],
) )
objc_library(
name = "MPPDetection",
srcs = ["sources/MPPDetection.m"],
hdrs = ["sources/MPPDetection.h"],
deps = [
":MPPCategory",
"//third_party/apple_frameworks:UIKit",
],
)

View File

@ -0,0 +1,103 @@
// Copyright 2023 The MediaPipe Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>
#import "mediapipe/tasks/ios/components/containers/sources/MPPCategory.h"
NS_ASSUME_NONNULL_BEGIN
/**
* Normalized keypoint represents a point in 2D space with x, y coordinates. x and y are normalized
* to [0.0, 1.0] by the image width and height respectively.
*/
NS_SWIFT_NAME(NormalizedKeypoint)
@interface MPPNormalizedKeypoint : NSObject
/** The (x,y) coordinates location of the normalized keypoint. */
@property(nonatomic, readonly) CGPoint location;
/** The optional label of the normalized keypoint. */
@property(nonatomic, readonly, nullable) NSString *label;
/** The optional score of the normalized keypoint. If score is absent, it will be equal to 0.0. */
@property(nonatomic, readonly) float score;
/**
* Initializes a new `MPPNormalizedKeypoint` object with the given location, label and score.
* You must pass 0.0 for `score` if it is not present.
*
* @param location The (x,y) coordinates location of the normalized keypoint.
* @param label The optional label of the normalized keypoint.
* @param score The optional score of the normalized keypoint. You must pass 0.0 for score if it
* is not present.
*
* @return An instance of `MPPNormalizedKeypoint` initialized with the given given location, label
* and score.
*/
- (instancetype)initWithLocation:(CGPoint)location
label:(nullable NSString *)label
score:(float)score NS_DESIGNATED_INITIALIZER;
- (instancetype)init NS_UNAVAILABLE;
+ (instancetype)new NS_UNAVAILABLE;
@end
/** Represents one detected object in the results of `MPPObjectDetector`. */
NS_SWIFT_NAME(Detection)
@interface MPPDetection : NSObject
/** An array of `MPPCategory` objects containing the predicted categories. */
@property(nonatomic, readonly) NSArray<MPPCategory *> *categories;
/** The bounding box of the detected object. */
@property(nonatomic, readonly) CGRect boundingBox;
/**
* An optional array of `MPPNormalizedKeypoint` objects associated with the detection. Keypoints
* represent interesting points related to the detection. For example, the keypoints represent the
* eyes, ear and mouth from the from detection model. In template matching detection, e.g. KNIFT,
* they can instead represent the feature points for template matching.
*/
@property(nonatomic, readonly, nullable) NSArray<MPPNormalizedKeypoint *> *keypoints;
/**
* Initializes a new `MPPDetection` object with the given array of categories, bounding box and
* optional array of keypoints;
*
* @param categories A list of `MPPCategory` objects that contain category name, display name,
* score, and the label index.
* @param boundingBox A `CGRect` that represents the bounding box.
* @param keypoints: An optional array of `MPPNormalizedKeypoint` objects associated with the
* detection. Keypoints represent interesting points related to the detection. For example, the
* keypoints represent the eyes, ear and mouth from the face detection model. In template matching
* detection, e.g. KNIFT, they can instead represent the feature points for template matching.
*
* @return An instance of `MPPDetection` initialized with the given array of categories, bounding
* box and `nil` keypoints.
*/
- (instancetype)initWithCategories:(NSArray<MPPCategory *> *)categories
boundingBox:(CGRect)boundingBox
keypoints:(nullable NSArray<MPPNormalizedKeypoint *> *)keypoints
NS_DESIGNATED_INITIALIZER;
- (instancetype)init NS_UNAVAILABLE;
+ (instancetype)new NS_UNAVAILABLE;
@end
NS_ASSUME_NONNULL_END

View File

@ -0,0 +1,68 @@
// Copyright 2023 The MediaPipe Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import "mediapipe/tasks/ios/components/containers/sources/MPPDetection.h"
@implementation MPPNormalizedKeypoint
- (instancetype)initWithLocation:(CGPoint)location
label:(nullable NSString *)label
score:(float)score {
self = [super init];
if (self) {
_location = location;
_label = label;
_score = score;
}
return self;
}
// TODO: Implement hash
- (BOOL)isEqual:(nullable id)object {
if (!object) {
return NO;
}
if (self == object) {
return YES;
}
if (![object isKindOfClass:[MPPNormalizedKeypoint class]]) {
return NO;
}
MPPNormalizedKeypoint *otherKeypoint = (MPPNormalizedKeypoint *)object;
return CGPointEqualToPoint(self.location, otherKeypoint.location) &&
(self.label == otherKeypoint.label) && (self.score == otherKeypoint.score);
}
@end
@implementation MPPDetection
- (instancetype)initWithCategories:(NSArray<MPPCategory *> *)categories
boundingBox:(CGRect)boundingBox
keypoints:(nullable NSArray<MPPNormalizedKeypoint *> *)keypoints {
self = [super init];
if (self) {
_categories = categories;
_boundingBox = boundingBox;
_keypoints = keypoints;
}
return self;
}
@end

View File

@ -61,3 +61,15 @@ objc_library(
"//mediapipe/tasks/ios/components/containers:MPPEmbeddingResult", "//mediapipe/tasks/ios/components/containers:MPPEmbeddingResult",
], ],
) )
objc_library(
name = "MPPDetectionHelpers",
srcs = ["sources/MPPDetection+Helpers.mm"],
hdrs = ["sources/MPPDetection+Helpers.h"],
deps = [
"//mediapipe/framework/formats:detection_cc_proto",
"//mediapipe/framework/formats:location_data_cc_proto",
"//mediapipe/tasks/ios/common/utils:NSStringHelpers",
"//mediapipe/tasks/ios/components/containers:MPPDetection",
],
)

View File

@ -0,0 +1,26 @@
// Copyright 2023 The MediaPipe Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#include "mediapipe/framework/formats/detection.pb.h"
#import "mediapipe/tasks/ios/components/containers/sources/MPPDetection.h"
NS_ASSUME_NONNULL_BEGIN
@interface MPPDetection (Helpers)
+ (MPPDetection *)detectionWithProto:(const mediapipe::Detection &)detectionProto;
@end
NS_ASSUME_NONNULL_END

View File

@ -0,0 +1,83 @@
// Copyright 2023 The MediaPipe Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import "mediapipe/tasks/ios/components/containers/utils/sources/MPPDetection+Helpers.h"
#import "mediapipe/framework/formats/location_data.pb.h"
#import "mediapipe/tasks/ios/common/utils/sources/NSString+Helpers.h"
static const NSInteger kDefaultCategoryIndex = -1;
namespace {
using DetectionProto = ::mediapipe::Detection;
using BoundingBoxProto = ::mediapipe::LocationData::BoundingBox;
} // namespace
@implementation MPPDetection (Helpers)
+ (MPPDetection *)detectionWithProto:(const DetectionProto &)detectionProto {
NSMutableArray<MPPCategory *> *categories =
[NSMutableArray arrayWithCapacity:(NSUInteger)detectionProto.score_size()];
for (int idx = 0; idx < detectionProto.score_size(); ++idx) {
NSInteger categoryIndex =
detectionProto.label_id_size() > idx ? detectionProto.label_id(idx) : kDefaultCategoryIndex;
NSString *categoryName = detectionProto.label_size() > idx
? [NSString stringWithCppString:detectionProto.label(idx)]
: nil;
NSString *displayName = detectionProto.display_name_size() > idx
? [NSString stringWithCppString:detectionProto.display_name(idx)]
: nil;
[categories addObject:[[MPPCategory alloc] initWithIndex:categoryIndex
score:detectionProto.score(idx)
categoryName:categoryName
displayName:displayName]];
}
CGRect boundingBox = CGRectZero;
if (detectionProto.location_data().has_bounding_box()) {
const BoundingBoxProto &boundingBoxProto = detectionProto.location_data().bounding_box();
boundingBox.origin.x = boundingBoxProto.xmin();
boundingBox.origin.y = boundingBoxProto.ymin();
boundingBox.size.width = boundingBoxProto.width();
boundingBox.size.height = boundingBoxProto.height();
}
NSMutableArray<MPPNormalizedKeypoint *> *normalizedKeypoints;
if (!detectionProto.location_data().relative_keypoints().empty()) {
normalizedKeypoints = [NSMutableArray
arrayWithCapacity:(NSUInteger)detectionProto.location_data().relative_keypoints_size()];
for (const auto &keypoint : detectionProto.location_data().relative_keypoints()) {
NSString *label = keypoint.has_keypoint_label()
? [NSString stringWithCppString:keypoint.keypoint_label()]
: nil;
CGPoint location = CGPointMake(keypoint.x(), keypoint.y());
float score = keypoint.has_score() ? keypoint.score() : 0.0f;
[normalizedKeypoints addObject:[[MPPNormalizedKeypoint alloc] initWithLocation:location
label:label
score:score]];
}
}
return [[MPPDetection alloc] initWithCategories:categories
boundingBox:boundingBox
keypoints:normalizedKeypoints];
}
@end

View File

@ -7,7 +7,5 @@ objc_library(
srcs = ["sources/MPPImage+TestUtils.m"], srcs = ["sources/MPPImage+TestUtils.m"],
hdrs = ["sources/MPPImage+TestUtils.h"], hdrs = ["sources/MPPImage+TestUtils.h"],
module_name = "MPPImageTestUtils", module_name = "MPPImageTestUtils",
deps = [ deps = ["//mediapipe/tasks/ios/vision/core:MPPImage"],
"//mediapipe/tasks/ios/vision/core:MPPImage",
],
) )

View File

@ -0,0 +1,38 @@
# Copyright 2023 The MediaPipe Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
package(default_visibility = ["//mediapipe/tasks:internal"])
licenses(["notice"])
objc_library(
name = "MPPObjectDetectionResult",
srcs = ["sources/MPPObjectDetectionResult.m"],
hdrs = ["sources/MPPObjectDetectionResult.h"],
deps = [
"//mediapipe/tasks/ios/components/containers:MPPDetection",
"//mediapipe/tasks/ios/core:MPPTaskResult",
],
)
objc_library(
name = "MPPObjectDetectorOptions",
srcs = ["sources/MPPObjectDetectorOptions.m"],
hdrs = ["sources/MPPObjectDetectorOptions.h"],
deps = [
":MPPObjectDetectionResult",
"//mediapipe/tasks/ios/core:MPPTaskOptions",
"//mediapipe/tasks/ios/vision/core:MPPRunningMode",
],
)

View File

@ -0,0 +1,49 @@
// Copyright 2023 The MediaPipe Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import <Foundation/Foundation.h>
#import "mediapipe/tasks/ios/components/containers/sources/MPPDetection.h"
#import "mediapipe/tasks/ios/core/sources/MPPTaskResult.h"
NS_ASSUME_NONNULL_BEGIN
/** Represents the detection results generated by `MPPObjectDetector`. */
NS_SWIFT_NAME(ObjectDetectionResult)
@interface MPPObjectDetectionResult : MPPTaskResult
/**
* The array of `MPPDetection` objects each of which has a bounding box that is expressed in the
* unrotated input frame of reference coordinates system, i.e. in `[0,image_width) x
* [0,image_height)`, which are the dimensions of the underlying image data.
*/
@property(nonatomic, readonly) NSArray<MPPDetection *> *detections;
/**
* Initializes a new `MPPObjectDetectionResult` with the given array of detections and timestamp (in
* milliseconds).
*
* @param detections An array of `MPPDetection` objects each of which has a bounding box that is
* expressed in the unrotated input frame of reference coordinates system, i.e. in `[0,image_width)
* x [0,image_height)`, which are the dimensions of the underlying image data.
* @param timestampMs The timestamp for this result.
*
* @return An instance of `MPPObjectDetectionResult` initialized with the given array of detections
* and timestamp (in milliseconds).
*/
- (instancetype)initWithDetections:(NSArray<MPPDetection *> *)detections
timestampMs:(NSInteger)timestampMs;
@end
NS_ASSUME_NONNULL_END

View File

@ -0,0 +1,28 @@
// Copyright 2023 The MediaPipe Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import "mediapipe/tasks/ios/vision/object_detector/sources/MPPObjectDetectionResult.h"
@implementation MPPObjectDetectionResult
- (instancetype)initWithDetections:(NSArray<MPPDetection *> *)detections
timestampMs:(NSInteger)timestampMs {
self = [super initWithTimestampMs:timestampMs];
if (self) {
_detections = detections;
}
return self;
}
@end

View File

@ -0,0 +1,71 @@
// Copyright 2023 The MediaPipe Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import <Foundation/Foundation.h>
#import "mediapipe/tasks/ios/core/sources/MPPTaskOptions.h"
#import "mediapipe/tasks/ios/vision/core/sources/MPPRunningMode.h"
#import "mediapipe/tasks/ios/vision/object_detector/sources/MPPObjectDetectionResult.h"
NS_ASSUME_NONNULL_BEGIN
/** Options for setting up a `MPPObjectDetector`. */
NS_SWIFT_NAME(ObjectDetectorOptions)
@interface MPPObjectDetectorOptions : MPPTaskOptions <NSCopying>
@property(nonatomic) MPPRunningMode runningMode;
/**
* The user-defined result callback for processing live stream data. The result callback should only
* be specified when the running mode is set to the live stream mode.
* TODO: Add parameter `MPPImage` in the callback.
*/
@property(nonatomic, copy) void (^completion)
(MPPObjectDetectionResult *result, NSInteger timestampMs, NSError *error);
/**
* The locale to use for display names specified through the TFLite Model Metadata, if any. Defaults
* to English.
*/
@property(nonatomic, copy) NSString *displayNamesLocale;
/**
* The maximum number of top-scored classification results to return. If < 0, all available results
* will be returned. If 0, an invalid argument error is returned.
*/
@property(nonatomic) NSInteger maxResults;
/**
* Score threshold to override the one provided in the model metadata (if any). Results below this
* value are rejected.
*/
@property(nonatomic) float scoreThreshold;
/**
* The allowlist of category names. If non-empty, detection results whose category name is not in
* this set will be filtered out. Duplicate or unknown category names are ignored. Mutually
* exclusive with categoryDenylist.
*/
@property(nonatomic, copy) NSArray<NSString *> *categoryAllowlist;
/**
* The denylist of category names. If non-empty, detection results whose category name is in this
* set will be filtered out. Duplicate or unknown category names are ignored. Mutually exclusive
* with categoryAllowlist.
*/
@property(nonatomic, copy) NSArray<NSString *> *categoryDenylist;
@end
NS_ASSUME_NONNULL_END

View File

@ -0,0 +1,41 @@
// Copyright 2023 The MediaPipe Authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
#import "mediapipe/tasks/ios/vision/object_detector/sources/MPPObjectDetectorOptions.h"
@implementation MPPObjectDetectorOptions
- (instancetype)init {
self = [super init];
if (self) {
_maxResults = -1;
_scoreThreshold = 0;
}
return self;
}
- (id)copyWithZone:(NSZone *)zone {
MPPObjectDetectorOptions *objectDetectorOptions = [super copyWithZone:zone];
objectDetectorOptions.scoreThreshold = self.scoreThreshold;
objectDetectorOptions.maxResults = self.maxResults;
objectDetectorOptions.categoryDenylist = self.categoryDenylist;
objectDetectorOptions.categoryAllowlist = self.categoryAllowlist;
objectDetectorOptions.displayNamesLocale = self.displayNamesLocale;
objectDetectorOptions.completion = self.completion;
return objectDetectorOptions;
}
@end

View File

@ -78,6 +78,7 @@ mediapipe_files(srcs = [
"pose.jpg", "pose.jpg",
"pose_detection.tflite", "pose_detection.tflite",
"pose_landmark_lite.tflite", "pose_landmark_lite.tflite",
"pose_landmarker.task",
"right_hands.jpg", "right_hands.jpg",
"right_hands_rotated.jpg", "right_hands_rotated.jpg",
"segmentation_golden_rotation0.png", "segmentation_golden_rotation0.png",
@ -187,6 +188,7 @@ filegroup(
"palm_detection_full.tflite", "palm_detection_full.tflite",
"pose_detection.tflite", "pose_detection.tflite",
"pose_landmark_lite.tflite", "pose_landmark_lite.tflite",
"pose_landmarker.task",
"selfie_segm_128_128_3.tflite", "selfie_segm_128_128_3.tflite",
"selfie_segm_144_256_3.tflite", "selfie_segm_144_256_3.tflite",
"selfie_segmentation.tflite", "selfie_segmentation.tflite",

Binary file not shown.

View File

@ -4,6 +4,11 @@ load("//mediapipe/framework/port:build_config.bzl", "mediapipe_ts_declaration")
package(default_visibility = ["//mediapipe/tasks:internal"]) package(default_visibility = ["//mediapipe/tasks:internal"])
mediapipe_ts_declaration(
name = "bounding_box",
srcs = ["bounding_box.d.ts"],
)
mediapipe_ts_declaration( mediapipe_ts_declaration(
name = "category", name = "category",
srcs = ["category.d.ts"], srcs = ["category.d.ts"],
@ -15,6 +20,15 @@ mediapipe_ts_declaration(
deps = [":category"], deps = [":category"],
) )
mediapipe_ts_declaration(
name = "detection_result",
srcs = ["detection_result.d.ts"],
deps = [
":bounding_box",
":category",
],
)
mediapipe_ts_declaration( mediapipe_ts_declaration(
name = "keypoint", name = "keypoint",
srcs = ["keypoint.d.ts"], srcs = ["keypoint.d.ts"],

View File

@ -0,0 +1,27 @@
/**
* Copyright 2023 The MediaPipe Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/** An integer bounding box, axis aligned. */
export declare interface BoundingBox {
/** The X coordinate of the top-left corner, in pixels. */
originX: number;
/** The Y coordinate of the top-left corner, in pixels. */
originY: number;
/** The width of the bounding box, in pixels. */
width: number;
/** The height of the bounding box, in pixels. */
height: number;
}

View File

@ -0,0 +1,33 @@
/**
* Copyright 2023 The MediaPipe Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import {BoundingBox} from '../../../../tasks/web/components/containers/bounding_box';
import {Category} from '../../../../tasks/web/components/containers/category';
/** Represents one detection by a detection task. */
export declare interface Detection {
/** A list of `Category` objects. */
categories: Category[];
/** The bounding box of the detected objects. */
boundingBox?: BoundingBox;
}
/** Detection results of a model. */
export interface DetectionResult {
/** A list of Detections. */
detections: Detection[];
}

View File

@ -56,6 +56,27 @@ jasmine_node_test(
deps = [":classifier_result_test_lib"], deps = [":classifier_result_test_lib"],
) )
mediapipe_ts_library(
name = "detection_result",
srcs = ["detection_result.ts"],
deps = [
"//mediapipe/framework/formats:detection_jspb_proto",
"//mediapipe/tasks/web/components/containers:detection_result",
],
)
mediapipe_ts_library(
name = "detection_result_test_lib",
testonly = True,
srcs = ["detection_result.test.ts"],
deps = [
":detection_result",
"//mediapipe/framework/formats:detection_jspb_proto",
"//mediapipe/framework/formats:location_data_jspb_proto",
"//mediapipe/tasks/web/components/containers:detection_result",
],
)
mediapipe_ts_library( mediapipe_ts_library(
name = "embedder_result", name = "embedder_result",
srcs = ["embedder_result.ts"], srcs = ["embedder_result.ts"],

View File

@ -0,0 +1,76 @@
/**
* Copyright 2023 The MediaPipe Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import 'jasmine';
import {Detection as DetectionProto} from '../../../../framework/formats/detection_pb';
import {LocationData} from '../../../../framework/formats/location_data_pb';
import {convertFromDetectionProto} from './detection_result';
// The OSS JS API does not support the builder pattern.
// tslint:disable:jspb-use-builder-pattern
describe('convertFromDetectionProto()', () => {
it('transforms custom values', () => {
const detection = new DetectionProto();
detection.addScore(0.1);
detection.addLabelId(1);
detection.addLabel('foo');
detection.addDisplayName('bar');
const locationData = new LocationData();
const boundingBox = new LocationData.BoundingBox();
boundingBox.setXmin(1);
boundingBox.setYmin(2);
boundingBox.setWidth(3);
boundingBox.setHeight(4);
locationData.setBoundingBox(boundingBox);
detection.setLocationData(locationData);
const result = convertFromDetectionProto(detection);
expect(result).toEqual({
categories: [{
score: 0.1,
index: 1,
categoryName: 'foo',
displayName: 'bar',
}],
boundingBox: {originX: 1, originY: 2, width: 3, height: 4}
});
});
it('transforms default values', () => {
const detection = new DetectionProto();
detection.addScore(0.2);
const locationData = new LocationData();
const boundingBox = new LocationData.BoundingBox();
locationData.setBoundingBox(boundingBox);
detection.setLocationData(locationData);
const result = convertFromDetectionProto(detection);
expect(result).toEqual({
categories: [{
score: 0.2,
index: -1,
categoryName: '',
displayName: '',
}],
boundingBox: {originX: 0, originY: 0, width: 0, height: 0}
});
});
});

View File

@ -0,0 +1,50 @@
/**
* Copyright 2023 The MediaPipe Authors. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import {Detection as DetectionProto} from '../../../../framework/formats/detection_pb';
import {Detection} from '../../../../tasks/web/components/containers/detection_result';
const DEFAULT_CATEGORY_INDEX = -1;
/** Converts a Detection proto into a Detection object. */
export function convertFromDetectionProto(source: DetectionProto): Detection {
const scores = source.getScoreList();
const indexes = source.getLabelIdList();
const labels = source.getLabelList();
const displayNames = source.getDisplayNameList();
const detection: Detection = {categories: []};
for (let i = 0; i < scores.length; i++) {
detection.categories.push({
score: scores[i],
index: indexes[i] ?? DEFAULT_CATEGORY_INDEX,
categoryName: labels[i] ?? '',
displayName: displayNames[i] ?? '',
});
}
const boundingBox = source.getLocationData()?.getBoundingBox();
if (boundingBox) {
detection.boundingBox = {
originX: boundingBox.getXmin() ?? 0,
originY: boundingBox.getYmin() ?? 0,
width: boundingBox.getWidth() ?? 0,
height: boundingBox.getHeight() ?? 0
};
}
return detection;
}

View File

@ -22,6 +22,7 @@ mediapipe_ts_library(
"//mediapipe/tasks/cc/core/proto:base_options_jspb_proto", "//mediapipe/tasks/cc/core/proto:base_options_jspb_proto",
"//mediapipe/tasks/cc/vision/object_detector/proto:object_detector_options_jspb_proto", "//mediapipe/tasks/cc/vision/object_detector/proto:object_detector_options_jspb_proto",
"//mediapipe/tasks/web/components/containers:category", "//mediapipe/tasks/web/components/containers:category",
"//mediapipe/tasks/web/components/processors:detection_result",
"//mediapipe/tasks/web/core", "//mediapipe/tasks/web/core",
"//mediapipe/tasks/web/vision/core:image_processing_options", "//mediapipe/tasks/web/vision/core:image_processing_options",
"//mediapipe/tasks/web/vision/core:vision_task_runner", "//mediapipe/tasks/web/vision/core:vision_task_runner",
@ -37,7 +38,9 @@ mediapipe_ts_declaration(
], ],
visibility = ["//visibility:public"], visibility = ["//visibility:public"],
deps = [ deps = [
"//mediapipe/tasks/web/components/containers:bounding_box",
"//mediapipe/tasks/web/components/containers:category", "//mediapipe/tasks/web/components/containers:category",
"//mediapipe/tasks/web/components/containers:detection_result",
"//mediapipe/tasks/web/core", "//mediapipe/tasks/web/core",
"//mediapipe/tasks/web/core:classifier_options", "//mediapipe/tasks/web/core:classifier_options",
"//mediapipe/tasks/web/vision/core:vision_task_options", "//mediapipe/tasks/web/vision/core:vision_task_options",

View File

@ -19,6 +19,7 @@ import {CalculatorOptions} from '../../../../framework/calculator_options_pb';
import {Detection as DetectionProto} from '../../../../framework/formats/detection_pb'; import {Detection as DetectionProto} from '../../../../framework/formats/detection_pb';
import {BaseOptions as BaseOptionsProto} from '../../../../tasks/cc/core/proto/base_options_pb'; import {BaseOptions as BaseOptionsProto} from '../../../../tasks/cc/core/proto/base_options_pb';
import {ObjectDetectorOptions as ObjectDetectorOptionsProto} from '../../../../tasks/cc/vision/object_detector/proto/object_detector_options_pb'; import {ObjectDetectorOptions as ObjectDetectorOptionsProto} from '../../../../tasks/cc/vision/object_detector/proto/object_detector_options_pb';
import {convertFromDetectionProto} from '../../../../tasks/web/components/processors/detection_result';
import {WasmFileset} from '../../../../tasks/web/core/wasm_fileset'; import {WasmFileset} from '../../../../tasks/web/core/wasm_fileset';
import {ImageProcessingOptions} from '../../../../tasks/web/vision/core/image_processing_options'; import {ImageProcessingOptions} from '../../../../tasks/web/vision/core/image_processing_options';
import {VisionGraphRunner, VisionTaskRunner} from '../../../../tasks/web/vision/core/vision_task_runner'; import {VisionGraphRunner, VisionTaskRunner} from '../../../../tasks/web/vision/core/vision_task_runner';
@ -26,15 +27,13 @@ import {ImageSource, WasmModule} from '../../../../web/graph_runner/graph_runner
// Placeholder for internal dependency on trusted resource url // Placeholder for internal dependency on trusted resource url
import {ObjectDetectorOptions} from './object_detector_options'; import {ObjectDetectorOptions} from './object_detector_options';
import {Detection} from './object_detector_result'; import {ObjectDetectorResult} from './object_detector_result';
const IMAGE_STREAM = 'input_frame_gpu'; const IMAGE_STREAM = 'input_frame_gpu';
const NORM_RECT_STREAM = 'norm_rect'; const NORM_RECT_STREAM = 'norm_rect';
const DETECTIONS_STREAM = 'detections'; const DETECTIONS_STREAM = 'detections';
const OBJECT_DETECTOR_GRAPH = 'mediapipe.tasks.vision.ObjectDetectorGraph'; const OBJECT_DETECTOR_GRAPH = 'mediapipe.tasks.vision.ObjectDetectorGraph';
const DEFAULT_CATEGORY_INDEX = -1;
export * from './object_detector_options'; export * from './object_detector_options';
export * from './object_detector_result'; export * from './object_detector_result';
export {ImageSource}; // Used in the public API export {ImageSource}; // Used in the public API
@ -44,7 +43,7 @@ export {ImageSource}; // Used in the public API
/** Performs object detection on images. */ /** Performs object detection on images. */
export class ObjectDetector extends VisionTaskRunner { export class ObjectDetector extends VisionTaskRunner {
private detections: Detection[] = []; private result: ObjectDetectorResult = {detections: []};
private readonly options = new ObjectDetectorOptionsProto(); private readonly options = new ObjectDetectorOptionsProto();
/** /**
@ -163,13 +162,13 @@ export class ObjectDetector extends VisionTaskRunner {
* @param image An image to process. * @param image An image to process.
* @param imageProcessingOptions the `ImageProcessingOptions` specifying how * @param imageProcessingOptions the `ImageProcessingOptions` specifying how
* to process the input image before running inference. * to process the input image before running inference.
* @return The list of detected objects * @return A result containing a list of detected objects.
*/ */
detect(image: ImageSource, imageProcessingOptions?: ImageProcessingOptions): detect(image: ImageSource, imageProcessingOptions?: ImageProcessingOptions):
Detection[] { ObjectDetectorResult {
this.detections = []; this.result = {detections: []};
this.processImageData(image, imageProcessingOptions); this.processImageData(image, imageProcessingOptions);
return [...this.detections]; return this.result;
} }
/** /**
@ -181,46 +180,21 @@ export class ObjectDetector extends VisionTaskRunner {
* @param timestamp The timestamp of the current frame, in ms. * @param timestamp The timestamp of the current frame, in ms.
* @param imageProcessingOptions the `ImageProcessingOptions` specifying how * @param imageProcessingOptions the `ImageProcessingOptions` specifying how
* to process the input image before running inference. * to process the input image before running inference.
* @return The list of detected objects * @return A result containing a list of detected objects.
*/ */
detectForVideo( detectForVideo(
videoFrame: ImageSource, timestamp: number, videoFrame: ImageSource, timestamp: number,
imageProcessingOptions?: ImageProcessingOptions): Detection[] { imageProcessingOptions?: ImageProcessingOptions): ObjectDetectorResult {
this.detections = []; this.result = {detections: []};
this.processVideoData(videoFrame, imageProcessingOptions, timestamp); this.processVideoData(videoFrame, imageProcessingOptions, timestamp);
return [...this.detections]; return this.result;
} }
/** Converts raw data into a Detection, and adds it to our detection list. */ /** Converts raw data into a Detection, and adds it to our detection list. */
private addJsObjectDetections(data: Uint8Array[]): void { private addJsObjectDetections(data: Uint8Array[]): void {
for (const binaryProto of data) { for (const binaryProto of data) {
const detectionProto = DetectionProto.deserializeBinary(binaryProto); const detectionProto = DetectionProto.deserializeBinary(binaryProto);
const scores = detectionProto.getScoreList(); this.result.detections.push(convertFromDetectionProto(detectionProto));
const indexes = detectionProto.getLabelIdList();
const labels = detectionProto.getLabelList();
const displayNames = detectionProto.getDisplayNameList();
const detection: Detection = {categories: []};
for (let i = 0; i < scores.length; i++) {
detection.categories.push({
score: scores[i],
index: indexes[i] ?? DEFAULT_CATEGORY_INDEX,
categoryName: labels[i] ?? '',
displayName: displayNames[i] ?? '',
});
}
const boundingBox = detectionProto.getLocationData()?.getBoundingBox();
if (boundingBox) {
detection.boundingBox = {
originX: boundingBox.getXmin() ?? 0,
originY: boundingBox.getYmin() ?? 0,
width: boundingBox.getWidth() ?? 0,
height: boundingBox.getHeight() ?? 0
};
}
this.detections.push(detection);
} }
} }

View File

@ -14,27 +14,6 @@
* limitations under the License. * limitations under the License.
*/ */
import {Category} from '../../../../tasks/web/components/containers/category'; export {BoundingBox} from '../../../../tasks/web/components/containers/bounding_box';
export {Category} from '../../../../tasks/web/components/containers/category';
export {Category}; export {Detection, DetectionResult as ObjectDetectorResult} from '../../../../tasks/web/components/containers/detection_result';
/** An integer bounding box, axis aligned. */
export declare interface BoundingBox {
/** The X coordinate of the top-left corner, in pixels. */
originX: number;
/** The Y coordinate of the top-left corner, in pixels. */
originY: number;
/** The width of the bounding box, in pixels. */
width: number;
/** The height of the bounding box, in pixels. */
height: number;
}
/** Represents one object detected by the `ObjectDetector`. */
export declare interface Detection {
/** A list of `Category` objects. */
categories: Category[];
/** The bounding box of the detected objects. */
boundingBox?: BoundingBox;
}

View File

@ -179,56 +179,29 @@ describe('ObjectDetector', () => {
}); });
it('transforms results', async () => { it('transforms results', async () => {
const detectionProtos: Uint8Array[] = []; const detection = new DetectionProto();
// Add a detection with all optional properties
let detection = new DetectionProto();
detection.addScore(0.1); detection.addScore(0.1);
detection.addLabelId(1); const locationData = new LocationData();
detection.addLabel('foo'); const boundingBox = new LocationData.BoundingBox();
detection.addDisplayName('bar');
let locationData = new LocationData();
let boundingBox = new LocationData.BoundingBox();
boundingBox.setXmin(1);
boundingBox.setYmin(2);
boundingBox.setWidth(3);
boundingBox.setHeight(4);
locationData.setBoundingBox(boundingBox); locationData.setBoundingBox(boundingBox);
detection.setLocationData(locationData); detection.setLocationData(locationData);
detectionProtos.push(detection.serializeBinary());
// Add a detection without optional properties const binaryProto = detection.serializeBinary();
detection = new DetectionProto();
detection.addScore(0.2);
locationData = new LocationData();
boundingBox = new LocationData.BoundingBox();
locationData.setBoundingBox(boundingBox);
detection.setLocationData(locationData);
detectionProtos.push(detection.serializeBinary());
// Pass the test data to our listener // Pass the test data to our listener
objectDetector.fakeWasmModule._waitUntilIdle.and.callFake(() => { objectDetector.fakeWasmModule._waitUntilIdle.and.callFake(() => {
verifyListenersRegistered(objectDetector); verifyListenersRegistered(objectDetector);
objectDetector.protoListener!(detectionProtos, 1337); objectDetector.protoListener!([binaryProto], 1337);
}); });
// Invoke the object detector // Invoke the object detector
const detections = objectDetector.detect({} as HTMLImageElement); const {detections} = objectDetector.detect({} as HTMLImageElement);
expect(objectDetector.fakeWasmModule._waitUntilIdle).toHaveBeenCalled(); expect(objectDetector.fakeWasmModule._waitUntilIdle).toHaveBeenCalled();
expect(detections.length).toEqual(2); expect(detections.length).toEqual(1);
expect(detections[0]).toEqual({ expect(detections[0]).toEqual({
categories: [{ categories: [{
score: 0.1, score: 0.1,
index: 1,
categoryName: 'foo',
displayName: 'bar',
}],
boundingBox: {originX: 1, originY: 2, width: 3, height: 4}
});
expect(detections[1]).toEqual({
categories: [{
score: 0.2,
index: -1, index: -1,
categoryName: '', categoryName: '',
displayName: '', displayName: '',

View File

@ -31,6 +31,7 @@ from setuptools.command import install
__version__ = 'dev' __version__ = 'dev'
IS_WINDOWS = (platform.system() == 'Windows') IS_WINDOWS = (platform.system() == 'Windows')
IS_MAC = (platform.system() == 'Darwin')
MP_ROOT_PATH = os.path.dirname(os.path.abspath(__file__)) MP_ROOT_PATH = os.path.dirname(os.path.abspath(__file__))
MP_DIR_INIT_PY = os.path.join(MP_ROOT_PATH, 'mediapipe/__init__.py') MP_DIR_INIT_PY = os.path.join(MP_ROOT_PATH, 'mediapipe/__init__.py')
MP_THIRD_PARTY_BUILD = os.path.join(MP_ROOT_PATH, 'third_party/BUILD') MP_THIRD_PARTY_BUILD = os.path.join(MP_ROOT_PATH, 'third_party/BUILD')
@ -343,11 +344,39 @@ class BuildExtension(build_ext.build_ext):
def run(self): def run(self):
_check_bazel() _check_bazel()
for ext in self.extensions: if IS_MAC:
self._build_binary(ext) for ext in self.extensions:
target_name = self.get_ext_fullpath(ext.name)
# Build x86
self._build_binary(
ext,
['--cpu=darwin_x86_64', '--ios_multi_cpus=i386,x86_64,armv7,arm64'],
)
x86_name = self.get_ext_fullpath(ext.name)
# Build Arm64
ext.name = ext.name + '.arm64'
self._build_binary(
ext,
['--cpu=darwin_arm64', '--ios_multi_cpus=i386,x86_64,armv7,arm64'],
)
arm64_name = self.get_ext_fullpath(ext.name)
# Merge architectures
lipo_command = [
'lipo',
'-create',
'-output',
target_name,
x86_name,
arm64_name,
]
if subprocess.call(lipo_command) != 0:
sys.exit(-1)
else:
for ext in self.extensions:
self._build_binary(ext)
build_ext.build_ext.run(self) build_ext.build_ext.run(self)
def _build_binary(self, ext): def _build_binary(self, ext, extra_args=None):
if not os.path.exists(self.build_temp): if not os.path.exists(self.build_temp):
os.makedirs(self.build_temp) os.makedirs(self.build_temp)
bazel_command = [ bazel_command = [
@ -359,6 +388,8 @@ class BuildExtension(build_ext.build_ext):
'--action_env=PYTHON_BIN_PATH=' + _normalize_path(sys.executable), '--action_env=PYTHON_BIN_PATH=' + _normalize_path(sys.executable),
str(ext.bazel_target + '.so'), str(ext.bazel_target + '.so'),
] ]
if extra_args:
bazel_command += extra_args
if not self.link_opencv and not IS_WINDOWS: if not self.link_opencv and not IS_WINDOWS:
bazel_command.append('--define=OPENCV=source') bazel_command.append('--define=OPENCV=source')
if subprocess.call(bazel_command) != 0: if subprocess.call(bazel_command) != 0:

View File

@ -67,13 +67,7 @@ def external_files():
http_file( http_file(
name = "com_google_mediapipe_BUILD", name = "com_google_mediapipe_BUILD",
sha256 = "d2b2a8346202691d7f831887c84e9642e974f64ed67851d9a58cf15c94b1f6b3", sha256 = "d2b2a8346202691d7f831887c84e9642e974f64ed67851d9a58cf15c94b1f6b3",
urls = ["https://storage.googleapis.com/mediapipe-assets/BUILD?generation=166187566369397616783235763936531678737479599640"], urls = ["https://storage.googleapis.com/mediapipe-assets/BUILD?generation=1661875663693976167832357639365316787374795996401679955080207504"],
)
http_file(
name = "com_google_mediapipe_BUILD_orig",
sha256 = "d86b98b82e00dd87cd46bd1429bf5eaa007b500c1a24d9316b73309f2e6c8df8",
urls = ["https://storage.googleapis.com/mediapipe-assets/BUILD.orig?generation=1679955080207504"],
) )
http_file( http_file(
@ -262,6 +256,12 @@ def external_files():
urls = ["https://storage.googleapis.com/mediapipe-assets/dummy_gesture_recognizer.task?generation=1665707319890725"], urls = ["https://storage.googleapis.com/mediapipe-assets/dummy_gesture_recognizer.task?generation=1665707319890725"],
) )
http_file(
name = "com_google_mediapipe_dynamic_input_classifier_tflite",
sha256 = "fb34b05e1cd4081f3c2bb882092f617efb19266b3353d51b3790a172cae09784",
urls = ["https://storage.googleapis.com/mediapipe-assets/dynamic_input_classifier.tflite?generation=1680543275416843"],
)
http_file( http_file(
name = "com_google_mediapipe_efficientdet_lite0_v1_json", name = "com_google_mediapipe_efficientdet_lite0_v1_json",
sha256 = "7a9e1fb625a6130a251e612637fc546cfc8cfabfadc7dbdade44c87f1d8996ca", sha256 = "7a9e1fb625a6130a251e612637fc546cfc8cfabfadc7dbdade44c87f1d8996ca",
@ -306,8 +306,8 @@ def external_files():
http_file( http_file(
name = "com_google_mediapipe_expected_pose_landmarks_prototxt", name = "com_google_mediapipe_expected_pose_landmarks_prototxt",
sha256 = "75dfd2825fc23f51e3906f3a0a050caa8ae9f502cc358af1e7c9fda7ea89c9a5", sha256 = "0bb27e9d9729c4171419abf7edd746b4234cb91198d663f3a4363248a49dad1a",
urls = ["https://storage.googleapis.com/mediapipe-assets/expected_pose_landmarks.prototxt?generation=1679955083449778"], urls = ["https://storage.googleapis.com/mediapipe-assets/expected_pose_landmarks.prototxt?generation=1680543279295598"],
) )
http_file( http_file(
@ -934,12 +934,24 @@ def external_files():
urls = ["https://storage.googleapis.com/mediapipe-assets/pose_expected_detection.pbtxt?generation=1678737492211540"], urls = ["https://storage.googleapis.com/mediapipe-assets/pose_expected_detection.pbtxt?generation=1678737492211540"],
) )
http_file(
name = "com_google_mediapipe_pose_expected_expanded_rect_pbtxt",
sha256 = "babb2a2d50077f6fa9ee15e30d81abb6e98a920e35acad7542bb3d27b5ce7ffd",
urls = ["https://storage.googleapis.com/mediapipe-assets/pose_expected_expanded_rect.pbtxt?generation=1680543294008098"],
)
http_file( http_file(
name = "com_google_mediapipe_pose_jpg", name = "com_google_mediapipe_pose_jpg",
sha256 = "c8a830ed683c0276d713dd5aeda28f415f10cd6291972084a40d0d8b934ed62b", sha256 = "c8a830ed683c0276d713dd5aeda28f415f10cd6291972084a40d0d8b934ed62b",
urls = ["https://storage.googleapis.com/mediapipe-assets/pose.jpg?generation=1678737494661975"], urls = ["https://storage.googleapis.com/mediapipe-assets/pose.jpg?generation=1678737494661975"],
) )
http_file(
name = "com_google_mediapipe_pose_landmarker_task",
sha256 = "ca4137626f0dc04f87893ccf2ad01949a3b1d4b55fa85ba957dde44a29dd956e",
urls = ["https://storage.googleapis.com/mediapipe-assets/pose_landmarker.task?generation=1680543298177615"],
)
http_file( http_file(
name = "com_google_mediapipe_pose_landmark_full_tflite", name = "com_google_mediapipe_pose_landmark_full_tflite",
sha256 = "e9a5c5cb17f736fafd4c2ec1da3b3d331d6edbe8a0d32395855aeb2cdfd64b9f", sha256 = "e9a5c5cb17f736fafd4c2ec1da3b3d331d6edbe8a0d32395855aeb2cdfd64b9f",
@ -1258,12 +1270,6 @@ def external_files():
urls = ["https://storage.googleapis.com/mediapipe-assets/vocab_with_index.txt?generation=1661875977280658"], urls = ["https://storage.googleapis.com/mediapipe-assets/vocab_with_index.txt?generation=1661875977280658"],
) )
http_file(
name = "com_google_mediapipe_w_avg_npy",
sha256 = "a044e35609986d18a972532f2980e939832b5b7d559659959d11ecc752a58bbe",
urls = ["https://storage.googleapis.com/mediapipe-assets/w_avg.npy?generation=1679955100435717"],
)
http_file( http_file(
name = "com_google_mediapipe_yamnet_audio_classifier_with_metadata_tflite", name = "com_google_mediapipe_yamnet_audio_classifier_with_metadata_tflite",
sha256 = "10c95ea3eb9a7bb4cb8bddf6feb023250381008177ac162ce169694d05c317de", sha256 = "10c95ea3eb9a7bb4cb8bddf6feb023250381008177ac162ce169694d05c317de",
@ -1276,60 +1282,6 @@ def external_files():
urls = ["https://storage.googleapis.com/mediapipe-assets/yamnet_embedding_metadata.tflite?generation=1668295071595506"], urls = ["https://storage.googleapis.com/mediapipe-assets/yamnet_embedding_metadata.tflite?generation=1668295071595506"],
) )
http_file(
name = "com_google_mediapipe_decoder_fingerprint_pb",
sha256 = "0bf6239c4855d78edb60f3349b46cdb2c6f83def64f1b31589b6e298e5cbec3c",
urls = ["https://storage.googleapis.com/mediapipe-assets/decoder/fingerprint.pb?generation=1679955102906559"],
)
http_file(
name = "com_google_mediapipe_decoder_keras_metadata_pb",
sha256 = "1631ee698455aea52d4467fe6118800718a86ec49c29f4f3c904785b72f425ff",
urls = ["https://storage.googleapis.com/mediapipe-assets/decoder/keras_metadata.pb?generation=1679955105294959"],
)
http_file(
name = "com_google_mediapipe_decoder_saved_model_pb",
sha256 = "b424d30c63548e93390b2944b9bd9dc29773a56197bb462d3bd9e7a0bd1270ff",
urls = ["https://storage.googleapis.com/mediapipe-assets/decoder/saved_model.pb?generation=1679955107808916"],
)
http_file(
name = "com_google_mediapipe_discriminator_fingerprint_pb",
sha256 = "1fa6201d253c9218f7054138b9ce273266ce431e00cbce2d74d557f6b97223fd",
urls = ["https://storage.googleapis.com/mediapipe-assets/discriminator/fingerprint.pb?generation=1679955110094297"],
)
http_file(
name = "com_google_mediapipe_discriminator_keras_metadata_pb",
sha256 = "59a8601790d615dd37ec24e788743ce737e9999ce6ea6593fcf1ee43f674987f",
urls = ["https://storage.googleapis.com/mediapipe-assets/discriminator/keras_metadata.pb?generation=1679955112486389"],
)
http_file(
name = "com_google_mediapipe_discriminator_saved_model_pb",
sha256 = "280d6097a9b3d4c3756e028c597fe3d3c76eb14f76f24d49a22ed7b6df1e3878",
urls = ["https://storage.googleapis.com/mediapipe-assets/discriminator/saved_model.pb?generation=1679955114873053"],
)
http_file(
name = "com_google_mediapipe_encoder_fingerprint_pb",
sha256 = "06cb4319f8178edf447a7a2442e89303a14a48cc4fc5ae27354eac2ba11ae120",
urls = ["https://storage.googleapis.com/mediapipe-assets/encoder/fingerprint.pb?generation=1679955117213209"],
)
http_file(
name = "com_google_mediapipe_encoder_keras_metadata_pb",
sha256 = "8b1429ee95c130fad0c78077b2b544cd03c9e288658aae93e81df4959b84009e",
urls = ["https://storage.googleapis.com/mediapipe-assets/encoder/keras_metadata.pb?generation=1679955119546778"],
)
http_file(
name = "com_google_mediapipe_encoder_saved_model_pb",
sha256 = "ce48392c71485ecd9b142b46e54442581a299df5560102337038b76a62e02a09",
urls = ["https://storage.googleapis.com/mediapipe-assets/encoder/saved_model.pb?generation=1679955122069362"],
)
http_file( http_file(
name = "com_google_mediapipe_gesture_embedder_keras_metadata_pb", name = "com_google_mediapipe_gesture_embedder_keras_metadata_pb",
sha256 = "c76b856101e2284293a5e5963b7c445e407a0b3e56ec63eb78f64d883e51e3aa", sha256 = "c76b856101e2284293a5e5963b7c445e407a0b3e56ec63eb78f64d883e51e3aa",
@ -1342,24 +1294,6 @@ def external_files():
urls = ["https://storage.googleapis.com/mediapipe-assets/gesture_embedder/saved_model.pb?generation=1668550484904822"], urls = ["https://storage.googleapis.com/mediapipe-assets/gesture_embedder/saved_model.pb?generation=1668550484904822"],
) )
http_file(
name = "com_google_mediapipe_mapping_fingerprint_pb",
sha256 = "6320890f1b9a57e5f4e50e3b56d96fd39d815aa2de51dd1c9b635aa6107d982b",
urls = ["https://storage.googleapis.com/mediapipe-assets/mapping/fingerprint.pb?generation=1679955124430234"],
)
http_file(
name = "com_google_mediapipe_mapping_keras_metadata_pb",
sha256 = "22582a2ec1d4883b52f50e628c1a2d69a2610b38d72a48a0bd9939c26be304f6",
urls = ["https://storage.googleapis.com/mediapipe-assets/mapping/keras_metadata.pb?generation=1679955126858694"],
)
http_file(
name = "com_google_mediapipe_mapping_saved_model_pb",
sha256 = "6a79de45d00f49110304bf0a6746bc717c45f77824cad22690f700f2fbdc1470",
urls = ["https://storage.googleapis.com/mediapipe-assets/mapping/saved_model.pb?generation=1679955129259768"],
)
http_file( http_file(
name = "com_google_mediapipe_mobilebert_tiny_keras_metadata_pb", name = "com_google_mediapipe_mobilebert_tiny_keras_metadata_pb",
sha256 = "cef8131a414c602b9d4742ac57f4f90bc5d8a42baec36b65deece884e2d0cf0f", sha256 = "cef8131a414c602b9d4742ac57f4f90bc5d8a42baec36b65deece884e2d0cf0f",
@ -1408,42 +1342,6 @@ def external_files():
urls = ["https://storage.googleapis.com/mediapipe-assets/object_detection_saved_model/saved_model.pb?generation=1661875999264354"], urls = ["https://storage.googleapis.com/mediapipe-assets/object_detection_saved_model/saved_model.pb?generation=1661875999264354"],
) )
http_file(
name = "com_google_mediapipe_decoder_variables_variables_data-00000-of-00001",
sha256 = "d720ddf354036f17fa210951f9ebfb009453b244913a493f494f1441cfc2eca3",
urls = ["https://storage.googleapis.com/mediapipe-assets/decoder/variables/variables.data-00000-of-00001?generation=1679955132326947"],
)
http_file(
name = "com_google_mediapipe_decoder_variables_variables_index",
sha256 = "245f69af6e53fb8b163059fe9936f57b68a7844e15d696393fcddf94c771dfcc",
urls = ["https://storage.googleapis.com/mediapipe-assets/decoder/variables/variables.index?generation=1679955134518344"],
)
http_file(
name = "com_google_mediapipe_discriminator_variables_variables_data-00000-of-00001",
sha256 = "50b00e1898a573588fb0d5d24d74346d99b7153b5d79441d0350c2c6ca89fb02",
urls = ["https://storage.googleapis.com/mediapipe-assets/discriminator/variables/variables.data-00000-of-00001?generation=1679955138489595"],
)
http_file(
name = "com_google_mediapipe_discriminator_variables_variables_index",
sha256 = "e5cb4be6442a5741504ce7da9487445637ad89b1f4b6a993bb9e762c7bd5621d",
urls = ["https://storage.googleapis.com/mediapipe-assets/discriminator/variables/variables.index?generation=1679955140891136"],
)
http_file(
name = "com_google_mediapipe_encoder_variables_variables_data-00000-of-00001",
sha256 = "09bcd1e2f1c6261bd1842af2da95651d54c9b4b9343eb9b8f0004a97f9bc84bf",
urls = ["https://storage.googleapis.com/mediapipe-assets/encoder/variables/variables.data-00000-of-00001?generation=1679955144875765"],
)
http_file(
name = "com_google_mediapipe_encoder_variables_variables_index",
sha256 = "964f5ac6ced7b19f76b7856d9dad47594a5b2fa89c52840f82996b809372aec9",
urls = ["https://storage.googleapis.com/mediapipe-assets/encoder/variables/variables.index?generation=1679955147123313"],
)
http_file( http_file(
name = "com_google_mediapipe_gesture_embedder_variables_variables_data-00000-of-00001", name = "com_google_mediapipe_gesture_embedder_variables_variables_data-00000-of-00001",
sha256 = "c156c9654c9ffb1091bb9f06c71080bd1e428586276d3f39c33fbab27fe0522d", sha256 = "c156c9654c9ffb1091bb9f06c71080bd1e428586276d3f39c33fbab27fe0522d",
@ -1456,18 +1354,6 @@ def external_files():
urls = ["https://storage.googleapis.com/mediapipe-assets/gesture_embedder/variables/variables.index?generation=1668550490691823"], urls = ["https://storage.googleapis.com/mediapipe-assets/gesture_embedder/variables/variables.index?generation=1668550490691823"],
) )
http_file(
name = "com_google_mediapipe_mapping_variables_variables_data-00000-of-00001",
sha256 = "4187055e7f69fcc913ee2b11151a56149dda3017c75621d1e160596bde874c07",
urls = ["https://storage.googleapis.com/mediapipe-assets/mapping/variables/variables.data-00000-of-00001?generation=1679955149680908"],
)
http_file(
name = "com_google_mediapipe_mapping_variables_variables_index",
sha256 = "a04fcae7083715613f93ac89943f5fe1f5ba2e6efb9efd14eee7314f25502e4a",
urls = ["https://storage.googleapis.com/mediapipe-assets/mapping/variables/variables.index?generation=1679955152034297"],
)
http_file( http_file(
name = "com_google_mediapipe_mobilebert_tiny_assets_vocab_txt", name = "com_google_mediapipe_mobilebert_tiny_assets_vocab_txt",
sha256 = "07eced375cec144d27c900241f3e339478dec958f92fddbc551f295c992038a3", sha256 = "07eced375cec144d27c900241f3e339478dec958f92fddbc551f295c992038a3",