mediapipe

Cross-platform, customizable ML solutions for live and streaming media.

android audio-processing c-plus-plus calculator computer-vision deep-learning framework graph-based graph-framework inference machine-learning mediapipe mobile-development perception pipeline-framework stream-processing video-processing

Go to file

MediaPipe Team a641ea12e1 Update gesture recognizer to new mediapipe tasks pipeline PiperOrigin-RevId: 493950564		2022-12-08 11:32:42 -08:00
.github	Tweaked the issue templates.	2022-12-06 15:14:23 -08:00
docs	Internal change	2022-12-06 19:48:05 -08:00
mediapipe	Update gesture recognizer to new mediapipe tasks pipeline	2022-12-08 11:32:42 -08:00
third_party	Create Build Rules for Apple Frameworks	2022-12-06 15:18:45 -08:00
.bazelrc	Project import generated by Copybara.	2022-05-05 19:57:20 +00:00
.bazelversion	Project import generated by Copybara.	2022-06-28 12:11:05 +00:00
.dockerignore	Project import generated by Copybara.	2019-06-16 16:06:57 -07:00
.gitignore	Set up the Open Source build for MediaPipe Tasks Web	2022-11-04 10:59:32 -07:00
build_android_examples.sh	Project import generated by Copybara.	2022-03-21 12:12:39 -07:00
build_desktop_examples.sh	Project import generated by Copybara.	2022-03-21 12:12:39 -07:00
build_ios_examples.sh	Project import generated by Copybara.	2022-03-21 12:12:39 -07:00
BUILD.bazel	Set up the Open Source build for MediaPipe Tasks Web	2022-11-04 10:59:32 -07:00
CONTRIBUTING.md	Project import generated by Copybara.	2021-12-13 15:56:02 -08:00
Dockerfile	Install node and npm on Docker as a prerequisite to open sourcing the Web files	2022-11-02 16:45:52 -07:00
LICENSE	Project import generated by Copybara.	2019-06-16 16:06:57 -07:00
MANIFEST.in	Project import generated by Copybara.	2022-09-06 21:46:17 +00:00
package.json	Open Source the first set of MediaPipe Tasks tests for Web	2022-12-07 12:15:34 -08:00
README.md	Project import generated by Copybara.	2022-09-06 21:46:17 +00:00
requirements.txt	Enable using tasks metadata components in MediaPipe PyPI packages.	2022-11-04 09:49:13 -07:00
setup_android_sdk_and_ndk.sh	Project import generated by Copybara.	2022-06-28 12:11:05 +00:00
setup_opencv.sh	Project import generated by Copybara.	2021-09-01 18:15:31 -07:00
setup.py	Copy metatdata_schema.fbs into PyPI packages.	2022-11-10 17:48:48 -08:00
tsconfig.json	Open Source the first set of MediaPipe Tasks tests for Web	2022-12-07 12:15:34 -08:00
WORKSPACE	Update Bazel dependencies for Apple	2022-12-07 15:25:56 -08:00
yarn.lock	Open Source the first set of MediaPipe Tasks tests for Web	2022-12-07 12:15:34 -08:00

README.md

layout	title	nav_order
default	Home	1

Live ML anywhere

MediaPipe offers cross-platform, customizable ML solutions for live and streaming media.


End-to-End acceleration: Built-in fast ML inference and processing accelerated even on common hardware	Build once, deploy anywhere: Unified solution works across Android, iOS, desktop/cloud, web and IoT

Ready-to-use solutions: Cutting-edge ML solutions demonstrating full power of the framework	Free and open source: Framework and solutions both under Apache 2.0, fully extensible and customizable

ML solutions in MediaPipe

Face Detection	Face Mesh	Iris	Hands	Pose	Holistic

Hair Segmentation	Object Detection	Box Tracking	Instant Motion Tracking	Objectron	KNIFT

	Android	iOS	C++	Python	JS	Coral
Face Detection	✅	✅	✅	✅	✅	✅
Face Mesh	✅	✅	✅	✅	✅
Iris	✅	✅	✅
Hands	✅	✅	✅	✅	✅
Pose	✅	✅	✅	✅	✅
Holistic	✅	✅	✅	✅	✅
Selfie Segmentation	✅	✅	✅	✅	✅
Hair Segmentation	✅		✅
Object Detection	✅	✅	✅			✅
Box Tracking	✅	✅	✅
Instant Motion Tracking	✅
Objectron	✅		✅	✅	✅
KNIFT	✅
AutoFlip			✅
MediaSequence			✅
YouTube 8M			✅

See also MediaPipe Models and Model Cards for ML models released in MediaPipe.

Getting started

To start using MediaPipe solutions with only a few lines code, see example code and demos in MediaPipe in Python and MediaPipe in JavaScript.

To use MediaPipe in C++, Android and iOS, which allow further customization of the solutions as well as building your own, learn how to install MediaPipe and start building example applications in C++, Android and iOS.

The source code is hosted in the MediaPipe Github repository, and you can run code search using Google Open Source Code Search.

Publications

Bringing artworks to life with AR in Google Developers Blog
Prosthesis control via Mirru App using MediaPipe hand tracking in Google Developers Blog
SignAll SDK: Sign language interface using MediaPipe is now available for developers in Google Developers Blog
MediaPipe Holistic - Simultaneous Face, Hand and Pose Prediction, on Device in Google AI Blog
Background Features in Google Meet, Powered by Web ML in Google AI Blog
MediaPipe 3D Face Transform in Google Developers Blog
Instant Motion Tracking With MediaPipe in Google Developers Blog
BlazePose - On-device Real-time Body Pose Tracking in Google AI Blog
MediaPipe Iris: Real-time Eye Tracking and Depth Estimation in Google AI Blog
MediaPipe KNIFT: Template-based feature matching in Google Developers Blog
Alfred Camera: Smart camera features using MediaPipe in Google Developers Blog
Real-Time 3D Object Detection on Mobile Devices with MediaPipe in Google AI Blog
AutoFlip: An Open Source Framework for Intelligent Video Reframing in Google AI Blog
MediaPipe on the Web in Google Developers Blog
Object Detection and Tracking using MediaPipe in Google Developers Blog
On-Device, Real-Time Hand Tracking with MediaPipe in Google AI Blog
MediaPipe: A Framework for Building Perception Pipelines

Videos

YouTube Channel

Events

Community

Awesome MediaPipe - A curated list of awesome MediaPipe related frameworks, libraries and software
Slack community for MediaPipe users
Discuss - General community discussion around MediaPipe

Alpha disclaimer

MediaPipe is currently in alpha at v0.7. We may be still making breaking API changes and expect to get to stable APIs by v1.0.

Contributing

We welcome contributions. Please follow these guidelines.

We use GitHub issues for tracking requests and bugs. Please post questions to the MediaPipe Stack Overflow with a mediapipe tag.