cassdlcm/mediapipe
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge 78814a902d into ae05ad04b3

Browse Source
This commit is contained in:
DJ 2021-05-11 11:53:58 -05:00 committed by GitHub
commit 28460e4099
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 172 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
docs/getting_started/javascript.md
View File

@ -24,6 +24,8 @@ Solution | NPM Package | Example
[Holistic][Ho-pg] | [@mediapipe/holistic][Ho-npm] | [mediapipe.dev/demo/holistic][Ho-demo]
[Pose][P-pg] | [@mediapipe/pose][P-npm] | [mediapipe.dev/demo/pose][P-demo]
Note: Also see the [Holistic Remote demo][Ho-Re], which explores more options in the Holistic Demo. <!--This doesn't have a CodePen or NPM repository yet.-->
Click on a solution link above for more information, including API and code
snippets.
@ -62,11 +64,10 @@ structured as `<major>.<minor>.<build>`. To prevent breaking changes from
affecting your work, restrict your request to a `<minor>` number. e.g.,
`@mediapipe/holistic@0.1`.
[Ho-pg]: ../solutions/holistic#javascript-solution-api
[F-pg]: ../solutions/face_mesh#javascript-solution-api
[Fd-pg]: ../solutions/face_detection#javascript-solution-api
[H-pg]: ../solutions/hands#javascript-solution-api
[P-pg]: ../solutions/pose#javascript-solution-api
[Ho-pg]: ../solutions/holistic.md#javascript-solution-api
[F-pg]: ../solutions/face_mesh.md#javascript-solution-api
[H-pg]: ../solutions/hands.md#javascript-solution-api
[P-pg]: ../solutions/pose.md#javascript-solution-api
[Ho-npm]: https://www.npmjs.com/package/@mediapipe/holistic
[F-npm]: https://www.npmjs.com/package/@mediapipe/face_mesh
[Fd-npm]: https://www.npmjs.com/package/@mediapipe/face_detection
@ -86,6 +87,7 @@ affecting your work, restrict your request to a `<minor>` number. e.g.,
[H-pen]: https://code.mediapipe.dev/codepen/hands
[P-pen]: https://code.mediapipe.dev/codepen/pose
[Ho-demo]: https://mediapipe.dev/demo/holistic
[Ho-Re]: https://mediapipe.dev/demo/holistic_remote
[F-demo]: https://mediapipe.dev/demo/face_mesh
[Fd-demo]: https://mediapipe.dev/demo/face_detection
[H-demo]: https://mediapipe.dev/demo/hands

163
docs/getting_started/js_utils.md Normal file
View File

@ -0,0 +1,163 @@
---
layout: default
title: JavaScript Utilities
parent: MediaPipe in JavaScript
nav_order: 1
---
# JavaScript Utils for MediaPipe
{: .no_toc }
1. TOC
{:toc}
---
## Utilities
MediaPipe offers utilities for enabling users to use the MediaPipe Examples with ease, and conveniently integrate with existing code.
The utilities MediaPipe currently has are ~
* [Drawing Utilities][draw-npm],
* [Camera Utilities][cam-npm], and
* [Control Panel][ctrl-npm] Utilities.
### Drawing Utilities
These are for drawing on canvas using output from MediaPipe's solutions.
Currently, this includes three functions ~
1. *drawLandmarks*
```js
drawLandmarks(canvasContext, landmarks, {
fillColor: fillColor,
color: color,
lineWidth: lineWidth
}
```
This function is for drawing landmarks.
The value for the `landmarks` parameter can be found using in the `results` array.
2. *drawConnectors*
```js
drawConnectors(canvasContext, landmarks, LANDMARK_CONSTANTS, {
color: color,
lineWidth: lineWidth
})
```
This is a function used for plotting connectors in various MediaPipe Solutions.
The `LANDMARK_CONSTANTS` are specific for every solution, so you could choose these there.
3. *drawRectangle*
```js
drawRectangle(canvasContext, {
width: width,
height: height,
xCenter: xCenter,
yCenter: yCenter,
rotation: rotation
}, {
color: color
})
```
This allows drawing of a rectangle, with a certain width, height and rotation given the coordinates of its location.
### Camera Utilities
This module has a `Camera` object that can be initialized and used easily, like below.
```js
// Our input frames will come from here.
const videoElement =
document.getElementsByClassName('input_video')[0];
const camera = new Camera(videoElement, {
onFrame: async () => {
// Send it to a demo, for example, hands
await hands.send({image: videoElement});
},
// Set the width of camera to be rendered
width: 1280,
// And set the height
height: 720
});
// Start the camera.
// Note: It will ask the user for permission, and will error out if this is not given.
camera.start();
```
### Control Panel Utilities
To showcase and monitor the model in elegant ways, MediaPipe has this Control Panel module.
This panel is also convenient to use and set up.
One of the most interesting features is the built-in `FPS` object that allows us to monitor the FPS or Frames per Second.
> Note that the event handler function mentioned in the code can be [referred to here][H-code]
```js
// Initialize the FPS control
const fpsControl = new FPS();
// In the event handler function onResults()
function onResults(results) {
// Tick the FPS, i.e., set Incrementation breaks
fpsControl.tick();
// then do the required processing on the results object
// Pass in a <div>
new ControlPanel(controlsElement, {
// Whether to invert the camera, defaults to false
selfieMode: true,
// Maximum Number of Hands to detect, a placeholder, only when using Hands API
maxNumHands: 2,
// Minimum Confidence score
minDetectionConfidence: 0.5,
// Minimum Tracking score
minTrackingConfidence: 0.5
})
.add([
// a StaticText is simply a label
new StaticText({title: 'MediaPipe Hands'}),
// Add the FPS control to the Control Panel
fpsControl,
// a Toggle is one with only 2 options (i.e., true or false)
new Toggle({title: 'Selfie Mode', field: 'selfieMode'}),
// A slider can have multiple options. This will have the options 1,2,3 and 4.
// Range determines maximum and minimum allowed value
// Step determines the difference between two options
new Slider(
{title: 'Max Number of Hands', field: 'maxNumHands', range: [1, 4], step: 1}),
new Slider({
title: 'Min Detection Confidence',
field: 'minDetectionConfidence',
range: [0, 1],
step: 0.01
}),
new Slider({
title: 'Min Tracking Confidence',
field: 'minTrackingConfidence',
range: [0, 1],
step: 0.01
}),
])
// This is run when an option is updated.
// the options object will contain parameters for all the above in a array.
.on(options => {
videoElement.classList.toggle('selfie', options.selfieMode);
hands.setOptions(options);
});
```
[H-code]: ../solutions/hands.md#code
[Ho-pg]: ../solutions/holistic.md#javascript-solution-api
[F-pg]: ../solutions/face_mesh.md#javascript-solution-api
[H-pg]: ../solutions/hands.md#javascript-solution-api
[P-pg]: ../solutions/pose.md#javascript-solution-api
[Ho-npm]: https://www.npmjs.com/package/@mediapipe/holistic
[F-npm]: https://www.npmjs.com/package/@mediapipe/face_mesh
[H-npm]: https://www.npmjs.com/package/@mediapipe/hands
[P-npm]: https://www.npmjs.com/package/@mediapipe/pose
[draw-npm]: https://www.npmjs.com/package/@mediapipe/pose
[cam-npm]: https://www.npmjs.com/package/@mediapipe/pose
[ctrl-npm]: https://www.npmjs.com/package/@mediapipe/pose
[Ho-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/holistic
[F-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/face_mesh
[H-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/hands
[P-jsd]: https://www.jsdelivr.com/package/npm/@mediapipe/pose
[Ho-pen]: https://code.mediapipe.dev/codepen/holistic
[F-pen]: https://code.mediapipe.dev/codepen/face_mesh
[H-pen]: https://code.mediapipe.dev/codepen/hands
[P-pen]: https://code.mediapipe.dev/codepen/pose
[Ho-demo]: https://mediapipe.dev/demo/holistic
[F-demo]: https://mediapipe.dev/demo/face_mesh
[H-demo]: https://mediapipe.dev/demo/hands
[P-demo]: https://mediapipe.dev/demo/pose
[npm]: https://www.npmjs.com/package/@mediapipe
[codepen]: https://code.mediapipe.dev/codepen

2
docs/solutions/hands.md
View File

@ -316,6 +316,8 @@ Supported configuration options:
</html>
```
<br id=code></br>
```javascript
<script type="module">
const videoElement = document.getElementsByClassName('input_video')[0];