cassdlcm/mediapipe
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
930 Commits 10 Branches 56 Tags 829 MiB
5687d19dec
Commit Graph

885 Commits

Author SHA1 Message Date
Jiuqiang Tang
955f090f9f Retire the visibility group "//mediapipe/framework:mediapipe_internal". 
PiperOrigin-RevId: 493687025
 2022-12-07 13:08:33 -08:00
Sebastian Schmidt
2811e0c5c8 Open Source the first set of MediaPipe Tasks tests for Web 
PiperOrigin-RevId: 493673279
 2022-12-07 12:15:34 -08:00
Jiuqiang Tang
3c0ddf16b4 Fix an incorrect model sanity check in the object detector graph. 
PiperOrigin-RevId: 493663592
 2022-12-07 11:38:54 -08:00
Jiuqiang Tang
80c605459c Open up framework visibility. 
PiperOrigin-RevId: 493660013
 2022-12-07 11:26:16 -08:00
Sebastian Schmidt
d84eec387b Add missing import to InferenceCalculator.proto 
PiperOrigin-RevId: 493649869
 2022-12-07 10:52:17 -08:00
Khanh LeViet
d9688b769f Hide internal APIs from mediapipe pip package's API docs. 
PiperOrigin-RevId: 493607984
 2022-12-07 08:15:25 -08:00
Nikolay Chirkov
aad797197b TensorV1 EGL.h include fix. 
PiperOrigin-RevId: 493596083
 2022-12-07 07:16:59 -08:00
Camillo Lugaresi
523d16dffa Make GpuBuffer a shared_ptr to a storage collection 
PiperOrigin-RevId: 493519590
 2022-12-06 23:56:19 -08:00
Sebastian Schmidt
1167f61f98 Remove generic Options template argument from TaskRunner 
PiperOrigin-RevId: 493462947
 2022-12-06 18:04:31 -08:00
MediaPipe Team
576c6da173 Internal change 
PiperOrigin-RevId: 493427500
 2022-12-06 15:24:32 -08:00
Sebastian Schmidt
fca0f5806b Create Build Rules for Apple Frameworks 
PiperOrigin-RevId: 493426040
 2022-12-06 15:18:45 -08:00
Sebastian Schmidt
0f32072804 Move ISSUE_TEMPLATAE files to .github folder 
PiperOrigin-RevId: 493405734
 2022-12-06 14:03:26 -08:00
Khanh LeViet
cdc14522e2 Added issue templates for MP Preview. 
PiperOrigin-RevId: 493389856
 2022-12-06 13:03:10 -08:00
Jiuqiang Tang
6deef1a5f1 Allow specifying tag_suffix in the templated CreateModelResources method. 
PiperOrigin-RevId: 493375701
 2022-12-06 12:10:06 -08:00
Jiuqiang Tang
c6e6f9e0b9 Fix aar breakage caused by missing "//mediapipe/tasks/cc/vision/image_segmenter/proto:segmenter_options_java_proto_lite". 
PiperOrigin-RevId: 493357585
 2022-12-06 11:08:01 -08:00
Sebastian Schmidt
ab0b0ab558 Change visibility for MP Tasks Web to public 
PiperOrigin-RevId: 493343996
 2022-12-06 10:26:16 -08:00
Jiuqiang Tang
fb0b96115f Open up mediapipe core calculators' visibility. 
PiperOrigin-RevId: 493314353
 2022-12-06 08:40:57 -08:00
Camillo Lugaresi
b0b38a0013 Internal change 
PiperOrigin-RevId: 493313240
 2022-12-06 08:31:46 -08:00
Jiuqiang Tang
1761cdcef4 Fix aar breakage caused by missing "//mediapipe/tasks/java/com/google/mediapipe/tasks/components/containers:normalized_landmark". 
PiperOrigin-RevId: 493204770
 2022-12-05 22:16:46 -08:00
MediaPipe Team
af43687f2e Open-sources a unit test. 
PiperOrigin-RevId: 493184055
 2022-12-05 20:11:07 -08:00
MediaPipe Team
3174b20fbe Move segmentation calculator and options out of 'components' folder. 
PiperOrigin-RevId: 493157929
 2022-12-05 17:37:49 -08:00
Jiuqiang Tang
1e76d47a71 Checks if a custom global resource provider is used as the first step of loading the model resources on all platforms. 
PiperOrigin-RevId: 493141519
 2022-12-05 16:20:20 -08:00
MediaPipe Team
3ad03bee0b Adds missing visibility rule. 
PiperOrigin-RevId: 493118880
 2022-12-05 14:50:35 -08:00
MediaPipe Team
69b27b246a Adds a public function for creating TaskRunner instances. 
PiperOrigin-RevId: 493109736
 2022-12-05 14:18:17 -08:00
Adam Cozzette
4f8eaee20f Internal change 
PiperOrigin-RevId: 493065632
 2022-12-05 11:35:33 -08:00
MediaPipe Team
35bb18945f Better handling of empty packets in vector calculators. 
PiperOrigin-RevId: 493000695
 2022-12-05 07:25:14 -08:00
Sebastian Schmidt
e457039fc6 Don't inherit from GraphRunner 
PiperOrigin-RevId: 492584486
 2022-12-02 16:20:19 -08:00
Sebastian Schmidt
da9587033d Move shared code to TaskRunner 
PiperOrigin-RevId: 492534879
 2022-12-02 12:42:45 -08:00
Sebastian Schmidt
dabc2af15b Fix base bath loading in Fileset resolver 
PiperOrigin-RevId: 492526041
 2022-12-02 12:06:48 -08:00
MediaPipe Team
768d2dc548 Separate web and java api landmark and world landmark to two classes. This makes the platforms interface consistent. 
PiperOrigin-RevId: 492332990
 2022-12-01 16:49:47 -08:00
Jiuqiang Tang
ead41132a8 Load model file content from model file path with the help of GetResourceContents in browsers. 
This can handle the model files that are provided via a custom ResourceProviderFn.

PiperOrigin-RevId: 492308453
 2022-12-01 15:03:57 -08:00
Jiuqiang Tang
af990c3da1 Open up the visibility of "//mediapipe/java/com/google/mediapipe/framework/image:image". 
PiperOrigin-RevId: 492308109
 2022-12-01 15:00:25 -08:00
Khanh LeViet
fd79f18aeb Make BaseOptions to pass absolute path to C++ layer. 
PiperOrigin-RevId: 492296573
 2022-12-01 14:14:51 -08:00
Camillo Lugaresi
40eb0e6385 Internal change 
PiperOrigin-RevId: 492276913
 2022-12-01 12:59:41 -08:00
Hadon Nash
1e2cb2b359 Internal change 
PiperOrigin-RevId: 492253867
 2022-12-01 11:34:55 -08:00
MediaPipe Team
02aa162c9e Rename gesture_recognizer test_data to testdata to be consistent with rest of model_maker 
PiperOrigin-RevId: 492246728
 2022-12-01 11:13:19 -08:00
Sebastian Schmidt
e685ac9344 Re-use classifier options for ObjectDetector 
PiperOrigin-RevId: 492206856
 2022-12-01 08:48:09 -08:00
Jiuqiang Tang
3ee37800e2 Depending on "inference_calculator_cpu" when the mediapipe tasks can only support cpu inference. 
PiperOrigin-RevId: 492205954
 2022-12-01 08:43:30 -08:00
Jiuqiang Tang
e7eee27c1c Remove the deleted library "mediapipe/tasks/java/com/google/mediapipe/tasks/components/processors:embedderoptions" from mediapipe_tasks_aar's android_library deps list. 
PiperOrigin-RevId: 492200061
 2022-12-01 08:17:12 -08:00
Sebastian Schmidt
a430939fe4 Document RunningMode 
PiperOrigin-RevId: 492193299
 2022-12-01 07:45:12 -08:00
MediaPipe Team
01010fa248 Internal change 
PiperOrigin-RevId: 492188196
 2022-12-01 07:17:19 -08:00
MediaPipe Team
29c7702984 Inline formerly nested 'ClassifierOptions' in Java classifier APIs. 
PiperOrigin-RevId: 492173060
 2022-12-01 05:52:15 -08:00
Jiuqiang Tang
460aee7933 Make mediapipe_tasks_aar's android_library depend on "//third_party:androidx_annotation". 
PiperOrigin-RevId: 492092487
 2022-11-30 20:45:02 -08:00
Camillo Lugaresi
fcd2d2c5af Internal change 
PiperOrigin-RevId: 491733850
 2022-11-29 14:14:37 -08:00
Michael Hays
88173948ee Internal change 
PiperOrigin-RevId: 491724816
 2022-11-29 13:39:35 -08:00
Sebastian Schmidt
09740130e8 Use naturalWidth and naturalHeight for image data 
PiperOrigin-RevId: 491694147
 2022-11-29 11:35:03 -08:00
Jiuqiang Tang
c8a413bb4e Open up mediapipe framework's visibility. 
PiperOrigin-RevId: 491672877
 2022-11-29 10:19:44 -08:00
Camillo Lugaresi
cc11b45228 Remove unneeded GPU_SHARED side packet for GlSurfaceSink 
PiperOrigin-RevId: 491487092
 2022-11-28 17:55:30 -08:00
Camillo Lugaresi
fc526374ab Use GpuResources in GpuTestBase and update GpuBufferMultiPoolTest 
PiperOrigin-RevId: 491486495
 2022-11-28 17:51:03 -08:00
MediaPipe Team
e987b69f39 Add alternative method to determine unique kernel cache path 
PiperOrigin-RevId: 491476293
 2022-11-28 16:52:16 -08:00
MediaPipe Team
7b74fd53f5 Verify that kernel cache is only used when OpenCL is active 
PiperOrigin-RevId: 491463306
 2022-11-28 15:47:55 -08:00
MediaPipe Team
26a7ca5c64 fix typo and minor formatting issues 
PiperOrigin-RevId: 491453662
 2022-11-28 15:05:28 -08:00
MediaPipe Team
b65c40b302 Internal change 
PiperOrigin-RevId: 491441446
 2022-11-28 14:17:23 -08:00
Sebastian Schmidt
342f95fa20 Typo fix 
PiperOrigin-RevId: 491434987
 2022-11-28 13:54:09 -08:00
MediaPipe Team
c48ca1f674 internal change 
PiperOrigin-RevId: 491429214
 2022-11-28 13:33:57 -08:00
Sebastian Schmidt
153edc59a1 Add support for browsers without SIMD 
PiperOrigin-RevId: 491371277
 2022-11-28 09:54:22 -08:00
MediaPipe Team
395d9d8ea2 Instantiate GetDetectionVectorItemCalculator variant of GetVectorItemCalculator<>. 
PiperOrigin-RevId: 491123314
 2022-11-27 00:07:10 -08:00
Camillo Lugaresi
0bdb48ceb1 Use kUtilityFramebuffer in GlCalculatorHelper 
All calculators using the same context can share a single framebuffer object.

PiperOrigin-RevId: 490605074
 2022-11-23 16:19:31 -08:00
MediaPipe Team
41a7f9d7d6 Internal change 
PiperOrigin-RevId: 490595529
 2022-11-23 15:24:50 -08:00
MediaPipe Team
bfa57310c4 Move TextPreprocessing to "processors" folder. 
PiperOrigin-RevId: 490532670
 2022-11-23 10:19:45 -08:00
Camillo Lugaresi
54d1744c8f Remove DrishtiGraphGPUData, add MetalSharedResources 
This class is unused except by the Metal helper; let's narrow it down and simplify gpu_shared_data.

PiperOrigin-RevId: 490531767
 2022-11-23 10:15:17 -08:00
Camillo Lugaresi
3c53ec2cdb Do not expose DrishtiGraphGPUData.h in public header 
This class is an implementation detail.

PiperOrigin-RevId: 490530823
 2022-11-23 10:11:32 -08:00
MediaPipe Team
b5189758f7 Move ImagePreprocessing to "processors" folder. 
PiperOrigin-RevId: 490444821
 2022-11-23 02:08:37 -08:00
Sebastian Schmidt
c5ce523697 Add cosine APIs to Embedder tasks 
PiperOrigin-RevId: 490444597
 2022-11-23 02:04:44 -08:00
Sebastian Schmidt
05681fc0e1 Internal 
PiperOrigin-RevId: 490439195
 2022-11-23 01:29:37 -08:00
Camillo Lugaresi
a55839de51 This storage only needs a "done writing" callback on simulator, so only set it there 
- When not on simulator, we pass nullptr instead of a do-nothing callback.
- The callback is no longer a method, but a function. Only the CVPixelBuffer is captured.

PiperOrigin-RevId: 490380248
 2022-11-22 18:09:09 -08:00
Sebastian Schmidt
3bbc0e9af9 Internal change 
PiperOrigin-RevId: 490375672
 2022-11-22 17:39:54 -08:00
MediaPipe Team
837225c53d Internal change 
PiperOrigin-RevId: 490374976
 2022-11-22 17:36:15 -08:00
Camillo Lugaresi
8ba9d87e66 Update ImageFrameToGpuBufferCalculator to use api2 and GpuBuffer conversions 
PiperOrigin-RevId: 490374387
 2022-11-22 17:32:33 -08:00
MediaPipe Team
fac97554df Small TS audio API improvement 
PiperOrigin-RevId: 490374083
 2022-11-22 17:27:47 -08:00
Camillo Lugaresi
efa9e737f8 Use current context if available in ConvertToImageFrame 
If we're already running in a GlContext, there's no need to go back to the producer context, which may be different.

PiperOrigin-RevId: 490373829
 2022-11-22 17:24:03 -08:00
MediaPipe Team
7ce4aa6592 Internal change 
PiperOrigin-RevId: 490366250
 2022-11-22 16:40:38 -08:00
Camillo Lugaresi
515d00fc22 Internal change 
PiperOrigin-RevId: 490349260
 2022-11-22 15:18:21 -08:00
vrabaud
872d1afda7 Internal change 
PiperOrigin-RevId: 490196129
 2022-11-22 03:12:38 -08:00
Camillo Lugaresi
d648926155 Just reset the fb binding to 0 in ReadTexture 
This saves a get operation. We already have precedent in lots of other MediaPipe code where we just reset bindings to 0.

PiperOrigin-RevId: 490170691
 2022-11-22 00:36:54 -08:00
Camillo Lugaresi
bacbac8d92 Use kUtilityFramebuffer in ReadTexture 
This avoids creating a temporary framebuffer each time.

PiperOrigin-RevId: 490163892
 2022-11-21 23:59:44 -08:00
Camillo Lugaresi
a8b7761022 Define a kUtilityFramebuffer context attachment 
A framebuffer object is often needed to render to a texture or read data from it. Currently we create one in each GlCalculatorHelper, but that is redundant (we only need one per context, and multiple calculators can share the same context). Other times, the code that needs to use this doesn't own a helper. For both reasons, this should be attached to the context.

We could just make this a member of GlContext since it's so common. However, I figured we might as well use the attachment system.

PiperOrigin-RevId: 490160214
 2022-11-21 23:29:27 -08:00
Camillo Lugaresi
54a684717f Internal change 
PiperOrigin-RevId: 490159674
 2022-11-21 23:24:16 -08:00
MediaPipe Team
d43d0ff615 Internal change 
PiperOrigin-RevId: 490089940
 2022-11-21 15:47:45 -08:00
MediaPipe Team
adddf2c2ab Extracted common test helper functions out from the unittest into a sharable 
library. Also migrated away from OpenCVX.

PiperOrigin-RevId: 490074410
 2022-11-21 14:39:56 -08:00
MediaPipe Team
652423a23d Internal change 
PiperOrigin-RevId: 490053179
 2022-11-21 13:06:47 -08:00
MediaPipe Team
7f0134eecb Internal change 
PiperOrigin-RevId: 490041386
 2022-11-21 12:16:00 -08:00
MediaPipe Team
7acbf557a1 Cleanup after migration to new classification output format. 
PiperOrigin-RevId: 489921603
 2022-11-21 01:57:05 -08:00
MediaPipe Team
13c6b9a8c6 Allow kernel cache path to be specified without trailing path delimiter 
PiperOrigin-RevId: 489891079
 2022-11-20 22:20:18 -08:00
Camillo Lugaresi
3ac7f6a216 Simplify image creation in PacketCreator 
Use more existing functions, remove redundant code, remove direct use of RuntimeException.

PiperOrigin-RevId: 489868983
 2022-11-20 19:32:21 -08:00
MediaPipe Team
6cf464636b Internal change 
PiperOrigin-RevId: 489842199
 2022-11-20 14:26:22 -08:00
Scott Zhu
a367753eda Internal change 
PiperOrigin-RevId: 489824381
 2022-11-20 10:41:20 -08:00
MediaPipe Team
bdf4078e89 Internal change 
PiperOrigin-RevId: 489752009
 2022-11-19 21:14:24 -08:00
Camillo Lugaresi
a33cb1e05e Check that Java buffer supports direct access before using it 
If the buffer is not created with allocateDirect, JNI APIs will return a data pointer of nullptr and a capacity of -1. This can cause a crash when we access it.

Also clean up the code to raise exceptions instead of just logging errors and returning nullptr.

PiperOrigin-RevId: 489751312
 2022-11-19 21:04:48 -08:00
MediaPipe Team
977ee4272e Add public visibility to the model maker public API. 
PiperOrigin-RevId: 489701768
 2022-11-19 10:53:21 -08:00
Sebastian Schmidt
bbcbd5fc6c Audio Embedder for Web 
PiperOrigin-RevId: 489669966
 2022-11-19 04:49:24 -08:00
Sebastian Schmidt
e853f04b79 Create AudioTaskRunner 
PiperOrigin-RevId: 489613573
 2022-11-18 19:55:08 -08:00
Camillo Lugaresi
eb8ef1ace0 Use shared_from_this in GlTextureBuffer::GetReadView, GetWriteView 
This ensures that the callbacks in GlTextureView won't call an expired object, even if user code holds a GlTextureView after releasing the buffer.

Note that GlTextureBuffer is not always held by a shared_ptr, but it always is when GpuBuffer calls GetRead/WriteView on it. An alternative solution would have been to have GpuBuffer pass its shared_ptr<GlTextureBuffer> to the view method, which could have been implemented with some compile-time logic to detect whether the method expects such an argument. However, that doesn't seem necessary.

PiperOrigin-RevId: 489611843
 2022-11-18 19:42:34 -08:00
MediaPipe Team
bbd5da7971 Added the gray scale image support for the ImageToTensorCalculator on CPU. 
PiperOrigin-RevId: 489593917
 2022-11-18 17:12:57 -08:00
MediaPipe Team
524ac3ca61 Internal change for Model Maker 
PiperOrigin-RevId: 489540387
 2022-11-18 12:48:30 -08:00
Jiuqiang Tang
1b594a0310 Return error status when any tflite input and output tensor doesn't have valid dimensionality information that is needed to allocate Gl/Metal buffer before calling ModifyGraphWithDelegate. 
PiperOrigin-RevId: 489539740
 2022-11-18 12:44:55 -08:00
Sebastian Schmidt
71ae496a20 Add AudioEmbedder documentation 
PiperOrigin-RevId: 489532283
 2022-11-18 12:12:41 -08:00
MediaPipe Team
e2052a6a51 Rename embedding postprocessor "configure" method for consistency with classification postprocessor. 
PiperOrigin-RevId: 489518257
 2022-11-18 11:13:10 -08:00
Adam Cozzette
ac212c1507 Internal change 
PiperOrigin-RevId: 489502255
 2022-11-18 10:08:26 -08:00
MediaPipe Team
03d388fecf Add hand landmark named index constants 
PiperOrigin-RevId: 489498248
 2022-11-18 09:52:44 -08:00
MediaPipe Team
2f361e2f47 Internal change 
PiperOrigin-RevId: 489486417
 2022-11-18 08:53:43 -08:00
Jiuqiang Tang
e046982a3c Internal change 
PiperOrigin-RevId: 489484898
 2022-11-18 08:47:09 -08:00
Hadon Nash
ae44012c0c Allowing BypassCalculator to accept InputSidePackets. 
PiperOrigin-RevId: 489483992
 2022-11-18 08:43:02 -08:00
Camillo Lugaresi
efcdedbd59 Remove redundant _ios targets 
PiperOrigin-RevId: 489355333
 2022-11-17 18:16:58 -08:00
Copybara-Service
cd056e114a Merge pull request #3866 from kinaryml:fix-tasks-api-python-311-errors 
PiperOrigin-RevId: 489346167
 2022-11-17 17:20:23 -08:00
MediaPipe Team
6f3cb340e1 Internal change 
PiperOrigin-RevId: 489345940
 2022-11-17 17:16:25 -08:00
MediaPipe Team
ab3a5f0fbf Make MuxCalculator with DefaultInputStreamHandler to handle graph closure gracefully 
PiperOrigin-RevId: 489336722
 2022-11-17 16:30:35 -08:00
MediaPipe Team
a7bd725e65 Internal change 
PiperOrigin-RevId: 489331826
 2022-11-17 16:09:15 -08:00
MediaPipe Team
1fb0902aa0 Update gesture_recognizer test 
PiperOrigin-RevId: 489301508
 2022-11-17 14:04:30 -08:00
Sebastian Schmidt
3ccf7308e0 Add shared options for Text and Audio Tasks 
PiperOrigin-RevId: 489186644
 2022-11-17 05:28:27 -08:00
Hadon Nash
ea4989b6f1 Internal change 
PiperOrigin-RevId: 489135553
 2022-11-17 00:07:59 -08:00
kinaryml
5a6837d034 Fix errors that will occur in python 3.11 2022-11-16 22:08:52 -08:00
Sebastian Schmidt
899c87466e Add MP Tasks entrypoints 
PiperOrigin-RevId: 489110875
 2022-11-16 20:57:33 -08:00
Camillo Lugaresi
6fc277ee1c Internal change 
PiperOrigin-RevId: 489091534
 2022-11-16 18:38:03 -08:00
Sebastian Schmidt
e66e88802c Change NPM Bundle to ESM 
PiperOrigin-RevId: 489091370
 2022-11-16 18:34:53 -08:00
MediaPipe Team
90eb4a19d8 Internal change 
PiperOrigin-RevId: 489088227
 2022-11-16 18:16:32 -08:00
MediaPipe Team
b6b72d5e4e Add MuxCalculator test case where graph is being closed while SELECT has not been received. 
PiperOrigin-RevId: 489061902
 2022-11-16 15:56:43 -08:00
Sebastian Schmidt
3cdf0f6536 Fix a crash that occurred when a model returns fewer vector elements than before 
PiperOrigin-RevId: 489041814
 2022-11-16 14:38:37 -08:00
MediaPipe Team
74474d859e Update image_classifier demo with new ImageClassifierOption changes 
PiperOrigin-RevId: 489031381
 2022-11-16 14:00:39 -08:00
MediaPipe Team
cdd44e77b7 Internal change 
PiperOrigin-RevId: 488969539
 2022-11-16 10:07:40 -08:00
MediaPipe Team
dc9578d226 Internal change 
PiperOrigin-RevId: 488946809
 2022-11-16 08:30:20 -08:00
Sebastian Schmidt
f7aef677fc Add running mode to all vision tasks 
PiperOrigin-RevId: 488816785
 2022-11-15 19:02:21 -08:00
Camillo Lugaresi
febfc2029b Annotate plane argument 
PiperOrigin-RevId: 488813363
 2022-11-15 18:53:09 -08:00
Camillo Lugaresi
63e2089639 Deprecate a bunch of old stuff in GlCalculatorHelper 
PiperOrigin-RevId: 488813296
 2022-11-15 18:51:56 -08:00
Camillo Lugaresi
1979801a92 Remove GlCalculatorHelperImpl; merge with GlCalculatorHelper 
Originally, there were multiple implementations of GlCalculatorHelperImpl, depending on the platform and underlying GL APIs. These have all been refactored into other components, and the remaining code in this class is unified and much reduced in size. We can get rid of this implementation detail now.

PiperOrigin-RevId: 488813220
 2022-11-15 18:50:45 -08:00
Camillo Lugaresi
8b319e963a Add comment explaining ViewProvider 
This was only documented via examples (e.g. ViewProvider<GlTextureView>), but it's better to explain it properly in the header where the base case is defined.

PiperOrigin-RevId: 488813144
 2022-11-15 18:48:54 -08:00
Camillo Lugaresi
a28ccb0964 Remove unnecessary forward declarations 
PiperOrigin-RevId: 488813066
 2022-11-15 18:43:36 -08:00
Camillo Lugaresi
13b4b825d7 Remove std::shared_ptr<GpuBuffer> argument from GetRead/WriteView 
PiperOrigin-RevId: 488813004
 2022-11-15 18:41:53 -08:00
Camillo Lugaresi
1c0a1d0aab Remove shared_ptr<GpuBuffer> member from GlTextureView 
This only exists to support GlTexture's GetFrame API. It can be moved into GlTexture.

PiperOrigin-RevId: 488812896
 2022-11-15 18:40:15 -08:00
Camillo Lugaresi
767cc2ee3c More comments on gpu_buffer_storage 
This gives a basic explanation of the role of storages and views, and provides some details on how to implement a new storage type.

PiperOrigin-RevId: 488812807
 2022-11-15 18:38:29 -08:00
Camillo Lugaresi
4c874fe4cd Allow conversion of GlTextureBuffer to CVPixelBufferRef 
This means that, if an iOS application sends in a GlTextureBuffer but expects a CVPixelBufferRef as output, everything will work even if the graph just forwards the same input. Also, access by Metal calculators will also work transparently.

PiperOrigin-RevId: 488812748
 2022-11-15 18:36:45 -08:00
Camillo Lugaresi
fe66de3714 Internal change 
PiperOrigin-RevId: 488812677
 2022-11-15 18:32:54 -08:00
MediaPipe Team
2f77bf44e3 Use train_data to evaluate accuracy of unit test for gesture_recognizer due to limited dataset size. 
PiperOrigin-RevId: 488808942
 2022-11-15 18:10:28 -08:00
Camillo Lugaresi
b308c0dd5e Implement CVPixelBufferRef access as a view. 
PiperOrigin-RevId: 488798216
 2022-11-15 17:18:07 -08:00
Camillo Lugaresi
4bda012bba Factor out gl_texture_util 
PiperOrigin-RevId: 488797985
 2022-11-15 17:13:22 -08:00
Camillo Lugaresi
77b3edbb67 Internal change 
PiperOrigin-RevId: 488797407
 2022-11-15 17:09:43 -08:00
Camillo Lugaresi
7e19bbe35c Internal change 
PiperOrigin-RevId: 488795920
 2022-11-15 17:01:33 -08:00
Camillo Lugaresi
1beca61650 Register GlTextureBuffer pool with GpuBuffer 
First crack at hooking up pools with the GpuBufferStorage system. Will most likely be superseded later, but for now this works with minimal code impact: just overwrite the factory for a storage type with one that uses the pool.

PiperOrigin-RevId: 488783854
 2022-11-15 16:19:48 -08:00
Camillo Lugaresi
583d27636b Factor out ReusablePool 
PiperOrigin-RevId: 488783477
 2022-11-15 16:18:20 -08:00
Camillo Lugaresi
53d015af08 Generic MultiPool template 
PiperOrigin-RevId: 488783176
 2022-11-15 16:15:42 -08:00
Camillo Lugaresi
b9fa2e3496 Make it possible to override the SimplePool factory used by MultiPool 
This means MultiPool no longer needs a SetFlushPlatformCaches method, which was too specific to the CVPixelBufferPool.

PiperOrigin-RevId: 488783003
 2022-11-15 16:14:27 -08:00
Camillo Lugaresi
267476657d MultiPool options header refactoring 
Passing MultiPool options to the base pool factories means that we don't have to specialize which options we pass to them.

PiperOrigin-RevId: 488782861
 2022-11-15 16:13:05 -08:00
Camillo Lugaresi
7ef3185ecb Allow customizing MultiPool options 
These don't need to be constants.

PiperOrigin-RevId: 488782713
 2022-11-15 16:11:54 -08:00
Camillo Lugaresi
f13903b7c5 Call SimplePool methods directly 
This removes redundant helper functions in GpuBufferMultiPool.

PiperOrigin-RevId: 488782516
 2022-11-15 16:10:34 -08:00
Camillo Lugaresi
0c4522cb9f Move flush hook to CvPixelBufferPoolWrapper constructor 
This unifies the implementation of GpuBufferMultiPool::GetBufferFromSimplePool.

PiperOrigin-RevId: 488782173
 2022-11-15 16:07:17 -08:00
Camillo Lugaresi
a4fe3eb094 Add CreateBufferWithoutPool method to base pools 
This may not fit exactly in a pool class, but it makes it easy for the multi-pool to find the appropriate method by depending only on the type of the base pool. For the CVPixelBuffer case, the buffer type is CFHolder<CVPixelBufferRef>, and it seems even less appropriate to specialize that template to add such a method there. An alternative would be to allow defining a creation function separately.

PiperOrigin-RevId: 488782054
 2022-11-15 16:05:53 -08:00
Camillo Lugaresi
0d273dd11a Factor out CvPixelBufferPoolWrapper 
This is platform-specific and does not need to live in the main multi_pool sources.

PiperOrigin-RevId: 488781934
 2022-11-15 16:04:27 -08:00
Camillo Lugaresi
ab2dd779e7 Factor out CvTextureCacheManager 
This is a platform-specific component that is only used with CVPixelBufferPool.

PiperOrigin-RevId: 488781757
 2022-11-15 16:02:53 -08:00
MediaPipe Team
fae55910f4 Enable absl::string_view kCalculatorName 
PiperOrigin-RevId: 488781493
 2022-11-15 15:59:09 -08:00
Camillo Lugaresi
a520d6cc38 Remove FORCE_CONTIGUOUS_PIXEL_BUFFER_ON_IPHONE_SIMULATOR 
This workaround code is no longer necessary, as per the comment.

PiperOrigin-RevId: 488777606
 2022-11-15 15:41:53 -08:00
Camillo Lugaresi
3c71c64be1 Remove shared_ptr from SimplePool definition 
This makes the types more explicit and will help with factoring out platform-specific code.

PiperOrigin-RevId: 488775470
 2022-11-15 15:32:57 -08:00
Camillo Lugaresi
a67069156e Use flat_hash_map in ResourceCache 
This is the recommended hashmap in most cases.

PiperOrigin-RevId: 488772031
 2022-11-15 15:23:31 -08:00
MediaPipe Team
38b636f7ee Internal change 
PiperOrigin-RevId: 488770794
 2022-11-15 15:19:48 -08:00
Camillo Lugaresi
7a87546c30 Internal change 
PiperOrigin-RevId: 488761646
 2022-11-15 14:39:58 -08:00
MediaPipe Team
e65f21e2d8 Update the docstring to make it consistent with the model option update. 
PiperOrigin-RevId: 488761331
 2022-11-15 14:36:48 -08:00
Sebastian Schmidt
496720308c Migrate remaining MP Tasks Libraries to ts_declarations 
PiperOrigin-RevId: 488752799
 2022-11-15 14:08:15 -08:00
MediaPipe Team
1689112b23 Improve model_util_test code. 
PiperOrigin-RevId: 488752497
 2022-11-15 14:04:39 -08:00
MediaPipe Team
f14645cb06 Model maker gesture recognizer test changes 
PiperOrigin-RevId: 488702055
 2022-11-15 10:50:32 -08:00
MediaPipe Team
ebba119f15 Add Java ImageEmbedder API. 
PiperOrigin-RevId: 488588010
 2022-11-15 01:26:00 -08:00
Camillo Lugaresi
6f54308c25 Internal change 
PiperOrigin-RevId: 488552135
 2022-11-14 21:42:41 -08:00
Jiuqiang Tang
87dff8142c Fix a typo. 
PiperOrigin-RevId: 488519113
 2022-11-14 18:13:50 -08:00
Sebastian Schmidt
9bd8b432c3 Add typings support to d.ts package 
PiperOrigin-RevId: 488519074
 2022-11-14 18:10:25 -08:00
Jiuqiang Tang
f16e63694e Build embedding tasks into tasks AARs. 
PiperOrigin-RevId: 488509942
 2022-11-14 17:18:12 -08:00
Sebastian Schmidt
a12bc3fd0e Add IIFE bundles for NPM packages 
PiperOrigin-RevId: 488504360
 2022-11-14 16:51:58 -08:00
Jiuqiang Tang
794f64db55 Fix the wrong path of "text_embedder_graph_options_java_proto_lite". 
PiperOrigin-RevId: 488490050
 2022-11-14 15:51:05 -08:00
Jiuqiang Tang
cce1751dbf Expose tasks components API in PyPI packages. 
PiperOrigin-RevId: 488487666
 2022-11-14 15:41:11 -08:00
Jiuqiang Tang
e714e656fe Update python documentation. 
PiperOrigin-RevId: 488482142
 2022-11-14 15:18:47 -08:00
Sebastian Schmidt
c027373688 Internal change 
PiperOrigin-RevId: 488481286
 2022-11-14 15:15:22 -08:00
Sebastian Schmidt
11270d0c93 Image Embedder for Web 
PiperOrigin-RevId: 488468214
 2022-11-14 14:26:15 -08:00
Jiuqiang Tang
b4fba6fe61 MediaPipe Tasks AudioEmbedder Java API 
PiperOrigin-RevId: 488456442
 2022-11-14 13:42:55 -08:00
Sebastian Schmidt
ca7b5e9d8b Fix Script loading 
PiperOrigin-RevId: 488440736
 2022-11-14 12:40:41 -08:00
Copybara-Service
9a2af2f2a1 Merge pull request #3853 from kinaryml:audio-embedder-python 
PiperOrigin-RevId: 488434586
 2022-11-14 12:16:45 -08:00
Sebastian Schmidt
b00236e86e Hand Landmarker Web API 
PiperOrigin-RevId: 488434079
 2022-11-14 12:13:27 -08:00
MediaPipe Team
34daba4747 Add Java TextEmbedder API. 
PiperOrigin-RevId: 488427327
 2022-11-14 11:49:56 -08:00
Jiuqiang Tang
b40b2ade14 Fix typos. 
PiperOrigin-RevId: 488416345
 2022-11-14 11:07:59 -08:00
Jiuqiang Tang
4b5c3521af Dividing the timestamp by 1000 when returning a "none" result object from GestureRecognizer and HandLandmarker APIs. 
PiperOrigin-RevId: 488409077
 2022-11-14 10:44:20 -08:00
MediaPipe Team
c7d531ebb2 AddTarget -> ConnectTo and documentation 
PiperOrigin-RevId: 488407930
 2022-11-14 10:40:06 -08:00
MediaPipe Team
05cb40ff79 MediaPipe TextEmbedder task for web 
PiperOrigin-RevId: 488373613
 2022-11-14 08:34:09 -08:00
MediaPipe Team
badaccfb04 Internal change 
PiperOrigin-RevId: 488333493
 2022-11-14 05:05:03 -08:00
Nikolay Chirkov
bc6240e989 Zero-initialize id<MTLDevice> etc. members in Tensor 
PiperOrigin-RevId: 488308585
 2022-11-14 02:31:15 -08:00
Sebastian Schmidt
6610ca72ba Add ts_declaration rule for OSS 
PiperOrigin-RevId: 488307893
 2022-11-14 02:27:21 -08:00
Sebastian Schmidt
8d9cdb86dc Internal change 
PiperOrigin-RevId: 488294157
 2022-11-14 01:25:57 -08:00
kinaryml
a2bb58382d Added AudioEmbedder API and tests along with fixing a couple of typos in AudioClassifier 2022-11-14 01:20:45 -08:00
Jiuqiang Tang
6c0ca947de MediaPipe Tasks Audio embedder C++ API. 
PiperOrigin-RevId: 488273381
 2022-11-13 23:10:23 -08:00
Copybara-Service
0dfa91a166 Merge pull request #3846 from kinaryml:text-embedder-python 
PiperOrigin-RevId: 488198025
 2022-11-13 10:44:54 -08:00
Jiuqiang Tang
8aa59faaa4 Internal change 
PiperOrigin-RevId: 488193046
 2022-11-13 09:51:48 -08:00
Alan Kelly
bb6fa85703 Enable unsigned quantized infererence using XNNPACK. 
PiperOrigin-RevId: 488179656
 2022-11-13 07:10:04 -08:00
Liam Miller-Cushon
da36468409 Internal change 
PiperOrigin-RevId: 488065083
 2022-11-12 10:13:58 -08:00
MediaPipe Team
aafbf73c0a Renames model options in TextClassifier. 
PiperOrigin-RevId: 488063797
 2022-11-12 10:02:19 -08:00
kinaryml
a7ed160a8e Fixed a bug in embedding_result 2022-11-12 08:55:56 -08:00
kinaryml
a3788a23bc Removed unused code in image_embedder_test 2022-11-12 07:48:55 -08:00
kinaryml
157092d93e Removed unused dataclasses 2022-11-12 07:47:32 -08:00
kinaryml
a8103629c7 Updated Text Embedder API 2022-11-12 07:42:46 -08:00
Kinar R
acd9c280c0
Merge branch 'master' into text-embedder-python 2022-11-12 21:08:38 +05:30
Jiuqiang Tang
bf6c8a0b63 Expose ImageEmbedder APIs in PyPI packages. 
PiperOrigin-RevId: 488033833
 2022-11-12 04:35:22 -08:00
Copybara-Service
5d9ea88815 Merge pull request #3845 from kinaryml:image-embedder-python 
PiperOrigin-RevId: 487950862
 2022-11-11 17:03:00 -08:00
MediaPipe Team
c2ac040a6c Adds a public import API for TextClassifier. 
PiperOrigin-RevId: 487949023
 2022-11-11 16:52:16 -08:00
MediaPipe Team
8ec83d2aa0 Clarify AssetManager usage 
PiperOrigin-RevId: 487935478
 2022-11-11 15:49:50 -08:00
Sebastian Schmidt
c7030ac7fa Use CommonJS for NPM package 
PiperOrigin-RevId: 487902199
 2022-11-11 13:25:25 -08:00
Sebastian Schmidt
20a6f15f18 Improvements to NPM package 
PiperOrigin-RevId: 487901715
 2022-11-11 13:22:01 -08:00
Kinar R
fae77fc742
Update text_embedder_test.py 2022-11-12 01:27:20 +05:30
MediaPipe Team
340d7651af Internal change 
PiperOrigin-RevId: 487881149
 2022-11-11 11:53:41 -08:00
Hadon Nash
a83d87e157 Internal change 
PiperOrigin-RevId: 487880137
 2022-11-11 11:50:25 -08:00
Jiuqiang Tang
ce292c2a49 Fix a typo. 
PiperOrigin-RevId: 487872120
 2022-11-11 11:20:14 -08:00
MediaPipe Team
b0583a1821 Fixes SetAlphaCalculator silently failing to convert float inputs. 
PiperOrigin-RevId: 487868409
 2022-11-11 11:06:26 -08:00
Jiuqiang Tang
f329e38dc1 Map the "com_github_glog_glog" dependency of "com_google_sentencepiece" to mediapipe's "com_github_glog_glog_no_gflags". 
PiperOrigin-RevId: 487727239
 2022-11-10 22:12:42 -08:00
Kinar R
2d0b237711
Merge branch 'google:master' into text-embedder-python 2022-11-11 10:39:57 +05:30
Kinar R
f27068c6f2
Merge branch 'google:master' into image-embedder-python 2022-11-11 10:38:52 +05:30
MediaPipe Team
b4972ed4ae Update GestureRecognitionResult to GestureRecognizerResult. 
PiperOrigin-RevId: 487712873
 2022-11-10 20:25:31 -08:00
MediaPipe Team
3e05871f98 Open source Model Maker text tasks. 
PiperOrigin-RevId: 487706929
 2022-11-10 19:52:51 -08:00
Mark McDonald
8c8b668ba3 Fix comment for mediapipe license. 
The `"""` comment indicates a public docstring for the module, and will end up in the generated docs. By using a "private" comment (`#`) we will not document the license as part of the API.

The Apache license is noted in the footer of generated docs, and this is sufficient.

PiperOrigin-RevId: 487688719
 2022-11-10 18:06:35 -08:00
MediaPipe Team
ec327cedcb Internal Changes 
PiperOrigin-RevId: 487673720
 2022-11-10 16:48:13 -08:00
MediaPipe Team
2ea5184c51 Internal change 
PiperOrigin-RevId: 487663454
 2022-11-10 16:04:22 -08:00
Sebastian Schmidt
1a59f0a217 Prevent renaming of properties used in API 
PiperOrigin-RevId: 487661625
 2022-11-10 15:58:47 -08:00
Jiuqiang Tang
b3ac1c1387 Changes to create MediaPipe Tasks Audio AAR. 
PiperOrigin-RevId: 487661296
 2022-11-10 15:55:33 -08:00
Jiuqiang Tang
d6ba5e4bbe Map the "com_github_glog_glog" dependency of "com_google_audio_tools" to mediapipe's "com_github_glog_glog_no_gflags". 
PiperOrigin-RevId: 487651038
 2022-11-10 15:11:57 -08:00
Copybara-Service
bbf4ff0300 Merge pull request #3832 from kinaryml:update-classification-tasks-output-stream 
PiperOrigin-RevId: 487638077
 2022-11-10 14:22:31 -08:00
Hadon Nash
d93caf00de Fixing a bug in the BypassCalculator "passthrough" output stream ids. 
PiperOrigin-RevId: 487633708
 2022-11-10 14:05:47 -08:00
MediaPipe Team
8ec4427bd7 Add option, result types and utils for Java embedders. 
PiperOrigin-RevId: 487615327
 2022-11-10 12:53:00 -08:00
Sebastian Schmidt
aeb2466844 Pick TFLite delegate for Web CPU 
PiperOrigin-RevId: 487611446
 2022-11-10 12:37:42 -08:00
Jiuqiang Tang
bb5782ee2f Correct the timestamp in the vision tasks result objects. 
PiperOrigin-RevId: 487597999
 2022-11-10 11:46:16 -08:00
Jiuqiang Tang
da4d455d0c Implement MediaPipe Tasks AudioClassifier Java API. 
PiperOrigin-RevId: 487570148
 2022-11-10 10:10:03 -08:00
Jiuqiang Tang
2ce3a9719e Remove the problematic packet release calls in OutputHandler::run. 
OutputHandler::run is installed as a MultiStreamCallback into the TaskRunner and it's invoked in the Graph::CallbackToJava() jni method.

In CallbackToJava(), the packets are removed in RemovePacket() calls after the multi stream callback is invoked. The current logic lets the MultiStreamCallback release the packets first and leaves null pointers to RemovePacket() to remove, which may cause the app crash. The fix is to remove the packet release calls in the MultiStreamCallback and always leave the packet release work to RemovePacket().

PiperOrigin-RevId: 487558809
 2022-11-10 09:27:35 -08:00
Kinar R
0a6e21c212
Merge branch 'master' into image-embedder-python 2022-11-10 16:04:28 +05:30
kinaryml
0e9b925726 Fixed some typos and revised image embedder tests 2022-11-10 02:30:17 -08:00
kinaryml
1604908a59 Added files needed for the text embedder's implementation and tests 2022-11-10 02:16:51 -08:00
MediaPipe Team
0b12aa9435 Internal change 
PiperOrigin-RevId: 487466061
 2022-11-10 01:32:14 -08:00
MediaPipe Team
0ac604d507 Add timestamp aggregation to EmbeddingPostprocessingGraph. 
PiperOrigin-RevId: 487463848
 2022-11-10 01:20:45 -08:00
MediaPipe Team
f11c757629 Update C++ HandLandmarksDetectionResult to HandLandmarkerResult. 
PiperOrigin-RevId: 487443827
 2022-11-09 23:20:15 -08:00
Yuqi Li
d2142e86a9 Metadata Writer: Add Bert metadata writer in Text Classifier. 
PiperOrigin-RevId: 487354439
 2022-11-09 14:53:12 -08:00
MediaPipe Team
116b4bb6c4 Internal change 
PiperOrigin-RevId: 487338991
 2022-11-09 13:57:13 -08:00
MediaPipe Team
7066ee231e Hand Landmarker Java API and unit test. 
PiperOrigin-RevId: 487307858
 2022-11-09 11:55:31 -08:00
kinaryml
7ec0d8cf3b Added tolerance for vector coordinate values in embeddings 2022-11-09 11:53:30 -08:00
Jiuqiang Tang
6f66a7fde4 Rename GestureRecognitionResult to GestureRecognizerResult. 
PiperOrigin-RevId: 487306292
 2022-11-09 11:49:29 -08:00
kinaryml
17bb174444 Refactored embeddings to embedding_result 2022-11-09 11:42:32 -08:00
Jiuqiang Tang
4e568a1c3d Expose HandLandmarker API in PyPI packages. 
PiperOrigin-RevId: 487294501
 2022-11-09 11:06:33 -08:00
Sebastian Schmidt
84895316d2 Add pre-compiled WASM files to NPM packages 
PiperOrigin-RevId: 487283112
 2022-11-09 10:28:09 -08:00
Copybara-Service
0a5534204f Merge pull request #3829 from kinaryml:hand-landmarker-python 
PiperOrigin-RevId: 487281485
 2022-11-09 10:23:38 -08:00
kinaryml
dc30cf9732 Updated embeddings container to use a timestamp and made parameters - head_index,head_name optional 2022-11-08 23:51:55 -08:00
kinaryml
d327b3649d Fixed some typos in methods and comments 2022-11-08 23:47:58 -08:00
kinaryml
dd6fdedd5f Added some sanity tests 2022-11-08 23:15:58 -08:00
Kinar R
36c50ff8f3
Merge branch 'google:master' into image-embedder-python 2022-11-09 12:41:55 +05:30
kinaryml
7e251bc6b6 Merge branch 'hand-landmarker-python' of https://github.com/kinaryml/mediapipe into hand-landmarker-python 2022-11-08 23:07:30 -08:00
Kinar R
812fa2cc70
Merge branch 'google:master' into hand-landmarker-python 2022-11-09 12:37:06 +05:30
kinaryml
46f135e54d Renamed HandLandmarksDetectionResult to HandLandmarkerResult 2022-11-08 23:06:28 -08:00
MediaPipe Team
b4e1833dd0 Internal change 
PiperOrigin-RevId: 487125366
 2022-11-08 20:35:46 -08:00
MediaPipe Team
a5bcb97d88 Adds an AverageWordVecModel. 
PiperOrigin-RevId: 487104909
 2022-11-08 18:29:21 -08:00
MediaPipe Team
c31aaa94a6 Adds a BertClassifier. 
PiperOrigin-RevId: 487086744
 2022-11-08 16:49:46 -08:00
Sebastian Schmidt
669d539551 NPM package definitions for MediaPipe Tasks 
PiperOrigin-RevId: 487071334
 2022-11-08 15:46:05 -08:00
Lu Wang
253ff0f85c Update the Java doc for model asset for BaseOptions 
PiperOrigin-RevId: 487059371
 2022-11-08 14:57:27 -08:00