diff --git a/README.md b/README.md index 56d5f15ec..c36179178 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ ![MediaPipe](mediapipe/docs/images/mediapipe_small.png?raw=true "MediaPipe logo") ======================================================================= -[MediaPipe](http://mediapipe.dev) is a framework for building multimodal (eg. video, audio, any time series data) applied ML pipelines. With MediaPipe, a perception pipeline can be built as a graph of modular components, including, for instance, inference models (e.g., TensorFlow, TFLite) and media processing functions. http://mediapipe.dev +[MediaPipe](http://mediapipe.dev) is a framework for building multimodal (eg. video, audio, any time series data) applied ML pipelines. With MediaPipe, a perception pipeline can be built as a graph of modular components, including, for instance, inference models (e.g., TensorFlow, TFLite) and media processing functions. ![Real-time Face Detection](mediapipe/docs/images/realtime_face_detection.gif) diff --git a/mediapipe/docs/gpu.md b/mediapipe/docs/gpu.md index 6c2b07cc1..d17733266 100644 --- a/mediapipe/docs/gpu.md +++ b/mediapipe/docs/gpu.md @@ -36,7 +36,7 @@ calculator derived from base class GlSimpleCalculator. The GPU calculator `LuminanceCalculator` is shown as an example. The method `LuminanceCalculator::GlRender` is called from `GlSimpleCalculator::Process`. -``` +```c++ // Converts RGB images into luminance images, still stored in RGB format. // See GlSimpleCalculator for inputs, outputs and input side packets. class LuminanceCalculator : public GlSimpleCalculator { diff --git a/mediapipe/docs/how_to_questions.md b/mediapipe/docs/how_to_questions.md index d05fc1f68..ddc6c80c6 100644 --- a/mediapipe/docs/how_to_questions.md +++ b/mediapipe/docs/how_to_questions.md @@ -23,7 +23,7 @@ You can see an example in: ### How to visualize perception results -The [`AnnotationOverlayCalculator`] allows perception results, such as boudning +The [`AnnotationOverlayCalculator`] allows perception results, such as bounding boxes, arrows, and ovals, to be superimposed on the video frames aligned with the recognized objects. The results can be displayed in a diagnostic window when running on a workstation, or in a texture frame when running on device. You can @@ -85,7 +85,7 @@ The first approach has the advantage of leveraging [`CalculatorGraphConfig`] processing tools such as "subgraphs". The second approach has the advantage of allowing active calculators and packets to remain in-flight while settings change. Mediapipe contributors are currently investigating alternative approaches -to achieve both of these adantages. +to achieve both of these advantages. ### How to process realtime input streams diff --git a/mediapipe/docs/install.md b/mediapipe/docs/install.md index 1a53ac02a..f91beda1b 100644 --- a/mediapipe/docs/install.md +++ b/mediapipe/docs/install.md @@ -6,6 +6,7 @@ Choose your operating system: - [Installing on Debian and Ubuntu](#installing-on-debian-and-ubuntu) - [Installing on CentOS](#installing-on-centos) - [Installing on macOS](#installing-on-macos) +- [Installing on Windows Subsystem for Linux (WSL)](#installing-on-windows-subsystem-for-linux-wsl) - [Installing using Docker](#installing-using-docker) - [Setting up Android SDK and NDK](#setting-up-android-sdk-and-ndk) @@ -299,6 +300,125 @@ Required libraries # Hello World! ``` +### Installing on Windows Subsystem for Linux (WSL) + +1. Follow + [the instruction](https://docs.microsoft.com/en-us/windows/wsl/install-win10) + to install Windows Sysystem for Linux (Ubuntu) + +2. Install Windows ADB and start the ADB server in Windows + + Note: Window’s and WSL’s adb versions must be the same version, e.g., if WSL + has ADB 1.0.39, you need to download the corresponding Windows ADB from + [here](https://dl.google.com/android/repository/platform-tools_r26.0.1-windows.zip). + +3. Launch WSL + + Note: All the following steps will be executed in WSL. The Windows directory + of the Linux Subsystem can be found in + C:\Users\YourUsername\AppData\Local\Packages\CanonicalGroupLimited.UbuntuonWindows_SomeID\LocalState\rootfs\home + +4. Install the needed packages + + ```bash + username@DESKTOP-TMVLBJ1:~$ sudo apt-get update && sudo apt-get install -y --no-install-recommends build-essential git python zip adb openjdk-8-jdk + ``` + +5. Install Bazel + + ```bash + username@DESKTOP-TMVLBJ1:~$ curl -sLO --retry 5 --retry-max-time 10 \ + https://storage.googleapis.com/bazel/0.27.0/release/bazel-0.27.0-installer-linux-x86_64.sh && \ + sudo mkdir -p /usr/local/bazel/0.27.0 && \ + chmod 755 bazel-0.27.0-installer-linux-x86_64.sh && \ + sudo ./bazel-0.27.0-installer-linux-x86_64.sh --prefix=/usr/local/bazel/0.27.0 && \ + source /usr/local/bazel/0.27.0/lib/bazel/bin/bazel-complete.bash + + username@DESKTOP-TMVLBJ1:~$ /usr/local/bazel/0.27.0/lib/bazel/bin/bazel version && \ + alias bazel='/usr/local/bazel/0.27.0/lib/bazel/bin/bazel' + ``` + +6. Checkout mediapipe repository + + ```bash + username@DESKTOP-TMVLBJ1:~$ git clone https://github.com/google/mediapipe.git + + username@DESKTOP-TMVLBJ1:~$ cd mediapipe + ``` + +7. Install OpenCV + + Option 1. Use package manager tool to install the pre-compiled OpenCV + libraries. + + ```bash + username@DESKTOP-TMVLBJ1:~/mediapipe$ sudo apt-get install libopencv-core-dev libopencv-highgui-dev \ + libopencv-imgproc-dev libopencv-video-dev + ``` + + Option 2. Run [`setup_opencv.sh`] to automatically build OpenCV from source + and modify MediaPipe's OpenCV config. + + Option 3. Follow OpenCV's + [documentation](https://docs.opencv.org/3.4.6/d7/d9f/tutorial_linux_install.html) + to manually build OpenCV from source code. + + Note: You may need to modify [`WORKSAPCE`] and [`opencv_linux.BUILD`] to + point MediaPipe to your own OpenCV libraries, e.g., if OpenCV 4 is installed + in "/usr/local/", you need to update the "linux_opencv" new_local_repository + rule in [`WORKSAPCE`] and "opencv" cc_library rule in [`opencv_linux.BUILD`] + like the following: + + ```bash + new_local_repository( + name = "linux_opencv", + build_file = "@//third_party:opencv_linux.BUILD", + path = "/usr/local", + ) + + cc_library( + name = "opencv", + srcs = glob( + [ + "lib/libopencv_core.so*", + "lib/libopencv_highgui.so*", + "lib/libopencv_imgcodecs.so*", + "lib/libopencv_imgproc.so*", + "lib/libopencv_video.so*", + "lib/libopencv_videoio.so*", + + ], + ), + hdrs = glob(["include/opencv4/**/*.h*"]), + includes = ["include/opencv4/"], + linkstatic = 1, + visibility = ["//visibility:public"], + ) + + ``` + +8. Run the hello world desktop example + + ```bash + username@DESKTOP-TMVLBJ1:~/mediapipe$ export GLOG_logtostderr=1 + + # Need bazel flag 'MEDIAPIPE_DISABLE_GPU=1' as desktop GPU is currently not supported + username@DESKTOP-TMVLBJ1:~/mediapipe$ bazel run --define 'MEDIAPIPE_DISABLE_GPU=1' \ + mediapipe/examples/desktop/hello_world:hello_world + + # Should print: + # Hello World! + # Hello World! + # Hello World! + # Hello World! + # Hello World! + # Hello World! + # Hello World! + # Hello World! + # Hello World! + # Hello World! + ``` + ### Installing using Docker This will use a Docker image that will isolate mediapipe's installation from the rest of the system. diff --git a/mediapipe/framework/calculator_runner.cc b/mediapipe/framework/calculator_runner.cc index 157afaacf..8b5239883 100644 --- a/mediapipe/framework/calculator_runner.cc +++ b/mediapipe/framework/calculator_runner.cc @@ -216,6 +216,10 @@ mediapipe::Counter* CalculatorRunner::GetCounter(const std::string& name) { return graph_->GetCounterFactory()->GetCounter(name); } +std::map CalculatorRunner::GetCountersValues() { + return graph_->GetCounterFactory()->GetCounterSet()->GetCountersValues(); +} + ::mediapipe::Status CalculatorRunner::BuildGraph() { if (graph_ != nullptr) { // The graph was already built. diff --git a/mediapipe/framework/calculator_runner.h b/mediapipe/framework/calculator_runner.h index 59b20c548..7fe12eba2 100644 --- a/mediapipe/framework/calculator_runner.h +++ b/mediapipe/framework/calculator_runner.h @@ -127,6 +127,9 @@ class CalculatorRunner { // Returns a graph counter. mediapipe::Counter* GetCounter(const std::string& name); + // Returns all graph counters values. + std::map GetCountersValues(); + private: static const char kSourcePrefix[]; static const char kSinkPrefix[]; diff --git a/mediapipe/framework/counter_factory.cc b/mediapipe/framework/counter_factory.cc index 0eac61777..0c3536602 100644 --- a/mediapipe/framework/counter_factory.cc +++ b/mediapipe/framework/counter_factory.cc @@ -73,6 +73,16 @@ Counter* CounterSet::Get(const std::string& name) LOCKS_EXCLUDED(mu_) { return counters_[name].get(); } +std::map CounterSet::GetCountersValues() + LOCKS_EXCLUDED(mu_) { + absl::ReaderMutexLock lock(&mu_); + std::map result; + for (const auto& it : counters_) { + result[it.first] = it.second->Get(); + } + return result; +} + Counter* BasicCounterFactory::GetCounter(const std::string& name) { return counter_set_.Emplace(name, name); } diff --git a/mediapipe/framework/counter_factory.h b/mediapipe/framework/counter_factory.h index d1d09df60..23b631056 100644 --- a/mediapipe/framework/counter_factory.h +++ b/mediapipe/framework/counter_factory.h @@ -65,6 +65,9 @@ class CounterSet { // exist. Counter* Get(const std::string& name); + // Retrieves all counters names and current values from the internal map. + std::map GetCountersValues() LOCKS_EXCLUDED(mu_); + private: absl::Mutex mu_; std::map> counters_ GUARDED_BY(mu_);