mediapipe/docs/tools/visualizer.md
MediaPipe Team 350fbb2100 Project import generated by Copybara.
GitOrigin-RevId: d073f8e21be2fcc0e503cb97c6695078b6b75310
2021-02-27 03:30:05 -05:00

3.6 KiB

layout title parent nav_order
default Visualizer Tools 1

Visualizer

{: .no_toc }

  1. TOC {:toc}

To help users understand the structure of their calculator graphs and to understand the overall behavior of their machine learning inference pipelines, we have built the MediaPipe Visualizer that is available online.

  • A graph view allows users to see a connected calculator graph as expressed through a graph configuration that is pasted into the graph editor or uploaded. The user can visualize and troubleshoot a graph they have created.

    Startup screen

Working within the Editor

Getting Started:

The graph can be modified by adding and editing code in the Editor view.

Editor UI

  • Pressing the "New" button in the upper right corner will clear any existing code in the Editor window.

    New Button

  • Pressing the "Upload" button will prompt the user to select a local PBTXT file, which will overwrite the current code within the editor.

  • Alternatively, code can be pasted directly into the editor window.

  • Errors and informational messages will appear in the Feedback window.

    Error Msg

Understanding the Graph

The visualizer graph shows the connections between calculator nodes.

  • Streams exit from the bottom of the calculator producing the stream and enter the top of any calculator receiving the stream. (Notice the use of the key, "input_stream" and "output_stream").

    Stream UI

    Stream_code

  • Sidepackets work the same, except that they exit a node on the right and enter on the left. (Notice the use of the key, "input_side_packet" and "output_side_packet").

    Sidepacket UI

    Sidepacket_code

  • There are special nodes that represent inputs and outputs to the graph and can supply either side packets or streams.

    Special nodes

    Special nodes

Visualizing Subgraphs

The MediaPipe visualizer can display multiple graphs in separate tabs. If a graph has a type field in the top level of the graph's text proto definition, and that value of graph type is used as a calculator name in another graph, it is considered a subgraph by the visualizer and colored appropriately where it is used. Clicking on a subgraph will navigate to the corresponding tab which holds the subgraph's definition.

For instance, there are two graphs involved in MediaPipe Hands: the main graph (source pbtxt file) and its associated subgraph (source pbtxt file). To visualize them:

  • In the MediaPipe visualizer, click on the upload graph button and select the 2 pbtxt files to visualize (main graph and its associated subgraph).

    Upload graph button

    Choose the 2 files

  • There will be 2 additional tabs. The main graph tab is hand_detection_mobile.pbtxt.

    hand_detection_mobile_gpu.pbtxt

  • Clicking on the HandDetection node in purple redirects the view to the hand_detection_gpu.pbtxt tab.

    Hand detection subgraph