detectForVideo badly explained

[marcusx2]

Description of issue (what needs changing)

The documentation fo detectForVideo is lacking

Clear description

The documentation for https://ai.google.dev/edge/api/mediapipe/js/tasks-vision.objectdetector#objectdetectordetectforvideo is lacking. A couple of questions

1- timestamp is the timestamp of the current frame, in ms. In this example performance.now is used. What's the meaning of that. If I want to play a normal video of for example 3 seconds, and want to check every frame, I use performance.now? Because the example is not a "normal" video, it's a webcam

2- The detectForVideo uses a TexImageSource as input just like the normal detect. How is that? Is there any substantial internal difference between normal detect and detectForVideo, besides the video having a timestamp?

[HarishPermal]

Hi @marcusx2,

Thanks for the feedback on the documentation, here are some clarifications

As you mentioned in this example, performance.now() is used because a live camera feed doesn't have a predefined timeline like a video file. It provides a high-resolution timer relative to the page's origin, ensuring each captured frame gets a unique and increasing timestamp.

For a standard video file you should not use performance.now(). instead use the timestamp

And comming to the TexImageSource input both functions accept TexImageSource because, at any given moment a video frame is just an image. The primary difference lies in the runningMode of mediapipe task is configured with, detect is for the IMAGE mode treating each call as independent and detectForVideo is for VIDEO mode where mediapipe tasks can be stateful, they can leverage information from previous frames to optimize detection on the current frame.

I hope this answers you questions 😊

[marcusx2]

Hello again @HarishPermal . I'd greatly appreciate it if there was an example using detectForVideo on an actual standard video. That has a length and uses the timestamp....