Skip to content

Skip frame for Adaptive #283

Closed
Closed
Skip frame for Adaptive#283
Labels
feature
Milestone
0.6.1

Description

@MattFanto
MattFanto
opened on Aug 8, 2022

Description of Problem & Solution
I'm trying to process many high fps videos and unfortunately the video processing would be too low for our use case, I saw the skip frames option. Unfortunately it can't be applied to the Adaptive algorithm due to incompatibility with the mandatory scene stats.
I was wondering if it would be possible to create a more generic implementation of skip frames that can be used with the Adaptive detector.

Proposed Implementation:
Eventually adding a parameter to VideoManager, would allow skipping the frames directly there, creating an abstraction for the downstream detector so that the least amount of changes would be required.

Alternative Solutions:
The alternative solution in my case would be to reduce the FPS eagerly, with some libraries like FFmpeg. Unfortunately, that would be much more resource-consuming compared to the lazy approach above.

Activity

  1. Breakthrough commented on Aug 10, 2022

    @Breakthrough
    Breakthrough
    on Aug 10, 2022
    Owner

    I'm pursuing turning AdaptiveDetector into an online algorithm for the next release. This would also mean it no longer requires a StatsManager. It does mean some API changes as the way callbacks work right now (they assume a cut can be detected immediately), but that's something that should be resolved for the long term regardless.

    I'll use this as a tracking issue for that feature, but hope to have this done for v0.6.1.

    In general have you had to tweak the parameters you're using with Adaptive? If this proceeds as planned, I want to verify that the defaults are reasonable for most use cases, as I would like to essentially make what AdaptiveDetector does now the new default for v0.6.1.

    Thanks!

    Edit: Updated new approach (previously was to merge with ContentDetector, but they should remain separate, and usage examples point people to that as a default in the next release).

  2. Breakthrough
    added
    feature
    in progress
    on Aug 10, 2022
  3. Breakthrough
    added this to the v0.6.1 milestone on Aug 10, 2022

  4. MattFanto commented on Aug 10, 2022

    @MattFanto
    MattFanto
    on Aug 10, 2022
    Author

    Hi @Breakthrough thanks a million for your reply, great news looking forward to trying the new version.

    Regarding your question: at least to me the default are reasonable

  5. Breakthrough
    added a commit that references this issue on Aug 11, 2022

    [detector][adaptive] Make adaptive_ratio calculation online

    b9e129e

  6. Breakthrough commented on Aug 12, 2022

    @Breakthrough
    Breakthrough
    on Aug 12, 2022
    Owner

    I managed to make the transition, and AdaptiveDetector now doesn't require a StatsManager. Here's a beta version you can try if you're interested in testing (can be installed using pip):

    scenedetect-0.6.1.dev0.tar.gz

    Couple things remain as TODOs, but most of them are for the CLI and code organization - the above build should work just fine as is.

    • Had to change short option of -w/--frame-window to -f, using it instead for --weights
      • consider a different name for --weights or if this breaking change should proceed
    • Decide if this functionality should be merged with ContentDetector or remain separate
      • Arguments for merging:
        • it can be turned off by setting window size to zero
        • reduces code duplication when handling CLI and config file options
        • doesn't affect main score calculation
        • most users already are using ContentDetector/ detect-content so will experience improvement when updating
      • Arguments against merging
        • may cause changes in behavior for end users if turned on by default
        • detectors need to be refactored anyways
        • changing the documentation is easier / lower risk, and this can always be pursued in the future
      • Final decision: No merge
    • If merge is made, mark AdaptiveDetector/detect-adaptive as deprecated, otherwise
      update examples to use AdaptiveDetector/detect-adaptive instead of ContentDetector/detect-content
    • Change default min_content_val value to 18 (current default is 15) due to results of local testing

  7. MattFanto commented on Aug 24, 2022

    @MattFanto
    MattFanto
    on Aug 24, 2022
    Author

    Hi @Breakthrough , sorry for the late reply I just came back from holiday and thanks a million for the wheel.
    I'm adapting some parameter since I was using v0.5 and some changes are required to migrate to v0.6.x, in any case I already checked skip frame and it's working great.
    I'm available also to test a release candidate as soon as it is available

  8. MattFanto commented on Aug 30, 2022

    @MattFanto
    MattFanto
    on Aug 30, 2022
    Author

    Hi @Breakthrough , all working good I have adapted the params etc.. and I didn't see any regression, the performance improvement in our case is excellent, thanks a lot!
    May I ask you when are you going to release it to PyPI? I'm wondering from a deployment perspective if I should commit the wheel or I can just wait for the package in PyPI and install via pip

  9. Breakthrough commented on Aug 30, 2022

    @Breakthrough
    Breakthrough
    on Aug 30, 2022
    Owner

    I'm not sure on an exact timeline, but I hope to have a pre-release of v0.6.1 within the next couple weeks on PyPI. The final v0.6.1 release will probably be near the end of September October. Unfortunately I can't commit to any specific dates yet as I'm busy with IRL stuff the next couple weeks months.

  10. MattFanto commented on Aug 30, 2022

    @MattFanto
    MattFanto
    on Aug 30, 2022
    Author

    Thanks for the info, no problem at all. In any case I can use the wheel

  11. Breakthrough
    added
    status: completed
    and removed
    in progress
    on Nov 16, 2022
  12. Breakthrough
    closed this as completedon Nov 21, 2022

  13. danon commented on Oct 16, 2026

    @danon
    danon
    on Oct 16, 2026

    I have a similar problem with Threshold detector. In my case it never calls callback in detect_scenes().

  14. Breakthrough commented on Oct 16, 2026

    @Breakthrough
    Breakthrough
    on Oct 16, 2026 via email
    Owner
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions