Ubuntu 16.10 no longer supports gstreamer0.10

On Ubuntu 16.04, we should install libgstreamer0.10-dev to use processing-video (look at #69).

However, on Ubuntu 16.10, they are no longer provided, and only packages of gstreamer 1.0 series are provided. And GStreamer says "GStreamer 1.x has been the main stable series since 2012. GStreamer 0.10 is no longer maintained." (link)

And processing-video didn't run with gstreamer 1.0. It wrote out the following error messages (processing 3.3.2, processing-video 1.0.1).

UnsatisfiedLinkError: Error looking up function 'gst_date_get_type': /usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so: undefined symbol: gst_date_get_type
A library relies on native code that's not available.
Or only works properly when the sketch is run as a 32-bit application.

This would be because processing-video uses gstreamer-java. It says "Current version of the gstreamer-java is NOT compatible with: GStreamer 1.0 or above. this means the last usable version are GStreamer 0.10." (link)

How about using gst1-java-core instead of gstreamer-java?

Related links
- https://launchpad.net/ubuntu/yakkety/+source/gstreamer0.10
- https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=802812
For the moment, there is a dirty workaround here, which using .deb files of gstreamer 0.10 for Ubuntu 16.04.

Not only Ubuntu, but ArchLinux is also getting rid of gstreamer0.10. What's the state of the https://github.com/processing/processing-video/tree/gstreamer1.x or https://github.com/processing/processing-video/tree/gst1-java-core branches?

I see that the gst1-java-core branch uses a 0.9 snapshot from one year ago. There's a new one 5 months old (December 1st 2016) which has tons of fixes. I wish I had time to update the gst1-java-core jar and see what happens. https://github.com/gstreamer-java/gst1-java-core/releases

Since Video stopped working after gstreamer0.10 was removed, I cloned this repo, switched to the gst1-java-core branch, updated jna.jar and downloaded the latest gst1-java-core binary, and unfortunately does not build. The number of issues is not large, but I'm not familiar with gstreamer. Some method signatures have changed and some classes are gone (or maybe were renamed).

If anyone wants to try, this is what I did:

Go to your sketch folder, then to libraries:
cd libraries
Make a folder where to store the original video library
mkdirs ../libraries_off
Move the video library out, so we can clone the video repo
mv video ../libraries_off
Clone the video library repo
git clone https://github.com/processing/processing-video.git video
cd video
Switch to the gst1-java-core branch
git branch gst1-java-core
You need the Processing source code somewhere, and with the next line you tell the video library the location of the Processing source.
vi build.properties
Type this to try build the library
ant build

I opened IntelliJ idea, which helps a lot with auto-completing and suggesting fixes. Went to File > Project Structure > Libraries and added the processing/core/library folder so it finds the core Processing jar files.

Errors are highlighted in red, and it's easy to see what needs to be fixed. But it's not easy to see how :)

@hamoid

@hamoid These are the errors you are getting, right:

compile:
    [javac] Compiling 6 source files to /Users/andres/Documents/Processing/libraries/video/bin
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/Capture.java:40: error: package org.freedesktop.gstreamer.device does not exist
    [javac] import org.freedesktop.gstreamer.device.DeviceMonitor;
    [javac]                                        ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:47: error: package org.freedesktop.gstreamer.gl does not exist
    [javac] import org.freedesktop.gstreamer.gl.GLContext;
    [javac]                                    ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:48: error: package org.freedesktop.gstreamer.gl does not exist
    [javac] import org.freedesktop.gstreamer.gl.GLDisplay;
    [javac]                                    ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:49: error: package org.freedesktop.gstreamer.gl does not exist
    [javac] import org.freedesktop.gstreamer.gl.GLMemory;
    [javac]                                    ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:52: error: cannot find symbol
    [javac] import org.freedesktop.gstreamer.lowlevel.GstVideoAPI.VideoFrameStruct;
    [javac]                                                      ^
    [javac]   symbol:   class VideoFrameStruct
    [javac]   location: interface GstVideoAPI
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:53: error: cannot find symbol
    [javac] import org.freedesktop.gstreamer.lowlevel.GstVideoAPI.VideoInfoStruct;
    [javac]                                                      ^
    [javac]   symbol:   class VideoInfoStruct
    [javac]   location: interface GstVideoAPI
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/Capture.java:691: error: cannot find symbol
    [javac]     DeviceMonitor monitor = DeviceMonitor.createNew();
    [javac]     ^
    [javac]   symbol:   class DeviceMonitor
    [javac]   location: class Capture
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/Capture.java:691: error: cannot find symbol
    [javac]     DeviceMonitor monitor = DeviceMonitor.createNew();
    [javac]                             ^
    [javac]   symbol:   variable DeviceMonitor
    [javac]   location: class Capture
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/Capture.java:1098: error: Capture.NewSampleListener is not abstract and does not override abstract method newSample(AppSink) in NEW_SAMPLE
    [javac]   private class NewSampleListener implements AppSink.NEW_SAMPLE {
    [javac]           ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/Capture.java:1100: error: method does not override or implement a method from a supertype
    [javac]     @Override
    [javac]     ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/Movie.java:1066: error: Movie.NewSampleListener is not abstract and does not override abstract method newSample(AppSink) in NEW_SAMPLE
    [javac]   private class NewSampleListener implements AppSink.NEW_SAMPLE {
    [javac]           ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/Movie.java:1068: error: method does not override or implement a method from a supertype
    [javac]     @Override
    [javac]     ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:908: error: cannot find symbol
    [javac]     bus.connect(new Bus.NEED_CONTEXT() {
    [javac]                        ^
    [javac]   symbol:   class NEED_CONTEXT
    [javac]   location: class Bus
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:1194: error: MovieGL.NewSampleListener is not abstract and does not override abstract method newSample(AppSink) in NEW_SAMPLE
    [javac]   private class NewSampleListener implements AppSink.NEW_SAMPLE {
    [javac]           ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:1196: error: method does not override or implement a method from a supertype
    [javac]     @Override
    [javac]     ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:1209: error: package VideoInfoStruct does not exist
    [javac]       VideoInfoStruct.ByReference info =  org.freedesktop.gstreamer.Video.getVideoInfo(caps);
    [javac]                      ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:1209: error: cannot find symbol
    [javac]       VideoInfoStruct.ByReference info =  org.freedesktop.gstreamer.Video.getVideoInfo(caps);
    [javac]                                                                          ^
    [javac]   symbol:   method getVideoInfo(Caps)
    [javac]   location: class Video
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:1211: error: package VideoFrameStruct does not exist
    [javac]       VideoFrameStruct.ByReference frame = org.freedesktop.gstreamer.Video.mapVideoFrame(info,
    [javac]                       ^
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/MovieGL.java:1212: error: cannot find symbol
    [javac]                                            buffer, GstBufferAPI.GST_MAP_READ | GLMemory.GST_MAP_GL);
    [javac]                                                                                ^
    [javac]   symbol:   variable GLMemory
    [javac]   location: class MovieGL.NewSampleListener
    [javac] /Users/andres/Documents/Processing/libraries/video/src/processing/video/Video.java:149: error: cannot find symbol
    [javac]     Registry reg = Registry.getDefault();
    [javac]                            ^
    [javac]   symbol:   method getDefault()
    [javac]   location: class Registry
    [javac] 20 errors

@codeanticode

@codeanticode yes, looks like it. I tried finding solutions to some of those, but I could only go so far without studying gstreamer and its examples.

@hamoid

@hamoid as I've just replied in an email to @codeanticode most of those errors are because of additions that were made to the gst1-java-core API that didn't get submitted upstream. The other errors are because the API for the various AppSink listeners changed to better match GStreamer itself. Happy to help work out what needs to be added upstream and what needs to be fixed here.

@neilcsmith-net

I think your help would be very useful @neilcsmith-net :) Not sure how I can contribute. I could keep a fork which you guys observe, we discuss, and I follow your instructions...

@hamoid

@hamoid if you check @codeanticode gst1-java-core repo you'll see that it's 14 commits ahead and 87 commits behind upstream. Essentially the work required is to update the changes in the fork on top of the current code, and make the relevant pull requests upstream, which might require some changes to be accepted (by me, so I'm not going to make it that hard 😄 ).

https://github.com/codeanticode/gst1-java-core
https://github.com/codeanticode/gst1-java-core/commits/master

Of the errors listed above, Registry.getDefault() has become Registry.get() and the AppSink.NEW_SAMPLE listener needs to return FlowReturn.OK. Other than that I think the errors are all related to the missing commits from the fork.

Thanks! I cloned codeanticode's, then merged upstream. There were two conflicts:

gstreamer/lowlevel/GstVideoAPI.java upstream is minimal, where codeanticode's implements VideoFrameStruct and VideoInfoStruct. In the same file, VideoInfoAbi extends com.sun.jna.Union. Should I add jna?
gstreamer/MessageType.java upstream uses EXTENDED, codeanticode's MESSAGE_EXTENDED (I kept upstream version)

Should I clone codeanticode, then merge back upstream, or clone upstream, and then add codeanticode's changes?

@hamoid

@hamoid great! Not sure what you mean by adding JNA? JNA is already a requirement - nothing will work without it.

The changes will need to be made as pull requests upstream, but need to be split into specific functionality - ie. at least GL in one and Device stuff in another. Best thing is probably to stick with your fork for now and make sure everything is working, then I'll help with PR / merging upstream.

Actually, re-reading your question, if you want to supply the pull requests then probably better to clone upstream, then add codeanticode's changes in two or more branches - ie. a branch for GL and a branch for Device stuff, etc. Then make pull requests. Note that changes you make later to the branch will be reflected in the pull request, hence suggesting using specific branches for each.

Update: I made gst1-java-core build with codeanticode's changes before reading your comments. It was easy, so I'll do it again doing the branches and pull requests.

How do I decide what to put on the device branch, and what to put on the gl branch? I assume

../gl/ -> gl
../device/ -> device

But what about ../lowlevel/, Video*.java and many other files in the gstreamer/ folder?

@hamoid

@hamoid - Thanks for looking at this! 👍

The ../lowlevel/ folder is the actual native bindings. Basically, add only what is required to get the relevant branches to compile and run. Hopefully you won't end up with an overlap there - if you do, get back to me.

Again, with the Video*.java and other files in the gstreamer/root, add in only what is required for either branch to compile. There may be files not required by either branch - if they're not required to get the Processing Video library to work, let's deal with those later. If there are things there that don't directly relate to GL or Device, but are needed by the Processing Video library, again get back to me. Maybe @codeanticode can advise on exactly what is required, and what are non-requirement enhancements?

After two weeks vacation, I'm back :) I'll continue tomorrow.

@hamoid

@hamoid @neilcsmith-net Fantastic work on this issue, much appreciated! I started playing around to see if we can get this to a releasable state, and finally kiss 0.10 GStreamer goodbye for good. You find my WIP tree here (currently based on @hamoid's glAndDevice build): https://github.com/gohai/processing-video/commits/testing

BTW: Andres and I decided to rip out the half-baked GL code, since this was superseded by my GL Video library for the Raspberry Pi. So this means we can leave aside the Gl/opengl branch of gst1-java-core for now.

@gohai

@gohai great news! I am looking at pulling in the device probing stuff into a new gst1-java-core release soon. My personal fork has at least one memory leak fix. Let's make sure you have exactly what you need and get it into a full 0.9.1 release on Maven Central - I would really like you not to release with a snapshot! 😄

In terms of the device probing, when testing this I was confused why the video library is using a listener on the monitor bus rather than just calling getDevices()? I have a working example with device selection that seems to do what you'd want to do?

The OpenGL code will make its way into gst1-java-core very soon. While I agree that it would be good to get a feature comparable version of the Video library working with GStreamer 1.x sooner rather than later, the GL stuff is useful on other platforms too, so be good to not ignore completely. I for one will be looking to get OpenGL video support across as many platforms as possible into Praxis LIVE soon, so there's room for collaboration there.

@gohai

@gohai quick follow up - I think everything you need is now in the master branch of gst1-java-core. Please check and I'll release 0.9.1 ASAP.

@neilcsmith-net

@neilcsmith-net
I updated the gst1-java-core to your current master - thanks! Will definitely try to track your changes there and go for released versions.

Device probing: I made a note. But the much bigger issue for us is that GStreamer doesn't seem to have any DeviceMonitor backend for video devices on macOS, and I believe also Windows, currently. Which means that for the majority of P5 users we can't do list() :/

With your latest gst1-java-core we're still running into this incompatibility with playbin.getVideoSinkFrameRate(), btw. I stopped using it on our side for now.

@gohai

@gohai OK, I'll look to release 0.9.1 this coming week. I'll do some looking into device probing - latest GStreamer release notes suggest it should return something.

playbin.getVideoSinkFrameRate() is on my list to look at or remove. It currently relies on a changed API from GStreamer 0.10 -> 1.x. I haven't checked but I think you can get the framerate from the caps in the AppSink where you're currently getting width and height - assuming the struct returns Fraction OK?!

Return FlowReturn.OK not null or you may run into odd problems on certain platforms (the API was changed because of a bug report around this).

@neilcsmith-net

@neilcsmith-net I believe I saw something along those lines as well in the release notes, but this seems to only have been enumeration for Audio/Sink and Audio/Source on macOS - at least that are the only two devices that show up for me when I run gst-device-monitor-1.0.

Thanks for the pointers regarding framerate, I'll look into it.

@neilcsmith-net

@neilcsmith-net Getting the framerate from the caps worked out. I also made the change wrt FlowReturn.OK. Thanks!

I noticed two things: excessive seek() calls (via Processing's jump(), e.g. in the Scratch example) make the video become unresponsive after a while. Could there be a memory leak, or something sinister going on there? There is no logic to when it happens - we're not attempting to seek beyond eos, or something like that. Execution continues happily after the fateful seek(), but the NewSampleListener just won't get called any longer...

Any ideas?

The second observation was that I can't seem to be able to get a video file to play backwards in 1.x, and I remember that this issue also came up when I wrote the GL Video library. Curious: have you ever been successful doing that in Praxis Live?

@gohai

@gohai yes, both seek and reverse playback working fine (and you've seen it in action 😉 ). Although I'll test with the same video as the scratch example to be sure. The most relevant code is here - https://github.com/praxis-live/praxis/blob/master/praxis.video.gst1/src/net/neilcsmith/praxis/video/gst1/components/PlayBinDelegate.java#L126

Some things to note

the behaviour in the Scratch example of calling play, jump, pause is weird and might be causing an issue - try re-enabling the newPrerollListener and just scratching in the pause state. (if you keep it playing now does it continue working?)
all my calls down to GStreamer use Gst.getExecutor() to take things off the main Processing animation thread - getState() blocks (note my use of the timeout variant), although if you're sure execution is still happening afterwards that might not be the the problem with that particular example. It's still better not to block the animation thread waiting for GStreamer to do its thing though!
note the different seek calls for when the rate is reversed - you need to set the from and to values correctly (surprised this worked with 0.10, although maybe that behaviour did change)
a rate of zero will throw errors - note the hacky workaround! 😄

@gohai

@gohai for info, 0.9.1 is now tagged and available from the GitHub releases tab or Maven Central. Let me know if you run into any issues. Over to you now! 😄 Best wishes, Neil

@neilcsmith-net

@neilcsmith-net Thanks, integrated it here: https://github.com/gohai/processing-video

@hamoid

All: I built on top of the community came up with (thanks @hamoid @neilcsmith-net), and - 43 commits later - arrived at a version of the library that works with GStreamer 1.0 on all platforms and might be good enough for general use.

All details, including a link to a ZIP file for testing, can be found in this PR: #84

Please see if this works for you, and - if not - please consider sending small, self-contained PRs on top. I hope this way we can move this thing forward together!

@gohai

Many Thanks to all of you for all your hard work on this library!
I can report that I downloaded the @gohai code, copied it to the library folder and it works in Opensuse Tumbleweed with processing 3.3.7

I just noticed a funny bug: if I don't call background(0); in P2D mode, the program crashes with a timeout waiting to initialize the video.

Update: forget it. I can't reproduce it anymore.

Version 2.0 of the library (beta packages available under releases) uses gstreamer 1.x, so closing this issue.

Navigation Menu

Repository navigation

Ubuntu 16.10 no longer supports gstreamer0.10 #71

17 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Ubuntu 16.10 no longer supports gstreamer0.10 #71

Description

Activity

hamoid commented on May 2, 2017

hamoid commented on May 18, 2017

hamoid commented on Jun 3, 2017

codeanticode commented on Jun 4, 2017

hamoid commented on Jun 4, 2017

neilcsmith-net commented on Jun 4, 2017

hamoid commented on Jun 5, 2017

neilcsmith-net commented on Jun 5, 2017

hamoid commented on Jun 5, 2017

neilcsmith-net commented on Jun 5, 2017

neilcsmith-net commented on Jun 5, 2017

hamoid commented on Jun 9, 2017

hamoid commented on Jun 9, 2017

neilcsmith-net commented on Jun 9, 2017

hamoid commented on Jun 29, 2017

17 remaining items

gohai commented on Oct 31, 2017

neilcsmith-net commented on Nov 1, 2017

neilcsmith-net commented on Nov 3, 2017

gohai commented on Nov 5, 2017

neilcsmith-net commented on Nov 5, 2017

gohai commented on Nov 5, 2017

gohai commented on Nov 6, 2017

neilcsmith-net commented on Nov 7, 2017

neilcsmith-net commented on Nov 8, 2017

gohai commented on Nov 13, 2017

gohai commented on Nov 24, 2017

drnn1076 commented on Apr 23, 2018

hamoid commented on Apr 24, 2018

codeanticode commented on Apr 3, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions