The current project I am working on is about multimedia on MacOS. It includes replacing all multimedia functionality (in the player object, revVideoGrabber external etc) in LiveCode to use the latest Mac APIs (AVFoundation, introduced in OSX 10.7). All these multimedia capabilities rely on the QuickTime/QTKit APIs. But Apple recently deprecated QuickTime APIs. This means that apps that use QuickTime will be rejected in the AppStore submission.
So, after transitioning the existing Quicktime/QTKit code to AVFoundation, users should be able to submit a multimedia based app to the AppStore for their OS 10.7 customers and above.
The problem is that the new APIs do not provide the same functionality as the original QuickTime API, so we must ensure that we will provide our users with the same feature set they are currently used to. For example, the player control bar. QTKit’s controller is broken in OSX 10.7+ and AVFoundation does not have a controller until OSX 10.9. This means that we need to write our own custom controller.
I have almost finished implementing the custom controller, at least as far as the functionality is concerned. It probably needs a bit more work to look prettier!
Below are both the QTKit controller and our custom controller, as implemented so far. Note that the selection handles are broken in QTKit controller on OSX 10.7+
- QTKit controller for multimedia playback
- Custom controller for multimedia playback
The next stage is to update the LiveCode player object and all other multimedia functionality in the engine to use the newest AVFoundation APIs, as well as to implement a QTKit/AVFoundation version of the revVideoGrabber, since it too relies on the deprecated QuickTime API.
23 comments
Join the conversationrunrev - May 15, 2014
Panos talks on the new multimedia functionality in LiveCode, replacing outdated Quicktime APIs with AVFoundation http://t.co/0FQpU5JJjm
Faber Muñoz Perez - May 15, 2014
Good (y)
Paul Dupuis - May 15, 2014
I prefer yours. We heavily rely on the ability to have the user set start and end points for media clips that we can then read via script, store the selected section, and play it back on request (or export as a new media file via ExtendedQT external) at a later time. Your in/out point markers are much clearer than the current controller. Please consider adapting the “export” command syntax to exporting media clips (just like parts of images) as well in new media APIs.
App_Insight_ - May 15, 2014
“@runrev: replacing outdated Quicktime APIs with AVFoundation http://t.co/PPHwozTVIh” Yess! So long QuickTime!
Faber Muñoz Perez - May 15, 2014
Faber Muñoz Perez liked this on Facebook.
Devin Asay - May 15, 2014
How will multimedia functionality be implemented on other platforms, particularly Windows and Linux, now that Quicktime has been EOL’ed?
Ben - May 20, 2014
Hi Devin. Good question. Hopefully my response to Richards question above answers yours.
Josh - May 15, 2014
Are there any plans to implement any of the additional functionality of Trevor’s wonderful Enhanced QT External, for example, the ability to select a part of a movie in a player, copy, create a new player object, paste, then save as a new movie file?
Ben - May 20, 2014
Hi Josh. It is not in our immediate plans but as we expand our development staff it becomes easier to implement additional features like this. We’re working hard to open up the platform (open language) so that it becomes much easier for community members to write libraries and hook into lower level libraries so perhaps someone in the community will beat us to it!
lemvavoulo - May 15, 2014
LiveCode | Multimedia on MacOS: http://t.co/4n7L0dPCyr
Tuviah - May 15, 2014
QTKit is depreciated too as of last year. If was announced at WWDC and covered in a session. Everyone must move to AVKit and AVFoundation.
Dar - May 15, 2014
I’m glad this is moving along.
Kids love sounds when making games. (Who doesn’t?) They want overlapping sounds and multiple instances.
ben - May 20, 2014
Hi Dar, thanks for this feature request. It is in our plans though our first priority is to replace the existing basic multimedia across all platforms. There is multichannel sound on iOS and Android but we’d like to bring that to Desktop too.
Ouioui Leo - May 15, 2014
Ouioui Leo liked this on Facebook.
Richard Gaskin - May 16, 2014
Can you describe how this new player helps the process toward similar feature parity for the other 89% of our customers running Windows and Linux?
Ben - May 20, 2014
Hi Richard. The work on multimedia in 6.7 has seen us abstract the multimedia features into what we are calling the ‘platform dependent layer’ and reorganise the player object itself. We’ll be able to move on after 6.7 and implement the same platform dependent component for Windows Media and Linux. It is slightly less clear which API’s we’ll use on Linux as we’d like LiveCode to work out of the box. We have discussed including a Linux multimedia API with the source but there are licensing issues. We’ll keep the community posted as that project is planned for the other two platforms.
Peter Thirkell - May 18, 2014
Great to see this moving along. Will there be an option to use the “native” AV Foundation controller for users with Mavericks 10.9 (and above)?
Ben - May 20, 2014
Hi Peter, we’ll take a look and see what is possible. Thanks for the feature request.
Panagiotis Merakos - May 20, 2014
One of our kickstarter goals is to redo our multimedia across all platforms. The work we are working on for Mac will eventually be brought to Windows and Linux 🙂
Quentin Brown - June 3, 2014
Interesting to note that it was mentioned above that QTkit has been depreciated now for AVkit but this has not been addressed in any replies. It seems the goalposts are always moving but I hope we will not end up with an out of date platform before 7 is finished.
How much will the new platform dependant layer get in the way of someone wanting to develop something like a platform Independant player/multimedia library based on VLC/libAV/ffmpeg? Will they have to create new and separate methods and objects to use their library for multimedia or will there be some hooks provided to send calls to the native objects and multimedia features to be parsed by an alternative library? – the latter would seem preferable for many reasons – you could adapt existing apps to transition to other alternative libraries more easily as they became available or even have team members developing the new library whilst others are building the app to use them using the existing multimedia tools and then just changing a single line of code to use or test it with the new library..
Ben - June 4, 2014
Hi Quentin,
The platform layer is an abstraction that is designed to open up the implementation possibilities allowing us/you to connect to platform API’s or tie into independent libraries. Our eventual goal is to make it ‘plug-able’, with a LiveCode developer able to choose which module to load or even add their own.
In regards to the new implementation of multimedia on mac, we have a QTkit implementation for 10.6 and an AVFoundation implementation underway for 10.7 and above. It will be using the latest APIs when it is released.
Charles S - June 12, 2014
It’s great to hear that you’re working on media APIs. I was highly dependent on Livecode + EnhancedQT (as well as SMIL which is total legacy) as the magic behind my caption and video description editor, CapScribe (see http://www.inclusivemedia.ca/services/capscribe.shtml ). I’ve been working on CapScribe for the past 12 years. Things have been breaking down of late with the latest livecode releases and I’ve had to stop development.
Previously, EnhancedQT let me determine if video was still playing or not, insert text tracks into a QT file in order to provide captioning, and even embed text tracks into video.
Sadly text tracks in QT X are deprecated. Apple provides other ways for adding captions to QT files but I haven’t been able to access those approaches as they aren’t straight forward.
I also used EnhancedQT to add video description to QT video. This involved attaching additional audio clips either over existing QT video or spliced into the video.
Anything you’re doing to support these kinds of things at a code level would be greatly appreciated.
thanks,
Charles
Frustrated - April 19, 2016
What is the status of this effort, especially in light of Apple’s declaring QuickTime EOL with no further support? I depend heavily on the “player” function in LC picking up a UDP multicast stream and displaying in a window within a card, and without QT I cannot find an alternative that works.