Infinite LiveCode – The Second Stretch Goal – Full Cross Platform Audio Recording

The engine has long had high-level audio recording syntax provided by the record sound command and associated syntax:

record sound <filename>
record stop
record pause
record resume
the recording
the recordLoudness

the recordInput

the recordSampleSize
the recordChannels
the recordRate

the recordFormat
the recordCompression

the recordInputs
the recordFormats
the recordCompressionTypes

These pieces of syntax allow you to start and stop audio recording (to a specific file), control the low-level sample parameters (size, channels, rate) used to do the recording and specify what compression type to use and output file format.

However, this syntax has never been implemented for mobile and due to the gradual decline of QuickTime on both Mac and Windows platforms the current implementation isn’t really good enough to be widely useful.

During the port to Cocoa, we abstracted the implementation of the audio recording feature – indeed, we followed a ‘driver’ pattern where we defined a high-level C API which needs to be implemented to provide the audio recording feature. The ‘driver’ pattern is used where you have a piece of functionality you want to implement with a fixed (usually high-level) API, but where there might be multiple different implementations. In this case, the different implementations are a necessity as the way you record sound on different platforms uses vastly different system functions. Therefore, you separate out what you need to do into some high-level code which then calls into a well defined lower-level collection of functions. In the case of the audio recording feature in the engine, it sits upon this set of C functions:

void MCPlatformSoundRecorderCreate(MCPlatformSoundRecorderRef& r_recorder);
void MCPlatformSoundRecorderRetain(MCPlatformSoundRecorderRef recorder);
void MCPlatformSoundRecorderRelease(MCPlatformSoundRecorderRef recorder);
bool MCPlatformSoundRecorderIsRecording(MCPlatformSoundRecorderRef recorder);
double MCPlatformSoundRecorderGetLoudness(MCPlatformSoundRecorderRef recorder);
bool MCPlatformSoundRecorderStart(MCPlatformSoundRecorderRef recorder,
        MCStringRef filename);
void MCPlatformSoundRecorderStop(MCPlatformSoundRecorderRef recorder);
void MCPlatformSoundRecorderPause(MCPlatformSoundRecorderRef recorder);
void MCPlatformSoundRecorderResume(MCPlatformSoundRecorderRef recorder);
bool MCPlatformSoundRecorderListInputs(MCPlatformSoundRecorderRef recorder,
        MCPlatformSoundRecorderListInputsCallback callback, void *context);
bool MCPlatformSoundRecorderListCompressors(MCPlatformSoundRecorderRef recorder,
        MCPlatformSoundRecorderListCompressorsCallback callback, void *context);
void MCPlatformSoundRecorderGetConfiguration(MCPlatformSoundRecorderRef recorder,
        MCPlatformSoundRecorderConfiguration& r_config);
void MCPlatformSoundRecorderSetConfiguration(MCPlatformSoundRecorderRef recorder,
        const MCPlatformSoundRecorderConfiguration& config);

These C functions are what the upper parts of the engine call when the audio recording syntax is used – thus to implement audio recording for a specific platform you implement those functions using the appropriate APIs on the platform and all the high-level syntax in Script should work!

Each platform we support has its own way of managing audio recording:

• iOS and Mac use the AVAudioRecorder class
• Android uses the MediaRecorder class
• Windows uses the DirectShow APIs
• Linux uses gstreamer

These are all different foreign APIs we need to wrap!

For this Infinite LiveCode stretch goal we will:

• Tweak the engine to enable the ‘platform sound recorder’ abstraction in the engine to be written in Builder.
• Wrap the necessary foreign APIs on each platform.
• Implement the engine platform sound recorder interface for each platform in Builder.

This will mean that audio recording will work uniformly, and well, on all the platforms we currently support – and break the final dependence we have on QuickTime!

Further to the actual code, we will also make this feature ‘fully worked’ – we’ll provide associated documentation and training materials explaining how it all works so that you can start learning how to implement similar things yourselves. This will be a gold mine for seeing how to work with a range of differently constructed APIs which solve the same problem, as well as giving you insight into how different platforms offer these features.

Finally, remember, that this will all be open source code, built and included in the community edition – you will be able to take it, adapt it and help improve it!

We only need $8k more to ensure this stretch goal is met. Pledge for infinite LiveCode today.