Android audio recording library using Java FFI

In this example, we show you how to implement audio recording on Android using the new Java FFI capabilities in the latest DP of LiveCode 9.

The first step in figuring out how to create an Android audio recording library is to figure out what classes and methods are needed, which is tantamount to having a quick look at how it would be done in native code.

The API documentation for the MediaRecorder class has the following sample code:

MediaRecorder recorder = new MediaRecorder();
 recorder.setAudioSource(MediaRecorder.AudioSource.MIC);
 recorder.setOutputFormat(MediaRecorder.OutputFormat.THREE_GPP);
 recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
 recorder.setOutputFile(PATH_NAME);
 recorder.prepare();
 recorder.start();   // Recording is now started
 ...
 recorder.stop();
 recorder.reset();   // You can reuse the object by going back to setAudioSource() step
 recorder.release(); // Now the object cannot be reused

So in fact everything we need is included in the MediaRecorder class. We could wrap the entire class using an lcb-java specification and using the lc-compile-ffi-java tool, eg:

foreign package android.media

class MediaRecorder

constructor MediaRecorder()

final class method getAudioSourceMax() returns int
method getMaxAmplitude() returns int
method pause()
method prepare()
...

but this class is simple enough that we can just wrap the individual methods by hand.

This is going to be a library in the first instance, so we’ll start with a simple library shell:

library com.livecode.library.androidaudiorecorder

metadata version is "1.0.0"
metadata author is "LiveCode"
metadata title is "Android Audio Recorder"

-- It's safe to assume that any lcb module that uses java FFI bindings 
-- will need to use the following two modules:
use com.livecode.foreign
use com.livecode.java

end library

We wrap the MediaRecorder constructor using the binding string "java:android.media.MediaRecorder>new()". This uses new as the method name to indicate to the LCB VM that we want to call the constructor.

-- Define the foreign handler that binds to the MediaRecorder 
-- constructor
foreign handler _JNI_MediaRecorderConstructor() returns JObject binds to "java:android.media.MediaRecorder>new()"

We will need bindings for each of the methods we want to call on the MediaRecorder instance, namely

• setAudioSource(source as int)
• setOutputFormat(format as int)
• setAudioEncoder(encoder as int)
• setOutputFile(fileName as String)
• prepare()
• start()
• stop()
• reset()

All of these are instance methods of the MediaRecorder class with no return value. We wrap them as follows:

foreign handler _JNI_MediaRecorderSetRecordInput(in pRecorder as JObject, in pSource as JInt) returns nothing binds to "java:android.media.MediaRecorder>setAudioSource(I)V"
foreign handler _JNI_MediaRecorderSetRecordFormat(in pRecorder as JObject, in pFormat as JInt) returns nothing binds to "java:android.media.MediaRecorder>setOutputFormat(I)V"
foreign handler _JNI_MediaRecorderSetRecordEncoder(in pRecorder as JObject, in pEncoder as JInt) returns nothing binds to "java:android.media.MediaRecorder>setAudioEncoder(I)V"
foreign handler _JNI_MediaRecorderSetRecordOutputFile(in pRecorder as JObject, in pFile as JString) returns nothing binds to "java:android.media.MediaRecorder>setOutputFile(Ljava/lang/String;)V"
foreign handler _JNI_MediaRecorderPrepare(in pRecorder as JObject) returns nothing binds to "java:android.media.MediaRecorder>prepare()V"
foreign handler _JNI_MediaRecorderStart(in pRecorder as JObject) returns nothing binds to "java:android.media.MediaRecorder>start()V"
foreign handler _JNI_MediaRecorderStop(in pRecorder as JObject) returns nothing binds to "java:android.media.MediaRecorder>stop()V"
foreign handler _JNI_MediaRecorderReset(in pRecorder as JObject) returns nothing binds to "java:android.media.MediaRecorder>reset()V"

Because they are all instance methods of the class, the foreign handler takes JObject as its first argument – this is the instance of the class on which to call the given method. Wrapped in this way, calling _JNI_MediaRecorderStart(mRecorder) is just the same as recorder.start() in Java (albeit going through the JNI rather than calling directly).

There are a very limited selection of java types here, so the parameter codes in the binding strings are relatively straightforward

• ()V is a method with no parameters and no return value
• (I)V is a method with a (Java) int parameter and no return value
• (Ljava/lang/String;)V is a method with a (Java) String parameter and no return value

All that’s left to replicate the simple example in the Android API is to find out the actual (integer) values of the constants MediaRecorder.AudioSource.MIC, MediaRecorder.OutputFormat.THREE_GPP, and MediaRecorder.AudioEncoder.AMR_NB. These can be found on the AudioSource, OutputFormat and AudioEncoder API docs. In this case, these constants all map to 1.

Putting it all together gives us two handlers that we want to make available in LiveCode Script:

-- This will store an instance of a MediaRecorder
private variable mRecorder as optional JObject

public handler AndroidStartRecording(in pFileName as String)
	unsafe
		-- Don't create a new recorder object if there already is one
		if mRecorder is nothing then
			put _JNI_MediaRecorderConstructor() into mRecorder
		end if
		_JNI_MediaRecorderSetRecordInput(mRecorder, 1)
		_JNI_MediaRecorderSetRecordFormat(mRecorder, 1)
		_JNI_MediaRecorderSetRecordEncoder(mRecorder, 1)
		
		-- Convert the filename to a java string
		variable tFileName as JString
		put StringToJString(pFileName) into tFileName
		
		_JNI_MediaRecorderSetRecordOutputFile(mRecorder, tFileName)
		_JNI_MediaRecorderPrepare(mRecorder)
		_JNI_MediaRecorderStart(mRecorder)
	end unsafe
end handler

public handler AndroidStopRecording(in pFileName as String)
	-- Don't do anything if there is no recorder object
	if mRecorder is nothing then
		throw "recording has not been started!"
		return
	end if
	
	unsafe
		_JNI_MediaRecorderStop(mRecorder)
		_JNI_MediaRecorderReset(mRecorder)
	end unsafe
end handler

There is one final piece of the puzzle here, which is that recording audio on Android requires a permission in the manifest. We can add this as metadata to the library as follows:

metadata android.permissions is "RECORD_AUDIO"

Et voila, a simple Android audio recording library.

library com.livecode.library.androidaudiorecorder

metadata version is "1.0.0"
metadata author is "LiveCode"
metadata title is "Android Audio Recorder"

metadata android.permissions is "RECORD_AUDIO"

use com.livecode.foreign
use com.livecode.java

foreign handler _JNI_MediaRecorderConstructor() returns JObject binds to "java:android.media.MediaRecorder>new()"
foreign handler _JNI_MediaRecorderSetRecordInput(in pRecorder as JObject, in pSource as JInt) returns nothing binds to "java:android.media.MediaRecorder>setAudioSource(I)V"
foreign handler _JNI_MediaRecorderSetRecordFormat(in pRecorder as JObject, in pFormat as JInt) returns nothing binds to "java:android.media.MediaRecorder>setOutputFormat(I)V"
foreign handler _JNI_MediaRecorderSetRecordEncoder(in pRecorder as JObject, in pEncoder as JInt) returns nothing binds to "java:android.media.MediaRecorder>setAudioEncoder(I)V"
foreign handler _JNI_MediaRecorderSetRecordOutputFile(in pRecorder as JObject, in pFile as JString) returns nothing binds to "java:android.media.MediaRecorder>setOutputFile(Ljava/lang/String;)V"
foreign handler _JNI_MediaRecorderPrepare(in pRecorder as JObject) returns nothing binds to "java:android.media.MediaRecorder>prepare()V"
foreign handler _JNI_MediaRecorderStart(in pRecorder as JObject) returns nothing binds to "java:android.media.MediaRecorder>start()V"
foreign handler _JNI_MediaRecorderStop(in pRecorder as JObject) returns nothing binds to "java:android.media.MediaRecorder>stop()V"
foreign handler _JNI_MediaRecorderReset(in pRecorder as JObject) returns nothing binds to "java:android.media.MediaRecorder>reset()V"

-- This will store an instance of a MediaRecorder
private variable mRecorder as optional JObject

public handler AndroidStartRecording(in pFileName as String)
	unsafe
		-- Don't create a new recorder object if there already is one
		if mRecorder is nothing then
			put _JNI_MediaRecorderConstructor() into mRecorder
		end if
		_JNI_MediaRecorderSetRecordInput(mRecorder, 1)
		_JNI_MediaRecorderSetRecordFormat(mRecorder, 1)
		_JNI_MediaRecorderSetRecordEncoder(mRecorder, 1)
		
		-- Convert the filename to a java string
		variable tFileName as JString
		put StringToJString(pFileName) into tFileName
		
		_JNI_MediaRecorderSetRecordOutputFile(mRecorder, tFileName)
		_JNI_MediaRecorderPrepare(mRecorder)
		_JNI_MediaRecorderStart(mRecorder)
	end unsafe
end handler

public handler AndroidStopRecording(in pFileName as String)
	-- Don't do anything if there is no recorder object
	if mRecorder is nothing then
		throw "recording has not been started!"
		return
	end if
	
	unsafe
		_JNI_MediaRecorderStop(mRecorder)
		_JNI_MediaRecorderReset(mRecorder)
	end unsafe
end handler
end library

To provide more options on input sources, encoders and output formats you could use array literals for the mapping from sources to their defined values

constant kRecordInputs is { \
 "Default" : 0, \
 "Mic" : 1, \
 "VoiceUplink" : 2, \
 "VoiceDownlink" : 3, \
 "VoiceCall" : 4, \
 "Camcorder" : 5, \
 "VoiceRecognition" : 6, \
 "VoiceCommunication" : 7, \
 "RemoteSubmix" : 8, \
 "Unprocessed" : 9 \
}
constant kRecordCompressionTypes is { \
 "Default" : 0, \
 "AMR NB" : 1, \
 "AMR WB" : 2, \
 "AAC" : 3, \
 "HE AAC" : 4, \
 "AAC ELD" : 5, \
 "Vorbis" : 6 \
}
constant kRecordFormats is { \
 "ThreeGPP" : 1, \
 "MPEG-4" : 2, \
 "AMR WB" : 4, \
 "AAC ADTS" : 6, \
 "WebM" : 9 \
}

and handlers to allow these to be set, for example:

public handler AndroidSetRecordInput(in pInputSource as String)
	if pInputSource is not among the keys of kRecordInputs then
		throw "unrecognised source"
		return
	end if
	
	unsafe
		if mRecorder is nothing then 
			put _JNI_MediaRecorderConstructor() into mRecorder
		end if
	
		_JNI_MediaRecorderSetRecordInput(mRecorder, kRecordInputs[pInputSource])
	end unsafe
end handler

You may also want to create a LiveCode Script wrapper that uses the current values of the recordChannels, the recordRate and the recordSampleSize global properties, by adding a handler

foreign handler _JNI_MediaRecorderSetRecordChannels(in pRecorder as JObject, in pChannels as JInt) returns nothing binds to "java:android.media.MediaRecorder>setAudioChannels(I)V"
foreign handler _JNI_MediaRecorderSetRecordRate(in pRecorder as JObject, in pRate as JInt) returns nothing binds to "java:android.media.MediaRecorder>setAudioSamplingRate(I)V"
foreign handler _JNI_MediaRecorderSetRecordSampleSize(in pRecorder as JObject, in pRate as JInt) returns nothing binds to "java:android.media.MediaRecorder>setAudioEncodingBitRate(I)V"

public handler AndroidSetRecordConfig(in pChannels as Integer, in pRate as Real, in pSampleSize as Integer)	
	unsafe
		if mRecorder is nothing then 
			put _JNI_MediaRecorderConstructor() into mRecorder
		end if
		_JNI_MediaRecorderSetRecordChannels(mRecorder, pChannels) 
		_JNI_MediaRecorderSetRecordRate(mRecorder, pRate)
		_JNI_MediaRecorderSetRecordSampleSize(mRecorder, pSampleSize)
	end unsafe
end handler

to your LCB library, and calling

AndroidSetRecordConfig the recordChannels, the recordRate, the recordSampleSize

in the LCS wrapper.