If you’re like me, and you come to LiveCode from more “mainstream” programming languages, such as Java (8 or 9) or Python (heck, even C++), then you might be missing some of the very nice functional features for operating on Arrays and Lists, such as maps, filters or even list comprehensions. In this blog post I’ll try and answer the question, whether implementing functional features in LiveCode is a dream?
Before we get to the code, we need to establish a few prerequisites : the most important element when dealing with these features is the ability to pass around function references to use for execution (or even lambdas in some languages). Now unfortunately, in LCS this is not available natively, so we need to settle for a workaround. Essentially, we will be using the “dispatch” and “do” methods of the language to emulate function references.
Ok, so let’s start by implementing list comprehensions (with quite a number of assumptions, but fully functional within these assumptions). If we look at python’s way of writing a list comprehension, it would be something like
def some_function(array):
return [x + 2 for x in array] The actual list comprehension in this snippet is
[x + 2 for x in array]It is our goal to replicate this syntax as closely as possible. To be more precise, we would like to write an interface that accepts and evaluates expressions of the following form:
[LCS_EXPRESSION for VAR_NAME in ARRAY_VAR]Since we’re starting from scratch in LCS implementation, we need to first parse such an expression into its main components. If we dissect it, we can observe that it contains two major components:
The expression that is applied to each element : x + 2
The generator from which we draw the x’s : for x in array
Now, here comes the first assumption that we’re making about the code : the source array must be available in the scope of the handler that contains the code for the comprehension.
Considering this, we can write the following code to parse this expression into its components:
function parseComprehension pComprehension
local tExpression, tGenerator
set the itemdelimiter to " for "
put char 2 to -1 of item 1 to -2 of pComprehension into tExpression
put char 1 to -2 of item -1 of pComprehension into tGenerator
local tReturnObject, tSourceObjectName, tSourceObject
set the itemdelimiter to space
put item -1 of tGenerator into tSourceObjectName
do "return " && tSourceObjectName in caller
put the result into tReturnObject["Source"]
put tExpression into tReturnObject["Expression"]
return tReturnObject
end parseComprehension
Now as you may very well notice, in this case we are not doing any parsing of the source expression. This is because we assume that that is valid LCS code and so we can just pass the parsing down to the LCS engine by using the “do” syntax. Essentially, the only fancy thing that goes on here is dumping the contents of our source array into a more generically named object to ensure what in specialty terms is called alpha equivalence.
Now that we parsed the expression, we need a way to get the result. A first implementation might look something like:
function executeComprehension pParsedComprehension
local tResult, x
-- run through all the elements of the array
repeat with tIdx = 1 to the number of elements in pParsedComprehension["Source"]
put pParsedComprehension["Source"][tIdx] into x
-- execute the code, and put into its place in the new array
put value(pParsedComprehension["Expression"]) into tResult[tIdx]
end repeat
return tResult
end executeComprehensionNow the more attentive reader will notice here that we assume that the variable name is “x”. This is somewhat of an unfair assumption to make, so we can extend the code with a separate function, that takes care of this by replacing whatever name the user gave to the variable with a predefined one.
This function would look something like
function substituteComprehension pExpression, pGenerator
set the itemdelimiter to "in"
local tVarName
put word 1 of item 1 of pGenerator into tVarName
return replaceText(pExpression, tVarName, "tVariable65537")
end substituteComprehension The choice of the variable name with which we make the substitution is tVariable65537, which is just formed of tVariable and a prime number large enough for it to be improbably used in another context.
Again, this is a function that we need to plug in to the parser, an updated version of which would be
function parseComprehension pComprehension
local tExpression, tGenerator, tSubstitutedExpression
set the itemdelimiter to " for "
put char 2 to -1 of item 1 to -2 of pComprehension into tExpression
put char 1 to -2 of item -1 of pComprehension into tGenerator
-- this is new, use the substitution mechanism to generate a new expression
put substituteComprehension(tExpression, tGenerator) into tSubstitutedExpression
local tReturnObject, tSourceObjectName, tSourceObject
set the itemdelimiter to space
put item -1 of tGenerator into tSourceObjectName
do "return " && tSourceObjectName in caller
put the result into tReturnObject["Source"]
-- notice here that we’re returning the new expression
put tSubstitutedExpression into tReturnObject["Expression"]
return tReturnObject
end parseComprehension Updating the runner is a simpler task :
function executeComprehension pParsedComprehension
local tResult, tVariable65537
repeat with tIdx = 1 to the number of elements in pParsedComprehension["Source"]
put pParsedComprehension["Source"][tIdx] into tVariable65537
put value(pParsedComprehension["Expression"]) into tResult[tIdx]
end repeat
return tResult
end executeComprehensionOk, now in order to give it a more humane interface, let’s create a wrapper. This is something that’s quite easy, we just do:
function comprehension pExpression
local tComp
global gExpressionToParse
put pExpression into gExpressionToPars
do "global gExpressionToParse; return parseComprehension(gExpressionToParse)" in caller
put the result into tComp
delete variable gExpressionToParse
return executeComprehension(tComp)
end comprehension The only trick here is that in order to get access to the scope of the caller, we need to do some manipulations with globals in order to properly access the variables that we wish.
All in all, this means we can get the result of executing a list comprehension just by doing:
put comprehension("[y * 2 for y in tA]") into tExecutionResultWhich, I hope you would agree is a very close form to what we have in python.
Let’s now have a look at the map function. Map is a function that takes an array and a unary function, returning a new array whose elements are obtained by applying that unary function elementwise to the initial array.
Using the dispatch mechanism, that is quite easy to implement:
function map pArray, pMethodName
local tResult
repeat with tIdx = 1 to the number of elements of pArray
dispatch function pMethodName to me with pArray[tIdx]
put the result into tResult[tIdx]
end repeat
return tResult
end map
Now someone might rightfully argue that implementing 6 lines of code into a separate function might be a waste of time, but speaking from experience I can say that especially as someone who has worked in python for a long time can find the absence of such features quite annoying, as they are baked in our workflow.
To end on a more filtered note, we can go ahead and implement filter, which is a close cousin of map, with one small difference. Filter returns a new array whose elements are members of the initial array that satisfy a given unary predicate. We would implement this something like:
function funcFilter pArray, pPredicateName
local tResult, tNewCount
put 1 into tNewCount
repeat for each element tElem in pArray
dispatch function pPredicateName to me with tElem
if the result then
put tElem into tResult[tNewCount]
add 1 to tNewCount
end if
end repeat
return tResult
end funcFilter
Notice how we named the function funcFilter, because filter is an existing keyword in LiveCode and so it would have clashed.
The sharp reader (or the one who has a lot of experience with functional features) might notice that it is possible to express maps and filters as list comprehensions, so we let’s try and implement that as a final step.
When it comes to maps, it’s quite easy to implement a map version that uses a list comprehension. All we need to do is:
function mapAsComprehension pArray, pMethodName
put pArray into tBuffer
local tComprehensionString
put "["& pMethodName & "(x) for x in tBuffer]" into tComprehensionString
return comprehension(tComprehensionString)
end mapAsComprehension The only special thing that we are doing here is that we need to store a reference to pArray in tBuffer, which is expected to be accessible within the scope of the script that is using this function.
Indeed, we can now see that list comprehensions are just syntactic sugar for maps. Any list comprehension
[f(x) for x in tA]
can be expressed as
map(tA, f)
To use the python notation.
When it comes to filters, things get slightly more complicated, due to the fact that our current implementation of the list comprehension mechanism is limited to simple generators, that do not allow any conditions to be imposed on the elements that are drawn from the source array.
[LCS_EXPRESSION for VAR_NAME in GLOBAL_VAR ]We can somewhat easily do this by extending the parsing and execution functions like this:
function parseComprehensionWithIf pComprehension
local tExpression, tGenerator, tSubstitutedExpression
set the itemdelimiter to " for "
put char 2 to -1 of item 1 to -2 of pComprehension into tExpression
put char 1 to -2 of item -1 of pComprehension into tGenerator
put substituteComprehension(tExpression, tGenerator) into tSubstitutedExpression
local tReturnObject, tSourceObjectName, tSourceObject
set the itemdelimiter to space
local tString
put "" into tString
if item 5 of tGenerator is "if" then
repeat with tIdx = 6 to the number of items of tGenerator
put item tIdx of tGenerator & space after tString
end repeat
put substituteComprehension(tString, tGenerator) into tReturnObject["Condition"]
put item 4 of tGenerator into tSourceObjectName
else
put item -1 of tGenerator into tSourceObjectName
end if
do "return " && tSourceObjectName in caller
put the result into tReturnObject["Source"]
put tSubstitutedExpression into tReturnObject["Expression"]
return tReturnObject
end parseComprehensionWithIfAnd update the execution mechanism to:
function executeComprehensionWithIf pParsedComprehension
if pParsedComprehension["Condition"] is empty then
return executeComprehension(pParsedComprehension)
end if
local tResult, tVariable65537, tCounter, tChecker
put 1 into tCounter
repeat with tIdx = 1 to the number of elements in pParsedComprehension["Source"]
put pParsedComprehension["Source"][tIdx] into tVariable65537
do "put " && pParsedComprehension["Condition"] && " into tChecker"
if tChecker then
put value(pParsedComprehension["Expression"]) into tResult[tCounter]
add 1 to tCounter
end if
end repeat
return tResult
end executeComprehensionWithIf
Now, for those of you who might be interested in playing around with this, I’m providing a sample stack that contains all these functions embedded into the stack code.
As a conclusion, I think we’ve definitely established that those people who move over to LiveCode from other languages such as Python can definitely use LiveCode in a not so complicated manner to get some of their creature comforts back, leading me to saying that functional features in LiveCode are NOT a dream 🙂 Indeed, the only bits that seem to be missing are maps/reduces and filtering arrays on more than just regular expressions (functionality that is already built in)
P.S. If you’re looking into switching to LiveCode, make sure to have a look at our cheat sheets (https://livecode.com/resources/cheat-sheets/) 🙂
Join the conversation