Racing to a Temporary Fix

Recently, I’ve been spending most of my time working on a Secret Project* for LiveCode 8.0.  But the remainder has been spent on my other areas of interest: Linux, server-side LiveCode, and security. Today, we’re going to touch on all of those topics.

Temporary files

You will often find yourself using temporary files. For example, sometimes your app will need to run some other program, and will need to provide it with an input file.  Or perhaps you receive data over the network, and need somewhere to stash it while waiting to decide whether to put the data into long-term storage.

LiveCode provides some tools for working with temporary files. In particular, there is the tempName function, which generates a uniquely-generated filename.

For example, if you write

put the tempName

in the message box on Linux, you might see something like:

/tmp/filevZr4DG

So, the simplest way to create a temporary file for your LiveCode server app to use might be something like:

-- Don’t do this
put the tempName into tTempFile -- 1. Generate a filename 
open file tTempFile for write -- 2. Open the file

Why do I say not to do this?  It’s because of a very “classic” programming problem called a “race condition”.  Read on for an explanation of what this is, how it can affect your app, and how to avoid it.

Race conditions

Modern operating systems can run more than one program at a time.  There are two ways in which they can do it:

1. if there are multiple CPUs available, then each of them can simultaneously run a program
2. the operating system can switch the program that each CPU runs at any time (this is called “pre-empting” a program)

Now suppose two programs are running, and suppose that there’s some resource that only one program can access at a time. If the two programs try and access the resource at the same time, then the one that happens to be very slightly faster will “win” the “race” and get to use the resource, while the slower program has to wait its turn.

File creation is a common cause of race conditions. Recall that if you open a file for writing. e.g. with

open file tFileName for write

but the file called tFilename doesn’t exist yet, the operating system will create it for you. But if another program creates a file called tFilename just before you try to open it, you’ll open that file instead of getting a newly-created file.

Let’s go back to the temporary file example, this time with some more comments.

-- *Please* don’t do this
put the tempName into tTempFile -- 1. Generate a filename

At this point, you know that tTempFile doesn’t exist. But don’t forget that other programs might be running on another CPU, or that your app might get pre-empted after step 1, and get stopped for a while.  This could mean that by the time that the next step happens:

open file tTempFile for write -- 2. Open the file

tTempFile might now exist, having been created by another program.

On Linux, and in certain circumstances on other platforms, any program running as any user can create files in the /tmp/ directory.  There are many examples of Bad Things that can happen if someone can control the files that your app writes to, including loss of data or even damage to your operating system.

You can find more in-depth discussion of this problem and some examples of the aforementioned Bad Things in the Mac Developer Library, and much much much more in David Wheeler’s Secure Programs HOWTO.

What can we do about it?

Fortunately, there’s something that can be done to work around the problem: you can use a temporary folder instead of a temporary file. However, the procedure is slightly more involved.  Let’s go through it step-by-step.

-- 1. Make sure that other users can’t read/write into the
--    directories you create
set the convertOctals to true
put the umask into tSaveUmask
set the umask to “0077”

There’s a special property called the umask which can be used control the security settings on the files and folders that your app creates.  Setting it to “0077” will ensure that only the user that your app’s running as can access any of the files or folders that your app creates (apart from system administrators).  Note that you need to enable the convertOctals first (or the security settings won’t be applied properly).  In this example we save the previous value of the umask so that we can restore it at the end.

-- 2. Keep trying to create temporary directories until we get
--    one that doesn’t exist
local tTempDir
repeat forever
 try
   put the tempName into tTempDir
   create folder tTempDir
   exit repeat
 catch e
   if there is a folder tTempDir then next repeat
   if there is a file tTempDir then next repeat
   throw e
 end try
end repeat

Unlike the open file command, the create folder command always causes an error if the requested folder already exists.  This way we can remove the race condition!  If another program simultaneously attempts to create the same tTempDir or a file with the same name, then we can easily detect it in the catch and try a new folder name. Of course, if some other error occurs, such as running out of disk space, then it is important to make sure not to block it.

-- 3. Now we can create temporary files in that directory
put tTempDir & “/tempFileNameHere” into tTempFile
open file tTempFile
-- ... and now use the temporary file! ... --

With the temporary directory having been successfully and securely created, you can go ahead and create as many files in it as you like, without any need to generate filenames or worry about race conditions (because no other non-administrator users can read or write the tTempDir).

-- 4. Reset the umask
set the umask to tSaveUmask

Hopefully this step is quite self-explanatory, unlike anything else I’ve written today.

Conclusion

Even something as simple as creating a temporary file can pose unexpected hazards, but LiveCode provides the tools to steer safely around them. For server-side LiveCode apps running on Linux, I recommend using temporary directories rather than temporary files to improve robustness and security.  For other apps on other platforms, the simple approach may be enough most of the time, but using a temporary directory won’t hurt!