Removing Quicktime Dependency from LiveCode’s Player Object

by Ian Macphail on May 26, 2016 4 comments

A recent blog post from computer security firm Trend Micro announced that they had found 2 potential security vulnerabilities in QuickTime for Windows, and that a fix for those issues would not be forthcoming as Apple have withdrawn support for the Windows version of QuickTime. Consequently they recommend Uninstalling QuickTime for Windows as soon as possible. This is also the recommendation of US-CERT, the US government’s Computer Emergency Readiness Team.

Links:
Trend Micro Blog Post
US-CERT Alert
Apple QT For Windows Download Page

How does this affect LiveCode and our users? Our media player object has been based on QuickTime, on Windows and Mac. There is fallback support to use the old Windows MCI system, however this has been around since the days of Windows 3.1 and is no longer supported by Microsoft. Clearly it is now necessary to remove our reliance on Quicktime and move to a modern implementation of the player object. Happily, we have just done that, and the solution is available to you in the latest LiveCode release.

read more
Ian MacphailRemoving Quicktime Dependency from LiveCode’s Player Object

How To Add a Web Browser to Your App

by Ian Macphail on November 10, 2015 8 comments

If you’re using the latest release of LiveCode 8.0 then the answer to that question is incredibly simple: just open up your stack and drag a browser control (the icon that looks like a little picture of the Earth) from the toolbar onto your stack. Then you can resize it however you like and open up the property inspector to set the url you want the browser to display. Easy!

read more
Ian MacphailHow To Add a Web Browser to Your App

The Evolution of Dragons

by Ian Macphail on April 22, 2015 No comments

One of the new features introduced in the upcoming LiveCode version 8 release is the ability to create self-contained custom controls called widgets. These widgets are written in a variant of LiveCode called LiveCode Builder (LCB) and have full control over their appearance and behaviour.

In a similar way to Hanson’s previous blog post on fractals, I’ll be showing you how to draw a fractal shape. However this time, I’ll show you how to do it with a custom widget control that can draw the fractal for us.

dragon

read more
Ian MacphailThe Evolution of Dragons

Getting Back up to Speed

by Ian Macphail on July 17, 2014 9 comments

Recently we’ve noticed a bit of sluggishness creeping into the LiveCode engine, particularly when it comes to graphics. Over the last few weeks I’ve been working with Michael to investigate and put into action some ways to get the engine back up to speed and get your apps whizzing again.

Looking into this problem, we’ve identified one of the big contributing factors as the introduction of support for high-density displays – where previously an app on the iPhone may have covered an area of 320×480 pixels, on a device with a retina display it now covers 640×960 pixels – that’s 4 times the number of pixels that need to be drawn when updating the screen! This is even more noticeable on mobile devices, where the processing power can’t match that of the desktop.

So what can we do to speed things up a bit? One option is to go multi-threaded. A thread is a block of code that can be made to run independently of the main program. By splitting a task up into multiple threads, different tasks can be made to run concurrently rather than one after the other. Most modern processors have multiple cores – that is multiple separate CPUs that are each capable of running a single thread all at the same time. We can take advantage of this by splitting up the job of redrawing the screen into several smaller tasks, i.e. by splitting up the area that needs redrawing and giving each smaller area to a separate thread. This promises to deliver a pretty big speed-up when redrawing.

One other option that I’ve been working on involves making sure we don’t do more work when drawing than we need to.

There are many ways to try and make computer programs work faster. You can try to find inefficiencies in your code that make things slower than they need to be – perhaps you have a loop in which the same value is computed repeatedly, in which case it may be more efficient to compute that value once and store it in a variable to be referenced later. Another strategy is to make better use of the resources available – the example being the use of multi-threading discussed above. However, if you know a particular operation is going to take a long time maybe you can find a way to avoid doing it altogether, or as little as you can manage.

In recent years we introduced a couple of new graphical features that allow you to add color gradients to graphic objects and graphic effects to everything. These features both require a lot of computation to draw, and as such they can be the slowest to draw when updating the screen. We spent quite a lot of time and effort on making these as fast as possible, but even then they can slow things down considerably when the area they cover is large.

So let’s say you have a card with a nice gradient as the background. You click a button that performs some operation then updates the contents of a field and maybe toggles some other buttons on or off. If this all happens as a single operation then each of those objects affected will have to be redrawn at the same time. Whenever a change happens to an object that requires it to be redrawn (changing the color of a graphic, or updating the contents of a field for example), its enclosing rectangle will be added to the redraw region.

A region is a graphical programming object that collects together multiple rectangles as a single area and allows several useful operations to be performed on that region. You can add or exclude rectangles from the region, or combine that region with another. This makes them pretty useful for defining the areas of a scene that we need to redraw.

Once we know what areas of the window need redrawing the engine will then tell the windowing system (the part of the operating system responsible for handling windows) that we want to redraw, after which the engine will receive a message back telling which areas of the window to update. This allows the OS to have areas of the window redrawn if for instance another window is passed over ours.

Now we need to draw to the window in the area given to us by the OS. In order to prevent flickering as different objects are drawn we use double-buffering. This is a technique where an offscreen image (the buffer) is created into which we draw, after which the complete image is then drawn to the window, so only the end result of the redraw operation is seen.

How then do we reduce the amount of redrawing that gets done? Well, previously the engine would confine the drawing area to a rectangle by setting the clipping rect of the graphics context that is used to draw to the buffer. The old graphics context would only allow the clipping area to be set to a rectangle, so if the area being redrawn was composed of smaller rectangles with lots of space between them, there was no way to prevent those unneeded areas between from also being drawn into. By using the new graphics context introduced by the graphical refactor, and updating it to accept a region as the clipping area, we can now ensure that only those areas that need updating are redrawn, with those areas in between getting skipped over.

Going back to our example, we can see that previously a much larger area (the rectangle encompassing all the affected controls) would have been redrawn, whereas now only the much smaller individual areas covering the affected controls will be drawn. For a stack with an expensive (in terms of time needed to draw) background this provides a significant saving in the time spent updating the window.

This was a pretty satisfying job for me – I got to update some more of the older code in the engine, replacing lots of platform-specific region code with a single region object that could be used on all platforms – and one that was successful in improving the experience of users of LiveCode developed applications.

read more
Ian MacphailGetting Back up to Speed

Matrix Multiplication and Fullscreen Mode

by Ian Macphail on May 9, 2014 6 comments

In the old days, things were pretty simple. If you wrote an app for an iPhone, you knew it would look the same on everyone else’s iPhone too. You could also write a version of your app to fit the fixed size of the iPad. And if you were feeling extra fancy even create an alternate layout for the different orientations.

Then along came Android. As an open source operating system not tied to any particular vendor, device manufacturers are free to make devices with any size or shape they want, (within reason, i.e. a rectangle). Consequently, You could be assured that whatever size you designed your app within, your users would end up running it on a device with totally different dimensions.

To tackle this problem we’ve introduced fullscreen modes that allow your app to scale to fit the bounds of the screen in a number of different ways, depending on whether your app will allow some parts of the stack to go offscreen or use as much screen space available while keeping all stack content visible.

Each fullscreen mode can result in different scaling values, as well as repositioning to keep the stack content centred on the screen. What we end up with then is a series of transforming operations to be applied when drawing the stack to convert the coordinates to their new size & position.

Within the engine we use a common mathematical trick to represent different types of coordinate transformations – matrix multiplication. I won’t go into too much detail on matrices, (you can read more here), but effectively what they provide is a way to combine multiple different types of geometrical operations into a single object. A matrix can represent a rotation, translation (movement along the x and y axes), or scaling operation. A sequence of such operations can be represented as the multiplication of their representative matrices together, producing a single matrix equivalent to the whole sequence.

The operations to fit a stack’s content within the visible screen space can then be defined in terms of a single matrix, which is used whenever it is necessary to convert from logical stack coordinates to the coordinate system of the view into which it will be drawn. We can use the same matrix to draw the stack, or to convert the location of mouse clicks or touch events, or to make sure pop-up menus and dialogs are positioned correctly regardless of the scale or position of the stack within its window. This ensures that the appearance of the stack and the location of any events related to it are kept consistent whatever the fullscreen mode might be.

read more
Ian MacphailMatrix Multiplication and Fullscreen Mode

Scaling up the details

by Ian Macphail on April 8, 2014 1 comment

In the previous post, I covered the update to the graphics layer. Updating the graphics library means we can now draw stacks at any scale, which allows us to add a whole slew of new and useful features. One of which is the ability to transparently support high-density displays.

Hi-DPI support

The introduction of the Retina display to iPhone/iPad created something of a problem for apps developed for the previous generation of devices. Apps created at the fixed size of these devices suddenly had four times the number of pixels to fit into (2x width and 2x height), and although they could still display at the same size, on the higher density displays it was apparent that they were not taking advantage of the available high resolution. The reason being that the apps were drawing at the old size, and then being scaled up – giving a stretched, pixelated appearance.

Rather than create a smaller image and scale it up, in order to draw cleanly and crisp on higher density displays you need to create an image with the same number of pixels as the display. This is a problem when your app is 320 x 480 in size, and the area into which you need to draw is now 640 x 960. Without scaled drawing support in the engine you would need to manually adjust everything on the stack to appear twice as big – from the size & position of each control, to the line width of any graphics, to the font size of any text appearing in the app.

To overcome these issues we could now use the new graphics library features to implement scaled drawing within the LiveCode engine. This automatic scaling is now included as of LiveCode 6.6 and is enabled by setting the "usePixelScaling" property. On mobile devices the degree of scaling can also be controlled by changing the "pixelScale" property.

The pixelScale controls the relationship between the logical size of the screen and the number of physical pixels in the display. This is quite straightforward on mobile devices, where there is only a single display. The old pre-Retina Apple devices have a 1:1 ratio of logical coordinates to pixels, retina devices have a 1:2 ratio. For a 320×480 stack to appear at the same size on both devices it should have a pixelScale of 1 on the non-Retina device, and a pixelScale of 2 on the Retina device. This happens automatically in the LiveCode engine – the pixelScale is set on startup to the appropriate value for the density of the display.

Getting it all to work

Implementing this within the engine was pretty challenging. If everything can be scaled, then there is no longer necessarily a 1:1 relationship between the size and location of objects on the stack, and their position on the screen as understood by the operating system. Unfortunately this was an assumption made in many different areas, such as the size and location of windows, to the position of mouse movements and clicks, to the apparent size of the screen itself. The situation is further complicated by the fact that some platforms expect coordinates in terms of screen pixels (Windows, Android), and others in terms of logical screen coordinates distinct from physical pixel sizes (iOS, OSX) so we end up having to think in terms of three separate coordinate systems: logical sizes within the engine, physical pixels, and logical (or not) coordinates used by the OS.

All this means that where code could previously use the same coordinates when referring to objects on a stack, or the position of the mouse cursor within a window, now that code had to be adapted to translate back & forth from one coordinate system to another. This was made somewhat easier by the progress of ongoing refactoring work to separate out window-related functionality from the tangle of stack code that made no distinction between a stack and the window that contains it. Now it was possible to define a scale factor between one and the other, and convert coordinates at the point of passing through from one to the other.

I think this really highlights the value of the work we’ve done to split apart the engine code into clearly defined areas based on a consistent and logical identification of distinct concepts that helps collect related code together and removes the implementation details from areas that are not directly related.

LiveCode 6.6.1 is now available. This is a stable release.

Getting this release
To upgrade to this release please select "check for updates" from the help menu in LiveCode or download the installers directly at: http://downloads.livecode.com/livecode/

read more
Ian MacphailScaling up the details