cherry: A GPU accelerated slide show daemon for Raspberry Pi

Sometime ago, I wrote a little daemon program for Raspberry Pi to continuously slide show a bunch of images. The project was intended to be commercial – utilizing Raspberry Pi with its powerful GPU and low power requirements and targeted shop owners who have full-HD monitors continuously flipping through a set of image advertisements 24-hours a week.

I have decided to release the project under public domain hence you may use it any way you like. The program uses OpenCV for image loading and hence supports all image formats that OpenCV can load. Slide show blending occurs in GPU using dispmanx APIs hence completely bypasses both the CPU and XWindow server for smooth performance.

Before you try to compile the source, make sure:

  1. OpenCV libraries are installed
  2. Understand that the program is intended to be a daemon
  3. Understand that the program can run without loading any display/window manager

Go to source repository

Load screen for cherry program

Load screen for cherry program

Tagged , , ,

Writing file container reader and decoder for QuickTime 7: FAQ

Having just written a file container reader and decoder for a custom video format for QuickTime 7 (and having to comb through scarce deprecated documentation manuals), it seemed appropriate to write a little FAQ for anyone else looking to do the same. If you are new to QuickTime component programming, skimming through this FAQ should save you days if not weeks. No thanks required, (:

Custom QuickTime component playing a WMV file

Custom QuickTime component playing a WMV file

Should I be writing a QuickTime 7 component?

QuickTime 7 components are plugins for 32-bit QuickTime world. They are intended to extend both QuickTime Player and programs which use QTKit (an interface for embedding QuickTime Player in user applications). Hence, if want QuickTime Player or your program that uses QTKit to play a custom format, you will have to write a plugin component. For newer applications (especially 64-bit ones), Apple recommends that you use AVFoundation instead. This FAQ does not cover AVFoundation.

Why not write QuickTime X component?

Apple hasn’t documented on how to write an QuickTime X component yet. Again, QuickTime X exists exclusively for 64-bit machines and applications.

What is the difference between QuickTime 7 and QuickTime X?

QuickTime 7

  • Always uses 32-bit binaries
  • Is installed by default in both 32-bit and 64-bit machine, but QuickTime 7 Player UI needs to be downloaded manually in 64-bit OSX. It appears in ‘Applications->Utilities’ folder after installation
  • On a 64-bit Mac, QTKit uses QuickTime 7 as a fallback if QuickTime X is unable to play a media format

QuickTime X

  • Always uses 64-bit binaries
  • Both its component and UI are installed by default in 64-bit¬†Mac

How long has Apple promised support for QuickTime 7?

There has been no word on how long the support will last. Even though a lot of APIs used by QuickTime 7 component have been marked deprecated, both Apple as well as large user applications continue to use and prefer QuickTime 7. There are also a number of components written for QuickTime 7 which are not ported to QuickTime X. Looking at this trend, QuickTime 7 should survive some major future OS releases.

What are the different components and which one should I write?

The components have a very confusing nomenclature. So, here’s a list of different components and what they do:

  • Import Component: If you intend to write a file container reader or want to allow users to convert custom format media file to QuickTime movie file (QuickTime’s mov file container format – NOTE: If an Apple documentation states ‘media file’ then its referencing this format) through QuickTime 7 Player, you should write this component.
  • Export Component: If you intend to allow QuickTime to export a QuickTime movie file to a custom format media file, you should write this component.
  • Image¬†Decompressor¬†Component: If you intend to decode video stream frame data passed by Import Component, you should write this component. Do note that this component can¬†also be written for static image decompressors such as JPEG reader.
  • Image Compressor Component: Opposite of what Image Decompressor does.
  • Data Handler Component: If you intend to add capability to read/write media data from a source not traditionally supported by OS, you should write this component. This component need not understand, verify or parse the incoming/outgoing data. A data handler¬†component is tasked by higher level components with reading and writing to a custom data source.
  • Derived Media Handler Component: If you intend to modify media samples after they are just ready to be drawn by Base Media Handler, you should write this component. At this point, you already know the media duration, picture size, format and other information. This is a high level component.

Where are QuickTime 7 components installed and how can I register mine?

For system wide installation: /System/Library/QuickTime
For current user only installation: ~/Library/QuickTime (The directory ‘QuickTime’ may not exist. If so, create it)

Registration¬†occurs automatically when you copy your ‘<ComponentName>.component’ bundle folder to any of the above installation location. No other action is required. Beware that, if your component had some important attributes (discussed later) missing and the registration failed – correcting the attribute in place will not trigger the component to register. Neither issuing a ‘touch’ command on the bundle or restarting your system will force the system to retry registration. The Component Manager system will attempt to register only when you copy a new bundle to those locations. Hence, it’s a good idea¬†to write a little post-build script which removes your previous copy and copies the newly built bundle after the XCode build process finishes to ease testing and avoid potential headaches.

How can I make sure my component has been registered?

There is a program (actually a sample program) called ‘Fiendishthngs‘, provided by Apple, that lists all the components registered in your system. If your component is registered properly, it should appear in the list. Do know that plugin development using component model is deprecated and should not be used for any other application than to extend QuickTime 7.

Can different component, for instance an importer and decoder, be in the same binary/bundle?

Yes.

How does QuickTime 7 invoke the components when asked to play a custom file format?

QuickTime 7 interacts with registered components by using Component Manager interface. QuickTime, using Component Manager, goes through all the registered components at the installation locations and tries to enumerate all importers which will handle the format by matching file extension. ¬†Information regarding whether a component is an importer or decompressor or any other type¬†is embedded in a compiled resource file (.rsrc files) inside the component bundle. The resource file also has information on what the entry point for the binary is called and what kind of frame data is outputted by the importer. Similarly, after the importer is done, QuickTime enumerates decoder by matching frame data type looking at the decoder’s resource file. If there are several decoders for a type, QuickTime will select the one which best matches the requirement (such as destination surface pixel depth, color types etc).

What does a typical component bundle contain?

/<ComponentName>.component

|-> Info.plist
|-> /MacOS -> <ComponentName> (executable file)
|-> /Resources -> <ComponentName>.rsrc (compiled resource file)

Where can I find a sample source code for writing a component?

Apple provides ‘ElectricImageComponent‘ as an example. Look into¬†‘EI_ImageDecompressor’ for an example for a decoder. It¬†also has sample an importer and an exporter. Perian and Xiph are two example open source projects – although they might be little large for newbies.

What does a typical XCode project for a component look like?

Let’s call our decoder ‘TestDecoder’. Its project would¬†look like:

/<ProjectName>.xcodeproj

|-> TestDecoderDispatch.h – Contains macros for auto generating boilerplate functions including entry point function and ‘CanDo’ function which calls the functions you implement
|-> TestDecoder.c – Contains implementation for the functions you want to handle
|-> TestDecoder.r – Contains atoms describing what component is implemented and how
|-> Info.plist – Infomation about the bundle such as manufacturer, product name etc

What do these files contain?

  1. TestDecoderDispatch.h – If ‘ComponentResult’ macro is used on a function name, it means you will have to implement it in ‘TestDecoder.c’. Use of ‘ComponentError’ macro means, the auto-generated function will return an error when called by Component Manager. Use of ‘ComponentNoError’ does the opposite. Use of ‘StdComponentCall’ calls the standard function implemented by the macro. Finally, use of ‘ComponentDelegate’ delegates the call to base component’s implementation.
  2. TestDecoder.c – Along with implementation of functions you promised in ‘TestDecoderDispatch.h’ by the use of ‘ComponentResult’ macro, ‘IMAGECODEC_BASENAME’ macro (along with others) ¬†is an important macro defined here. It specifies prefix for implemented functions. For instance, an¬†ASF format importer¬†might set its prefix as ‘ASFImporter’ hence the initialization function (called ‘MovieImportOpen()’ in documentations) should be called ‘ASFImporterOpen()’. Finally, each of the implemented functions have its first parameter as a pointer to a structure that you will have to define. This structure carries context data across the different functions. Do not use global static data storage to communicate among your functions as this will not make them thread safe.
  3. TestDecoder.r – Different type of components use different types of atom structure here to communicate about themselves to the Component Manager. Specifying different atoms allows the Component Manager to see what kind of component is implement, what kind of properties does it support, what kind of file type/mime is it associated with etc. An important function of this resource file is to specify the entry point for a component. You should name your entry point using the same prefix specified using ‘IMAGECODEC_BASENAME’ in TestDecoder.c. For instance, if the prefix specified there was ‘ASFImporter’, then the entry point name should be ‘ASFImporterComponentDispatch’. This file is complied by ‘Rez’ program internally in proxy of¬†XCode and the resultant compiled binary resource file ‘TestDecoder.rsrc’ is produced.

What settings are required for the project?

The only two requirements for a component project are¬†that the project type should be ‘Bundle’ and architecture to build must be set exclusively to ’32-bit’. Frameworks required are ‘QuickTime’ and ‘CoreServices’.

What are the potential pitfall for a component project?

  1. If you are using C++ instead of C, be sure to wrap framework header includes in ‘extern “C”‘. This is to make sure your binary’s entry point function’s name is not mangled by the C++ compiler.
  2. The contents of your ‘<Prefix>Dispatch.h’ must not be guarded using include only once macros/#pragma once.
  3. In a decoder project, when you assign ‘ImageSubCodecDecompressCapabilities *cap->decompressRecordSize’, an empty memory of that size will be available for your use in ‘BeginBand()’, ‘DrawBand()’ and ‘EndBand()’ pointed by¬†‘ImageSubCodecDecompressRecord *drp->userDecompressRecord’. This memory may be used to store per band/frame context information. You do not need to allocate memory yourself.

How does the importer pass data to the decoder?

In your ‘MovieImportDataRef()’ (or just ‘MovieImportFile()’ if you only support importing from files), create a media track and add samples to it using ‘AddMediaSampleReference()’ (or ‘AddMediaSample2()’ if your frame data is in memory) specifying the file offset and size for each frame data and passing the media track back to the caller using ‘usedTrack’ pointer. A ‘ImageDescription’ structure is also necessary which is attached with the media to carry information¬†like frame width, height etc across to the decoder. If you need¬†to carry extra custom data to the decoder, ‘AddImageDescriptionExtension()’ function can be used to append information to this structure. On the other end, QuickTime will read data from the file using the provided offset and size on to the memory and provide data for each frame to the decoder one by one.

Where is the codec data pointer and draw surface destination pointers on the decoder side?

You can access the codec data passed by the importer using ‘ImageSubCodecDecompressRecord * drp->codecData’ pointer in ‘BeginBand()’ and ‘DrawBand()’ parameter list. The size of this codec data is available only at ‘BeginBand()’ in ‘CodecDecompressParams *p->bufferSize’. To use this value in ‘DrawBand()’ carry it across using ‘ImageSubCodecDecompressRecord *drp->userDecompressRecord’. The surface to draw on is pointed to by ‘ImageSubCodecDecompressRecord *drp->baseAddr’.

I have written an importer but my importer takes a lot of time adding frame data so the player UI takes sometime to appear. How can I remedy this?

Usually the decoder is only invoked after the importer is done adding sample references. This is the cause of the delay. To avoid it, in your ‘MovieImportDataRef()’, set the ‘movieImportResultNeedIdles’ flag for ‘outFlags’. You will have to add a placeholder track until the actual frame data is imported. This is to allow QuickTime to calculate duration of your media. You should also implement ‘MovieImportSetIdleManager()’ to save the handle of an idle manager given by the system. An Idle Manager is used to get idle time from QuickTime for doing your work. The system will periodically call your ‘MovieImportIdle()’ function if the player is doing nothing – this is when you can do your importing. Finally, implement ‘MovieImportGetMaxLoadedTime()’ to tell QuickTime how much of the media is loaded.

Tagged , , , , ,

FFmpeg LGPL v3 binaries for Windows

It seems cross compiling FFmpeg for LGPL v3 license (i.e. with non-LGPL v3 components and non-free components stripped) for Windows not only is a lot of hassle but takes a whole part of a day. So, this post is to provide links to statically linked LGPL v3 FFmpeg executables for both 32 and 64 bit architectures.

Download binaries with Debug information (has both 32 and 64 bit binaries)
Download binaries without Debug information (has both 32 and 64 bit binaries)

FFmpeg version: N-62439-g5e379cd

Statically linked library versions:
libavutil      52. 76.100 / 52. 76.100
libavcodec     55. 58.103 / 55. 58.103
libavformat    55. 37.100 / 55. 37.100
libavdevice    55. 13.100 / 55. 13.100
libavfilter     4.  4.100 /  4.  4.100
libswscale      2.  6.100 /  2.  6.100
libswresample   0. 18.100 /  0. 18.100

Build configuration:
–arch=x86_64 –target-os=mingw32 –pkg-config=pkg-config –enable-avisynth –enable-libmp3lame –enable-version3 –enable-zlib –enable-librtmp –enable-libvorbis –enable-libtheora –enable-libspeex –enable-libopenjpeg –enable-gnutls –enable-libgsm –enable-libfreetype –enable-libopus –disable-w32threads –enable-libvo-aacenc –enable-bzlib –extra-cflags=-DPTW32_STATIC_LIB –enable-libopencore-amrnb –enable-libopencore-amrwb –enable-libvo-amrwbenc –enable-libschroedinger –enable-libvpx –enable-libilbc –enable-static –disable-shared –enable-libsoxr –enable-fontconfig –enable-libass –enable-libbluray –enable-iconv –enable-libtwolame –extra-cflags=-DLIBTWOLAME_STATIC –enable-libcaca –enable-libmodplug –extra-libs=-lstdc++ –extra-libs=-lpng –extra-cflags= –extra-cflags= –enable-runtime-cpudetect

Tagged , ,

Adding shutdown/restart pushbutton for Raspberry Pi

This article deals with adding a push button on the Raspberry Pi’s¬†GPIO pins and¬†writing a daemon that handles push button events. If we press the push button for less than 2 seconds, we want the daemon to¬†shutdown the system and if the push button is still depressed for 2 or more seconds, the daemon must restart the system. First we deal with the hardware part of the problem.

Things to know:

  • Raspberry Pi has a 3.3 V microcontroller which means 3.3 V is considered logic high
  • The GPIO pins do not have voltage/current protection, so working voltage must strictly be 3.3 V and current in individual pins must be less than 16 mA

Wiring up hardware

Before we can wire up the GPIO pins, we need to convert the male GPIO pins on the board to female. According to the internet, an IDE hard drive cable is a good candidate for this. Unfortunately, I couldn’t find this cable in my local electronics shop so I settled with the following connector.

IDC26WayConnector

IDC 26 Way Connector

I did have to bend the pins a little on the connector so that it grips the GPIO pin well. Scroll down to see how the connector fits on the GPIO pins.

If you are new to electronics, wiring up the 3.3 V source on the board directly to a GPIO pin and putting a switch between them seems to be a no brainer. Unfortunately, this has two problems. First, if an accidental short occurs, a¬† source voltage directly connected to a pin can cause large surge of current¬†– damaging the hardware. Raspberry Pi’s individual pin is rated to work with current less than about 16 mA. We want to keep the current well below this limit. To solve this problem, we add a resistance big enough to limit current.

Secondly, when the switch is open, the pin is in what’s called a floating state. This means that because the pin is connected to neither a voltage source or ground, the input pin is in an indeterminate state.¬†During this project, I once incorrectly wired the circuit to leave the input pin as floating. When I pushed the button, the program would correctly read the pin value as high because it was tried to a 3.3 voltage source. When released though, the program would still read high. But if¬†the program was asked to read the pin a second time, then the program would correctly read low. This strange behaviour is caused by a floating pin.

To avoid this problem, we need to configure our push button circuit such that the pin is driven to either¬†ground or¬†3.3 V depending on the state of the push button. This is done by using a pull-down resistor. We use a ‘pull-down’ resistor to pull the input pin to¬†ground, when the pin is disconnected from logic high voltage by the push button.

Hence, the following circuit diagram shows the final setup of the circuit for GPIO pins:

Raspberry Pi Push button circuit

Raspberry Pi Push button circuit

From the wiring diagram, we can see that when the switch is open, the input pin is connected to the ground. When switch is closed, the 3.3 V source is parallel to the 1.2 K‚Ą¶ load which means input pin will be at 3.3 V (because the source is connected in parallel to both the loads). The 10 K‚Ą¶ resistor is the pull-down resistor while 1.2 K‚Ą¶ resistor is a current protection resistor. Using the consequence of Ohm’s law –¬†Voltage = Resistance * Current –¬†we can¬†calculate that when Voltage = 3.3 V, using the resistance values used above gives us a current below the rated limit.

Wiring view

Wiring view Wiring view

Setting up software

wiringPi‘ is one of the most commonly used GPIO library for Raspberry Pi. Setting up this library is detailed in the library’s website.

Assuming you have correctly setup ‘wiringPi’ and GNU C/C++ tools in your Pi, we can begin setting up the daemon. A daemon (for Windows users – a Linux daemon is equivalent to Windows service) is a perfect candidate for us because:

  1. It can be configured to start automatically Pi boots
  2. Daemons run with ‘root’ privileges which are required for some functions
  3. Daemons don’t require interaction from the user

It would be a waste of space to discuss the boilerplate code common to all daemons. Comments in¬†source file discuss this in abundance. The most important part of the daemon is handling button push interrupt. When the push button is depressed, the voltage in input pin jumps from logic low to high which in turn calls¬†an interrupt handler. The first thing this handler does is disable further interrupts. Unfortunately, the ‘wiringPi’ does not support disabling an installed handler. As suggested by the author of the library, we use a little hack. Then we sleep our thread for 2 seconds after which we test the pin value again. If the pin is released, we initiate shutdown else a restart.

Source for daemon: https://github.com/sanje2v/buttonshutdown-daemon

Follow these steps to setup the daemon:

  1. Download and¬†build the daemon source using¬†‘build.sh’ on your Pi. The build script also handles moving the output daemon file to ‘/usr/sbin’.
  2. Next, you will need to install the service script. This script handles starting and stoping our daemon. To install this script, just copy the ‘buttonshutdown’ file to ‘/etc/init.d’ and set¬†appropriate permissions using ‘sudo chmod 755 /etc/init.d/buttonshutdown’.
  3. Finally, we want our daemon to start automatically at boot time. To do this, run the command ‘sudo update-rc.d buttonshutdown defaults’.
  4. At this point, the daemon is not running. We have merely registered it to run at every next system start-up. To run¬†the daemon now, use ‘sudo service buttonshutdown start’.
  5. Now pushing the push button for less than 2 secs should initiate a shutdown while anything more should restart the system.
  6. If for some reason the daemon fails to load, check ‘/var/log/syslog’ using ‘grep buttonshutdown-daemon /var/log/syslog’.
Tagged , ,
Follow

Get every new post delivered to your Inbox.

Join 70 other followers