Having just written a file container reader and decoder for a custom video format for QuickTime 7 (and having to comb through scarce deprecated documentation manuals), it seemed appropriate to write a little FAQ for anyone else looking to do the same. If you are new to QuickTime component programming, skimming through this FAQ should save you days if not weeks. No thanks required, (:
Should I be writing a QuickTime 7 component?
QuickTime 7 components are plugins for 32-bit QuickTime world. They are intended to extend both QuickTime Player and programs which use QTKit (an interface for embedding QuickTime Player in user applications). Hence, if want QuickTime Player or your program that uses QTKit to play a custom format, you will have to write a plugin component. For newer applications (especially 64-bit ones), Apple recommends that you use AVFoundation instead. This FAQ does not cover AVFoundation.
Why not write QuickTime X component?
Apple hasn’t documented on how to write an QuickTime X component yet. Again, QuickTime X exists exclusively for 64-bit machines and applications.
What is the difference between QuickTime 7 and QuickTime X?
QuickTime 7
- Always uses 32-bit binaries
- Is installed by default in both 32-bit and 64-bit machine, but QuickTime 7 Player UI needs to be downloaded manually in 64-bit OSX. It appears in ‘Applications->Utilities’ folder after installation
- On a 64-bit Mac, QTKit uses QuickTime 7 as a fallback if QuickTime X is unable to play a media format
QuickTime X
- Always uses 64-bit binaries
- Both its component and UI are installed by default in 64-bit Mac
How long has Apple promised support for QuickTime 7?
There has been no word on how long the support will last. Even though a lot of APIs used by QuickTime 7 component have been marked deprecated, both Apple as well as large user applications continue to use and prefer QuickTime 7. There are also a number of components written for QuickTime 7 which are not ported to QuickTime X. Looking at this trend, QuickTime 7 should survive some major future OS releases.
What are the different components and which one should I write?
The components have a very confusing nomenclature. So, here’s a list of different components and what they do:
- Import Component: If you intend to write a file container reader or want to allow users to convert custom format media file to QuickTime movie file (QuickTime’s mov file container format – NOTE: If an Apple documentation states ‘media file’ then its referencing this format) through QuickTime 7 Player, you should write this component.
- Export Component: If you intend to allow QuickTime to export a QuickTime movie file to a custom format media file, you should write this component.
- Image Decompressor Component: If you intend to decode video stream frame data passed by Import Component, you should write this component. Do note that this component can also be written for static image decompressors such as JPEG reader.
- Image Compressor Component: Opposite of what Image Decompressor does.
- Data Handler Component: If you intend to add capability to read/write media data from a source not traditionally supported by OS, you should write this component. This component need not understand, verify or parse the incoming/outgoing data. A data handler component is tasked by higher level components with reading and writing to a custom data source.
- Derived Media Handler Component: If you intend to modify media samples after they are just ready to be drawn by Base Media Handler, you should write this component. At this point, you already know the media duration, picture size, format and other information. This is a high level component.
Where are QuickTime 7 components installed and how can I register mine?
For system wide installation: /System/Library/QuickTime
For current user only installation: ~/Library/QuickTime (The directory ‘QuickTime’ may not exist. If so, create it)
Registration occurs automatically when you copy your ‘<ComponentName>.component’ bundle folder to any of the above installation location. No other action is required. Beware that, if your component had some important attributes (discussed later) missing and the registration failed – correcting the attribute in place will not trigger the component to register. Neither issuing a ‘touch’ command on the bundle or restarting your system will force the system to retry registration. The Component Manager system will attempt to register only when you copy a new bundle to those locations. Hence, it’s a good idea to write a little post-build script which removes your previous copy and copies the newly built bundle after the XCode build process finishes to ease testing and avoid potential headaches.
How can I make sure my component has been registered?
There is a program (actually a sample program) called ‘Fiendishthngs‘, provided by Apple, that lists all the components registered in your system. If your component is registered properly, it should appear in the list. Do know that plugin development using component model is deprecated and should not be used for any other application than to extend QuickTime 7.
Can different component, for instance an importer and decoder, be in the same binary/bundle?
Yes.
How does QuickTime 7 invoke the components when asked to play a custom file format?
QuickTime 7 interacts with registered components by using Component Manager interface. QuickTime, using Component Manager, goes through all the registered components at the installation locations and tries to enumerate all importers which will handle the format by matching file extension. Information regarding whether a component is an importer or decompressor or any other type is embedded in a compiled resource file (.rsrc files) inside the component bundle. The resource file also has information on what the entry point for the binary is called and what kind of frame data is outputted by the importer. Similarly, after the importer is done, QuickTime enumerates decoder by matching frame data type looking at the decoder’s resource file. If there are several decoders for a type, QuickTime will select the one which best matches the requirement (such as destination surface pixel depth, color types etc).
What does a typical component bundle contain?
/<ComponentName>.component
|-> Info.plist
|-> /MacOS -> <ComponentName> (executable file)
|-> /Resources -> <ComponentName>.rsrc (compiled resource file)
Where can I find a sample source code for writing a component?
Apple provides ‘ElectricImageComponent‘ as an example. Look into ‘EI_ImageDecompressor’ for an example for a decoder. It also has sample an importer and an exporter. Perian and Xiph are two example open source projects – although they might be little large for newbies.
What does a typical XCode project for a component look like?
Let’s call our decoder ‘TestDecoder’. Its project would look like:
/<ProjectName>.xcodeproj
|-> TestDecoderDispatch.h – Contains macros for auto generating boilerplate functions including entry point function and ‘CanDo’ function which calls the functions you implement
|-> TestDecoder.c – Contains implementation for the functions you want to handle
|-> TestDecoder.r – Contains atoms describing what component is implemented and how
|-> Info.plist – Infomation about the bundle such as manufacturer, product name etc
What do these files contain?
- TestDecoderDispatch.h – If ‘ComponentResult’ macro is used on a function name, it means you will have to implement it in ‘TestDecoder.c’. Use of ‘ComponentError’ macro means, the auto-generated function will return an error when called by Component Manager. Use of ‘ComponentNoError’ does the opposite. Use of ‘StdComponentCall’ calls the standard function implemented by the macro. Finally, use of ‘ComponentDelegate’ delegates the call to base component’s implementation.
- TestDecoder.c – Along with implementation of functions you promised in ‘TestDecoderDispatch.h’ by the use of ‘ComponentResult’ macro, ‘IMAGECODEC_BASENAME’ macro (along with others) is an important macro defined here. It specifies prefix for implemented functions. For instance, an ASF format importer might set its prefix as ‘ASFImporter’ hence the initialization function (called ‘MovieImportOpen()’ in documentations) should be called ‘ASFImporterOpen()’. Finally, each of the implemented functions have its first parameter as a pointer to a structure that you will have to define. This structure carries context data across the different functions. Do not use global static data storage to communicate among your functions as this will not make them thread safe.
- TestDecoder.r – Different type of components use different types of atom structure here to communicate about themselves to the Component Manager. Specifying different atoms allows the Component Manager to see what kind of component is implement, what kind of properties does it support, what kind of file type/mime is it associated with etc. An important function of this resource file is to specify the entry point for a component. You should name your entry point using the same prefix specified using ‘IMAGECODEC_BASENAME’ in TestDecoder.c. For instance, if the prefix specified there was ‘ASFImporter’, then the entry point name should be ‘ASFImporterComponentDispatch’. This file is complied by ‘Rez’ program internally in proxy of XCode and the resultant compiled binary resource file ‘TestDecoder.rsrc’ is produced.
What settings are required for the project?
The only two requirements for a component project are that the project type should be ‘Bundle’ and architecture to build must be set exclusively to ’32-bit’. Frameworks required are ‘QuickTime’ and ‘CoreServices’.
What are the potential pitfall for a component project?
- If you are using C++ instead of C, be sure to wrap framework header includes in ‘extern “C”‘. This is to make sure your binary’s entry point function’s name is not mangled by the C++ compiler.
- The contents of your ‘<Prefix>Dispatch.h’ must not be guarded using include only once macros/#pragma once.
- In a decoder project, when you assign ‘ImageSubCodecDecompressCapabilities *cap->decompressRecordSize’, an empty memory of that size will be available for your use in ‘BeginBand()’, ‘DrawBand()’ and ‘EndBand()’ pointed by ‘ImageSubCodecDecompressRecord *drp->userDecompressRecord’. This memory may be used to store per band/frame context information. You do not need to allocate memory yourself.
How does the importer pass data to the decoder?
In your ‘MovieImportDataRef()’ (or just ‘MovieImportFile()’ if you only support importing from files), create a media track and add samples to it using ‘AddMediaSampleReference()’ (or ‘AddMediaSample2()’ if your frame data is in memory) specifying the file offset and size for each frame data and passing the media track back to the caller using ‘usedTrack’ pointer. A ‘ImageDescription’ structure is also necessary which is attached with the media to carry information like frame width, height etc across to the decoder. If you need to carry extra custom data to the decoder, ‘AddImageDescriptionExtension()’ function can be used to append information to this structure. On the other end, QuickTime will read data from the file using the provided offset and size on to the memory and provide data for each frame to the decoder one by one.
Where is the codec data pointer and draw surface destination pointers on the decoder side?
You can access the codec data passed by the importer using ‘ImageSubCodecDecompressRecord * drp->codecData’ pointer in ‘BeginBand()’ and ‘DrawBand()’ parameter list. The size of this codec data is available only at ‘BeginBand()’ in ‘CodecDecompressParams *p->bufferSize’. To use this value in ‘DrawBand()’ carry it across using ‘ImageSubCodecDecompressRecord *drp->userDecompressRecord’. The surface to draw on is pointed to by ‘ImageSubCodecDecompressRecord *drp->baseAddr’.
I have written an importer but my importer takes a lot of time adding frame data so the player UI takes sometime to appear. How can I remedy this?
Usually the decoder is only invoked after the importer is done adding sample references. This is the cause of the delay. To avoid it, in your ‘MovieImportDataRef()’, set the ‘movieImportResultNeedIdles’ flag for ‘outFlags’. You will have to add a placeholder track until the actual frame data is imported. This is to allow QuickTime to calculate duration of your media. You should also implement ‘MovieImportSetIdleManager()’ to save the handle of an idle manager given by the system. An Idle Manager is used to get idle time from QuickTime for doing your work. The system will periodically call your ‘MovieImportIdle()’ function if the player is doing nothing – this is when you can do your importing. Finally, implement ‘MovieImportGetMaxLoadedTime()’ to tell QuickTime how much of the media is loaded.
Sanjeev's Blog 