WESTLEY
-------
Preamble:
Westley is the MLT projects XML serialisation/deserialisation format -
as such, it closely mirrors the internal structure of the MLT API.
Introduction:
A westley document is essentially a list of 'producers' - a producer is
an mlt object which generates mlt frames (images and associated audio
samples).
There are 3 types of producer:
* Basic Producers - these are typically file or device oriented feeds;
* Playlists - these are arrangements of multiple producers;
* Multitracks - these are the fx encapsulators.
In the mlt model, producers are created and attached to 'consumers' -
consumers are software playback components (such as SDL), or wrappers for
hardware drivers (such as bluefish) or even the westley serialising
consumer itself (the latter doesn't receive frames - it merely
interrogates the connected producer for its configuration).
Although westley was defined as a serialisation mechanism for instantiated
MLT components, this document will concentrate on the hand authoring of
westley documents.
Rules:
As shall become apparent through the remainder of this document, the basic
tenet of westley authoring is to organise the document in the following
manner:
1) create producer elements for each unique media clip in the project;
2) create playlists for each track;
3) create a multitrack and specify filters and transitions;
4) adding global filters.
While other uses of westley exist, the approach taken here is to maximise
efficiency for complex projects.
Basic Producers:
The simplest westley document is:
clip1.dv
The westley wrapping is of course superfluous here - loading this document
with MLT is identical to loading the clip directly.
Of course, you can specify additional properties. For example, consider an
MPEG file with multiple soundtracks - you could define a westley document to
ensure that the second audio track is loaded:
clip1.mpeg1
NB: This relies on the mpeg being handled by the avformat producer, rather
than the mcmpeg one. See services.txt for more details.
A more useful example comes with the pango producer for a text producer.
TODO: pango example...
Notes:
1) It is better not to specify in/out points when defining basic producers
as these can be specified in the playlists. The reasoning is that in/out
restricts the amount of the clip available, and could lead to the same clip
being loaded multiple times if you need different regions of the clip
elsewhere;
2) A westley can be specified as a resource, so westleys can naturally
encapsulate other westleys.
Playlists:
Playlists provide a 'collection' structure for producers. These can be used
to define 'tracks' in the multitrack approach, or simple playlists for
sequential, single track playout.
As an example, the following defines two basic producers and a playlist with 3
items:
clip1.dvclip2.dv
Here we see how the playlist defines the in/out points of the basic
producers.
Notes:
1) All in/out points are absolute frame positions relative to the producer
being appended to the playlist;
2) Westley documents are currently authored for a specific normalisation;
3) The last 'producer' in the document is the default for play out;
4) Playlists can reference the same producer multiple times. In/out regions
do not need to be contiguous - duplication and skipping is acceptable.
Interlude - Introducing Multitracks:
So far we've defined basic producers and playlists/tracks - the tractor is
the element that allows us to arrange our tracks and specify filters and
transitions. Similarly to a playlist, a tractor is a container.
Note that MLT doesn't see a filter or a transition as a producer in the
normal sense - filters and transitions are passive when it comes to seeking.
Internally, seeks are carried out on the producers. This is an important
point - MLT does not follow a traditional graph oriented model.
Visualising an MLT tractor and it's interaction with the consumer will
assist here:
+----------------------------------------------+
|tractor |
| +----------+ +-+ +-+ +-+ +-+ |
| |multitrack| |f| |f| |t| |t| |
| | +------+ | |i| |i| |r| |r| |
| | |track0|-|--->|l|- ->|l|- ->|a|--->|a|\ |
| | +------+ | |t| |t| |n| |n| \ |
| | | |e| |e| |s| |s| \ |
| | +------+ | |r| |r| |i| |i| \ | +--------+
| | |track1|-|- ->|0|--->|1|--->|t|--->|t|-----|--->|consumer|
| | +------+ | | | | | |i| |i| / | +--------+
| | | | | | | |o| |o| / | ^
| | +------+ | | | | | |n| |n| / | |
| | |track2|-|- ->| |- ->| |--->|0|- ->|1|/ | |
| | +------+ | | | | | | | | | | |
| +----------+ +-+ +-+ +-+ +-+ | |
+----------------------------------------------+ |
^ |
| |
+-----------+ |
|APPLICATION|--------------------------------------------+
+-----------+
Internally, all frames from all tracks pass through all the filters and
transitions - these are told which tracks to deal and which regions of the
tracks to work on.
Note that the application communicates with the producer - it can alter
playback speed, position, or even which producer is connected to which
consumer.
The consumer receives the first non-blank frame (see below). It has no say
in the order in which gets them (the sdl consumer when used with inigo might
appear to be an exception - it isn't - it simply has a route back to the
application to allow the application to interpret key presses).
Whether this is better or worse than a traditional graph approach is a moot
point. The author of this document likes it anyway :-).
Tractors:
To create a multitrack westley, we can use two playlists and introduce a
tractor. For the purposes of demonstration, I'll add a filter here too:
clip1.dvclip2.dv0greyscale
Here we see that blank frames are inserted into the first playlist and a
blank is provided at the beginning of the second - this can be visualised in
the traditional timeline widget as follows:
+-------+ +-------------+
|a | |a |
+-------+---+-------------+
|b |
+---+
Adding the filter on the top track, gives us:
+-------+ +-------------+
|a | |a |
+-------+---+-------------+
|greyscale |
--------+---+-------------+
|b |
+---+
Note that it's only applied to the visible parts of the top track.
The requirement to apply a filter to the output, as opposed to a specific
track leads us to the final item in the Rules section above. As an example,
let's assume we wish to watermark all output, then we could use the
following:
clip1.dvclip2.dv0greyscalewatermarkwatermark1.png
Here we employ another tractor and we define a single track (being the
tractor we previously defined) and apply a watermarking filter there.
This is simply provided as an example - the watermarking functionality could
be better handled at the playout stage itself (ie: as a filter automatically
placed between all producers and the consumer).
Tracks act like "layers" in an image processing program like the GIMP. The
bottom-most track takes highest priority and higher layers are overlays
and do not appear unless there are gaps in the lower layers or unless
a transition is applied that merges the tracks on the specifed region.
Practically speaking, for A/B video editing it does not mean too much,
and it will work as expected; however, as a general rule apply any CGI
(graphic overlays with pixbuf or titles with pango) on tracks higher than
your video tracks. Also, this means that any audio-only tracks that are
lower than your video tracks will play rather than the audio from the video
clip. Remember, nothing is affected like mixing or compositing until one
applies a transition or appropriate filter.
clip1.dvclip2.mpeg01luma01mix0.01.0
A "luma" transition is a video wipe processor that takes a greyscale bitmap
for the wipe definition. When one does not specify a bitmap, luma performs
a dissolve. The "mix" transition does an audio mix, but it interpolates
between the gain scaling factors between the start and end properties -
in this example, from 0.0 (none of track B) to 1.0 (all of track B).
Because the bottom track starts out with a gap specified using the
element, the upper track appears during the blank segment. See the demos and
services.txt to get an idea of the capabilities of the included transitions.
Flexibility:
The information presented above is considered the MLT Westley "normal"
form. This is the output generated by the westley consumer, for example,
when used with inigo. It is the output generated when you use the
"Westley to File" consumer in the demo script, which beginners will find
most useful for learning to use westley XML. This section describes
alternative forms the westley producer accepts.
First of all, the normal form is more of a linear format with producers
and playlists defined prior to their usage in a multitrack. Westley
also accepts a hierarchical format with producers as children of tracks
or playlist entries and with playlists as children of tracks:
Obviously, this example is meant to demonstrate hierarchy and not effective
use of playlist or multitrack!
Secondly, as part of error handling, westley is forgiving if you fail to
supply ,