Next: Blue Banana Up: Video Effects Previous: Auto Scale Contents Index
The Blend Algebra/Blend Program plugins provide the advanced feature of combining and modifying color pixels and transparencies of several tracks according to a mathematic algorithm written entirely by the user in the form of compact and simple piece of code. Such user defined algorithms (blend functions) are compiled, dynamically linked and immediately executed inside CINELERRA-GG on the fly without the need to restart the application, reload the project, reattach plugins, and so on.
The following description is based on the /cinelerra-5.1/doc/README.blendalg that accompanies the code by the author of both the code and the document. It is the authoriative documentation; the most detailed/original source and has additional commentary and programming information. If you encounter problems or confusion when using these plugins, always refer directly to that document here. If running CINELERRA-GG from an AppImage, testing has shown that it should work as well as if run from a build - just be aware of filename extensions that appear by default in the input field for the funtion name and change them accordingly.
The idea behind these plugins6 is that the user can program sufficiently short and simple blending algorithms, called Functions, by following detailed instructions with very little programming skills required. Blend Algebra and Blend Program come with a library of Functions (System Library) that mimic the 30 overlay effects found in the Patchbay or Overlay plugins. The user can modify these Functions as needed or create new ones. To look at these example functions, there is an Edit... button in the respective plugin dialog. You will need a text editor to do so, which by default is emacs if it is installed. If not, you can set your favorite editor via the corresponding environment variable in a terminal (export CIN_EDITOR='name editor') and then start CINELERRA-GG from that same terminal.
If you can not wait to get started, checkout these examples here. You will also find test projects to download to learn more about the plugins and to see the various ways they can be used to create your own functions.
Both plugins, Blend Algebra and Blend Program, are similar. They are described together along with their differences. The main difference between the two plugins is that Blend Algebra is like a function; it combines its arguments (tracks) and yields the result which is placed into the single track configured for output. However, Blend Program does not produce a function result, but instead directly modifies its arguments. If more than one track is to be modified, Blend Program is the only choice. If a single track is to be processed, Blend Program is the best choice and might run slightly faster. Any Blend Algebra function can be rewritten to become a Blend Program function. But if a function processes several tracks and modifies only one, it will run faster in Blend Algebra which has more controls in its plugin dialog.
How the plugin works is defined by the user. The word function is used to denote this for either of the two plugins throughout this description. These functions are for their plugins the same as the plugin shared objects in the main CINELERRA-GG binary. But these functions are simple enough that they can be easily written by users and manipulated in binary or in source form. The plugins make it possible to have a user library and a system library of functions. Thus, an experienced user can have a number of such functions to reuse in many projects.
The plugin is attached to a track as usual. Generally, it needs several tracks as arguments and it has to be attached to the additional tracks as a shared effect.
All the parameters described below are keyframable, including user functions themselves. The top area of the dialog menu deals with programmatic aspects. The middle area of the dialog is color specific. The bottom area of the dialog has controls for the argument track ordering, similar to that of the Overlay plugin.
export CIN_EDITOR=kate export CIN_EDITOR=gedit ... $ ~/cinelerra5/cinelerra-5.1/bin/./cin |
export CIN_EDITOR='konsole -e vi' $ ~/cinelerra5/cinelerra-5.1/bin/./cin |
There may be illegal floating point operations in the user's code, such as division by zero producing infinity, or square root from minus unity producing NaN (not-a-number). Even if only legal finite floating point numbers are computed, the results may go out of range. This dialog section controls what to do in such cases.
RGB: [0.0 .. 1.0] for R, G, B YUV: [0.0 .. 1.0] for Y, [-0.5 .. +0.5] for U, V HSV: [0.0 .. 360.0] for H, [0.0 .. 1.0] for S, V Transparency: [0.0 .. 1.0] for Alpha |
All the parameters are keyframable, including user functions themselves. That is, a plugin can have several functions attached at the same time and switch between them while rendering. The color and alpha channels of the substitution (key) color are interpolated between keyframes; the other parameters are switched.
The user library directory is that defined by the environment variable $CIN_USERLIB, or if not defined, $HOME/.bcast5lib as a fallback default. The functions for Blend Algebra reside there in the subdirectory dlfcn/ba and for Blend Program in dlfcn/bp (dlfcn stands for dynamic loaded function). Having user functions in such a library in a single place makes it convenient to reuse the same functions in different projects. The Copy to userlib button in the Attach... file selection dialog is handy to save such functions in the user library for future use.
The reason to define user library $HOME/.bcast5lib by default (and not $HOME/.bcast5) is the following. There is a common workaround: if a newer version of CINELERRA-GG does not start, to delete $HOME/.bcast5 completely. Doing so, all user functions would be deleted if they were saved there.
The system library is defined by the environment variable $CIN_DAT, the CINELERRA-GG installation path, and under the same subdirectories dlfcn/ba or dlfcn/bp. System library is meant for functions distributed with CINELERRA-GG.
When a complete project is to be exported, or fed to a render farm, it is often necessary to place all the resources under the same directory where the project .xml file resides. If some functions are used which reside elsewhere (for example, in a user library), they could not be found after exporting the project to another computer. The Copy to project button in the Attach... file selection dialog is used to copy such functions to the project directory in a convenient manner.
Blend Algebra/Blend Program functions are written in C, and with usage of a set of special cpp macros hiding from the user all the Cinelerra internals. This approach makes it easy to write functions of moderate complexity with almost no background in C programming. All macros defined for user functions are written in capital letters. A typical Blend Algebra function consists of the following logical blocks.
<optional static variable definitions> BLEND_ALGEBRA_INIT <macros and/or C statements for the INIT (preparation) phase will be executed once before processing of each frame> BLEND_ALGEBRA_PROC <macros and C statements for the PROC (processing) phase will be executed repeatedly for each pixel in the frame> BLEND_ALGEBRA_END |
The macro BLEND_ALGEBRA_STOP terminates execution of the INIT or PROC phase immediately and returns to the calling plugin.
The structure of a Blend Program function is the same, only the word ALGEBRA in all macros must be changed to PROGRAM. The following macros can be used in the INIT phase:
COLORSPACE_RGB COLORSPACE_YUV COLORSPACE_HSV |
Use any of these COLORSPACE statements to declare the working color space inside the user function. If there is no such declaration, the native color space of the project (RGB or YUV) will be used.
REQUIRE_TRACKS (<n of tracks>)
Declares the minimum required number of tracks the function uses. If the effect is attached to more tracks, it is fine. If there are less tracks, it is an error and the function is not executed. The absence of such declaration means that the function can process any number of tracks (one or more).
PARALLEL_SAFE
This declaration means that it is safe to execute the function in parallel. Without this statement the function will be executed sequentially, independent of the state of the parallelization checkbox in the plugin GUI.
The following special variables (macros) can be used for queries:
PARALLEL_REQUEST
1 == parallelization requested in the GUI
0 == parallelization switched off
TOTAL_TRACKS
The actual number of tracks passed to the function.
HAS_ALPHA
1 == the project color space has alpha channel (transparency)
WIDTH, HEIGHT
The dimensions of processed frames in numbers of pixels.
The following variables (macros) can be used in the PROC phase.
In addition to the next variables, TOTAL_TRACKS, HAS_ALPHA, WIDTH, HEIGHT can be used as well.
Color (and alpha) components of pixel from i-th track. Track numbering is 0-based, as standartized for C arrays. So, the first track has number 0, the last one - number TOTAL_TRACKS-1. All the color components are floating point values of the C type 'float'.
Here the letters R, G, B, H, S, V, etc. are for readability. If the user has declared COLORSPACE_RGB, and then written something like H(i), it does not mean that he should get the H-component of color autoconverted to the HSV color space. Actually R, Y, H are mutually identical to the 1-st color component, so are G, U, S to the 2-nd, and B, V to the 3-rd one.
R(i), G(i), B(i) Y(i), U(i), V(i) H(i), S(i), V(i) A(i) |
Corresponding color components of the key (substitution) color in the working color space of the function and the opacity slider.
KEY_R, KEY_G, KEY_B KEY_Y, KEY_U, KEY_V KEY_H, KEY_S, KEY_V KEY_A |
Color components for the result, for Blend Algebra only. Blend Programs return no result so do not have these definitions.
R_OUT, G_OUT, B_OUT Y_OUT, U_OUT, V_OUT H_OUT, S_OUT, V_OUT A_OUT |
Integer X and Y coordinates of the current pixel in its frame, in the ranges of [0 .. WIDTH] and [0 .. HEIGHT], respectively.
PIX_X, PIX_Y |
These macros clip color components of the i-th track to the bounds appropriate in the respective color space.
CLIP_RGB(i) CLIP_YUV(i) CLIP_HSV(i) CLIP_RGBA(i) CLIP_YUVA(i) CLIP_HSVA(i) CLIP_A(i) |
Clip all color components (including alpha) for all tracks.
CLIP_RGB_ALL CLIP_YUV_ALL CLIP_HSV_ALL |
Like clipping track color components, but for the result of a Blend Algebra function (not relevant for Blend Programs).
CLIP_RGB_OUT CLIP_YUV_OUT CLIP_HSV_OUT CLIP_RGBA_OUT CLIP_YUVA_OUT CLIP_HSVA_OUT CLIP_A_OUT |
Clipping is done according to the inherent bounds of the respective color space as follows:
RGB: [0.0 .. 1.0] for R, G, B YUV: [0.0 .. 1.0] for Y, [-0.5 .. +0.5] for U, V HSV: [0.0 .. 360.0] for H, [0.0 .. 1.0] for S, V Transparency: [0.0 .. 1.0] for Alpha |
ABS(x), SQR(x), MAX(x,y), MIN(x,y) absolute value, square, max and min values for floating point arguments. Can be used in any phase of a function.
TO_RAD(x), TO_DEG(x) conversion from degrees to radians and vice versa.
Both macros clip the 'x' argument to bounds between y and z. CLIP returns value leaving x unchanged. CLAMP assigns that value to x.
CLIP(x,y,z) CLAMP(x,y,z) |
All the macros are defined in the text header $CIN_DAT/dlfcn/BlendAlgebraStart for Blend Algebra and the analogous header BlendProgramStart for Blend Program in the same dlfcn subdirectory of the Cinelerra installation directory. These headers are prepended to the user functions during compilation. The user with a basic C knowledge can easily understand what happens in these headers.
Of course, you can use any valid C statements inside user functions. As they are linked with -lm, the standard C math library and with glibc, almost any mathematical or C library function can be called. C-style comments can also be used, and are welcome.
Compilation of Blend Algebra and Blend Program functions is carried out by a Perl script BlendAlgebraCompile.pl and BlendProgramCompile.pl. Both scripts are distributed in the dlfcn subdirectory of $CIN_DAT, the Cinelerra installation directory. Like the famous ContextManual.pl script of Context Help, the system script at the first call is copied into $CIN_CONFIG ($HOME/.bcast5) where it can be edited by the user and adapted according to their needs.
While a regular compilation, the script works in that directory where the function source file resides. First, the BlendAlgebraStart/BlendProgramStart header is prepended to the source to produce an intermediate file with the .c suffix. This C source is compiled by the system C compiler (gcc by default) and linked with -lm to produce the shared object ready for dynamic loading (attaching to the plugin).
Usually the script is executed by the respective CINELERRA-GG plugin when a user function is to be compiled. But the script can also be run by the user explicitly. Execution can be controlled by options starting with the '-' character, the first argument not starting with '-' being the function name to compile. The following options are recognized:
-compile do not check modification time, compile unconditional
-cfile do not remove intermediate .c file
-opt add optimizing options to compiler command line
-debug add debugging options to compiler command line
-warn add warning options to compiler command line
-edit open function source in text editor
-verbose verbose execution
-noapi do not copy itself to $HOME/.bcast5
-h, -help, -? print short help and current configuration
In the beginning of the script some variables can be redefined according to the user's needs: the C compiler executable, compiler options for optimization or debugging, and text editor executable. These environment variables are taken into account in various places in Blend Algebra/Blend Program plugins.
$CIN_CC: C compiler executable (default: gcc)
$CC: C compiler executable if $CIN_CC is not defined (default: gcc)
$CIN_EDITOR: text editor (default: emacs)
$CIN_DAT (set by Cinelerra binary): Cinelerra installation directory. The Blend Algebra / Blend Program system library with scripts, headers, and functions distributed with Cinelerra resides in the subdirectory $CIN_DAT/dlfcn.
$CIN_CONFIG (set by Cinelerra binary): user's config directory, usually
$HOME/.bcast5.
$CIN_USERLIB: user's library (default: $HOME/.bcast5lib). User's functions reside in the subdirectory $CIN_USERLIB/dlfcn.
In the distribution 30 Blend Algebra functions are provided for all the 30 CINELERRA-GG overlay modes according to the formula defined in the CinelerraGG manual. In principle, they are not needed as the built in overlayer engine is OpenGL accelerated and runs faster. Nevertheless, these functions can be handy as starting examples for users who may like to modify the formula in some way. Furthermore, their behavior differs from that of the standard overlayer in several aspects:
There are several example functions from totally different areas of application: a user programmable plugin is a really universal tool and can be handy in unexpected cases. A programmer can sometimes make use of this plugin when creating new effects. It has the big advantage that you can more quickly test different combinations without spending time to restart CINELERRA-GG, recompile it, reload project, and reattach plugins, etc. See the examples here.
There is the ydiff.ba function, actually representing the complete ydiff application running inside a plugin. In addition, there is a Transition example, and 2 more function examples showing use of the Blend Programs: the chromakey.bp and background.bp functions. They too serve as excellent starting points for further user experimentation. The usage of the provided functions is either self evident or briefly described in the comments of their source files.
NOTE: make install copies the functions into Cinelerra's bin directory in source (*.ba, *.bp) and binary (*.so) forms via cp -a, to preserve modification times of the files. Otherwise, if modification time of the function source accidentally becomes newer than that of its binary, the plugin will repeatedly try to recompile the function. If they are installed system-wide so that the user has no permission to modify them, such recompilations will fail.
User's functions can have bugs. Most probable bugs in a user function can be illegal arithmetic such as division by zero or logarithm of a negative number (FPE, floating exception), and out of bounds indexation of arrays (usually leading to SEGV). Most modern processors do not generate FPE. Instead, they generate NaN as the result of an illegal arithmetic instruction. After calling user functions, the plugins test the results not to be NaN and replace them with the configured substitution color if necessary.
SEGV is more problematic. If the user has written in his function, let's say, REQUIRE _TRACKS(2), and then accessed R(1000000), most likely the Cinelerra binary will crash. Although theoretically it could be possible to trap SEGV via sigaction()/setjmp() /longjmp(), in Cinelerra it is problematic. In a multithreaded application, which Cinelerra is, only one signal handler for all its threads at the same time is allowed, and Cinelerra already has a SEGV handler which can not be overwritten. So the user is responsible for his indexation bugs, which can easily occur in off-by-one errors, like referencing elements such as R(TOTAL_TRACKS) (the last legal element being R(TOTAL_TRACKS-1)), or R(i) where i has been decremented to -1.
The user is allowed to call any C function from the -lm and glibc libraries. But some functions are not thread safe (which means PARALLEL_SAFE macro should not be used) and some others have undesired side effects. For example, should the user write inside his function something like exit(R(0));, Cinelerra would immediately halt. The user is left to imagine what would happen after the following expressione:
R_OUT = R (system ("killall -9 X"));
|
Blend Algebra/Blend Program functions are not hardware accelerated which means that the function algorithm is not programmed as an OpenGL shader. However, functions can be parallelized using the load balancing engine of Cinelerra, and can be compiled with optimization (this is switched on by default). The default optimizing option (defined in BlendAlgebraCompile.pl and BlendProgramCompile.pl) is -O2. One can redefine it to -O3 or 0fast. It has to be noted that using 0fast or -ffast-math options can lead to ignoring some IEEE rules for floating point math, for example some intrinsic tests on infinities or NaN which can yield unpredictable results.
Although user's function can be compiled with debugging option such as -g or -ggdb, debugging functions can be tricky. The programmer can set a breakpoint into the function only after its code has been loaded in memory by the plugin, and set another breakpoint into plugin only after that plugin itself has been attached to a track. Moreover, a function can get detached from memory under some conditions, and then its breakpoints will be lost.
Debugging printout inside the PROC phase, although possible, would seem too verbose: the PROC phase would be called for full-HD footage 2073600 times per frame! Here perhaps one could make use of PIX_X, PIX_Y coordinates at some selected place like in the example:
if (PIX_X == 320 && PIX_Y == 200) |
The current implementation should be portable as long as you have a working C compiler such as gcc, the default to compile CINELERRA-GG, or clang which has been known to work; but other compilers may be compatible as well. Portability to other architectures that are Unix-like and ELF based should work, but not Windows.
NOTE: For developers who want to take advantage of creating a plugin to generate an effect in the same manner as Blend Algebra/Blend Program, please reference Unique Blend plugins workflow. This methodology makes it easier for the programmer to more quickly test different combinations without spending time to restart CINELERRA-GG, recompile it, reload your project, reattach plugins, and so on.
![\includegraphics[scale=0.5]{Blend_Program.png}](img28.gif)
