Detailed Setup Description

CAUTION , any exact command lines worked as of 01/2018 on a Fedora system. These can change over time and on different operating systems/levels. Always check/verify any command line before using.

Set up CINELERRA-GG

A CINELERRA-GG render farm is organized into a master node and any number of client nodes. The master node is the computer which is running the gui. The client nodes are anywhere else on the network with CINELERRA-GG installed and are run from the command line. Before you start the master node for CINELERRA-GG, you need to set up a shared filesystem on the disk storage node as this is the node that will have the common volume where all the data will be stored. The location of the project and its files should be the same in the client computers as in the master computer and to avoid problems of permissions, it is better to use the same user in master and clients. For example, if you have the project in /home/<user>/project-video you must create the same directory path on the clients, but empty. Sharing the directory of the location of your project on the master computer can be done with NFS as described next. Alternatively, you can look up on the internet how to use Samba to share a directory.

Create a shared filesystem and mount using NFS

(only for Network) All nodes in the render farm should use the same filesystem with the same paths to the project files on all of the master and client nodes. This is easiest to do by setting up an NFS shared disk system.

1. On each of the computers, install the nfs software if not already installed. For example, on Debian 9 you will need to run: (be sure to check/verify before using any command line):

   apt-get install nfs-kernel-server
   

2. On the computer that contains the disk storage to be shared, define the network filesystem. For example to export /tmp, edit the /etc/exports file to add the following line:

    192.168.1.0/24(rw,fsid=1,no_root_squash,sync,no_subtree_check)
   

3. Next reset the exported nfs directories using:

   exportfs -ra
   

   and you may have to start or restart nfs:

   systemctl restart nfs
   

4. Each of the render farm computers must mount the exported nfs target path. To see the exports which are visible from a client, login as root to the client machine and keyin:

   showmount -e <ip-addr> #using the ip address of the storage host
   

5. to access the host disk storage from the other computers in the render farm, mount the nfs export on the corresponding target path: (be sure to check/verify before using any command line):

   mount -t nfs <ip-addr>:/<path> <path>
   

   where <path> is the storage host directory, and <ip-addr> is the network address of the storage host. Because all of the computers must have the same directory path, create that same directory path with the same uid/gid/permissions on each storage client computer ahead of time.
6. To make this permanent across reboots on the client nodes, add the following line to /etc/fstab:

   {masternode}:/nfsshare /mnt nfs defaults 0 0
   

   You can make this permanent on the disk storage host BUT the command lines shown, which were correct in January 2018 on Fedora, may be different for your operating system or in the future. In addition if your network is not up, there may be numerous problems. If you make a mistake, your system may not boot. To make permanent, add the following line to /etc/fstab:

   192.168.1.12:/tmp /tmp nfs rw,async,hard,intr,noexec,noauto 0 0
   

   You will still have to mount the above manually because of the noauto parameter but you won’t have to remember all of the other necessary parameters. Depending on your expertise level, you can change that.

   Later, to remove access to the storage host filesystem:

   umount <path>
   

   Be aware that you may have to adjust any security or firewalls you have in place. Most firewalls will require extra rules to allow nfs access. Many have built-in configurations for this.

Configure Rendering on Master Node

There is 1 master node which is running the CINELERRA-GG gui and where the video will be edited and the command given to start up the rendering. Any number of client computers can be run from the command line only, so they can be headless since no X or any graphical libraries are needed. Of course, the CINELERRA-GG software must be installed on each of the client computers.

1. Assuming you already have CINELERRA-GG installed on the master node, start CINELERRA-GG by clicking on the icon or by typing the following command on the terminal screen: /cinelerra_path/cin.
2. Use the File pulldown Settings → Preferences, the Performance tab, to set up your Render Farm options in the Render Farm pane.
3. Check the Use render farm option. By default, once you enable the option of Render Farm, rendering is usually done using the render farm. Batch rendering can be done locally, or farmed.
4. Add the hostname or the IP address of each of the client nodes in the Hostname textbox and the port number that you want to use in the Port textbox. You can make sure a port number is not already in use by keying in on the command line:

   netstat -n -l -4 --protocol inet
   

   Next, click on the Add Nodes button and then you will see that host appear in the Nodes list box to the right. The X in the first column of the nodes box denotes that the node is active. To review the standard port allocations, check the /etc/services file.
5. Enter the total jobs that you would like to be used in the Total job textbox.
6. The default watchdog timer initial state is usually just fine but can be adjusted later if needed.
7. Click OK on the Preferences window when done.

Create Workflow

While working on the master computer, it is recommended that you keep all the resources being used on the same shared disk. Load your video/audio piece and do your editing and preparation. Add any desired plugins, such as a Title, to fine-tune your work. You want to make sure your video is ready to be rendered into the final product.

Start the Client Nodes

To start up the client nodes run CINELERRA-GG from the command line on each of the client computers using the following command:

/{cinelerra_pathname}/cin -d [port number]
# for example:
/mnt1/bin/cinelerra -d 401

This starts CINELERRA-GG in command prompt mode so that it listens to the specified port number for commands from the master node for rendering. When you start each of the clients up, you will see some messages scroll by as each client is created on that computer, such as:

RenderFarmClient::main_loop: client started
RenderFarmClient::main_loop: Session started from 127.0.0.1

As it completes its jobs, you will should see:

RenderFarmClientThread::run: Session finished

Render Using Render Farm

After you have followed the preceding steps, you are ready to use the render farm. Click on File → Render...which opens the render dialog. The most important point here is to use for the Output path / Select a file to render to a path/file name that is on the shared volume that is also mounted on the clients. Click on OK to render. The CINELERRA-GG program divides the timeline into the number of jobs specified by the user. These jobs are then dispatched to the various nodes depending upon the load balance. The first segment will always render on the master node and the other segments will be farmed out to the render nodes. Batch Rendering, as well as BD/DVD rendering, may use the render farm. Each line in the batchbay can enable/disable the render farm. Typically, video can be rendered into many file segments and concatenated, but normally audio is rendered as one monolithic file (not farmed).

Another performance feature which can use the Render Farm is Background Rendering. This is also enabled on the Preferences → Performances tab. The background render function generates a set of image files by pre-rendering the timeline data on the fly. As the timeline is update by editing, the image data is re-rendered to a background render storage path. The Render Farm will be used for this operation if it is enabled at the same time as the background render feature.

Assemble the Output Files

Once all of the computer jobs are complete, you can put the output files together by using the shell script, RenderMux (from the menubar scripts button just above FF), if the files were rendered using ffmpeg (see Menu Bar Shell Commands).

If you want to remain within the open project in CINELERRA-GG, you can load these files by creating a new track and specifying concatenate to existing tracks in the load dialog in the correct numerical order. File types which support direct copy can be concatenated into a single file by rendering to the same file format with render farm disabled as long as the track dimensions, output dimensions, and asset dimensions are equal.

Finally if you want to use ffmpeg from terminal externally to CINELERRA-GG you can go to the directory where the rendered files are, it creates a text file list.txt containing the list of all our files:

  	file 'name.webm001'
  	file 'name.webm002'
  	...
  	file 'name.webm00n'

and finally give the command

  	ffmpeg -f concat -i list.txt -c copy output.webm