How to create a photo slideshow synchronized with music (in Linux, of course)

If you ever tried to create a photo slideshow where the transitions between photos are synchronized with some background music, you probably found out, that it is quite difficult. After reading this tutorial, you will hopefully change your mind :o)

Tools you need:

• ImageMagick (http://www.imagemagick.org)
• Cinelerra (http://cinelerra.org, nice tutorials on Cinelerra are here: http://www.g-raffa.eu/Cinelerra/HOWTO/)
• mjpegtools, FFmpeg or other tools for rendering
• Perl
• These two scripts: cinelerra_transitions_on_labels.pl, cinelerra_transitions_on_labels_2vidtracks.pl (copy and paste both on your disk)

Configure Cinelerra:
Run Cinelerra, in the main window select "Settings -> Preferences", select "Playback" tab and in "Audio Out" section, set the "Playback buffer size" to the lowest possible value (in my case 2048). Then in the "Video Out" section, there is a "Video Driver" drop-down menu. This is a bit tricky. If your graphic card supports OpenGL, you can select "X11-OpenGL", but I experienced few crashes most probably because of this driver. So let's use "X11" or "X11-XV" until we really need the OpenGL (which will become handy after we add pan-and-zoom effect and fading between images, but let's start with "X11").

Edit: The default length of inserted images in Cinelerra is set to 1 frame, so to change that, go to "Settings -> Preferences -> Recording -> Images" and check "Import images with a duration of" and set "3.0000" seconds besides that.

Configure scripts:
If you open the scripts cinelerra_transitions_on_labels.pl and cinelerra_transitions_on_labels_2vidtracks.pl, you will find few configuration variables at the beginning. There are comments, so you should be able to set them properly. There are some default values, so you don't have to make any changes there.

Step-by-step:
Now we will create a slideshow in 1280x720 resolution.

1. Setup your project. Create a directory for your project. Let's say it is /home/miso/video1.  Now create directories images, images/1280x720, sound. Copy the pictures you want to use into the images directory and the background music into the sound directory.
2. Scale your images. To work with images in Cinelerra, it is better to scale them to the final resolution of the video:
   cd images
   for i in *.jpg; do
       convert $i -resize 1280x1280 1280x720/$i
   done
   cd ..
3. Create a Cinelerra project. Run Cinelerra, in the main window's menu select "File -> New" and set the "Audio tracks: 2", "Audio channels: 2", "Frame rate: 30", "Width: 1280", "Height: 720" and click "OK". Then select "File -> Load files". Go into the directory images/1280x720 and select all your images (using Shift key). At the bottom of the file selection dialog, set the "Insertion strategy" to "Concatenate to existing tracks". Then repeat this step for your music file. Now you should see your pictures in track "Video 1" and your music in "Audio 1" andl "Audio 2" (because it is stereo).
   
   
4. Turn off keyframing and set drag-and-drop editing. As shown on this image:
   
5. Order your images. Reorder your images if you want. Just drag any image on the timeline with your mouse and drop it between other two images. You can scale the timeline using "Shift + mouse wheel". Don't pay any attention to the duration of images for now.
   
6. Create beat-labels. Go to the start of your video (Home key) and play it (Spacebar). Don't look at your images, just listen to the music and when you detect a beat, press "L" key on your keyboard (lowercase). It will create a label, a green triangle on the timeline, which will be used as a transition point between two images. So let there enough time between labels (about 3 seconds). If you missed a beat, just click anywhere on the timeline and press "L". To remove a label, click on the label's green triangle on the timeline (it will get red) and press "L". Save the project "File -> Save" as slideshow.xml.
   
7. Align images on labels. Run this script in your terminal:
   ./cinelerra_transitions_on_labels.pl slideshow.xml
   Then in Cinelerra, go to "File -> Load files", select the slideshow.xml and set the "Insertion strategy" to "Replace current project". Images are now aligned on the labels. You can play your video to see, if there is no mistake. you can still add new labels and remove the old ones, but then save the project again and repeat this whole step with running the alignment script and reloading the project in Cinelerra.
   
8. Cross-fading and Ken-Burns (pan and zoom) effect. Run this script in your terminal:
   ./cinelerra_transitions_on_labels_2vidtracks.pl slideshow.xml
   A new file slideshow_2vidtracks.xml is created. Load the new project file in Cinelerra as in the previous step. Now your images are spread in two video tracks with cross-fading transitions and with a random pan and zoom. Animation keyframes are placed at each label's position.
   
   Change the video driver to "X11-OpenGL" (as described above in "Configure Cinelerra" part). Play your video. Since the panning and zooming is randomly generated, you will probably need to tune it. Turn on "Generate keyframes while tweaking" icon (it looks like a key: ) in the main window's toolbar right under the main menu. Select a green triangle of the label at the start of the image (it will get red as you click on it):
   
   Place your mouse cursor anywhere on the video track on which the affected image resides and press "Shift + Tab" on your keyboard (to arm only that track; arm more tracks with "Tab"). The track gets armed (a small red circle icon on the left turns on) which means that any editing will affect only that track.
   
   Now go to the "Compositor" window and in the left toolbar select "Adjust camera automation" icon and "Show tool info" icon. A small dialog with X (horizontal), Y (vertical), Z (zooming) values and alignments appears:
   
   We are now going to adjust the camera at that selected label. There are actually two overlapping images in the Compositor window, each with 50% visibility.
   
   Now click and drag with your mouse in the Compositor's video to adjust the camera position. X and Y values of one image will change (only the armed track is affected). To adjust zooming factor, set the Z value manually. When you are satisfied with the starting point of the image, select the label at the end of the image and adjust the camera the same way.
9. Render your video. you can find many tutorials on Cinelerra rendering, but I use this. Select "File -> Render", set the "File format" to "MPEG Audio", check "Render audio tracks" and configure it (with the icon next to the checkbox) to "Layer: III", "Kbits per second: 160" or whatever you like. Uncheck "Render video tracks". Set "Render range: project" and "Insertion strategy: Insert nothing". At the top of the dialog, set the correct filename for your mp3 and confirm the dialog. When the audio is rendered, you can render the video tracks. Select "File -> Render" again, but now set the "File format" to "YUV4MPEG Stream", uncheck "Render audio tracks" and check "Render video tracks". Configure video rendering with the icon next to the checkbox. Set the correct output path (filename extension should be ".m2v"), check "Use pipe" and set there this string:
   mpeg2enc -f 8 --verbose 0 --multi-thread 4  --aspect 3 --format 3  --video-bitrate 12000 --nonvideo-bitrate 384  --no-constraints  --video-buffer 448  --frame-rate 5  --interlace-mode 0  --force-b-b-p  --video-norm n  --keep-hf  --sequence-header-every-gop --min-gop-size 6 --max-gop-size 6  -o %
   Set the bitrates at your will. Confirm the dialog and after a while your video is ready. Now you need to merge the audio and video by running this command in your terminal:
   mplex -f 3 -b 2000 video.mp3 video.m2v -o video.mpg
   So your final rendered video file is "video.mpg". To transcode to other formats, I like to use avidemux.

Tips:

• To see the keyframes on the video tracks, select in the main menu "View -> Camera X, Y and Z", arm both video tracks and click on the icon "Fit all autos to display" (or press "Alt + f").
• While creating beat-labels, the waveform in the audio track may be useful.
• Cinelerra has lots of options and possibilities, so feel free to read some more Cinelerra tutorials.
• Keyframes in Cinelerra are interpolated with bezier curves. Therefore panning and zooming starts slowly, then gets faster and ends slowly again. If you want to make it linear, select the part you want to linearize (or press "Ctrl + A" to select the whole video), arm the tracks you want to change (with mouse and Tab key) and in the main menu select "Keyframes -> Straighten curves".

Notes:

• I tried to make use of an automatic beat-detection, but sometimes there are music rhythms where you don't want to put the transitions on the strongest beats. So doing it manually is probably the best way in this case.
• I wanted to keep the scripts as simple as possible, so that other people are able to easily modify them.

Example: