Thursday, December 2, 2010

How to create a photo slideshow synchronized with music (in Linux, of course)

If you ever tried to create a photo slideshow where the transitions between photos are synchronized with some background music, you probably found out, that it is quite difficult. After reading this tutorial, you will hopefully change your mind :o)

Tools you need:
Configure Cinelerra:
Run Cinelerra, in the main window select "Settings -> Preferences", select "Playback" tab and in "Audio Out" section, set the "Playback buffer size" to the lowest possible value (in my case 2048). Then in the "Video Out" section, there is a "Video Driver" drop-down menu. This is a bit tricky. If your graphic card supports OpenGL, you can select "X11-OpenGL", but I experienced few crashes most probably because of this driver. So let's use "X11" or "X11-XV" until we really need the OpenGL (which will become handy after we add pan-and-zoom effect and fading between images, but let's start with "X11").

Edit: The default length of inserted images in Cinelerra is set to 1 frame, so to change that, go to "Settings -> Preferences -> Recording -> Images" and check "Import images with a duration of" and set "3.0000" seconds besides that.

Configure scripts:
If you open the scripts cinelerra_transitions_on_labels.pl and cinelerra_transitions_on_labels_2vidtracks.pl, you will find few configuration variables at the beginning. There are comments, so you should be able to set them properly. There are some default values, so you don't have to make any changes there.

    Step-by-step:
    Now we will create a slideshow in 1280x720 resolution.
    1. Setup your project. Create a directory for your project. Let's say it is /home/miso/video1.  Now create directories images, images/1280x720, sound. Copy the pictures you want to use into the images directory and the background music into the sound directory.
    2. Scale your images. To work with images in Cinelerra, it is better to scale them to the final resolution of the video:
      cd images
      for i in *.jpg; do

          convert $i -resize 1280x1280 1280x720/$i
      done
      cd ..
    3. Create a Cinelerra project. Run Cinelerra, in the main window's menu select "File -> New" and set the "Audio tracks: 2", "Audio channels: 2", "Frame rate: 30", "Width: 1280", "Height: 720" and click "OK". Then select "File -> Load files". Go into the directory images/1280x720 and select all your images (using Shift key). At the bottom of the file selection dialog, set the "Insertion strategy" to "Concatenate to existing tracks". Then repeat this step for your music file. Now you should see your pictures in track "Video 1" and your music in "Audio 1" andl "Audio 2" (because it is stereo).

    4. Turn off keyframing and set drag-and-drop editing. As shown on this image:
    5. Order your images. Reorder your images if you want. Just drag any image on the timeline with your mouse and drop it between other two images. You can scale the timeline using "Shift + mouse wheel". Don't pay any attention to the duration of images for now.
    6. Create beat-labels. Go to the start of your video (Home key) and play it (Spacebar). Don't look at your images, just listen to the music and when you detect a beat, press "L" key on your keyboard (lowercase). It will create a label, a green triangle on the timeline, which will be used as a transition point between two images. So let there enough time between labels (about 3 seconds). If you missed a beat, just click anywhere on the timeline and press "L". To remove a label, click on the label's green triangle on the timeline (it will get red) and press "L". Save the project "File -> Save" as slideshow.xml.
    7. Align images on labels. Run this script in your terminal:
      ./cinelerra_transitions_on_labels.pl slideshow.xml
      Then in Cinelerra, go to "File -> Load files", select the slideshow.xml and set the "Insertion strategy" to "Replace current project". Images are now aligned on the labels. You can play your video to see, if there is no mistake. you can still add new labels and remove the old ones, but then save the project again and repeat this whole step with running the alignment script and reloading the project in Cinelerra.
    8. Cross-fading and Ken-Burns (pan and zoom) effect. Run this script in your terminal:
      ./cinelerra_transitions_on_labels_2vidtracks.pl slideshow.xml
      A new file slideshow_2vidtracks.xml is created. Load the new project file in Cinelerra as in the previous step. Now your images are spread in two video tracks with cross-fading transitions and with a random pan and zoom. Animation keyframes are placed at each label's position.

      Change the video driver to "X11-OpenGL" (as described above in "Configure Cinelerra" part). Play your video. Since the panning and zooming is randomly generated, you will probably need to tune it. Turn on "Generate keyframes while tweaking" icon (it looks like a key: ) in the main window's toolbar right under the main menu. Select a green triangle of the label at the start of the image (it will get red as you click on it):

      Place your mouse cursor anywhere on the video track on which the affected image resides and press "Shift + Tab" on your keyboard (to arm only that track; arm more tracks with "Tab"). The track gets armed (a small red circle icon on the left turns on) which means that any editing will affect only that track.

      Now go to the "Compositor" window and in the left toolbar select "Adjust camera automation" icon and "Show tool info" icon. A small dialog with X (horizontal), Y (vertical), Z (zooming) values and alignments appears:

      We are now going to adjust the camera at that selected label. There are actually two overlapping images in the Compositor window, each with 50% visibility.

      Now click and drag with your mouse in the Compositor's video to adjust the camera position. X and Y values of one image will change (only the armed track is affected). To adjust zooming factor, set the Z value manually. When you are satisfied with the starting point of the image, select the label at the end of the image and adjust the camera the same way.
    9. Render your video. you can find many tutorials on Cinelerra rendering, but I use this. Select "File -> Render", set the "File format" to "MPEG Audio", check "Render audio tracks" and configure it (with the icon next to the checkbox) to "Layer: III", "Kbits per second: 160" or whatever you like. Uncheck "Render video tracks". Set "Render range: project" and "Insertion strategy: Insert nothing". At the top of the dialog, set the correct filename for your mp3 and confirm the dialog. When the audio is rendered, you can render the video tracks. Select "File -> Render" again, but now set the "File format" to "YUV4MPEG Stream", uncheck "Render audio tracks" and check "Render video tracks". Configure video rendering with the icon next to the checkbox. Set the correct output path (filename extension should be ".m2v"), check "Use pipe" and set there this string:
      mpeg2enc -f 8 --verbose 0 --multi-thread 4  --aspect 3 --format 3  --video-bitrate 12000 --nonvideo-bitrate 384  --no-constraints  --video-buffer 448  --frame-rate 5  --interlace-mode 0  --force-b-b-p  --video-norm n  --keep-hf  --sequence-header-every-gop --min-gop-size 6 --max-gop-size 6  -o %
      Set the bitrates at your will. Confirm the dialog and after a while your video is ready. Now you need to merge the audio and video by running this command in your terminal:
      mplex -f 3 -b 2000 video.mp3 video.m2v -o video.mpg
      So your final rendered video file is "video.mpg". To transcode to other formats, I like to use avidemux.
    Tips:
    • To see the keyframes on the video tracks, select in the main menu "View -> Camera X, Y and Z", arm both video tracks and click on the icon "Fit all autos to display" (or press "Alt + f").
    • While creating beat-labels, the waveform in the audio track may be useful.
    • Cinelerra has lots of options and possibilities, so feel free to read some more Cinelerra tutorials.
    • Keyframes in Cinelerra are interpolated with bezier curves. Therefore panning and zooming starts slowly, then gets faster and ends slowly again. If you want to make it linear, select the part you want to linearize (or press "Ctrl + A" to select the whole video), arm the tracks you want to change (with mouse and Tab key) and in the main menu select "Keyframes -> Straighten curves".
    Notes:
    • I tried to make use of an automatic beat-detection, but sometimes there are music rhythms where you don't want to put the transitions on the strongest beats. So doing it manually is probably the best way in this case.
    • I wanted to keep the scripts as simple as possible, so that other people are able to easily modify them.
    Example:

      14 comments:

      Prost said...

      Hello Michal,

      Thanks for a very useful
      tutorial - I have successfully
      created a few slideshows using
      your method. I am also interested
      in automatic beat detection - could
      you please let me know what software
      you were using for that.

      Thanks,
      Prost

      Michal Fapšo said...

      Hi Prost, it is nice to hear that someone can use that tutorial.

      For an automatic beat-detection, you can try the Aubio library (http://aubio.org/). In the source package, there is a simple example "aubioonset" for beat-detection.

      Other options are bpmdj (http://bpmdj.yellowcouch.org/index.html) or mixxx (http://www.mixxx.org/).

      Craig said...

      This is great, not quite finished yet but just wondered if this is a typo:
      convert $i -resize 1280x1280 1280x720/$i

      Should the 1280x1280 not be 1280x720?

      Maybe you got away with this but I wanted to do 1920x1080 and found I couldn't get it resized correctly. Had to go back to the IM man to sort out what was wrong.

      Thanks for a great article though, commandline processing is so much quicker and easier, if you get the syntax and variables right!!

      Cheers,
      Craig

      Michal Fapšo said...

      Hi Craig, thanks for your comment.

      The resize argument 1280x1280 shoud be correct for both portrait and landscape orientation of photos. If you use -resize 1280x720, then it resizes a potrait photo to 720 pixels height.

      Maybe it would be better for potrait photos to keep the width at 1280 as for landscapes, so that you can fill the whole video frame with it.

      david hardy said...

      Hi, great tutorial, but it's not really working for me...when I run the 1st script, and try to load the new file, cinelerra crashes. I get " ERROR!
      Trying to insert edit at position, but failed: -12" in the command window. So there must be an issue with my labels not being in the right place, maybe 2 labels on one slide? Haven't figured it out yet...david

      Michal Fapšo said...

      Hi David, could you please send me your project's XML file? I haven't use this script for some time, so maybe it is not compatible with newer versions of Cinelerra. Which version do you use?

      david hardy said...

      I finally got the 1st script to work by carefully placing the labels just before the end of each slide - then, importantly - one just before the end of the last slide. I still can't get the 2nd script to work, though. It creates the crossfade transitions, but all but about .25 seconds of the 1st slide are missing. I'm happy with just the standard transitions for now - though it would be great to use the Ken Burns style.

      I am using Cinelerra 2.2-0.3. I'll try and send the XML file.

      Monica Lawrence said...


      Best freeware to upload photo slideshows with background music to YouTube. Image effects, direct YouTube upload! Free download & video how-to make photo slide.

      Cecilia said...

      Hi Michal,
      First of all, thank you for your work !
      I don't know what to do with the .pl scripts. I download them and I get .txt files. I turn them into .pl files by renaming them. Then I go to usr/bin. I can see a file called "perl" in it...so what must I do with the .pl files I've down loaded ? I can't even paste them in "bin". What can I do ?
      Thanks.
      Cecilia

      Michal Fapšo said...

      Hi Cecilia, it is described in the steps 7 and 8. You have to create a cinelerra project with labels on places where you want image to switch. Then your .pl files have to have an executable flag (chmod a+x *.pl) or prepend "perl " to the commands. Then run "perl ./cinelerra_transitions_on_labels.pl slideshow.xml" and then "perl ./cinelerra_transitions_on_labels_2vidtracks.pl slideshow.xml". If you still have problems running those scripts, maybe send me an email :o)

      Happy new year to all Cinelerra users!

      tbone said...

      Hi Michal

      Great tutorial you have here, but I can't get it to work!

      The problems seem to arise with the perl scripts. The first one doesn't work at all. It runs, but when I try to open the xml file in cinelerra afterwards it just won't open. In fact at this stage cinelerra just closes down!

      I got past this by adjusting the timing of the photo transition manually. The second perl script does a bit better, but it somehow loses the pictures in the process! So there are 2 video streams, with a white line switching between them.... but no pictures in either stream!

      Do you have any idea why this would be?

      Michal Fapšo said...

      Thanks tbone, I haven't use these scripts for about three years and there may be some bugs of course. I don't know when I will be able to correct them, so if you could do it, that would be great! If you need any help from me, I will gladly give you some hints on how to analyse and correct those bugs. Just write me an email to michal.fapso at gmail dot com

      Anonymous said...

      Hi Michal,

      Did you and tbone workout the issue with the perl scripts? I'm running into the same issue.

      Thanks
      Cujo

      Michal Fapšo said...

      Sorry Cujo, it is still not solved :o(