Losslessly split/merge AVI video files

[peter_b]

I know that there are plenty ways of splitting AVI files, but I needed it with a bit more reliability and precision than most others require for home use. The video was encoded using Huffyuv (so it's lossless and only keyframes), and the audio was PCM 16bit 48kHz.

So it should be possible to exactly cut the video at a given frame/sample and merge it back together later on, without a single bit getting lost.
Well, it seems that ffmpeg/mencoder/etc are not really optimized for this, but rather for handling lossy formats (like XviD, mpeg, etc..) - so I had to fiddle around with their options a bit.

Here's what I've managed so far:
1) Split the video every minute, at exactly the last frame of each minute (e.g. 00:00:00.00 - 00:00:59.24):

Code: Select all

ffmpeg -ss $OFFSET.000 -t (($INTERVAL - 1)).999 -i $INPUT -an -vcodec copy $OUTPUT.avi

About the variables (bash-style):

• $OFFSET: time offset where to start the video sequence. You'll need a loop around the above line to get multiple segments.
  $INTERVAL: time interval (in seconds) of each segment to extract.
  $INPUT: (full length) input video filename (e.g. "long_video.avi")
  $OUTPUT: output video filename of current video segment (e.g. "long_video-003.avi" for the 3rd segment)

About the parameters:

• -ss: Start at a given timestamp. Format: "sec.msec"
  -t: Only "play" (=extract) a certain interval. Format: "sec.msec"
  -i: The input video filename (full length)
  -vcodec copy: Copy the video data as-is. Don't transcode or touch it.
  -an: Do *not* copy any audio. Yes. that's right: No audio - just the video. You'll see later on, why.

2) Extract the full length audio in one piece out of the original video and split it separately:
This is necessary in order to *make sure* we don't lose a single audio sample during split/merge due to muxing-synchronization oddities, depending on the tools used.
First of all, extract the audio out of the AVI file:

Code: Select all

ffmpeg -i $VIDEO.avi -vn -acodec copy $AUDIO.wav

Now I'm using the awesome "swiss army knife of audio processing" tool "SoX" to split/merge the audio:

Code: Select all

sox $AUDIO.wav $SEGMENT_X.wav trim $OFFSET $INTERVAL

The variables are similar to the ones used for the video. I suppose you'll get the idea how it works.

Now, in order to merge everything back, you simply concatenate the audio files like this:

Code: Select all

sox $SEGMENT1 $SEGMENT2 ... $SEGMENT_N $OUTPUT_MERGED.wav

It's bit-proof. I've checked this, by inverting the original audio and mix it together with the merged copy. The outcome was a perfect flat-line audio.

I wish it was that easy with video with ffmpeg, but it seems that mencoder is easier for that:

Code: Select all

mencoder -oac copy -ovc copy $SEGMENT1 $SEGMENT2 ... $SEGMENT_N -o $OUTPUT_MERGED.avi

The parameters "-oac copy" and "ovc copy" are again: "copy audio/video as-is. Don't transcode. Don't touch"
Again, I've verified that this method does not drop or multiply frames, by embedding a visual timecode (min:sec.frame) into the original video before splitting it (I've used kdenlive's embed-timecode-option for this).

[peter_b]

Since recently, FFmpeg can also be used to concat and transcode multiple video segments in one step.
In my case, I'm using multiple FFv1/pcm_s16le segments (1500 frames each).

Here is an example of concatenating these "minute segments":
1) First, you create a small textfile which contains each segment filename, prefixed with "file" per line.
Like this:

Code: Select all

file 'test-00815-000.avi'
file 'test-00815-001.avi'
file 'test-00815-002.avi'
file 'test-00815-003.avi'

2) Now this list of 4 segments can be given as input-file parameter to FFmpeg:

Code: Select all

$ ffmpeg -f concat -i mylist.txt -vf 'movie=logo_image.tga [logo]; [in][logo] overlay [out]' -t 70 -b:v 3000k test2.mpg

As you can see, this even allows to do processing (e.g. add a logo overlay) and transcoding all in one step.

I have not yet verified frame/sample accuracy of this method, but from subjective checks it seems fine (no jumps, no clicks).

Links:

• FFmpeg wiki: Concatenating media files

[peter_b]

Here's another concatenation example, using bash to avoid an intermediate concat-list file:

Code: Select all

ffmpeg -f concat -i <(for f in ./*.avi; do echo "file '$PWD/$f'"; done) -c copy output.avi

Source: FFmpeg Wiki

[peter_b]

Updating this list of commands
Here's another way of using FFmpeg's built-in segment muxer for splitting files:

Code: Select all

$ ffmpeg -i VIDEO_IN.avi -c copy -f segment -segment_time 60 -reset_timestamps 1 -map 0 VIDEO_OUT-%03d.avi

Arguments:

• segment_time 60: Time in seconds. Here each segment has 60seconds duration.
• %03d.avi: This is printf-syntax for 3-digit-zero-padded index. Automatically increased by FFmpeg.

File output will be:
VIDEO_OUT-000.avi
VIDEO_OUT-001.avi
VIDEO_OUT-002.avi
...and so on...

I have verified that this can be used for a lossless roundtrip: segment + concat (using framemd5 and FFV1).