UsingPrograms

From HerzbubeWiki
Jump to: navigation, search

Manuell CDs und DVDs brennen

hdiutil

hdiutil ist ein Tool zum Anlegen, manipulieren und brennen von Disk Images. Viele seiner Funktionen können mit "Disk Utility.app" (Festplatten-Dienstprogramm) via GUI genutzt werden. Generell gibt es eine sehr gute man page für hdiutil.

hdiutil makehybrid -default-volume-name xxx -o yyy zzz

--> bastelt ein Hybrid-Image mit Filenamen yyy.iso mit dem Inhalt des Verzeichnis zzz und dem Volume-Titel xxx

hdiutil burn xxx.iso

--> brennt das Image auf Disk

hdiutil burn -list

--> listet die verfügbaren Devices auf (kann evt. als Input für cdrecord verwendet werden)


Umgang mit cdrecord und mkisofs (veraltet)

mkisofs -v -hfs -V "xxx" -R -J -probe yyy >zzz-image

--> bastelt ein HFS Volume Image mit dem Inhalt des Verzeichnis yyy und dem Volume-Titel xxx.

--> unbestätigt, aber evt. wahr: Volume-Titel sollte keine Spaces enthalten

cdrecord -dev=IOCompactDiscServices -v -speed=24 fs=8m -overburn -dao -eject xxx-image

--> IOCompactDiscServices ist das Device, unter dem der Record angesprochen werden kann

--> fs=8m gibt einen unnötig grossen Buffer an - aber man will ja sicher gehen

--> dao = disc at once; macht bei Overburning von grossen Images sicherlich Sinn


eject disk1 --> builtin CDROM eject disk2 --> external CDROM

From fink-users:

  • prepare Joliet compliant iso image from a directory:
mkisofs -J -R -V "name of the CD" -o output_file.iso somedirectory/
  • burn audio CD:
cdrecord dev=IOCompactDiscServices speed=12 -v -audio -pad somelistofrippedaudio(.aiff or .wav)

- to turn burnfree on, you need to add driveropts=burnfree to the cdrecord command

cd /mnt/stuff/Archiv/Warez/Photoshop
cdrecord -dev=IOCompactDiscServices -v -speed=24 fs=8m -dao -eject Photoshop.7.iso


wget

Kommando zum Saugen eines ganzen Verzeichnis-Baums von einer Site wget -rkp -l 0 http://foo.bar/dir1/index.html

Das gleiche, aber mit der Einschränkung nur Zeug unter dir1 zu holen -> für den Fall, dass auf irgendeiner Seite noch ein "klitzekleiner" Link besteht, der in einen anderen Verzeichnisbaum "ausbricht" wget -rkp -np -l 0 http://foo.bar/dir1/index.html

Kommando zum Holen einer einzigen Seite mit allen Prerequisites, auch wenn diese auf anderen Hosts liegen: wget -Hkp http://foo.bar/index.html

-r Rekursiver Download wget -r http://foo.bar/index.html

-H Host-übergreifender Download -L Nur relativen Links folgen (eine Seite könnte ja auch einen absoluten Link auf ein Dokument des gleichen Host enthalten) -np Niemals in der Verzeichnis-Hierarchie weiter als bis zum angegebenen Base-Verzeichnis springen.

-p Prerequisites downloaden, d.h. alles, was zum Ansehen einer Seite wichtig ist (Images, Sounds, Stylesheets)

-l Tiefe (depth). Default = 5. 0 = unendlich Damit wird angegeben, wieviele Hops von der Ursprungs-URL weggegangen werden darf. Mit Tiefe 2 wird 1.html->2.html->3.html weit gesprungen.

-k Links konvertieren für lokales Browsen. Relative Links auf Elemente, die nicht heruntergeladen worden sind, werden ins absolute Format http://... umgewandelt.

-c Mit abgebrochenem Download weitermachen (continue)


CVS

Die Umgebungsvariable CVSROOT sollte gesetzt sein:

export CVSROOT=/Volumes/data/Development/cvsroot

Creating a new CVS repository:

See: file://localhost/data/Infos,%20Manuals,%20Artikel/References/cederqvist-1.11.3.html/cvs_2.html#SEC23

Adding a new directory tree to CVS

cvs import -m "Initial import comment" target/directory/in/repository foobar start

See: file://localhost/data/Infos,%20Manuals,%20Artikel/References/cederqvist-1.11.3.html/cvs_3.html#SEC40

Tags

Tags are "artificial" revisions of a file. A normal revision may go up from 1.20 to 1.21 automatically after a file is committed. If you tag a file then you mark a given revision of a file with, without committing the file.

To assign tag to a file: cvs tag foobar.c To view all tags on a file: cvs status -v foobar.c To assign tag to all files in a directory and its subdirs: cvs tag . To checkout all files for tag : cvs checkout -r .

To assign tag to files only that already contain tag : cvs tag -r . To assign tag to revisions of files no later than : cvs tag -D . In addition to the last 2 commands: option -f specifies to tag the most recent revision of a file in the repository, instead of ignoring the file, if -r or -D don't produce a match.

See: file://localhost/data/Infos,%20Manuals,%20Artikel/References/cederqvist-1.11.3.html/cvs_4.html#SEC53

Steps to take before checking in to CVS

  1. Optionally: make clean to prevent binaries and other stuff from getting into CVS
  2. cvs add
  3. cvs remove
  4. cvs commit on project's top level


ImageMagick

  • Bilder konvertieren:
for i in *bmp; do
  convert -size 1024x768 "$i" -resize 1280x1024 -quality 90 "$i.jpg";mv *jpg ../jpg
done
  • Bilder skalieren (das Ratio wird beibehalten, falls die neue Grösse nicht passt, wählt convert selbst eine passende Grösse, die dem ursprünglichen Ratio am nächsten kommt):
for i in *jpg; do
  convert -sample 1600x1200 "$i" "new-$i"
done
  • Grösse bestimmen:
identify x.jpg

Weitere Infos: die ImageMagick Homepage hat Links zu verschiedenen Artikeln, z.B. http://www-106.ibm.com/developerworks/library/l-graf/?ca=dnt-428


rsync

Ein rsync Server verwendet Port 873

Beispiel für einen Aufruf auf der Kommandozeile:

rsync -e ssh -avz patrick@osgiliath:/var/backups ~/Desktop/backups
  • -e = shell, die verwendet werden soll
  • -a = archive (oder --archive), Kurzform fuer "recursive and preserve almost everything"
  • -v = verbose (oder --verbose)
  • -z = compress (oder --compress)
  • -n = simulation (oder --dry-run)
  • -q = quiet (oder --quiet)
  • Trailing Slash bei einem Verzeichnis = nur den Inhalt des Verzeichnisses
  • Kein Trailing Slash = das Verzeichnis selbst, inkl. alle Subverzeichnisse


Bei Verwendung in cron:

  • Option -v durch -q ersetzen
  • sowie rsync muss als Daemon auf der Remote Maschine laufen, wenn keine ssh Verbindungs-Optionen (= Passwort) in einem File gespeichert werden sollen.
  • der rsync Daemon muss das gewünschte Verzeichnis als Modul mit anonymous Zugriff exportieren, wenn keine rsync Verbindungs-Optionen (= Passwort) in einem File gespeichert werden sollen

Der crontab Eintrag kann z.B. so aussehen:

0 20,21,22,23,0 * * *  rsync -aqz osgiliath::backup ~/Desktop/backup

Wenn rsync als Daemon auf der Remote Maschine läuft, koennen folgende Kommandos nützlich sein:

  • Alle verfügbaren Module abrufen
rsync patrick@osgiliath::
  • Den Inhalt eines bestimmten Moduls auflisten
 rsync patrick@osgiliath::backup


Toolset für Sound-Verarbeitung

Tools installieren

Homebrew tools

Install via Homebrew:

  • libogg, libogg-shlibs
  • libvorbis0, libvorbis0-shlibs
  • lame, lame-dev, lame-shlibs
  • libmad, libmad-shlibs, madplay
  • flac-nox, flac-nox-bin, flac-nox-shlibs
  • libsndfile1, libsndfile1-dev, libsndfile1-shlibs
  • mppdec
  • ffmpeg. Note that several additional non-standard codecs can be installed - run brew info ffmpeg for information, and/or visit the FFmpeg wiki. I typically run brew install ffmpeg --with-fdk-aac --with-libmodplug --with-libvorbis --with-libvpx --with-openh264 --with-opus --with-theora --with-tools --with-two-lame --with-x265


sox

Download von http://www.sourceforge.net/projects/sox

Entpacken in /usr/local/src, danach wie gewohnt configure, make, make install.

Besonderheiten: damit die von fink bereit gestellten Libraries gefunden werden, müssen dem configure Script einige Compiler-Optionen mitgegeben werden:

./configure CPPFLAGS="-I/sw/include" LDFLAGS="-L/sw/lib"


mac (Monkey's Audio Compressor)

Download von http://www.sourceforge.net/projects/mac

Entpacken in /usr/local/src, danach wie gewohnt configure, make, make install

Besonderheiten: Zumindest die Version mac-3.99-u4-b5 hat einen Linker-Fehler mit gcc 4. Ein Tracker-Item (http://sourceforge.net/tracker/index.php?func=detail&aid=1377803&group_id=123827&atid=699624) weist darauf hin, dass das Problem dadurch umgangen werden kann, dass beim Linken gcc statt g++ verwendet wird. Auf meiner Maschine führt dieser Workaround zu anderen Linker-Fehlern, denen nicht beizukommen war. Auf der fink-users Mailing Liste (http://www.mail-archive.com/fink-users@lists.sourceforge.net/msg21840.html) war dann der endgültige Hinweis mit MACOSX_DEPLOYMENT_TARGET zu finden.

MACOSX_DEPLOYMENT_TARGET=10.3
./configure --prefix=/usr/local
make
make install
unset MACOSX_DEPLOYMENT_TARGET


shntool

Download von http://www.etree.org/shnutils/shntool/

Entpacken in /usr/local/src, danach wie gewohnt configure, make, make install

Hinweis: Dieses Tool muss als letztes der Serie installiert werden, da es eigentlich nur ein Front-End ist und deshalb davon abhängt, was bereits alles vorhanden ist.


Anwendungsfälle

CD Image im .ape Format in Tracks aufsplitten

Im folgenden Beispiel liegen die Track-Informationen in einem .cue File vor. Es wäre aber auch möglich, die Split-Punkte auf der Kommandozeile anzugeben.

shntool split -f CDImage.ape.cue CDImage.ape


Batch-Umwandlung von .wav Files in .mp3

for i in 01 02 03 04 05 06 07 08 09 10; do lame --preset extreme track$i.wav $i.mp3; done


Konvertierung mit ffmpeg

Folgendes Kommando konvertiert Sound Files, wobei das Input/Output Format anhand der File Extension (.wav, .mp3, etc.) erkannt wird:

ffmpeg -i <input file> <output file>


Konvertierung mit flac

Folgendes Kommando konvertiert eine .flac Datei in eine .aiff Datei, wobei der Name des Output Files explizit angegeben wird und das Format implizit aus der File Extension abgeleitet wird:

flac --decode --output-name=<output file>.aiff <input file>

Folgendes Kommando konvertiert mehrere Dateien von .flac in .wav

flac --decode *.flac

Ditto, aber das Ausgabe Format explizit als .aiff festlegen:

flac --decode --force-aiff *.flac


Spaceward Ho!

Special ship designs with the following Tech combinations:

  • WE 12, WA 15, SC 15, TE 1, MI 0
  • WE 11, WA 12, SC 12, TE 1, MI 0


Convert .001/.002 etc. to .bin/.cue

On some file sharing sites you may encounter archives (often .rar archives) that contain a lot of small files with extensions .001/.002 etc. These files are chunks of binary data that can be re-assembled into a set of .bin/.cue files like this:

  • Install 7-Zip
  • Extract the archive
  • Right-click on the .001 file
  • Select "Extract to here" from the context menu

The resulting .bin/.cue files will be placed in the current folder.


Convert .bin/.cue to .iso

Install bchunk via Homebrew/MacPorts/whatever, then run this command line:

bchunk foo.bin foo.cue foo.iso


Working with multimedia container files with ffmpeg

Introduction to containers

Container files can contain any number of data streams, and the same container file may contain streams composed of different data types: Audio data, video data, subtitles data, or just data without a specific type. Within the container, streams do not exist as fully separate chunks of data (e.g. the first 100 MB are audio, the next 700 MB are video, etc), but are all mixed together. The process of interleaving audio, video and other streams into a single coherent data stream is called "multiplexing", which usually is shortened to "muxing". When the content of the container file is played, the player software extracts the individual streams from the multiplexed data stream via a process usually called "demuxing".


Now if you use ffmpeg to manipulate a multimedia file, this muxing and demuxing also has to take place - even if you only want to to something simple as adding some metadata. ffmpeg in that case does not need to understand a given audio or video encoding format (the codec), but it must understand the container format, because the specification of that format determines how the individual streams need to be muxed.


Introduction to ffmpeg

For input files, the file extension does not seem to be relevant, apparently ffmpeg determines the container format by probing the input file's content. For instance, ffmpeg is happy to process .m4v files that were produced by HandBrake. However, for output files the file extension matters. For instance, to create an MP4 output file you must use the .mp4 file name extension. Specifically, the .m4v file extension does not work.


General ffmpeg command line principles:

  • Options are applied to the next specified input/output file. The order in which options appear therefore is important, and some options may appear more than once.
  • First specify global options, then specify input files with their options, then specify the output file with its options
  • ffmpeg refers to input files and streams via 0-based indices: The second input file has index 1, the third stream has index 2, the fifth stream in the first input file is referenced with "0:4". Streams can also be referred to by type: "a" for audio, "v" for video, "s" for subtitle, "d" for data and "t" for attachment. For instance, v:2 is the third video stream. See the man page for even more ways how to specify a stream.


References:

  • man ffmpeg


Other concepts

Keyframe, key frame 
From the Wikipedia article: "In video compression, a keyframe, also known as an "intra-frame", is a frame in which a complete image is stored in the data stream. In video compression, only changes that occur from one frame to the next are stored in the data stream, in order to greatly reduce the amount of information that must be stored. [...] Whenever a drastic change to the image occurs, such as when switching from one camera shot to another, or at a scene change, a keyframe must be created. The entire image for the frame must be output when the visual difference between the two frames is so great that representing the new image incrementally from the previous frame would require more data than recreating the whole image."


Homebrew installation of ffmpeg

When you install ffmpeg via Homebrew you can specify a lot of options to enable (or disable) certain codecs. To view all available options, run

brew info ffmpeg

The following command enables all options that were available at the time of writing (August 2018):

brew install ffmpeg --with-chromaprint --with-fdk-aac --with-fontconfig --with-freetype --with-frei0r --with-game-music-emu --with-libass --with-libbluray --with-libbs2b --with-libcaca --with-libgsm --with-libmodplug --with-librsvg --with-libsoxr --with-libssh --with-libvidstab --with-libvorbis --with-libvpx --with-opencore-amr --with-openh264 --with-openjpeg --with-openssl --with-opus --with-rtmpdump --with-rubberband --with-sdl2 --with-snappy --with-speex --with-srt --with-tesseract --with-theora --with-tools --with-two-lame --with-wavpack --with-webp --with-x265 --with-xz --with-zeromq --with-zimg


In a similar manner, the following command installs libav with all available options enabled:

brew install libav --with-fontconfig --with-frei0r --with-gnutls --with-libvo-aacenc --with-opencore-amr --with-openssl --with-rtmpdump --with-schroedinger --with-sdl --with-speex --with-theora


List content of a container

List the content of a container:

ffprobe foo.mp4

View detailed information for all streams in a container:

ffprobe -show_streams foo.mp4

View detailed information for a selected stream:

ffprobe -show_streams -select_streams 21 foo.mp4

Show information about each frame in a selected video stream. This can be useful for identifying keyframes, i.e. frames where a video can be split. Warning: This generates massive amounts of data!

ffprobe -show_frames -select_streams v:1 foo.mp4

To reduce the amount of data generated by the -show_frames option, the interval examined by ffprobe can be specified with the -read_intervals option. Read the man page for the exact syntax, here are a few useful examples:

# Read 6 seconds of frames, between seconds 5 and 10 (including 5 and 10)
ffprobe -show_frames -select_streams v -read_intervals 00:00:05%00:00:10 foo.mp4
ffprobe -show_frames -select_streams v -read_intervals 5%10 foo.mp4
ffprobe -show_frames -select_streams v -read_intervals 5%+6 foo.mp4
# Read from the start until second 10 (including 10)
ffprobe -show_frames -select_streams v -read_intervals %10 foo.mp4
# Read the first 5 seconds from the start
ffprobe -show_frames -select_streams v -read_intervals %+5 foo.mp4
# Read the first 20 frames from the start
ffprobe -show_frames -select_streams v -read_intervals %+#20 foo.mp4
# Read 20 frames after position 01:02:03
ffprobe -show_frames -select_streams v -read_intervals 01:02:03%+#20, foo.mp4


Copy streams from an input to an output file

Copy all streams from an input to an output file, effectively creating a duplicate of the original. This is a bit nonsensical, but illustrates the use of the "copy" pseudo-codec and a simple application of the -map option.

ffmpeg -i foo.mp4 -map 0 -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 bar.mp4

Discussion:

  • The "copy" pseudo-codec prevents decoding/encoding of data streams, instead it preserves the streams' data without change
  • The various "-copy..." options may or may not be necessary, but they all ensure that some aspect of the original input files remains unchanged
  • The -map option specifices which input stream(s) should be copied to the output file. In the example we only have a single input file, which is referenced by the index number 0. Because we didn't specify any streams, ffmpeg will copy all streams that it finds in the input file.
  • -map_metadata makes sure that all meta data from input file 0 is preserved. In theory, this option should not be necessary because the default should already preserve meta data. In practice, however, without the option I saw ffmpeg removing the meta data tag "creation_time".
  • ffmpeg always changes the major_brand metadata tag to "isom", even if it previously was set to "mp42" (e.g. by HandBrake). If this Trac entry is to be believed, the reason is that ffmpeg is capable of writing only mp41-compatible files, which results in major_brand to be displayed as "isom".
  • When you diff the stream listing from an output file with the stream listing from an input file, you might see that audio stream properties like bit_rate or max_bit_rate have changed. I can only hope that this is not a problem, and that ffmpeg has changed these values because they were incorrect in the first place.


A general problem is that ffmpeg does not preserve the handler_name meta data tag of audio streams - instead it changes the tag to "SoundHandler", regardless of the original value (e.g. HandBrake sets this to "Stereo" or "Surround"). This is an important issue because the handler name is used by player software such as VLC to indicate what kind of content the audio stream contains. The problem can be fixed by adding one -metadata option for each audio stream, like this: -metadata:s:a:0 handler=Stereo. This example sets the first audio stream's handler name. For a more complete example, in a file with 14 audio streams I ended up with the following spectacular (and cumbersome) command line.

ffmpeg -i foo.m4v -map 0 -codec copy -copy_unknown -copyts -map_metadata 0 -metadata:s:a:0 handler=Stereo -metadata:s:a:1 handler=Surround -metadata:s:a:2 handler=Stereo -metadata:s:a:3 handler=Stereo -metadata:s:a:4 handler=Stereo -metadata:s:a:5 handler=Stereo -metadata:s:a:6 handler=Stereo -metadata:s:a:7 handler=Stereo -metadata:s:a:8 handler=Stereo -metadata:s:a:9 handler=Stereo -metadata:s:a:10 handler=Stereo -metadata:s:a:11 handler=Surround -metadata:s:a:12 handler=Stereo -metadata:s:a:13 handler=Surround bar.mp4


Merge selected streams from several input files

Here's an example that merges selected streams from two input files. The input files both contain the same video but have different audio streams and subtitle tracks. Such a scenario can occur if you rip a DVD with HandBrake, and the DVD contains two titles with the same video.

ffmpeg \
       -i "foo.m4v" \
       -i "bar.m4v" \
       -map_metadata 0 \
       -map_metadata 1 \
       -map 0:v:0 \
       -map 0:a:0 -map 0:a:1 -map 0:s:1 \
       -map 0:a:10 -map 0:a:11 -map 0:a:12 -map 0:a:13 -map 0:s:2 -map 0:s:3 \
       -map 0:s:0 \
       -map 1:a:2 -map 1:a:3 -map 1:a:4 -map 1:a:5 -map 1:a:6 -map 1:a:7 -map 1:a:8 -map 1:a:9 \
       -metadata:s:a:0 handler="Stereo" -metadata:s:a:1 handler="Surround" -metadata:s:a:2 handler="Stereo" -metadata:s:a:3 handler="Surround" \
       -metadata:s:a:4 handler="Stereo" -metadata:s:a:5 handler="Surround" \
       -metadata:s:a:6 handler="Stereo" -metadata:s:a:7 handler="Stereo" -metadata:s:a:8 handler="Stereo" -metadata:s:a:9 handler="Stereo" \
       -metadata:s:a:10 handler="Stereo" -metadata:s:a:11 handler="Stereo" -metadata:s:a:12 handler="Stereo" -metadata:s:a:13 handler="Stereo" \
       -codec copy -copyinkf -copy_unknown -copyts \
       "baz.mp4"

Discussion:

  • The order in which the -map options appear determines the order in which ffmpeg generates output streams
  • The -metadata options refer to output streams
  • A description of the output streams:
    • One video stream from the first file. The video stream from the second file is ignored.
    • Two audio streams and one subtitle track from the first file. These contain the English language audio and subtitles. There are two audio streams because HandBrake generates one stream with codec "AAC" and one stream with codec "AC3 Passthru" from the same original DVD audio.
    • Four audio streams and two subtitle tracks from the first file. These contain the German language audio and subtitles. One pair of audio streams are for "Dolby Digital 5.1 EX", the other pair are for "DTS ES 6.1". The first subtitle track contains the regular German subtitles, the other contains subtitles for the hearing-impaired.
    • One subtitle track from the first file. This contains the "Foreign Audio Search" subtitles generated by HandBrake.
    • Eight audio streams from the second file. These contain 4 different audio commentaries. Each audio commentary is comprised of two audio streams.


Here's an example that concatenates multiple segments of a video into one single video file. The command also concatenates several audio streams and one subtitle track. The command is based on this ffmpeg wiki page. It uses the so-called "concat demuxer", which is documented here.

ls *segment*.mp4

foo, segment 1.mp4
foo, segment 2.mp4
foo, segment 3.mp4
[...]

for f in ./*segment*.mp4; do echo "file '$f'" >segmentlist.txt; done

ffmpeg \
       -f concat -safe 0 -i segmentlist.txt \
       -map_metadata 0 \
       -map 0:0 \
       -map 0:1 -map 0:2 \
       -map 0:3 \
       -metadata:s:a:0 handler="Stereo" -metadata:s:a:1 handler="Stereo" \
       -codec copy  -copyinkf -copy_unknown -copyts \
       "foo, all segments.mp4"

Discussion:

  • The -f option is used to force the format of the next input (or output) file. ffmpeg normally auto detects the format for input files and guesses the format from the file extension for output files. In this case we must explicitly specify the "concat" format to let ffmpeg know that it will have to use the "concat" demuxer on the files listed in the text file.
  • The -safe 0 option is not documented on the man page. The option is here because the command line shown here is based on an example from the ffmpeg wiki page on concatenation. The wiki page says that "The -safe 0 above is not required if the paths are relative."
  • The -i segmentlist.txt option specifies an input file that is a "virtual concatenation script". This together with the -f concat option creates a virtual input "file" that consists of the files in the text file concatenated together.
  • The -map_metadata, -map and -metadata options all relate to the virtual input "file" created by the concat demuxer. They are explained in more detail further up in the notes for previous examples.


Cutting video without transcoding

This example copies all streams from an input to an output file, but cuts off everything before the 08:04 mark and after the 01:32:26 mark.

ffmpeg -i foo.mp4 -ss 00:08:04 -to 01:32:26 -map 0 -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 bar.mp4

Discussion:

  • The -ss option performs a seek operation. The option can be used as an input option (before the -i option) or as an output option (after the -i option), which makes a big difference when the copy pseudo-codec is used. If used as an input option the seek is fast but not accurate: ffmpeg in that case seeks to the nearest keyframe (which is fast because ffmpeg does not have to decode the stream) and begins copying from there. If used as an output option the seek is slow but accurate: ffmpeg actually decodes the stream and is therefore capable of copying from the exact seek position that was specified. For details see the ffmpeg Trac page on seeking.
  • The -to option specifies the position where to stop copying. Because the -ss option was used as an output option, the timestamps from the original input file remain intact and we can use such a timestamp for -to. If -ss were used as an input option, the timestamps seen by -to would begin at zero and we would have to specify 01:24:22 (01:32:26 minus 00:08:04).
  • Timestamps and durations can be expressed as either "HH:MM:SS[.m]", or just "S[.m]". In both cases, "m" is an optional fraction of seconds.

The following example is equivalent to the one above, the only difference is that we use the -t option instead of the -to option. The -t option specifies the duration to copy and is independent of whether the -ss option is used as an input or output option.

ffmpeg -i foo.mp4 -ss 00:08:04 -t 01:24:22 -map 0 -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 bar.mp4


The problem with frame-accurate cutting (i.e. when you use -ss as an output option) is that if you don't pay attention the output video might not start with a keyframe. The result in that case is that when you view the video the first few seconds probably look garbled, until the next keyframe occurs in the output video and the video player software is able to display a correct image. The workflow to find the nearest keyframe to cut on looks like this:

  • Find a timestamp where you want to start cutting. This can be very simple e.g. by viewing the video in VLC, or some other video player software, and noting down the time where you would like to cut.
  • Use ffprobe to inspect a window of about +/- 5 seconds around the desired cut position. Let's say you would like to cut at position 00:05:00, then the command would be this:
ffprobe -show_frames -select_streams v -read_intervals 00:04:55%+10 foo.mp4 >frames.txt
  • In the output search for a keyframe (key_frame=1) that is closest to the desired cut position. If the cut position is 00:05:00 then the keyframe would be one that is located around pkt_dts_time=300.000000 (5 minutes = 300 seconds).
  • Note down the pkt_dts_time value of the frame BEFORE the keyframe. So if we assume that the keyframe nearest the desired cutting location has pkt_dts_time=296.020000, then its predecessing frame might have pkt_dts_time=296.000000.
  • Use the pkt_dts_time value to start the actual cutting. For some reason ffmpeg appears to start outputting frames not at the exact -ss location, but at one frame after.
ffmpeg -i foo.mp4 -ss 296.000000 -to 01:32:26 -map 0 -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 bar.mp4
  • Before you do the real cutting, you could make a small cut of 10 seconds and then examine the first frame of the result with ffprobe:
ffprobe -show_frames -select_streams v -read_intervals %+#1 bar.mp4


Troubleshooting

Data stream not understood

I encountered the problem that an .m4v file generated by HandBrake contained a stream which caused ffmpeg to barf. The stream in question was this:

Stream #0:21(eng): Data: bin_data (text / 0x74786574)

The error message printed by ffmpeg was this:

[mp4 @ 0x7f8b1b889600] Unknown hldr_type for text, writing dummy values   

And when I examined the output file I saw that ffmpeg had created a new pseudo-stream just before the original data stream:

Stream #0:21(eng): Data: none (stts / 0x73747473)

But apparently the stream was improperly added, because on examining the output ffprobe also printed this error message quite early during its output:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fabff021000] overread end of atom 'stsd' by 232 bytes

I was able to apparently fix the issue by adding the option -map -0:d after the original -map 0. The new command line looked like this:

ffmpeg -i foo.mp4 -map 0 -map -0:d -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 bar.mp4

With this additional option, ffmpeg preserves the data stream perfectly. I must admit, however, that I don't really understand why, because the prefixing "-" character is supposed to remove a mapping from an already established prior mapping, which in my understanding should cause ffmpeg to remove the stream entirely from the output. Well, I can only say that I found the "solution" in this pull request for the youtube-dl project, right at the bottom.


Recipe for ripping TV shows

This recipe builds on the information and commands in the previous sections, notably the sections "List content of a container" and "Cutting video without transcoding". For background detail information read these sections.


Template actions:

  • Transfer video content from iPad to laptop
  • In a terminal session, cd to the folder where the video content was copied to
  • For every video file, run the following series of commands interspersed with interactive actions using VLC:
VIDEO_FILE=10x3

ffprobe -show_frames -select_streams v -read_intervals 00:05:11%+10 *$VIDEO_FILE* 2>/dev/null | awk 'BEGIN{FS="="}{if ($1!="key_frame" && $1!="best_effort_timestamp_time"){next}; if ($1=="key_frame"){printIt=$2}; if (printIt=="1") {print}}'

ffmpeg -i *$VIDEO_FILE* -ss 312.000000 -t 10 -map 0 -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 cut.mp4

ffprobe -show_frames -select_streams v -read_intervals %+#1 cut.mp4 2>/dev/null | grep -i key_frame

rm -f cut.mp4; ffmpeg -i *$VIDEO_FILE* -ss 312.000000 -to 01:13:15 -map 0 -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 cut.mp4


Make the template actions do the right thing:

  • Set the VIDEO_FILE environment variable to a substring that uniquely identifies the file name. In the example above, "10x3" could refer to episode 10 of season 3.
  • Find the rough start position and the final end position for the cut
    • Open the video file in VLC
    • Find the start position from where you want to start the cutting. Note down the position. This is the rough start position A.
    • Find the end position where you want to end the cutting. Note down the position. This is the final end position B.
  • Find the final start position for the cut
    • In the first ffprobe command, replace "00:05:11" with position A, minus 5 seconds
    • Run the command. The command prints all keyframe positions in a window +/- 5 seconds around the cut position.
    • In the output search for the keyframe that is closest to the desired cut position. From the keyframe position take the next-lower integral number (e.g. 296.020000 => take 296.000000, or 296.000000 => take 295.000000) and note down that number. This is the final start position C.
  • Double-check that you noted down the correct final start position
    • In the second ffmpeg command, replace "312.000000" with position C, then run the command. This generates a short 10-second test cut of the video starting at the desired cut position.
    • Run the second ffprobe command. The commands prints information about the first frame in the test cut This should be a keyframe. If you noted down the correct final start position the command prints a line. If the command prints nothing, then the first frame that resulted from the cutting was not a keyframe and you need to adjust the final start position.
    • Although the test cut correctly starts at a keyframe, it's still possible that for some reason you noted down an entirely wrong final start position. Open the cut.mp4 file with VLC to manually verify that the final start position is correct.
  • Perform the actual cutting
    • In the final ffmpeg command, replace "312.000000" with position C and "01:13:15" with position B
    • Run the command.
    • This takes a while. While the cutting is in progress you can already start inspecting the next video file.
    • When the cutting is finished, open cut.mp4 with VLC to review the result. If you are happy, rename/move the file to its final destination.


Content creation

Xbox Game Bar (Windows)

Press Windows + G to launch the XBox Game Bar. Here you have various widgets that let you take screenshots or record video and/or audio.

Videos are placed in this folder:

C:\Users\<username>\Videos\Captures


OBS (Open Broadcaster Software)

OBS Studio is a cross-platform, open source software that lets the user record and/or livestream audio and video.

References: