UsingPrograms

From HerzbubeWiki
Jump to: navigation, search

Manuell CDs und DVDs brennen

hdiutil

hdiutil ist ein Tool zum Anlegen, manipulieren und brennen von Disk Images. Viele seiner Funktionen können mit "Disk Utility.app" (Festplatten-Dienstprogramm) via GUI genutzt werden. Generell gibt es eine sehr gute man page für hdiutil.

hdiutil makehybrid -default-volume-name xxx -o yyy zzz

--> bastelt ein Hybrid-Image mit Filenamen yyy.iso mit dem Inhalt des Verzeichnis zzz und dem Volume-Titel xxx

hdiutil burn xxx.iso

--> brennt das Image auf Disk

hdiutil burn -list

--> listet die verfügbaren Devices auf (kann evt. als Input für cdrecord verwendet werden)


Umgang mit cdrecord und mkisofs (veraltet)

mkisofs -v -hfs -V "xxx" -R -J -probe yyy >zzz-image

--> bastelt ein HFS Volume Image mit dem Inhalt des Verzeichnis yyy und dem Volume-Titel xxx.

--> unbestätigt, aber evt. wahr: Volume-Titel sollte keine Spaces enthalten

cdrecord -dev=IOCompactDiscServices -v -speed=24 fs=8m -overburn -dao -eject xxx-image

--> IOCompactDiscServices ist das Device, unter dem der Record angesprochen werden kann

--> fs=8m gibt einen unnötig grossen Buffer an - aber man will ja sicher gehen

--> dao = disc at once; macht bei Overburning von grossen Images sicherlich Sinn


eject disk1 --> builtin CDROM eject disk2 --> external CDROM

From fink-users:

  • prepare Joliet compliant iso image from a directory:
mkisofs -J -R -V "name of the CD" -o output_file.iso somedirectory/
  • burn audio CD:
cdrecord dev=IOCompactDiscServices speed=12 -v -audio -pad somelistofrippedaudio(.aiff or .wav)

- to turn burnfree on, you need to add driveropts=burnfree to the cdrecord command

cd /mnt/stuff/Archiv/Warez/Photoshop
cdrecord -dev=IOCompactDiscServices -v -speed=24 fs=8m -dao -eject Photoshop.7.iso


wget

Kommando zum Saugen eines ganzen Verzeichnis-Baums von einer Site wget -rkp -l 0 http://foo.bar/dir1/index.html

Das gleiche, aber mit der Einschränkung nur Zeug unter dir1 zu holen -> für den Fall, dass auf irgendeiner Seite noch ein "klitzekleiner" Link besteht, der in einen anderen Verzeichnisbaum "ausbricht" wget -rkp -np -l 0 http://foo.bar/dir1/index.html

Kommando zum Holen einer einzigen Seite mit allen Prerequisites, auch wenn diese auf anderen Hosts liegen: wget -Hkp http://foo.bar/index.html

-r Rekursiver Download wget -r http://foo.bar/index.html

-H Host-übergreifender Download -L Nur relativen Links folgen (eine Seite könnte ja auch einen absoluten Link auf ein Dokument des gleichen Host enthalten) -np Niemals in der Verzeichnis-Hierarchie weiter als bis zum angegebenen Base-Verzeichnis springen.

-p Prerequisites downloaden, d.h. alles, was zum Ansehen einer Seite wichtig ist (Images, Sounds, Stylesheets)

-l Tiefe (depth). Default = 5. 0 = unendlich Damit wird angegeben, wieviele Hops von der Ursprungs-URL weggegangen werden darf. Mit Tiefe 2 wird 1.html->2.html->3.html weit gesprungen.

-k Links konvertieren für lokales Browsen. Relative Links auf Elemente, die nicht heruntergeladen worden sind, werden ins absolute Format http://... umgewandelt.

-c Mit abgebrochenem Download weitermachen (continue)


CVS

Die Umgebungsvariable CVSROOT sollte gesetzt sein:

export CVSROOT=/Volumes/data/Development/cvsroot

Creating a new CVS repository:

See: file://localhost/data/Infos,%20Manuals,%20Artikel/References/cederqvist-1.11.3.html/cvs_2.html#SEC23

Adding a new directory tree to CVS

cvs import -m "Initial import comment" target/directory/in/repository foobar start

See: file://localhost/data/Infos,%20Manuals,%20Artikel/References/cederqvist-1.11.3.html/cvs_3.html#SEC40

Tags

Tags are "artificial" revisions of a file. A normal revision may go up from 1.20 to 1.21 automatically after a file is committed. If you tag a file then you mark a given revision of a file with, without committing the file.

To assign tag to a file: cvs tag foobar.c To view all tags on a file: cvs status -v foobar.c To assign tag to all files in a directory and its subdirs: cvs tag . To checkout all files for tag : cvs checkout -r .

To assign tag to files only that already contain tag : cvs tag -r . To assign tag to revisions of files no later than : cvs tag -D . In addition to the last 2 commands: option -f specifies to tag the most recent revision of a file in the repository, instead of ignoring the file, if -r or -D don't produce a match.

See: file://localhost/data/Infos,%20Manuals,%20Artikel/References/cederqvist-1.11.3.html/cvs_4.html#SEC53

Steps to take before checking in to CVS

  1. Optionally: make clean to prevent binaries and other stuff from getting into CVS
  2. cvs add
  3. cvs remove
  4. cvs commit on project's top level


ImageMagick

  • Bilder konvertieren:
for i in *bmp; do
  convert -size 1024x768 "$i" -resize 1280x1024 -quality 90 "$i.jpg";mv *jpg ../jpg
done
  • Bilder skalieren (das Ratio wird beibehalten, falls die neue Grösse nicht passt, wählt convert selbst eine passende Grösse, die dem ursprünglichen Ratio am nächsten kommt):
for i in *jpg; do
  convert -sample 1600x1200 "$i" "new-$i"
done
  • Grösse bestimmen:
identify x.jpg

Weitere Infos: die ImageMagick Homepage hat Links zu verschiedenen Artikeln, z.B. http://www-106.ibm.com/developerworks/library/l-graf/?ca=dnt-428


rsync

Ein rsync Server verwendet Port 873

Beispiel für einen Aufruf auf der Kommandozeile:

rsync -e ssh -avz patrick@osgiliath:/var/backups ~/Desktop/backups
  • -e = shell, die verwendet werden soll
  • -a = archive (oder --archive), Kurzform fuer "recursive and preserve almost everything"
  • -v = verbose (oder --verbose)
  • -z = compress (oder --compress)
  • -n = simulation (oder --dry-run)
  • -q = quiet (oder --quiet)
  • Trailing Slash bei einem Verzeichnis = nur den Inhalt des Verzeichnisses
  • Kein Trailing Slash = das Verzeichnis selbst, inkl. alle Subverzeichnisse


Bei Verwendung in cron:

  • Option -v durch -q ersetzen
  • sowie rsync muss als Daemon auf der Remote Maschine laufen, wenn keine ssh Verbindungs-Optionen (= Passwort) in einem File gespeichert werden sollen.
  • der rsync Daemon muss das gewünschte Verzeichnis als Modul mit anonymous Zugriff exportieren, wenn keine rsync Verbindungs-Optionen (= Passwort) in einem File gespeichert werden sollen

Der crontab Eintrag kann z.B. so aussehen:

0 20,21,22,23,0 * * *  rsync -aqz osgiliath::backup ~/Desktop/backup

Wenn rsync als Daemon auf der Remote Maschine läuft, koennen folgende Kommandos nützlich sein:

  • Alle verfügbaren Module abrufen
rsync patrick@osgiliath::
  • Den Inhalt eines bestimmten Moduls auflisten
 rsync patrick@osgiliath::backup


Toolset für Sound-Verarbeitung

Tools installieren

Homebrew tools

Install via Homebrew:

  • libogg, libogg-shlibs
  • libvorbis0, libvorbis0-shlibs
  • lame, lame-dev, lame-shlibs
  • libmad, libmad-shlibs, madplay
  • flac-nox, flac-nox-bin, flac-nox-shlibs
  • libsndfile1, libsndfile1-dev, libsndfile1-shlibs
  • mppdec
  • ffmpeg. Note that several additional non-standard codecs can be installed - run brew info ffmpeg for information, and/or visit the FFmpeg wiki. I typically run brew install ffmpeg --with-fdk-aac --with-libmodplug --with-libvorbis --with-libvpx --with-openh264 --with-opus --with-theora --with-tools --with-two-lame --with-x265


sox

Download von http://www.sourceforge.net/projects/sox

Entpacken in /usr/local/src, danach wie gewohnt configure, make, make install.

Besonderheiten: damit die von fink bereit gestellten Libraries gefunden werden, müssen dem configure Script einige Compiler-Optionen mitgegeben werden:

./configure CPPFLAGS="-I/sw/include" LDFLAGS="-L/sw/lib"


mac (Monkey's Audio Compressor)

Download von http://www.sourceforge.net/projects/mac

Entpacken in /usr/local/src, danach wie gewohnt configure, make, make install

Besonderheiten: Zumindest die Version mac-3.99-u4-b5 hat einen Linker-Fehler mit gcc 4. Ein Tracker-Item (http://sourceforge.net/tracker/index.php?func=detail&aid=1377803&group_id=123827&atid=699624) weist darauf hin, dass das Problem dadurch umgangen werden kann, dass beim Linken gcc statt g++ verwendet wird. Auf meiner Maschine führt dieser Workaround zu anderen Linker-Fehlern, denen nicht beizukommen war. Auf der fink-users Mailing Liste (http://www.mail-archive.com/fink-users@lists.sourceforge.net/msg21840.html) war dann der endgültige Hinweis mit MACOSX_DEPLOYMENT_TARGET zu finden.

MACOSX_DEPLOYMENT_TARGET=10.3
./configure --prefix=/usr/local
make
make install
unset MACOSX_DEPLOYMENT_TARGET


shntool

Download von http://www.etree.org/shnutils/shntool/

Entpacken in /usr/local/src, danach wie gewohnt configure, make, make install

Hinweis: Dieses Tool muss als letztes der Serie installiert werden, da es eigentlich nur ein Front-End ist und deshalb davon abhängt, was bereits alles vorhanden ist.


Anwendungsfälle

CD Image im .ape Format in Tracks aufsplitten

Im folgenden Beispiel liegen die Track-Informationen in einem .cue File vor. Es wäre aber auch möglich, die Split-Punkte auf der Kommandozeile anzugeben.

shntool split -f CDImage.ape.cue CDImage.ape


Batch-Umwandlung von .wav Files in .mp3

for i in 01 02 03 04 05 06 07 08 09 10; do lame --preset extreme track$i.wav $i.mp3; done


Konvertierung mit ffmpeg

Folgendes Kommando konvertiert Sound Files, wobei das Input/Output Format anhand der File Extension (.wav, .mp3, etc.) erkannt wird:

ffmpeg -i <input file> <output file>


Konvertierung mit flac

Folgendes Kommando konvertiert eine .flac Datei in eine .aiff Datei, wobei der Name des Output Files explizit angegeben wird und das Format implizit aus der File Extension abgeleitet wird:

flac --decode --output-name=<output file>.aiff <input file>

Folgendes Kommando konvertiert mehrere Dateien von .flac in .wav

flac --decode *.flac

Ditto, aber das Ausgabe Format explizit als .aiff festlegen:

flac --decode --force-aiff *.flac


Spaceward Ho!

Special ship designs with the following Tech combinations:

  • WE 12, WA 15, SC 15, TE 1, MI 0
  • WE 11, WA 12, SC 12, TE 1, MI 0


Convert .001/.002 etc. to .bin/.cue

On some file sharing sites you may encounter archives (often .rar archives) that contain a lot of small files with extensions .001/.002 etc. These files are chunks of binary data that can be re-assembled into a set of .bin/.cue files like this:

  • Install 7-Zip
  • Extract the archive
  • Right-click on the .001 file
  • Select "Extract to here" from the context menu

The resulting .bin/.cue files will be placed in the current folder.


Convert .bin/.cue to .iso

Install bchunk via Homebrew/MacPorts/whatever, then run this command line:

bchunk foo.bin foo.cue foo.iso


Working with multimedia container files with ffmpeg

Introduction to containers

Container files can contain any number of data streams, and the same container file may contain streams composed of different data types: Audio data, video data, subtitles data, or just data without a specific type. Within the container, streams do not exist as fully separate chunks of data (e.g. the first 100 MB are audio, the next 700 MB are video, etc), but are all mixed together. The process of interleaving audio, video and other streams into a single coherent data stream is called "multiplexing", which usually is shortened to "muxing". When the content of the container file is played, the player software extracts the individual streams from the multiplexed data stream via a process usually called "demuxing".


Now if you use ffmpeg to manipulate a multimedia file, this muxing and demuxing also has to take place - even if you only want to to something simple as adding some metadata. ffmpeg in that case does not need to understand a given audio or video encoding format (the codec), but it must understand the container format, because the specification of that format determines how the individual streams need to be muxed.


Introduction to ffmpeg

For input files, the file extension does not seem to be relevant, apparently ffmpeg determines the container format by probing the input file's content. For instance, ffmpeg is happy to process .m4v files that were produced by HandBrake. However, for output files the file extension matters. For instance, to create an MP4 output file you must use the .mp4 file name extension. Specifically, the .m4v file extension does not work.


General ffmpeg command line principles:

  • Options are applied to the next specified input/output file. The order in which options appear therefore is important, and some options may appear more than once.
  • First specify global options, then specify input files with their options, then specify the output file with its options
  • ffmpeg refers to input files and streams via 0-based indices: The second input file has index 1, the third stream has index 2, the fifth stream in the first input file is referenced with "0:4". Streams can also be referred to by type: "a" for audio, "v" for video, "s" for subtitle, "d" for data and "t" for attachment. For instance, v:2 is the third video stream. See the man page for even more ways how to specify a stream.


References:

  • man ffmpeg


Homebrew installation of ffmpeg

When you install ffmpeg via Homebrew you can specify a lot of options to enable (or disable) certain codecs. To view all available options, run

brew info ffmpeg

The following command enables all options that were available at the time of writing (August 2018):

brew install ffmpeg --with-chromaprint --with-fdk-aac --with-fontconfig --with-freetype --with-frei0r --with-game-music-emu --with-libass --with-libbluray --with-libbs2b --with-libcaca --with-libgsm --with-libmodplug --with-librsvg --with-libsoxr --with-libssh --with-libvidstab --with-libvorbis --with-libvpx --with-opencore-amr --with-openh264 --with-openjpeg --with-openssl --with-opus --with-rtmpdump --with-rubberband --with-sdl2 --with-snappy --with-speex --with-srt --with-tesseract --with-theora --with-tools --with-two-lame --with-wavpack --with-webp --with-x265 --with-xz --with-zeromq --with-zimg


In a similar manner, the following command installs libav with all available options enabled:

brew install libav --with-fontconfig --with-frei0r --with-gnutls --with-libvo-aacenc --with-opencore-amr --with-openssl --with-rtmpdump --with-schroedinger --with-sdl --with-speex --with-theora


Command line examples

List the content of a container:

ffprobe foo.mp4

View detailed information for all streams in a container:

ffprobe -show_streams foo.mp4

View detailed information for a selected stream:

ffprobe -show_streams -select_streams 21 foo.mp4


Copy all streams from an input to an output file, effectively creating a duplicate of the original. This is a bit nonsensical, but illustrates the use of the "copy" pseudo-codec and a simple application of the -map option.

ffmpeg -i foo.mp4 -map 0 -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 bar.mp4

Discussion:

  • The "copy" pseudo-codec prevents decoding/encoding of data streams, instead it preserves the streams' data without change
  • The various "-copy..." options may or may not be necessary, but they all ensure that some aspect of the original input files remains unchanged
  • The -map option specifices which input stream(s) should be copied to the output file. In the example we only have a single input file, which is referenced by the index number 0. Because we didn't specify any streams, ffmpeg will copy all streams that it finds in the input file.
  • -map_metadata makes sure that all meta data from input file 0 is preserved. In theory, this option should not be necessary because the default should already preserve meta data. In practice, however, without the option I saw ffmpeg removing the meta data tag "creation_time".
  • ffmpeg always changes the major_brand metadata tag to "isom", even if it previously was set to "mp42" (e.g. by HandBrake). If this Trac entry is to be believed, the reason is that ffmpeg is capable of writing only mp41-compatible files, which results in major_brand to be displayed as "isom".
  • When you diff the stream listing from an output file with the stream listing from an input file, you might see that audio stream properties like bit_rate or max_bit_rate have changed. I can only hope that this is not a problem, and that ffmpeg has changed these values because they were incorrect in the first place.


A general problem is that ffmpeg does not preserve the handler_name meta data tag of audio streams - instead it changes the tag to "SoundHandler", regardless of the original value (e.g. HandBrake sets this to "Stereo" or "Surround"). This is an important issue because the handler name is used by player software such as VLC to indicate what kind of content the audio stream contains. The problem can be fixed by adding one -metadata option for each audio stream, like this: -metadata:s:a:0 handler=Stereo. This example sets the first audio stream's handler name. For a more complete example, in a file with 14 audio streams I ended up with the following spectacular (and cumbersome) command line.

ffmpeg -i foo.m4v -map 0 -codec copy -copy_unknown -copyts -map_metadata 0 -metadata:s:a:0 handler=Stereo -metadata:s:a:1 handler=Surround -metadata:s:a:2 handler=Stereo -metadata:s:a:3 handler=Stereo -metadata:s:a:4 handler=Stereo -metadata:s:a:5 handler=Stereo -metadata:s:a:6 handler=Stereo -metadata:s:a:7 handler=Stereo -metadata:s:a:8 handler=Stereo -metadata:s:a:9 handler=Stereo -metadata:s:a:10 handler=Stereo -metadata:s:a:11 handler=Surround -metadata:s:a:12 handler=Stereo -metadata:s:a:13 handler=Surround bar.mp4


Here's an example that merges selected streams from two input files. The input files both contain the same video but have different audio streams and subtitle tracks. Such a scenario can occur if you rip a DVD with HandBrake, and the DVD contains two titles with the same video.

ffmpeg \
       -i "foo.m4v" \
       -i "bar.m4v" \
       -map_metadata 0 \
       -map_metadata 1 \
       -map 0:v:0 \
       -map 0:a:0 -map 0:a:1 -map 0:s:1 \
       -map 0:a:10 -map 0:a:11 -map 0:a:12 -map 0:a:13 -map 0:s:2 -map 0:s:3 \
       -map 0:s:0 \
       -map 1:a:2 -map 1:a:3 -map 1:a:4 -map 1:a:5 -map 1:a:6 -map 1:a:7 -map 1:a:8 -map 1:a:9 \
       -metadata:s:a:0 handler="Stereo" -metadata:s:a:1 handler="Surround" -metadata:s:a:2 handler="Stereo" -metadata:s:a:3 handler="Surround" \
       -metadata:s:a:4 handler="Stereo" -metadata:s:a:5 handler="Surround" \
       -metadata:s:a:6 handler="Stereo" -metadata:s:a:7 handler="Stereo" -metadata:s:a:8 handler="Stereo" -metadata:s:a:9 handler="Stereo" \
       -metadata:s:a:10 handler="Stereo" -metadata:s:a:11 handler="Stereo" -metadata:s:a:12 handler="Stereo" -metadata:s:a:13 handler="Stereo" \
       -codec copy -copyinkf -copy_unknown -copyts \
       "baz.mp4"

Discussion:

  • The order in which the -map options appear determines the order in which ffmpeg generates output streams
  • The -metadata options refer to output streams
  • A description of the output streams:
    • One video stream from the first file. The video stream from the second file is ignored.
    • Two audio streams and one subtitle track from the first file. These contain the English language audio and subtitles. There are two audio streams because HandBrake generates one stream with codec "AAC" and one stream with codec "AC3 Passthru" from the same original DVD audio.
    • Four audio streams and two subtitle tracks from the first file. These contain the German language audio and subtitles. One pair of audio streams are for "Dolby Digital 5.1 EX", the other pair are for "DTS ES 6.1". The first subtitle track contains the regular German subtitles, the other contains subtitles for the hearing-impaired.
    • One subtitle track from the first file. This contains the "Foreign Audio Search" subtitles generated by HandBrake.
    • Eight audio streams from the second file. These contain 4 different audio commentaries. Each audio commentary is comprised of two audio streams.


Here's an example that concatenates multiple segments of a video into one single video file. The command also concatenates several audio streams and one subtitle track. The command is based on this ffmpeg wiki page. It uses the so-called "concat demuxer", which is documented here.

ls *segment*.mp4

foo, segment 1.mp4
foo, segment 2.mp4
foo, segment 3.mp4
[...]

for f in ./*segment*.mp4; do echo "file '$f'" >segmentlist.txt; done

ffmpeg \
       -f concat -safe 0 -i segmentlist.txt \
       -map_metadata 0 \
       -map 0:0 \
       -map 0:1 -map 0:2 \
       -map 0:3 \
       -metadata:s:a:0 handler="Stereo" -metadata:s:a:1 handler="Stereo" \
       -codec copy  -copyinkf -copy_unknown -copyts \
       "foo, all segments.mp4"

Discussion:

  • The -f option is used to force the format of the next input (or output) file. ffmpeg normally auto detects the format for input files and guesses the format from the file extension for output files. In this case we must explicitly specify the "concat" format to let ffmpeg know that it will have to use the "concat" demuxer on the files listed in the text file.
  • The -safe 0 option is not documented on the man page. The option is here because the command line shown here is based on an example from the ffmpeg wiki page on concatenation. The wiki page says that "The -safe 0 above is not required if the paths are relative."
  • The -i segmentlist.txt option specifies an input file that is a "virtual concatenation script". This together with the -f concat option creates a virtual input "file" that consists of the files in the text file concatenated together.
  • The -map_metadata, -map and -metadata options all relate to the virtual input "file" created by the concat demuxer. They are explained in more detail further up in the notes for previous examples.


Troubleshooting

Data stream not understood

I encountered the problem that an .m4v file generated by HandBrake contained a stream which caused ffmpeg to barf. The stream in question was this:

Stream #0:21(eng): Data: bin_data (text / 0x74786574)

The error message printed by ffmpeg was this:

[mp4 @ 0x7f8b1b889600] Unknown hldr_type for text, writing dummy values   

And when I examined the output file I saw that ffmpeg had created a new pseudo-stream just before the original data stream:

Stream #0:21(eng): Data: none (stts / 0x73747473)

But apparently the stream was improperly added, because on examining the output ffprobe also printed this error message quite early during its output:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fabff021000] overread end of atom 'stsd' by 232 bytes

I was able to apparently fix the issue by adding the option -map -0:d after the original -map 0. The new command line looked like this:

ffmpeg -i foo.mp4 -map 0 -map -0:d -codec copy -copyinkf -copy_unknown -copyts -map_metadata 0 bar.mp4

With this additional option, ffmpeg preserves the data stream perfectly. I must admit, however, that I don't really understand why, because the prefixing "-" character is supposed to remove a mapping from an already established prior mapping, which in my understanding should cause ffmpeg to remove the stream entirely from the output. Well, I can only say that I found the "solution" in this pull request for the youtube-dl project, right at the bottom.