MKVnator - Automatic MKV creation with your NAS/server

Discussion of advanced MakeMKV functionality, expert mode, conversion profiles
Post Reply
mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Thu Aug 01, 2019 10:37 am

I wanted to speed up the process of adding new movies to my Plex collection. By that the following steps should be done automatically:
(- create disk backup, must be done manually as the discs do not contain enough data to determine specific movies or we use an external database)
- batch convert multiple discs
- select the correct movie title or episode titles
- auto select audio and subtitle languages
- auto create AC3 audio track if not available (because my setup does not support DTS)
- auto naming of subtitles
- auto export forced subtitles as SRT
- let my Synology NAS do this work

The plan is to use a pure Bash script and use Docker apps. I never used Bash before, so feel free to give feedback on coding style or mistakes.

Current status of MKVnator
- the NAS does the work
- batch convert multiple blu-ray disc backups (not DVDs and not TV series!)
- select the movie title that is bigger than 10GB (size is changeable)
- auto select audio and subtitle languages (default setting is "ger,eng,tur")
- auto create AC3 audio track if not available (optional)
- the first language is set as default track (with the default setting it would be "ger")
- auto naming subtitles as "Forced" that are smaller than 3MB (size is changeable)
- forced subtitle tracks get the flag "forced"
- forced subtitles are exported as SUP file (later we want to convert them to SRT)
- if multiple titles bigger than 10GB were found, the disc will be skipped (now you can force a specific title)
- if the disc backup folder contains the file "force_mpls_00030.txt" it will create the mkv out of the title that uses the playlist "00030.mpls"
- no disc backup gets lost, so you can repeat on mistakes (default setting is moving to a recycle folder, auto delete is optional)

As you can see this project has an early stage, but I will update it as time permits.

Requirements
- Server or NAS that supports docker (I plan to add non-docker support as well)
- decrypted backups of your discs manually created by MakeMKV and located on your NAS under "/volume1/video/Rips/" (or you change the default MKVnators settings). This is how it looks for me:
2019-08-01 12_33_23.jpg
2019-08-01 12_33_23.jpg (25.31 KiB) Viewed 1945 times

How-To
1.) install MakeMKV through docker and start it as "makemkv", set "/storage" and "/output" to the same folder as MKVnators "storage_folder" and "output_folder". My makemkv docker settings:
2019-08-01 12_23_50.jpg
2019-08-01 12_23_50.jpg (73.21 KiB) Viewed 1945 times
2.) install MKVtoolnix through docker and start it as "mkvtoolnix", set "/storage" to the same folder as MKVnators "output_folder" (yes the output folder as this contains the MKV files!).
3.) check the settings inside of MKVnator.sh (eg the default "local_folder" is "/volume1/video/" and the default "storage_folder" is "Rips/"). Note: all folders except of "storage_folder" and "output_folder" are created automatically
4.) add MKVnator.sh to your NAS and create a task through your scheduler. At first you should disable the schedule and test the execution manually. The Synology NAS sends the result to my email address:
2019-08-01 12_29_55.jpg
2019-08-01 12_29_55.jpg (28.15 KiB) Viewed 1945 times

MKVnator.sh Script

View next post:
Last edited by mgutt on Sat Aug 10, 2019 11:09 am, edited 24 times in total.

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Thu Aug 01, 2019 10:38 am

Code: Select all

#!/bin/sh

# #####################################
# MKVnator v0.8
# 
# Notes:
# - Your storage folder "/volume1/video/Rips/" must contain Blu-Ray disc backups generated by MakeMKV GUI or makemkvcon command line
# - Check the settings to change folders, languages and sizes
# - The name of the disc folder will be used for the generated MKV filename
# 
# Changelog:
# 0.8
# - New function: Add AC3 audio track if MKV does not contain it. Its generated through the DTS audio track in default language. The setting to enable it is mkv_audio_add_ac3 (default is false)
# - Bug fix regarding arrays containing unquoted wildcards and question marks (https://unix.stackexchange.com/questions/534858/why-does-shopt-s-nullglob-remove-a-string-with-question-mark-in-an-array-elemen)
# 0.7
# - Bug fix in forced subtitle auto naming
# - New setting "makemkv_minlength" added (default is 600 seconds)
# - Bug fix for empty loops (shopt -s nullglob)
# - Skip remux if no audio track or subtitle would be removed
# - Skip auto subtitle naming if MKV already contains subtitle names
# - Remuxed MKV files are renamed to ".rmx.mkv"
# - MKV files with auto named subtitles are renamed to ".sub.mkv"
# - Multiple code lines have been rewritten to take advantage of arrays as app parameters (like "mkvpropedit_options")
# 0.6
# - New function: Determine forced subtitle by exporting it as sup file and check its filesize. It must be smaller than the new setting br_sub_forced_max_size. Default is 3 MB.
# 0.5
# - Forcing has been extended: The user can force mkv creation of a specific title by adding a file named "force_mpls00040.txt" to the disc folder
# - Forcing by title id is still supported but not recommend as MakeMKV GUI returns different title ids than makemkvcon (https://makemkv.com/forum/viewtopic.php?f=10&t=20081)
# - You can force titles with the keywords mpls, m2ts, size, id, chapters, etc This means "force_size_12.5.txt" forces all titles with a size of 12.5 GB.
# 0.4
# - New function: Remux the MKV file by choosing subtitle and HD/Core audio languages through the settings.
# - New function: Move files to recycle folder instead of deleting
# 0.3
# - New function: The user can force a specific title by adding a file named "force_title02.txt" to the disc folder
# - buggy "wrong naming conditions" fixed
# - add disc type "BR" or "DVD" to mkv filename
# - added "delete_disc_folder_source" setting to preserve the source files
# 0.2
# - already existing disc folders had wrong path
# - removing source had wrong path
# 
# Todo:
# - optional: add ac3 audio track if it does not exist
# - auto naming of subtitles
# - add similar support for ignoreForcedSubtitlesFlag="true", setFirstAudioTrackAsDefault="true", setFirstSubtitleTrackAsDefault="true", setFirstForcedSubtitleTrackAsDefault="true"
# - add similar support for -sel:all,+sel:(ger|eng|tur|nolang),-sel:(core),-sel:mvcvideo,=100:all,-10:ger,+sel:attachment
# - skip tv series by checking sXXeYY, YYYY(-|.| )MM(-|.| )DD or DD(-|.| )MM(-|.| )YYYY in folder name https://support.plex.tv/articles/naming-and-organizing-your-tv-show-files/
# - add support for tv series
# - skip mkvextract -J if mkv_langs=all and mkv_audio=hd,core
# - skip mkvextract -o if all audio tracks have been selected so no audio track needs to be removed
# - skip mkvextract if mkv filename ends with " (1).mkv"
# - create docker container on-the-fly
# - kill docker container after the last disc/mkv has been finished
# - add docker alternative: local app execution
# - make MKVnator save against race conditions. mkdir should be atomic enough.
# - add more keywords to forcing: playlist, streams, stream
# - the forcing keyword of the text file is "force", we should support it case-insensitive
# - we need error handling if moving folders with mv targets non-empty folder
# - scan an existing collection of mkv files and check if audio tracks should be removed, the naming is wrong, etc
# - add containing languages to the mkv filename
# - makemkvcon --minlength should be part of the MKVnator settings
# - use disc name / title to automatically obtain IMDB/TMDB id/rating
# - we should replace mkvmerge_audio_tracks against an array and implode it later https://stackoverflow.com/a/9429887/318765
# - remove subtitle duplicates (some movies contain two identical subtitles)
# - export subtitles sup with mkvextract, monitor subtitleX.sup filesize and kill process if subtitleX exceeds forced subtitle maximum. Maybe this speeds up the process. Needs benchmarking.
# - write all errors and message to a log file
# - before skipping multiple title discs we check if one of the disc titles contain multiple languages (some discs have fake titles containing only the english audio track)
# - before skipping multiple title discs we check if one of the disc titles contain chapters (some discs have fake titles without chapters)
# - setting to keep all exported subtitle SUP files?
# - setting to reset all already existing subtitle names and flags of a mkv file?
# - determine describing audio for blinds by creating subtitles for the first 30 seconds of each audio track (should return describing words regarding movie logos etc.)
# - DVD VOB subs could be converted through Subtitle Edit or VobSub2SRT of ruediger (or fork of bubonic that supports Tesseract 4)
# #####################################

# ######### Settings ##################
br_title_min_size="10GB"
br_sub_forced_max_size="3MB"
mkv_langs="ger,eng,tur" # or use "all" to preserve all audio and subtitle languages. The first listed language is set as default audio track.
mkv_audio="hd" # "hd" preserves DTS-HD/True HD tracks and "core" preserves core tracks. Use "hd,core" to preserve both
mkv_audio_add_ac3=false
makemkv_minlength=600 # the minimum lenght of a disc title in seconds
local_folder="/volume1/video/"
storage_folder="Rips/" # the same folder must be set for makemkv's docker "/storage"
output_folder="Rips_mkv/" # the same folder must be set for makemkv's docker "/output"
no_title_folder="Rips_no_title_found/"
multi_title_folder="Rips_multiple_title_found/"
already_exists_folder="Rips_already_exist/"
wrong_naming_folder="Rips_wrong_naming/"
makemkv_error_folder="Rips_makemkv_error/"
makemkv_recycle_folder="Rips_recycle/"
mkvextract_recycle_folder="Rips_mkv_recycle/"
docker_storage_folder="/storage/"
docker_output_folder="/output/"
delete_disc_folder_source=false
delete_mkv_source=false
# #####################################

# ######### Script ####################
# filter user input
br_title_min_size=$(echo "$br_title_min_size" | tr -d '[:space:]')
br_title_min_size=${br_title_min_size:0:(-2)}
# convert GB to MB
if [ ${br_title_min_size:(-2)} == "GB" ]; then
    br_title_min_size=$(awk "BEGIN { print $br_title_min_size*1000}") # we need to use awk as bc does not exist in default synology installations
fi
br_title_min_size=${br_title_min_size%.*} # convert to integer (float comparison has low code readability without bc https://stackoverflow.com/a/8654332/318765)
br_sub_forced_max_size="${br_sub_forced_max_size//[!0-9.]/}" # float filtering (https://stackoverflow.com/a/19724571/318765)
br_sub_forced_max_size=$(awk "BEGIN { print $br_sub_forced_max_size*1000000}") # convert MB to Bytes
# create needed deep links
storage_folder="$local_folder$storage_folder"
output_folder="$local_folder$output_folder"
no_title_folder="$local_folder$no_title_folder"
multi_title_folder="$local_folder$multi_title_folder"
already_exists_folder="$local_folder$already_exists_folder"
wrong_naming_folder="$local_folder$wrong_naming_folder"
makemkv_error_folder="$local_folder$makemkv_error_folder"
makemkv_recycle_folder="$local_folder$makemkv_recycle_folder"
mkvextract_output_folder="$local_folder$mkvextract_output_folder"
mkvextract_recycle_folder="$local_folder$mkvextract_recycle_folder"
# checks
if [ ! -d "$storage_folder" ]; then
    echo "Storage folder $storage_folder not found!"
    exit 1
fi
# write test without -w https://unix.stackexchange.com/a/159658/101920
if >> "${output_folder}write.test"; then
    rm "${output_folder}write.test"
else
    echo "Not able to write to ${output_folder}!"
    exit 1
fi
# create folders
mkdir -p "$no_title_folder"
mkdir -p "$multi_title_folder"
mkdir -p "$already_exists_folder"
mkdir -p "$wrong_naming_folder"
mkdir -p "$makemkv_error_folder"
mkdir -p "$makemkv_recycle_folder"
mkdir -p "$mkvextract_output_folder"
# get mkv information
shopt -s nullglob # avoid empty directory errors (https://unix.stackexchange.com/questions/56051/avoiding-errors-due-to-unexpanded-asterisk)
for mkv_folder in "$output_folder"*; do
    for mkv_filename in "$mkv_folder/"*.mkv; do
        # skip finished mkv files
        if [[ $mkv_audio_add_ac3 == "true" ]]; then
            if [[ "${mkv_filename:(-8)}" == ".ac3.mkv" ]]; then
                echo "'$mkv_filename' has been skipped (already contains AC3 audio)"
                continue
            elif [[ "${mkv_filename:(-8)}" == ".dts.mkv" ]]; then
                echo "'$mkv_filename' has been skipped (contains unsupported DTS audio)"
                continue
            fi
        elif [[ "${mkv_filename:(-8)}" == ".sub.mkv" ]]; then
            echo "'$mkv_filename' has been skipped (already contains named subtitles)"
            continue
        fi
        mkv_folder="$(basename "$mkv_folder")/"
        mkv_filename=$(basename "$mkv_filename")
        docker_mkv_filename="${docker_storage_folder}${mkv_folder}${mkv_filename}"
        mkv_info="$(docker exec mkvtoolnix /usr/bin/mkvmerge -J "$docker_mkv_filename")"
        echo "MKV information of '${output_folder}${mkv_folder}${mkv_filename}' has been obtained"
        break 2;
    done
done
shopt -u nullglob # its important to reset this setting (https://unix.stackexchange.com/questions/534858/why-does-shopt-s-nullglob-remove-a-string-with-question-mark-in-an-array-elemen)
# get mkv tracks
unset track_names track_codecs track_langs # make sure they do not exist
declare -A track_names track_codecs track_langs
mkvmerge_remux_needed=""
mkvmerge_sub_tracks=()
mkvmerge_audio_tracks=()
mkvmerge_default_track=""
default_lang="${mkv_langs:0:3}" # first language is used as default language
echo "The default language is '$default_lang'"
forced_tracks=()
default_tracks=()
last_track_bits=""
last_track_channels=0
best_dts_track=""
best_dts_channels=0
best_ac3_track=""
best_ac3_channels=0
sub_has_name=false
while read -r line ; do
    echo $line
    # Note: Instead we could use "jq -r" to parse JSON, but I do not know if it is available on all NAS or servers
    track_codec_name=$(echo $line | grep -oP '^.*?(?=\")')
    track_id=$(echo $line | grep -oP '(?<="id": )[0-9]+')
    track_bits=$(echo $line | grep -oP '(?<="audio_bits_per_sample": )[0-9]+')
    track_channels=$(echo $line | grep -oP '(?<="audio_channels": )[0-9]+')
    track_codec_id=$(echo $line | grep -oP '(?<="codec_id": ").*?[^\\](?=\",)')
    track_lang=$(echo $line | grep -oP '(?<="language": ")[a-z]+')
    track_name=$(echo $line | grep -oP '(?<="track_name": ").*?[^\\](?=\",)') # most flexible way of getting a JSON value (https://stackoverflow.com/a/6852427/318765)
    track_default=$(echo $line | grep -oP '(?<="default_track": )(true|false)')
    track_forced=$(echo $line | grep -oP '(?<="forced_track": )(true|false)')
    track_type=$(echo $line | grep -oP '(?<=")[a-z]+$')
    # collect
    track_codecs[$track_id]=$track_codec_id
    track_langs[$track_id]=$track_lang
    # collect forced track ids
    if [[ $track_forced == "true" ]]; then
        forced_tracks+=("$track_id")
    fi
    # collect default track ids
    if [[ $track_default == "true" ]]; then
        default_tracks+=("$track_id")
    fi
    # collect track names
    if [[ -n $track_name ]]; then
        track_names[$track_id]=$track_name
    fi
    # add subtitles in prefered languages
    if [[ $track_type == "subtitles" ]]; then
        if [[ -n $track_name ]]; then
            sub_has_name=true
        fi
        if [[ $mkv_langs == "all" ]] || [[ $mkv_langs == *"$track_lang"* ]]; then
            mkvmerge_sub_tracks+=("$track_id")
            continue
        fi
    fi
    # audio tracks
    if [[ $track_type == "audio" ]]; then
        echo "Track ID #$track_id is an audio track  with the language '$track_lang'"
        # check if we need to add an AC3 audio track
        if [[ $mkv_audio_add_ac3 == "true" ]] && [[ $best_ac3_channels -lt 6 ]] && [[ $track_lang == $default_lang ]]; then
            echo "Track ID #$track_id has the default language '$default_lang' and it is allowed to convert it to AC3"
            # find the best DTS track
            if [[ $track_codec_id == "A_DTS" ]]; then
                echo "Track ID #$track_id is a DTS track"
                # the track with the most channels has the highest priority
                if [[ $track_channels -ge $best_dts_channels ]]; then
                    echo "Track ID #$track_id has $track_channels channels"
                    # Consider this track if it has more channels than the last one
                    if [[ $track_channels -gt $best_ac3_channels ]]; then
                        echo "Track ID #$track_id has more channels than the last one ($best_ac3_channels)"
                        # HD track has higher priority than Core track
                        if [[ -z $best_dts_track ]] || [[ -n $track_bits ]]; then
                            echo "Track ID #$track_id is the best DTS audio track by now"
                            best_dts_track=$track_id
                            best_dts_channels=$track_channels
                        fi
                    fi
                fi
            # find the best AC3 track
            elif [[ $track_codec_id == "A_AC3" ]] || [[ $track_codec_id == "A_EAC3" ]]; then
                echo "Track ID #$track_id is a DTS track"
                # the track with the most channels has the highest priority
                if [[ $track_channels -ge $best_ac3_channels ]]; then
                    echo "Track ID #$track_id has $track_channels channels"
                    # HD track overwrites Core track
                    if [[ -z $best_ac3_track ]] || [[ -n $track_bits ]]; then
                        echo "Track ID #$track_id is the best AC3 audio track by now"
                        best_ac3_track=$track_id
                        best_ac3_channels=$track_channels
                    fi
                    # skip DTS to AC3 conversion if AC3 has 6 channels or has as much channels as DTS (AC3 5.1 Core > DTS-HD 7.1)
                    if [[ $best_ac3_channels -ge $best_dts_channels ]] || [[ $best_ac3_channels -ge 6 ]]; then
                        echo "Track ID #$track_id is good enough. Adding an AC3 audio track is not necessary"
                        best_dts_track=""
                        best_dts_channels=0
                    fi
                fi
            else
                echo "$track_codec_id is unknown. Please contact the MKVnator developer."
                exit 1
            fi
            # should we use a switch and add A_TRUEHD and A_MLP? What about breaking the script if an unknown codec has been found?
        fi
        # remove hd audio
        if [[ $mkv_audio != *"hd"* ]] && [[ -n $track_bits ]]; then
            last_track_channels="$track_channels"
            last_track_bits="$track_bits"
            mkvmerge_remux_needed=1
            continue
        fi
        # remove core audio
        if [[ $mkv_audio != *"core"* ]] && [[ -z $track_bits ]] && [[ -n $last_track_bits ]] && [[ $last_track_channels == $track_channels ]]; then
            last_track_channels="$track_channels"
            last_track_bits="$track_bits"
            mkvmerge_remux_needed=1
            continue
        fi
        # add audio in prefered languages
        if [[ $mkv_langs == "all" ]] || [[ $mkv_langs == *"$track_lang"* ]]; then
            mkvmerge_audio_tracks+=("$track_id")
            # set default audio track
            if [[ -z $mkvmerge_default_track ]] && [[ $track_lang == $default_lang ]]; then
                mkvmerge_default_track="$track_id"
            fi
        fi
        last_track_channels="$track_channels"
        last_track_bits="$track_bits"
    fi
done < <(echo "$mkv_info" | 
        tr -d '\n' | # we need to remove line breaks with "tr" to force grep to return one-liners
        grep -oP '(?<=codec": ").*?"type": "[a-z]+') # Regex is faster than looping through all lines
# remux mkv
if [[ -n $mkvmerge_remux_needed ]]; then
    mkvmerge_options=()
    # set remux destination filename
    mkvmerge_options+=("-o")
    mkvmerge_options+=("${docker_mkv_filename:0:(-4)}.rmx.mkv") # replace ".mkv"
    # set audio tracks for remux
    if [[ ${#mkvmerge_audio_tracks[@]} -ge 1 ]]; then
        mkvmerge_options+=("-a")
        mkvmerge_options+=($(IFS=, ; echo "${mkvmerge_audio_tracks[*]}")) # implode (https://stackoverflow.com/a/9429887/318765)
    fi
    # set subtitles for remux
    if [[ ${#mkvmerge_sub_tracks[@]} -ge 1 ]]; then
        mkvmerge_options+=("-s")
        mkvmerge_options+=($(IFS=, ; echo "${mkvmerge_sub_tracks[*]}")) # implode (https://stackoverflow.com/a/9429887/318765)
    fi
    # set mkv sourcefile
    mkvmerge_options+=("$docker_mkv_filename")
    # remux mkv file
    docker exec mkvtoolnix /usr/bin/mkvmerge "${mkvmerge_options[@]}"
    # remove source
    if [[ $delete_mkv_source == "true" ]]; then
        rm -rf "${output_folder}${mkv_folder}${mkv_filename}"
        echo "'${output_folder}${mkv_folder}${mkv_filename}' has been deleted"
    else
        mkdir -p "${mkvextract_recycle_folder}${mkv_folder}"
        mv "${output_folder}${mkv_folder}${mkv_filename}" "${mkvextract_recycle_folder}${mkv_folder}${mkv_filename}"
        echo "'${output_folder}${mkv_folder}${mkv_filename}' has been moved to '${mkvextract_recycle_folder}${mkv_folder}${mkv_filename}'"
    fi
    # one job per script exection is enough
    exit 0
# export and determine forced subtitles
elif [[ ${#mkvmerge_sub_tracks[@]} -ge 1 ]] && [[ $sub_has_name == "false" ]]; then
    mkvpropedit_options=()
    mkvextract_options=()
    mkvextract_options+=("tracks")
    for id in "${mkvmerge_sub_tracks[@]}"; do
        mkvextract_options+=("${id}:${docker_storage_folder}${mkv_folder}subtitle${id}.sup")
        # remove existing subtitle forced flags
        if [[ -v "$forced_tracks[$id]" ]]; then
            mkvpropedit_options+=("--edit")
            mkvpropedit_options+=("track:$[$id+1]") # mkvpropedit starts counting at 1 (https://mkvtoolnix.download/doc/mkvpropedit.html#mkvpropedit.edit_selectors)
            mkvpropedit_options+=("--set")
            mkvpropedit_options+=("flag-forced=0")
       fi
    done
    # export all subtitles
    docker exec mkvtoolnix /usr/bin/mkvextract "$docker_mkv_filename" "${mkvextract_options[@]}"
    echo "Successfully extracted all subtitles"
    # determine forced subtitle by filesize
    shopt -s nullglob
    for filename in "${output_folder}${mkv_folder}"*.sup; do
        filesize=$(stat -c%s "$filename")
        # we found a forced subtitle
        if [ $br_sub_forced_max_size -ge $filesize ]; then
            echo "'$filename' is small enough to be a forced subtitle"
            track_id=$(echo $filename | grep -oP '(?<=subtitle)[0-9](?=\.sup)')
            # every whitespace separated parameter must be a new array element (or it returns "Error: More than one file name has been given")
            mkvpropedit_options+=("--edit")
            mkvpropedit_options+=("track:$[$track_id+1]")
            mkvpropedit_options+=("--set")
            mkvpropedit_options+=("flag-forced=1")
            mkvpropedit_options+=("--edit")
            mkvpropedit_options+=("track:$[$track_id+1]")
            mkvpropedit_options+=("--set")
            mkvpropedit_options+=("name=Forced")
            filename_new="${filename%.*}.${track_langs[$track_id]}.forced.sup"
            mv "$filename" "$filename_new"
            echo "'$filename' has been renamed to '$filename_new'"
        # delete all other exported subtitles
        else
            rm -rf "$filename"
            echo "'$filename' has been deleted"
        fi
    done
    shopt -u nullglob
    # set flags
    echo "Set flags with '${mkvpropedit_options[@]}'"
    docker exec mkvtoolnix /usr/bin/mkvpropedit "$docker_mkv_filename" "${mkvpropedit_options[@]}"
    # one job per script exection is enough
    exit 0
# extract dts audio
elif [[ $mkv_audio_add_ac3 == "true" ]] && [[ -n $best_dts_track ]]; then
    echo "Add AC3 audio to ${output_folder}${mkv_folder}${mkv_filename} using Track ID #$best_dts_track"
    # more channels need more work
    if [[ $best_dts_channels -gt 6 ]]; then
        echo "MKVnator does not support DTS tracks with more than 6 channels (planned feature)!"
        mv "${output_folder}${mkv_folder}${mkv_filename}" "${output_folder}${mkv_folder}${mkv_filename:0:(-4)}.dts.mkv"
        # extract dts audio track
        # track_name=$(echo "${track_names[$best_dts_track]}" | tr "[:upper:]" "[:lower:]") # to lower case (https://stackoverflow.com/a/2264451/318765)
        # track_name=${track_name// /_} # replace whitespace (https://stackoverflow.com/a/1706459/318765)
        # dts_filename="${docker_storage_folder}${mkv_folder}.audio${best_dts_track}.${track_langs[$best_dts_track]}.${track_name}.dts"
        # mkvextract_options=()
        # mkvextract_options+=("tracks")
        # mkvextract_options+=("${best_dts_track}:$dts_filename")
        # docker exec mkvtoolnix /usr/bin/mkvextract "$docker_mkv_filename" "${mkvextract_options[@]}"
        # example:
        # ffmpeg -i track.dts -acodec pcm_s24le output-file.w64
        # or:
        # https://stackoverflow.com/a/9728732/318765
        # after that we need to split w64 into wav (or possible in one step?)
        # https://stackoverflow.com/a/41249477/318765
        # after that we are able to mix the surround wavs with the back surround wavs:
        # https://stackoverflow.com/a/35330616/318765
        # convert it to multi-channel w64 (is it possible to produce a 5.1 AC3 file?)
        # https://trac.ffmpeg.org/wiki/AudioChannelManipulation#a6mono5.1
        exit 1
    else
        docker_options="run \
                --name=ffmpeg \
                -e TZ=Europe/Berlin \
                -e USER_ID=1000 \
                -e GROUP_ID=100 \
                -p 5803:5803 \
                -v ${output_folder}:/storage:rw \
                jrottenberg/ffmpeg"
        ffmpeg_src_filename="${docker_storage_folder}${mkv_folder}${mkv_filename}"
        ffmpeg_dst_filename="${ffmpeg_src_filename:0:(-4)}.ac3.mkv"
        set -f # disable filename globbing (https://unix.stackexchange.com/a/534859/101920)
        ffmpeg_options=(
                        -y # overwrite output file
                        -nostats # disable progress
                        -loglevel error
                        -map 0:v # select video stream
                        -map 0:$best_dts_track # add new ac3 audio track at first
                        -map 0:a # add all audio tracks (including the best dts audio track)
                        -map 0:s? # add all subtitles (if present)
                        -map 0:d? # add all data (if present)
                        -map 0:t? # add all attachments (if present)
                        -c copy # use the codec "copy" (no re-encoding) for all selected tracks
                        # beware: The index of "-c" (codec) and "-b" (bitrate) does not target the track index of the source
                        #         Instead it targets the new index created through the multiple "-map" commands.
                        #         This means "-c:a:0" will re-encode "-map 0:a:1"
                        -c:1 ac3 # the best dts track will be re-encoded as "ac3"
                        -b:1 640k # the best dts track will be re-encoded with 640 kbit/s (Blu-Ray standard, for DVD we need 448k (https://gdion.wordpress.com/2016/06/12/simple-dts-to-ac3-using-ffmpeg/)
        )
        set +f
        docker $docker_options -i "$ffmpeg_src_filename" ${ffmpeg_options[@]} "$ffmpeg_dst_filename"
        echo docker "$docker_options -i $ffmpeg_src_filename ${ffmpeg_options[@]} $ffmpeg_dst_filename"
        docker stop ffmpeg
        docker rm ffmpeg
        # todo:
        # - mkvpropedit set uid of new ac3 track (a trick to mark this audio track as non-original)
        # - remove encoder tags?
    fi
    # remove source
    if [[ $delete_mkv_source == "true" ]]; then
        rm -rf "${output_folder}${mkv_folder}${mkv_filename}"
        echo "'${output_folder}${mkv_folder}${mkv_filename}' has been deleted"
    else
        mkdir -p "${mkvextract_recycle_folder}${mkv_folder}"
        mv "${output_folder}${mkv_folder}${mkv_filename}" "${mkvextract_recycle_folder}${mkv_folder}${mkv_filename}"
        echo "'${output_folder}${mkv_folder}${mkv_filename}' has been moved to '${mkvextract_recycle_folder}${mkv_folder}${mkv_filename}'"
    fi
    # one job per script exection is enough
    exit 0
fi
# get next disc folder
shopt -s nullglob
for disc_folder in "$storage_folder"*; do
    break;
done
shopt -u nullglob
# storage folder is empty
if [[ -z $disc_folder ]] || [[ ${disc_folder:(-1)} == "*" ]]; then
    echo "$storage_folder does not contain any disc backups (Result: '$disc_folder')"
    # at this point we could stop the makemkv docker container
    exit 0
fi
# skip disc if not named properly
disc_folder_new="${wrong_naming_folder}${disc_name}/"
if [[ ! $disc_folder =~ "("[0-9]{4}")" ]]; then
    mv "$disc_folder" "$disc_folder_new"
    echo "The disc folder name '$disc_folder' does not contain a year and was moved to '$wrong_naming_folder'!"
    exit 1
fi
# feel free to add own naming conditions:
#if [[ ! $disc_folder =~ "FSK"[0-9]{1,2} ]]; then
#    mv "$disc_folder" "$disc_folder_new"
#    echo "The disc folder name '$disc_folder' does not contain an age rating and was moved to '$wrong_naming_folder'!"
#    exit 1
#fi
#if [[ ! $disc_folder =~ "IMDB"[0-9]{1}\.[0-9]{1} ]]; then
#    mv "$disc_folder" "$disc_folder_new"
#    echo "The disc folder name '$disc_folder' does not contain an IMDB rating and was moved to '$wrong_naming_folder'!"
#    exit 1
#fi
disc_name=$(basename "$disc_folder")
docker_disc_folder="${docker_storage_folder}${disc_name}/"
# get titles information
disc_info="$(docker exec makemkv /opt/makemkv/bin/makemkvcon --minlength=600 --noscan -r info file:"$docker_disc_folder")"
# makemkv errors
disc_folder_new="${makemkv_error_folder}${disc_name}/"
if [[ -z $disc_info ]]; then
    mv "$disc_folder" "$disc_folder_new"
    echo "makemkvcon returned nothing about '$disc_folder'"
    exit 1
fi
if [[ $disc_info =~ "Failed to open disc" ]]; then
    mv "$disc_folder" "$disc_folder_new"
    echo "makemkvcon fails reading the disc folder $disc_folder (docker: ${docker_disc_folder})"
    echo "$disc_info"
    # note: never delete/rename/move the storage_folder as long the container is running. This will crash the container or freeze its content.
    exit 1
fi
# obtain disc type
if [[ $disc_info == *"Blu-ray disc"* ]]; then
    disc_type="BR"
else
    disc_type="DVD"
fi
# get forced title
unset force # go for sure it was not declared before
declare -A force
shopt -s nullglob
for filename in "$disc_folder"/*.txt; do
    filename="$(basename "$filename")"
    filename="${filename%.*}" # filename without extension (https://stackoverflow.com/a/965072/318765)
    if [[ ${filename:0:5} == "force" ]]; then
        filename="${filename:5}" # remove "force"
        force_key="title_${filename//[!a-zA-Z]/}"
        force_value="${filename//[!0-9]/}" # integer filtering (https://stackoverflow.com/a/19724571/318765)
        force[$force_key]=$force_value
    fi
done
shopt -u nullglob
title_ids=()
# search for title
while read -r line ; do
    # obtain title data
    title_name=$(echo $line | grep -oP '(?<=,2,0,").*?(?=")')
    title_id=$(echo $line | grep -oP '(?<=_t)[0-9]+(?=\.mkv)')
    title_size=$(echo $line | grep -oP '(?<=")[0-9.]+ (MB|GB)')
    title_size_symbol=${title_size:(-2)}
    title_size=${title_size:0:(-2)}
    title_mpls=$(echo $line | grep -oP '(?<=")[0-9]+(?=\.mpls)')
    title_m2ts=$(echo $line | grep -oP '(?<=,26,0,").*?(?=")')
    title_chapters=$(echo $line | grep -oP '(?<=- )[0-9]+(?= chapter)')
    # set forced title
    for key in "${!force[@]}"; do
        force_value=$((10#${force["$key"]})) # remove leading zero (https://stackoverflow.com/a/11130324/318765)
        if [[ -n ${!key} ]]; then
            title_value=$((10#${!key}))
        else
            title_value=${!key}
        fi
        if [[ "$force_value" == "$title_value" ]]; then # variable variables (https://stackoverflow.com/a/10757531/318765)
            title_ids+=("$title_id")
        fi
    done
    # size check
    if [[ ${#force[@]} == 0 ]]; then
        if [ "$title_size_symbol" == "GB" ]; then
            title_size=$(awk "BEGIN { print $title_size*1000}")
        fi
        title_size=${title_size%.*} # convert to integer
        if [ $title_size -ge $br_title_min_size ]; then
            title_ids+=("$title_id")
        fi
    fi
# we need to remove line breaks with "tr" to force grep to return one-liners (Note: Regex multiple lines was much faster than looping through all lines)
done < <(echo "$disc_info" | 
        tr -d '\n' | # we need to remove line breaks with "tr" to force grep to return one-liners
        grep -oP 'TINFO.*?SINFO') # Regex is faster than looping through all lines
# skip disc folder if no title has been found
if [[ ${#title_ids[@]} == 0 ]]; then
    disc_folder_new="${no_title_folder}${disc_name}/"
    echo "$disc_folder"
    echo "$disc_folder_new"
    mv "$disc_folder" "$disc_folder_new"
    echo "Because '$disc_name' does not contain a title that was big enough it was moved to '$no_title_folder'"
    echo "MKVnator needs your help now:"
    echo "- Reduce 'br_title_min_size' in MKVnator settings to a lower value"
    echo "- move '$disc_name' back to $storage_folder"
    exit 1
fi
# skip the disc folder with multiple titles (until MKVnator supports tv series)
if [[ ${#title_ids[@]} -ge 2 ]]; then
    disc_folder_new="${multi_title_folder}${disc_name}/"
    mv "$disc_folder" "$disc_folder_new"
    echo "'$disc_name' has been skipped because of multiple possible titles and was moved to '$multi_title_folder'"
    echo "MKVnator needs your help now:"
    echo "- manually check the disc folder to determine the correct title (eg use MakeMKV GUI and compare segments/mpls files through VLC)"
    echo "- create a file named '${disc_folder_new}force_mpls_00030.txt' to force the title that uses the playlist 00030.mpls"
    echo "- move the disc folder back to '$storage_folder'"
    exit 1
fi
# at this point only one title id is left
title_id=${title_ids[0]}
# create new disc folder for mkv output
disc_folder_new="${output_folder}${disc_name}/"
# check if this disc has been already converted
if [ -d "$disc_folder_new" ]; then
    disc_folder_new="${already_exists_folder}${disc_name}/"
    mv "$disc_folder" "$disc_folder_new"
    echo "'$disc_folder_new' already exists and '$disc_name' has been moved to '$already_exists_folder'"
    echo "MKVnator needs your help now:"
    echo "- manually check why '$disc_name' already exists in '$output_folder'"
    echo "- rename the disc name folder"
    echo "- move the disc folder back to '$storage_folder'"
    exit 1
fi
mkdir -p "$disc_folder_new"
docker_disc_folder_new="${docker_output_folder}${disc_name}/"
# create mkv
docker exec makemkv /opt/makemkv/bin/makemkvcon --minlength="$makemkv_minlength" --noscan mkv file:"$docker_disc_folder" "$title_id" "$docker_disc_folder_new"
echo "MKV has been created under '$docker_disc_folder_new'"
# rename mkv file
shopt -s nullglob
for mkv_filename in "$disc_folder_new"*; do
    mkv_filename_new="${disc_folder_new}${disc_name} ${disc_type}.mkv"
    mv "$mkv_filename" "$mkv_filename_new"
    echo "MKV has been renamed to '$mkv_filename_new'"
    if [ -f "$mkv_filename_new" ]; then
        echo "Title '$title_id' of disc '$disc_name' has been successfully converted to '$mkv_filename_new'!"
        # remove source
        if [[ $delete_disc_folder_source == "true" ]]; then
            rm -rf "$disc_folder"
            echo "'$disc_folder' has been deleted"
        else
            mv "$disc_folder" "$makemkv_recycle_folder"
            echo "'$disc_folder' has been moved to '$makemkv_recycle_folder'"
        fi
    else
        echo "Something went wrong creating '$mkv_filename_new'!"
        exit 1
    fi
    break;
done
shopt -u nullglob
Last edited by mgutt on Sat Aug 10, 2019 11:26 am, edited 2 times in total.

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Thu Aug 01, 2019 10:40 am

this post is reserved for later usage

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Fri Aug 02, 2019 12:24 am

Version 0.3 has been released:
# - the user can force a specific title by adding a file named "force_title02.txt" to the disc folder
# - buggy "wrong naming conditions" fixed
# - add disc type "BR" or "DVD" to mkv filename
# - added "delete_disc_folder_source" setting to preserve the source files

New function: Forcing title
If a disc contains multiple titles bigger than 10GB it will be moved by MKVnator to "/Rips_multiple_title_found". Now you need to determine the correct title by yourself. One example is the german Blu-Ray of "American Gangster". You can do that by using the MakeMKV GUI -> File -> Open file... -> "Rips_multiple_title_found/American Gangster (2007)/BDMV/index.bdmv". After MakeMKV loads the disc it contains two relevant titles:
2019-08-02 00_51_13.jpg
2019-08-02 00_51_13.jpg (38.23 KiB) Viewed 1919 times

Its really easy to verify that title #0 is the correct one as only this contains multiple audio track languages:
2019-08-02 00_52_04.jpg
2019-08-02 00_52_04.jpg (39.1 KiB) Viewed 1919 times

while title #1 contains only one english track:
2019-08-02 00_52_14.jpg
2019-08-02 00_52_14.jpg (44.6 KiB) Viewed 1919 times

So I placed an empty text file named "force_title0.txt" in "/volume1/video/Rips_multiple_title_found/American Gangster (2007)/" and moved this folder back to "/volume1/video/Rips/". The next time MKVnator.sh runs it will find this file and force the MKV creation by using title #0. So this bypasses the title size and multiple title checks.
Last edited by mgutt on Tue Aug 06, 2019 10:26 am, edited 2 times in total.

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Sat Aug 03, 2019 11:59 am

Version 0.4 has been released:
# - New function: Remux the MKV file by choosing subtitle and HD/Core audio languages through the settings.
# - New function: Move files to recycle folder instead of deleting
The next step is done. Created MKVs are remuxed through mkvextract docker. By that we have two new settings:

Code: Select all

mkv_langs="ger,eng,tur" # use "all" to preserve all audio and subtitle languages, first listed language is set as default
mkv_audio="hd" # "hd" preserves DTS-HD/True HD tracks and "core" preserves core tracks. Use "hd,core" to preserve both
With this settings we remux this:
2019-08-03 13_49_54.jpg
2019-08-03 13_49_54.jpg (70 KiB) Viewed 1875 times
to this:
2019-08-03 13_50_21.jpg
2019-08-03 13_50_21.jpg (30.9 KiB) Viewed 1875 times
As "ger" is the first language set in mkv_langs it will be set as default audio track.

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Sat Aug 03, 2019 5:11 pm

# - the user can force a specific title by adding a file named "force02.txt" to the disc folder
This does not work at the moment because makemkvcon returns the title in a different order than MakeMKV GUI :shock:

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Sun Aug 04, 2019 1:36 am

Version 0.5 has been released:

Code: Select all

# - Forcing has been extended: The user can force a specific title by adding a file named "force_mpls00040.txt" to the disc folder
# - Forcing by title id is still supported but not recommend as MakeMKV GUI returns different title ids than makemkvcon (https://makemkv.com/forum/viewtopic.php?f=10&t=20081)
# - You can force titles with the keywords mpls, m2ts, size, id, chapters, etc This means "force_size_12.5.txt" forces all titles with a size of 12.5 GB.
This solves the forcing issue that MakeMKV GUI returns different title ids compared to makemkvcon. I suggest now to use a text file called "force_mpls_00001.mpls" to force the title with this playlist.

Later, after MKVnator supports TV series, you can force multiple titles as well. For example you can export all tracks with 20 chapters like "forcechapters20.txt". As you can see its not important how the filename is formatted. The only requirement is that it starts with the word "force", followed by the key and an integer or floating number. For example "force-size-24.0.txt" works as well to select all titles with a size of "24.0 GB".

Example:
2019-08-04 03_03_26.jpg
2019-08-04 03_03_26.jpg (18.5 KiB) Viewed 1851 times

The screenshots shows that leading zeros aren't important as well ;)

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Sun Aug 04, 2019 8:28 pm

I did a bigger test run and it works really good now. Rips with wrong names or multiple titles are moved and the others have been all converted and remuxed to MKV without errors:
2019-08-04 22_04_52.jpg
2019-08-04 22_04_52.jpg (56 KiB) Viewed 1831 times

Its time for the next steps:
# - adding missing ac3 audio track
# - auto naming of subtitles (at first only forced subtitles and as long the biggest advantage "OCR" is not solved we determine it by file size)
And there are much more ideas:
# - add similar support for ignoreForcedSubtitlesFlag="true", setFirstAudioTrackAsDefault="true", setFirstSubtitleTrackAsDefault="true", setFirstForcedSubtitleTrackAsDefault="true"
# - add similar support for -sel:all,+sel:(ger|eng|tur|nolang),-sel:(core),-sel:mvcvideo,=100:all,-10:ger,+sel:attachment
# - add debug mode so a user can create some logs
# - skip tv series by checking sXXeYY, YYYY(-|.| )MM(-|.| )DD or DD(-|.| )MM(-|.| )YYYY in folder name https://support.plex.tv/articles/naming ... how-files/
# - add support for tv series
# - skip mkvextract -J if mkv_langs=all and mkv_audio=hd,core
# - skip mkvextract -o if all audio tracks have been selected so no audio track needs to be removed
# - skip mkvextract if mkv filename ends with " (1).mkv"
# - create docker container on-the-fly
# - kill docker container after the last disc/mkv has been finished
# - add docker alternative: local app execution
# - make MKVnator save against race conditions. mkdir should be atomic enough.
# - add more keywords to forcing: playlist, streams, stream
# - the forcing keyword of the text file is "force", we should support it case-insensitive
# - we need error handling if moving folders with mv targets non-empty folder
# - scan an existing collection of mkv files and check if audio tracks should be removed, the naming is wrong, etc
# - add containing languages to the mkv filename
# - makemkvcon --minlength should be part of the MKVnator settings

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Mon Aug 05, 2019 12:53 am

As many movies do not contain forced subtitles at all, its not a solution to automatically set the smallest subtitle track as forced. Instead I thought using the biggest subtitle filesize of all my movies as a limiter.

I used this command to print the filesize of all forced SRT subtitles:

Code: Select all

while IFS= read -r -d '' filename; do
    filesize=$(stat -c%s "$filename")
    echo "$filename has a size of $filesize bytes."
done < <(find /volume1/video/Filme -name "*.srt" -print0)
By that I found the biggest for Inglourious Basterds. So extracted its forced SUP track:

Code: Select all

docker exec mkvtoolnix /usr/bin/mkvextract "/storage/video2.mkv" tracks 7:"/storage/subtitle7.sup"
It has a size of 8.5 MB. This is really huge for a forced subtitle track as some of the other movies with general subtitles have a filesize of 12.5 MB. I need to check more movies to find out if < 9 MB is a good rule.

EDIT: The 2nd biggest forced subtitle has Kill Bill 1 with 2.5 MB. And most movies have < 1,5 MB. Maybe this is enough to realize a 99% hit rate. We will see...

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Tue Aug 06, 2019 1:16 am

Version 0.6 has been released:

Code: Select all

# - New function: Determine forced subtitle by exporting it as sup file and check its filesize. It must be smaller than the new setting br_sub_forced_max_size. Default is 3 MB.
Before:
2019-08-06 03_03_27.jpg
2019-08-06 03_03_27.jpg (29.04 KiB) Viewed 1800 times

After:
2019-08-06 03_04_05.jpg
2019-08-06 03_04_05.jpg (58.48 KiB) Viewed 1800 times

As you can see it preserves the forced SUP file as well. This could be important for the future step converting it to SRT. Later we could think about preserving all subtitles files to determine their different names through OCR and text filters.

Possible solutions for converting to SRT are:
https://hub.docker.com/r/fresta/subtitleedit/
https://www.videohelp.com/software/BDSup2Sub
https://www.videohelp.com/software/SupRip

The setting br_sub_forced_max_size needs more testing. Of course we need other values for DVD rips. But as DVDs are not supported by MKVnator at the moment this is at the end of the list ;)

The next step of MKVnator is to add an AC3 (Dolby Digital) audio track if the MKV does not contain it for the default lang.

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Wed Aug 07, 2019 10:32 am

Something is wrong with the auto naming of forced subtitles. This movie contains two forced subtitles. As you can see MKVnator determined track 5 and track 7 as "Forced" and exported the SUP files:
2019-08-07 12_25_43.jpg
2019-08-07 12_25_43.jpg (69.44 KiB) Viewed 1741 times

If I open subtitle5.eng.forced.sup with Subtitle Edit its correct as well as it contains only 9 lines:
2019-08-07 12_29_44.jpg
2019-08-07 12_29_44.jpg (39.66 KiB) Viewed 1741 times

But if I open the movie file in Subtitle Edit and select the 5th track it is a "Regular" subtitle with 1393 lines. Instead the 6th track is the "Forced" subtitle. I'm afraid that "mkvmerge -J" returns a different order than "mkvpropedit". I will test that.

EDIT: Ok, solved it. Its not a different sorting. mkvpropedit does not start counting at zero like mkvmerge:
https://mkvtoolnix.download/doc/mkvprop ... _selectors
track:n
If the parameter n is a number then the nth track will be selected. The track order is the same that mkvmerge(1)'s --identify option outputs.

Numbering starts at 1.
:roll:

Will be solved in the next version :wink:

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Wed Aug 07, 2019 11:27 am

Version 0.7 has been released:
# - Bux fix in forced subtitle auto naming
# - New setting "makemkv_minlength" added (default is 600 seconds)
# - Bug fix for empty loops (shopt -s nullglob)
# - Skip remux if no audio track or subtitle would be removed
# - Skip auto subtitle naming if MKV already contains subtitle names
# - Remuxed MKV files are renamed to ".rmx.mkv"
# - MKV files with auto named subtitles are renamed to ".sub.mkv"
# - Multiple code lines have been rewritten to take advantage of arrays as app parameters (like "mkvpropedit_options")
As you can see the bug stated in my last post has been solved. The most important new functionallity is that MKVnator skips auto subtitle naming if the source mkv file already contains named subtitle. By that we avoid overwriting your manual work of the past. If it finds such MKVs it renames them to ".sub.mkv" to skip them in the next run.

One of the later versions will have a final "cleanup" that renames all .rmx.mkv and .sub.mkv back to .mkv again. This is only a temporay filename for MKVnator knowing which files can be skipped.

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Sat Aug 10, 2019 11:26 am

Version 0.8 has been released:

Code: Select all

# - New function: Add AC3 audio track if MKV does not contain it. Its generated through the DTS audio track in default language. The setting to enable it is mkv_audio_add_ac3 (default is false)
# - Bug fix regarding arrays containing unquoted wildcards and question marks (https://unix.stackexchange.com/questions/534858/why-does-shopt-s-nullglob-remove-a-string-with-question-mark-in-an-array-elemen)
Wow, the usage of ffmpeg is really crazy. But finally I understood the syntax and was able to realize the new feature of adding automatically a AC3 audio track if the MKV does not contain it.

At first the MKV information is analized:
- Does it already contain a german AC3 track with 6 or more channels? Then skip.
- Does it contain a german DTS track (the changeable default language of MKVnator) and is it the best we could find (most channels, best quality)? Then use it for generating the AC3 track.
- Has the best DTS track as much or less channels as the best AC3 track? Then skip.

These are the used ffmpeg commands:

Code: Select all

-y # overwrite output file
-nostats # disable progress
-loglevel error
-map 0:v # select video stream
-map 0:$best_dts_track # add new ac3 audio track at first
-map 0:a # add all audio tracks (including the best dts audio track)
-map 0:s? # add all subtitles (if present)
-map 0:d? # add all data (if present)
-map 0:t? # add all attachments (if present)
-c copy # use the codec "copy" (no re-encoding) for all selected tracks
# beware: The index of "-c" (codec) and "-b" (bitrate) does not target the track index of the source
#         Instead it targets the new index created through the multiple "-map" commands.
#         This means "-c:1" will re-encode "-map 0:$best_dts_track"
-c:1 ac3 # the best dts track will be re-encoded as "ac3"
-b:1 640k # the best dts track will be re-encoded with 640 kbit/s (Blu-Ray standard, for DVD we need 448k (https://gdion.wordpress.com/2016/06/12/simple-dts-to-ac3-using-ffmpeg/)
As you can see it adds the new AC3 track as the first audio track and leaves all other existing tracks untouched.

To enhance the usability of MKVnator the ffmpeg docker container is this time created through the script itself:

Code: Select all

        docker_options="run \
                --name=ffmpeg \
                -e TZ=Europe/Berlin \
                -e USER_ID=1000 \
                -e GROUP_ID=100 \
                -p 5803:5803 \
                -v ${output_folder}:/storage:rw \
                jrottenberg/ffmpeg"
...
        docker $docker_options -i "$ffmpeg_src_filename" ${ffmpeg_options[@]} "$ffmpeg_dst_filename"
        docker stop ffmpeg
        docker rm ffmpeg
The same will be done in one of the next versions with MKVToolnix and MakeMKV containers so a user only needs to download these docker containers.

mgutt
Posts: 65
Joined: Sun May 05, 2019 6:38 pm

Re: MKVnator - Automatic MKV creation with your NAS/server

Post by mgutt » Sun Aug 11, 2019 11:36 pm

Trying to find a solution to convert Blu-Ray SUP to SRT by Linux command line ends without success. Subtitle Edit does not support OCR through command line.

The next idea was to find a tool that extracts the images of the SUP file and use Tesseract 4 in an additional step to OCR the texts. A linux solution I found was OGMRip that contains subp2png, but it supports only VobSub:
http://ogmrip.sourceforge.net/en/manual.html

It seems there is nothing that works without a GUI and/or requiring Windows. But I found an alternative that requires only a webserver and is able to extract PNGs out of SUP files:
https://github.com/SjorsO/sup

I'll play around with that and try to create an API.

Post Reply