Category Archives: Programming

Extracting GPS data from the GoPro 7

28th July 2019 Karl 2 Comments

While extracting the telemetry data from the GoPro is reasonably well documented I have found some gaps for getting the extracting utilities installed and when extracting and combining data from multiple files. These notes are for a Debian/Ubuntu installation in a BASH Shell.

Installing the gopro-utils

As I couldn’t find any straightforward instructions for installation, I’ll be going through everything I needed to do to get it working, you may have some of these packages installed already.

sudo apt update
sudo apt upgrade
sudo apt install ffmpeg golang gpsbabel git

sudo apt update

sudo apt upgrade

sudo apt install ffmpeg golang gpsbabel git

Now to get the gopro-utils and install them, I’m placing the source files into my Downloads directory, as well as the GPS data extractor we’ll be adding the other telemetry tools too, this is all a bit long winded.

cd ~/Downloads
git clone https://github.com/JuanIrache/gopro-utils
cd gopro-utils
go get github.com/JuanIrache/gopro-utils/telemetry
go get github.com/mlouielu/gpxgo/gpx
cd bin/gopro2gpx
go build gopro2gpx.go
sudo cp gopro2gpx /usr/local/bin/
cd ../bin/gopro2geojson/
go build gopro2geojson.go
sudo cp gopro2geojson /usr/local/bin/
cd ../gopro2json/
go build gopro2json.go
sudo cp gopro2json /usr/local/bin/
cd ../gpmd2csv/
go build gpmd2csv.go
sudo cp gpmd2csv /usr/local/bin/
cd ../gpmd2info/
go build gpmd2info.go
sudo cp gpmd2info /usr/local/bin/
cd ../gps2kml/
go build gps2kml.go
sudo cp gps2kml /usr/local/bin/

cd ~/Downloads

git clone https://github.com/JuanIrache/gopro-utils

cd gopro-utils

go get github.com/JuanIrache/gopro-utils/telemetry

go get github.com/mlouielu/gpxgo/gpx

cd bin/gopro2gpx

go build gopro2gpx.go

sudo cp gopro2gpx /usr/local/bin/

cd ../bin/gopro2geojson/

go build gopro2geojson.go

sudo cp gopro2geojson /usr/local/bin/

cd ../gopro2json/

go build gopro2json.go

sudo cp gopro2json /usr/local/bin/

cd ../gpmd2csv/

go build gpmd2csv.go

sudo cp gpmd2csv /usr/local/bin/

cd ../gpmd2info/

go build gpmd2info.go

sudo cp gpmd2info /usr/local/bin/

cd ../gps2kml/

go build gps2kml.go

sudo cp gps2kml /usr/local/bin/

Extracting the Data

You will need to find which stream in the video recording the data has been saved to, to find this use ffprobe to examine the recording and look for the stream that contains GoPro MET, for example:

$ ffprobe GX090155.MP4
[snip]
    Stream #0:3(eng): Data: none (gpmd / 0x646D7067), 40 kb/s (default)
    Metadata:
      creation_time   : 2019-07-28T11:09:25.000000Z
      handler_name    : GoPro MET
[snip]

$ ffprobe GX090155.MP4

[snip]

Stream #0:3(eng): Data: none (gpmd / 0x646D7067), 40 kb/s (default)

Metadata:

creation_time : 2019-07-28T11:09:25.000000Z

handler_name : GoPro MET

[snip]

You can see that what we are wanting is on stream 3, as far as I can tell this stays the same every time, I don’t know if it is different for other GoPro models.

This bash script extracts the GPS data in GPX format from all the GoPro GX recordings in the directory, other options have been commented out, if you are using Garmin VIRB edit there is also an option for use with that. The script creates two files, one that contains the raw data and another with the desired GPS data, the GPS output file has the same name as the recording, but in lowercase with a .gpx extension.

#!/bin/bash
FILES="GX*.MP4"
STREAM="0:3"
for MP4FILE in $FILES; do
   OUTFILE=$(basename "$MP4FILE" | cut -d. -f1)
   
   OUTFILE=$(echo "$OUTFILE" | tr '[:upper:]' '[:lower:]')
   BINFILE="$OUTFILE.bin"
   ffmpeg -y -i "$MP4FILE" -codec copy -map $STREAM -f rawvideo "$BINFILE"

   #gpmd2csv -i "$BINFILE" -o "$OUTFILE.csv"
   gopro2gpx -i "$BINFILE" -a 500 -f 2 -o "$OUTFILE.gpx"
   #gopro2gpx -i "$BINFILE" -a 200 -f 3 -o "$OUTFILE-virb.gpx"
   #gopro2json -i "$BINFILE" -o "$OUTFILE.json"
done

#!/bin/bash

FILES="GX*.MP4"

STREAM="0:3"

for MP4FILE in $FILES; do

OUTFILE=$(basename "$MP4FILE" | cut -d. -f1)

OUTFILE=$(echo "$OUTFILE" | tr '[:upper:]' '[:lower:]')

BINFILE="$OUTFILE.bin"

ffmpeg -y -i "$MP4FILE" -codec copy -map $STREAM -f rawvideo "$BINFILE"

#gpmd2csv -i "$BINFILE" -o "$OUTFILE.csv"

gopro2gpx -i "$BINFILE" -a 500 -f 2 -o "$OUTFILE.gpx"

#gopro2gpx -i "$BINFILE" -a 200 -f 3 -o "$OUTFILE-virb.gpx"

#gopro2json -i "$BINFILE" -o "$OUTFILE.json"

done

Merging GPX files

As the GoPro splits recordings into 4GB blocks, when extracting you will get a single GPX file for each recording. Many pages found by Google say that to create a single track from these all you need to do is append the files into one big file. This is wrong, what you end up with is a single file with many short tracks, when what you are after is one long track covering the entire journey. This bash script uses gpsbabel to create single merged file from the extracted GPX data, it creates a file called “gpsoutput.gpx”.

#!/bin/bash
FILES="gx*.gpx"
ff=""
for f in $FILES; do
    ff="$ff -f $f"
done
gpsbabel -t -i gpx $ff -o gpx -x track,merge,title="GOPRO COMBINED LOG"  -F gpsoutpt.gpx

#!/bin/bash

FILES="gx*.gpx"

ff=""

for f in $FILES; do

ff="$ff -f $f"

done

gpsbabel -t -i gpx $ff -o gpx -x track,merge,title="GOPRO COMBINED LOG" -F gpsoutpt.gpx

The next stage will be to write a script that combines all these and completes the job in one easy process.

Links and Sources

Web Page to extract GoPro data: https://tailorandwayne.com/gopro-telemetry-extractor/ Easy to use for individual files
https://github.com/JuanIrache/gopro-utils
GoPro Forum Extracting the metadata in a useful format
GPSBabel, Free software for GPS data conversion and transfer: https://www.gpsbabel.org/

Arduino, Electronics, Hardware, Programming

The Fridge Door is Open

29th August 2018 Karl

My fridge door tends to rebound when closed staying open a smidgen and letting all the cold out. Rather than just checking that the door is properly shut, I thought it about time to have a microcontroller make a noise when the door has been left open too long.

This circuit uses an Arduino compatible Teensy LC for all the work, it has a phototransistor to sense the state of the fridge door light, a couple of LED’s one to indicate the power and another that comes on when the door is open. There is also Piezo buzzer to make an annoying noise after forty five seconds of door open time. The unit runs of a 3.7v rechargeable Lithium-ion battery and I have added a recharging circuit that takes power via the Teensy’s 5v USB port.

Circuitry

Note: These diagrams show 3.7v as the supply voltage. The Teensy LC can only tolerate a maximum 3.3V on the data pins, so these circuits are driven from the 3V output on the Teensy. They will all work without modification on the 5v Arduino Uno.

I have used a phototransistor to detect the fridge door light, there are two variants of this circuit light activated or dark activated, the 100k resistor can be replaced with a 100k variable if you need to adjust the sensitivity, the 330k resistor provides a weak pull-down on the output. The phototransistor is being used in switch mode to provide a logical output (rather than active mode which provides an output relative to the amount of light), so the output is connected to a digital input on the Arduino. The BC547 transistor is half of the darlington pair to provide extra gain on the output.

I chose the light activated switch, either will do but will provide different logical outputs to your controller. The circuit is enclosed in a small box inside the fridge and connected by ribbon cable to the controller, the ribbon cable is flat and does not upset the magnetic ‘seal’ on the fridge door.

To make some noise I used a piezo buzzer from an old computer, this is driven through a transistor as the Teensy does not provide enough current to drive it directly.

There is also a push button to provide a reset function if the buzzer is sounding while the door is open.

I have also added two LED’s, one to show power and anther that illuminates when the door is opened.

The final circuit if for recharging the battery, it connects to the 5V connection on the Teensy LC so charges the battery when the USB connection is in use. This has been copied from the MCP3831T datasheet.

Software

This uses an interrupt to listen for the light sensor, when the state changes, the door open pin is read to determine if the door is open or not. If it is then a timer is started, this gives you forty five seconds to complete your task before the alarm sounds. With the door closed the timer is stopped and set back to zero. If the sounder goes off while you are rummaging in the fridge the reset button can be pressed, this restarts the timer from zero again.

#define TEST_LED 13
#define FRIDGE_DOOR 6
#define DOOR_LED 8
#define POWER_LED 7
#define RESET_BTN 9
#define BUZZER 10

volatile boolean doorOpen = true;   // when the light is on
volatile boolean doorTrigger = false;
volatile unsigned long lastFridgeMillis = 0;
volatile unsigned long fridgeTimeoutMillis = 10;

boolean SerialDebug = false;
unsigned long doorOpenDelay = 45000; // 45 second timeout before buzzer sounds
unsigned long doorOpenTime = 0;
unsigned long startTime;
unsigned long previousToneMillis = 0;
unsigned long toneInterval = 20;
boolean soundTheAlarm = false;
boolean outputTone = false;
int toneMelody[5] = { 1600, 1900, 2000, 2100, 1800 };
const int toneMax = 5; 

unsigned long lastDoorOpenTime = 1;
char pout[80]; // for testing

// interrupt function
void fridgeDoor() {
  if ((millis() - lastFridgeMillis) > fridgeTimeoutMillis) {
    doorOpen = digitalRead(FRIDGE_DOOR); // goes high when dark
    lastFridgeMillis = millis();
  }
}

// see if the door is open or closed
void fridgeDoorOpen() {
  if (doorOpen == true && doorTrigger == false) {
    doorTrigger = true;
    startTime = millis();

    if (SerialDebug) {
      sprintf(pout, "door open: %d", doorOpen);
      Serial.println(pout);
    }
    digitalWrite(DOOR_LED, HIGH);
    return;
  }

  if (doorOpen == false && doorTrigger == true) {
    doorTrigger = false;

    if (SerialDebug) {
      sprintf(pout, "door closed: %d", doorOpen);
      Serial.println(pout);
    }
    startTime = 0;
    doorOpenTime = 0;
    soundTheAlarm = false;
    digitalWrite(DOOR_LED, LOW);
    return;
  }

}

// make some noise
void soundAlarm() {

  if (soundTheAlarm == false) {
    return;
  }

  int ii = 0;
  while (ii < 6) {
    if (soundTheAlarm == false) {
      break;
    }
    int i = 0;
    while (i < toneMax) {
      if (soundTheAlarm == false) {
        break;
      }

      unsigned long currentMillis = millis();
      if((currentMillis - previousToneMillis) > 50) {
        previousToneMillis = currentMillis;
        tone(BUZZER,toneMelody[i]-380,60);
        i++;
        }
    }
    ii++;
  }

}

// door open? then increment the timer - sound the alarm 
void fridgeDoorOpenTimer() {

  if (doorOpen == false && doorTrigger == false) {
    return;
  }

  if (digitalRead(RESET_BTN) == HIGH) {
    digitalWrite(POWER_LED, LOW);
    while (digitalRead(RESET_BTN) == HIGH) {
      delay(10);
    }

    if (SerialDebug) {
      Serial.println("reset");
    }
    startTime = millis();
    soundTheAlarm = false;
    digitalWrite(POWER_LED, HIGH);
  }

  // checking once a second
  doorOpenTime = millis() - startTime;
  if (doorOpenTime % 1000 == 0 && doorOpenTime != lastDoorOpenTime) {
    if (SerialDebug) {
      Serial.println(doorOpenTime);
    }

    if (doorOpenTime > doorOpenDelay) {
      if (SerialDebug) {
        Serial.println("Awwooga Awwooga - fridge door open!");
      }
      soundTheAlarm = true;
      soundAlarm();
    }

    lastDoorOpenTime = doorOpenTime;
  }

}


void setup() {

  pinMode(TEST_LED, OUTPUT);
  digitalWrite(TEST_LED, HIGH);

  if (SerialDebug) {
    Serial.begin(9600);
  }

  delay(2000);

  pinMode(DOOR_LED, OUTPUT);
  pinMode(POWER_LED, OUTPUT);
  pinMode(BUZZER, OUTPUT);

  if (SerialDebug) {
    Serial.println("ready");
  }
  digitalWrite(POWER_LED, HIGH);

  pinMode(FRIDGE_DOOR, INPUT);
  attachInterrupt(digitalPinToInterrupt(FRIDGE_DOOR), fridgeDoor, CHANGE);

  pinMode(RESET_BTN, INPUT);
  digitalWrite(TEST_LED, LOW);
}

void loop() {
  fridgeDoorOpen();
  fridgeDoorOpenTimer();
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

#define TEST_LED 13

#define FRIDGE_DOOR 6

#define DOOR_LED 8

#define POWER_LED 7

#define RESET_BTN 9

#define BUZZER 10

volatile boolean doorOpen = true; // when the light is on

volatile boolean doorTrigger = false;

volatile unsigned long lastFridgeMillis = 0;

volatile unsigned long fridgeTimeoutMillis = 10;

boolean SerialDebug = false;

unsigned long doorOpenDelay = 45000; // 45 second timeout before buzzer sounds

unsigned long doorOpenTime = 0;

unsigned long startTime;

unsigned long previousToneMillis = 0;

unsigned long toneInterval = 20;

boolean soundTheAlarm = false;

boolean outputTone = false;

int toneMelody[5] = { 1600, 1900, 2000, 2100, 1800 };

const int toneMax = 5;

unsigned long lastDoorOpenTime = 1;

char pout[80]; // for testing

// interrupt function

void fridgeDoor() {

if ((millis() - lastFridgeMillis) > fridgeTimeoutMillis) {

doorOpen = digitalRead(FRIDGE_DOOR); // goes high when dark

lastFridgeMillis = millis();

}

// see if the door is open or closed

void fridgeDoorOpen() {

if (doorOpen == true && doorTrigger == false) {

doorTrigger = true;

startTime = millis();

if (SerialDebug) {

sprintf(pout, "door open: %d", doorOpen);

Serial.println(pout);

}

digitalWrite(DOOR_LED, HIGH);

return;

}

if (doorOpen == false && doorTrigger == true) {

doorTrigger = false;

if (SerialDebug) {

sprintf(pout, "door closed: %d", doorOpen);

Serial.println(pout);

}

startTime = 0;

doorOpenTime = 0;

soundTheAlarm = false;

digitalWrite(DOOR_LED, LOW);

return;

}

// make some noise

void soundAlarm() {

if (soundTheAlarm == false) {

return;

}

int ii = 0;

while (ii < 6) {

if (soundTheAlarm == false) {

break;

}

int i = 0;

while (i < toneMax) {

if (soundTheAlarm == false) {

break;

}

unsigned long currentMillis = millis();

if((currentMillis - previousToneMillis) > 50) {

previousToneMillis = currentMillis;

tone(BUZZER,toneMelody[i]-380,60);

i++;

}

ii++;

}

// door open? then increment the timer - sound the alarm

void fridgeDoorOpenTimer() {

if (doorOpen == false && doorTrigger == false) {

return;

}

if (digitalRead(RESET_BTN) == HIGH) {

digitalWrite(POWER_LED, LOW);

while (digitalRead(RESET_BTN) == HIGH) {

delay(10);

}

if (SerialDebug) {

Serial.println("reset");

}

startTime = millis();

soundTheAlarm = false;

digitalWrite(POWER_LED, HIGH);

}

// checking once a second

doorOpenTime = millis() - startTime;

if (doorOpenTime % 1000 == 0 && doorOpenTime != lastDoorOpenTime) {

if (SerialDebug) {

Serial.println(doorOpenTime);

}

if (doorOpenTime > doorOpenDelay) {

if (SerialDebug) {

Serial.println("Awwooga Awwooga - fridge door open!");

}

soundTheAlarm = true;

soundAlarm();

}

lastDoorOpenTime = doorOpenTime;

}

void setup() {

pinMode(TEST_LED, OUTPUT);

digitalWrite(TEST_LED, HIGH);

if (SerialDebug) {

Serial.begin(9600);

}

delay(2000);

pinMode(DOOR_LED, OUTPUT);

pinMode(POWER_LED, OUTPUT);

pinMode(BUZZER, OUTPUT);

if (SerialDebug) {

Serial.println("ready");

}

digitalWrite(POWER_LED, HIGH);

pinMode(FRIDGE_DOOR, INPUT);

attachInterrupt(digitalPinToInterrupt(FRIDGE_DOOR), fridgeDoor, CHANGE);

pinMode(RESET_BTN, INPUT);

digitalWrite(TEST_LED, LOW);

}

void loop() {

fridgeDoorOpen();

fridgeDoorOpenTimer();

}

Links and Sources

Practical Electronics for Inventors, Third Edition. Paul Scherz and Simon Monk, pub. McGraw Hill, 2013. ISBN 978-0-07-177133-7
A pre-made USB LiPo Battery Charger – 3.7V Single Cell from Hobbytronics or from Sparkfun
Battery Charger MCP3831T Datasheet
TEPT4400 Phototransistor Datasheet

APPLE, Computer, linux, Programming, Raspberry Pi

Extracting MP3 audio from video files

29th July 2018 Karl

Here is a small Bash script that converts any supported ffmpeg video format; such as .MKV, .MP4 or .MOV and extracts the audio to an .MP3 file, It will also split that MP3 file into chunks and put them in a convenient directory. You will need to install ffmpeg and mp3splt for your particular platform.

Example Usage:

./mkv2mp3 "big fat file.mkv"

1	./mkv2mp3 "big fat file.mkv"

This uses ffmpeg to convert “big fat file.mkv” to “big fat file.mp3” and then uses mp3splt to create a directory “big fat file” containing the files 01 – big fat file.mp3, 02 – big fat file.mp3, etc. The MP3 files will be encoded at 128k Constant Bit Rate and each file will be around 50 minutes in length. To install in Debian/Ubuntu use: sudo apt-get install ffmpeg mp3splt

#!/bin/bash
INFILE=$1
BITRATE="128k"
LENGTH="50.0"

FILENAME=$(basename "$INFILE" | cut -d. -f1)
if [[ -z "${FILENAME// }" ]]; then
   echo "filename missing"
   exit 1
fi

echo "$FILENAME"
ffmpeg -i "$INFILE" -vn -acodec libmp3lame -b:a "$BITRATE" "$FILENAME".mp3
mp3splt -t "$LENGTH" -d "$FILENAME" -a -o @n+-+@f "$FILENAME".mp3

#!/bin/bash

INFILE=$1

BITRATE="128k"

LENGTH="50.0"

FILENAME=$(basename "$INFILE" | cut -d. -f1)

if [[ -z "${FILENAME// }" ]]; then

echo "filename missing"

exit 1

echo "$FILENAME"

ffmpeg -i "$INFILE" -vn -acodec libmp3lame -b:a "$BITRATE" "$FILENAME".mp3

mp3splt -t "$LENGTH" -d "$FILENAME" -a -o @n+-+@f "$FILENAME".mp3

mp3splt can find the audio in a quiet region near where the split is desired rather than midway through a word, this should make for much cleaner playback across tracks.

Alternative Method

This script gives the same results but uses ffmpeg to split the large MP3 file and then adds track numbering metadata using id3v2. To install in Debian/Ubuntu use: sudo apt-get install ffmpeg id3v2

#!/bin/bash
INFILE="$1"
BITRATE="128k"
LENGTH="3000"    # 50 minutes in seconds

FILENAME=$(basename "$INFILE" | cut -d. -f1)
if [[ -z "${FILENAME// }" ]]; then
   echo "filename missing"
   exit 1
fi

echo "$FILENAME"
MP3FILE="$FILENAME".mp3
ffmpeg -y -i "$INFILE" -vn -acodec libmp3lame -b:a "$BITRATE" "$MP3FILE"

OUTDIR="$FILENAME"
mkdir "$OUTDIR"

ffmpeg -y -i "$MP3FILE" -vn -f segment -segment_time "$LENGTH" -c copy "$OUTDIR"/%02d-"$MP3FILE"

TRACK=1
IFS=$'\n'
FILES=($(find "$OUTDIR" -name "*.mp3" | sort -n))
for file in "${FILES[@]}"; do
  TN=$(printf %02d $TRACK)
  id3v2 --track "$TN" --TIT2 "$FILENAME" "$file"
  TRACK=$((TRACK+1))
done

#!/bin/bash

INFILE="$1"

BITRATE="128k"

LENGTH="3000" # 50 minutes in seconds

FILENAME=$(basename "$INFILE" | cut -d. -f1)

if [[ -z "${FILENAME// }" ]]; then

echo "filename missing"

exit 1

echo "$FILENAME"

MP3FILE="$FILENAME".mp3

ffmpeg -y -i "$INFILE" -vn -acodec libmp3lame -b:a "$BITRATE" "$MP3FILE"

OUTDIR="$FILENAME"

mkdir "$OUTDIR"

ffmpeg -y -i "$MP3FILE" -vn -f segment -segment_time "$LENGTH" -c copy "$OUTDIR"/%02d-"$MP3FILE"

TRACK=1

IFS=$'\n'

FILES=($(find "$OUTDIR" -name "*.mp3" | sort -n))

for file in "${FILES[@]}"; do

TN=$(printf %02d $TRACK)

id3v2 --track "$TN" --TIT2 "$FILENAME" "$file"

TRACK=$((TRACK+1))

done

Creating an Audiobook

Taking this further, I was thinking that it would be nice to have these converted into the M4B Audiobook format for use on my elderly iPod. The script below assumes that you have processed the files as above and have added metadata tags using a tool like mp3tag (yes I know this is for Windows).

To complete this we need to: Combine the multiple MP3 files into one big file, or read the original big file then convert that to M4B format at 96K bit and add chapter marks every ten minutes. For this I have used ffmpeg v3.2.12 and libmp4v2 (for the mp4chaps utility), to install in Debian/Ubuntu use: sudo apt-get install libmp4v2-dev mp4v2-utils ffmpeg

This script works best from a single MP3 file rather than from those that have been re-combined back into a single file, recombining the files caused ffmpeg to exclaim “invalid packet size” and “invalid data” errors. It is able to tell the difference between a directory and a single MP3 and processes the file accordingly, don’t forget to add metadata tags and cover art before you run the script.

#!/bin/bash
INFILE="$1"

BITRATE="96k"
MP3TITLE=""
MP3ARTIST=""
MP3ALBUM=""
MP3COMMENT=""
MP3YEAR=""
MP3PERFORMER=""

####
####  FUNCTIONS
####

function setMP3variables() {
  TAG="$1"
  VALUE="$2"

  case "$TAG" in
    title)
      MP3TITLE="$VALUE"
      ;;
    artist)
      MP3ARTIST="$VALUE"
      ;;
    comment)
      MP3COMMENT="$VALUE"
      ;;
    album_artist)
      MP3PERFORMER="$VALUE"
      ;;
    album)
      MP3ALBUM="$VALUE"
      ;;
    date)
      MP3YEAR="$VALUE"
      ;;
   esac
}

## get the tag text from the metadata
## if the line contains an equals (=) then split by the first equals.
function getMP3tags() {
  MP3METADATA="$1"
  while read -r line
  do
    case "$line" in
      *=*)
        TAG=${line%%"="*}
        VALUE=${line#*"="}
        setMP3variables "$TAG" "$VALUE"
        ;;
    esac
  done < "$MP3METADATA"

  if [[ ! -z "${MP3TITLE// }" ]]; then
    OUTFILE="$MP3TITLE".mp3
    M4BFILE="$MP3TITLE".m4b
  fi
}

####
####   BEGINS
####

FILENAME=$(basename "$INFILE" | cut -d. -f1)
if [[ -z "${FILENAME// }" ]]; then
   echo "filename missing"
   exit 1
fi
OLDIFS=$IFS
OUTFILE="$FILENAME"_out.mp3
M4BFILE="${OUTFILE:0:-4}".m4b
METADATA="${OUTFILE:0:-4}".metadata
JPGFILE="${OUTFILE:0:-4}".jpg


## check if input is a diretory
if [ -d "$INFILE" ]; then
  # get the name of the first MP3 file in the directory
  FIRSTFILE=$(find "$INFILE" -name "*.mp3" | sort -n | head -1)
  FIRSTFILE=$(basename "$FIRSTFILE")

  if [[ -z "${FIRSTFILE// }" ]]; then
    echo "no mp3 files in directory"
    exit 1
  fi

  ## save the MP3 tag metadata from the first file
  ffmpeg -y -i "$INFILE/$FIRSTFILE" -f ffmetadata "$METADATA"
  ## save out the cver art image
  ffmpeg -y -i "$INFILE/$FIRSTFILE" -an -vcodec copy "$JPGFILE"
  ## process the tag metadata - put it into the $MP3* strings, set $OUTFILE and $LOGFILE from the title
  getMP3tags "$METADATA"

  ## make a bar seperated list of the MP3 files that are to be combined
  MP3FILES=""
  IFS=$'\n'
  FILES=($(find "$INFILE" -name "*.mp3" | sort -n))
  for files in "${FILES[@]}"; do
    MP3FILES+="$files|"
  done
  MP3FILES=${MP3FILES:0:-1}

  ## combine all the MP3 files into one big file
  ## $OUTFILE is set using either the $FILENAME or $MP3TITLE if that has been used
  ffmpeg -y -i "concat:$MP3FILES" -vn -vsync 2 -acodec copy "$OUTFILE"
else
  ## process a single MP3 file
  ## save the MP3 tag metadata from the first file
  ffmpeg -y -i "$INFILE" -f ffmetadata "$METADATA"
  ffmpeg -y -i "$INFILE" -an -vcodec copy "$JPGFILE"
  ## process the tag metadata - put it into the $MP3* strings, set $OUTFILE and $LOGFILE from the title
  getMP3tags "$METADATA"
  #ffmpeg -y -i "$INFILE" -vn -vsync 2 -acodec copy "$OUTFILE"
  OUTFILE="$INFILE"
fi

## if there is a cover image, these are are the options for adding it back in
WITHJPG=()
if [ -e "$JPGFILE" ]; then
   IFS=$OLDIFS
   WITHJPG=(-i "$JPGFILE" -metadata:s:v comment="Cover (front)")
fi

## create the audiobook M4B file
## and write the metadata into the output file
ffmpeg -y -i "$OUTFILE" -i "$METADATA" "${WITHJPG[@]}" -map_metadata 1 -id3v2_version 3 -acodec aac -b:a "$BITRATE" -strict -2 -f mp4 "$M4BFILE"
## add the chapters every 10 minutes
mp4chaps -e 600 "$M4BFILE"

## tidy up
rm "$MP3METADATA"
if [ -e "$JPGFILE" ]; then
   rm "$JPGFILE"
fi
#rm "$OUTFILE"
echo "finished"

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

#!/bin/bash

INFILE="$1"

BITRATE="96k"

MP3TITLE=""

MP3ARTIST=""

MP3ALBUM=""

MP3COMMENT=""

MP3YEAR=""

MP3PERFORMER=""

####

#### FUNCTIONS

####

function setMP3variables() {

TAG="$1"

VALUE="$2"

case "$TAG" in

title)

MP3TITLE="$VALUE"

;;

artist)

MP3ARTIST="$VALUE"

;;

comment)

MP3COMMENT="$VALUE"

;;

album_artist)

MP3PERFORMER="$VALUE"

;;

album)

MP3ALBUM="$VALUE"

;;

date)

MP3YEAR="$VALUE"

;;

esac

}

## get the tag text from the metadata

## if the line contains an equals (=) then split by the first equals.

function getMP3tags() {

MP3METADATA="$1"

while read -r line

case "$line" in

*=*)

TAG=${line%%"="*}

VALUE=${line#*"="}

setMP3variables "$TAG" "$VALUE"

;;

esac

done < "$MP3METADATA"

if [[ ! -z "${MP3TITLE// }" ]]; then

OUTFILE="$MP3TITLE".mp3

M4BFILE="$MP3TITLE".m4b

}

####

#### BEGINS

####

FILENAME=$(basename "$INFILE" | cut -d. -f1)

if [[ -z "${FILENAME// }" ]]; then

echo "filename missing"

exit 1

OLDIFS=$IFS

OUTFILE="$FILENAME"_out.mp3

M4BFILE="${OUTFILE:0:-4}".m4b

METADATA="${OUTFILE:0:-4}".metadata

JPGFILE="${OUTFILE:0:-4}".jpg

## check if input is a diretory

if [ -d "$INFILE" ]; then

# get the name of the first MP3 file in the directory

FIRSTFILE=$(find "$INFILE" -name "*.mp3" | sort -n | head -1)

FIRSTFILE=$(basename "$FIRSTFILE")

if [[ -z "${FIRSTFILE// }" ]]; then

echo "no mp3 files in directory"

exit 1

## save the MP3 tag metadata from the first file

ffmpeg -y -i "$INFILE/$FIRSTFILE" -f ffmetadata "$METADATA"

## save out the cver art image

ffmpeg -y -i "$INFILE/$FIRSTFILE" -an -vcodec copy "$JPGFILE"

## process the tag metadata - put it into the $MP3* strings, set $OUTFILE and $LOGFILE from the title

getMP3tags "$METADATA"

## make a bar seperated list of the MP3 files that are to be combined

MP3FILES=""

IFS=$'\n'

FILES=($(find "$INFILE" -name "*.mp3" | sort -n))

for files in "${FILES[@]}"; do

MP3FILES+="$files|"

done

MP3FILES=${MP3FILES:0:-1}

## combine all the MP3 files into one big file

## $OUTFILE is set using either the $FILENAME or $MP3TITLE if that has been used

ffmpeg -y -i "concat:$MP3FILES" -vn -vsync 2 -acodec copy "$OUTFILE"

else

## process a single MP3 file

## save the MP3 tag metadata from the first file

ffmpeg -y -i "$INFILE" -f ffmetadata "$METADATA"

ffmpeg -y -i "$INFILE" -an -vcodec copy "$JPGFILE"

## process the tag metadata - put it into the $MP3* strings, set $OUTFILE and $LOGFILE from the title

getMP3tags "$METADATA"

#ffmpeg -y -i "$INFILE" -vn -vsync 2 -acodec copy "$OUTFILE"

OUTFILE="$INFILE"

## if there is a cover image, these are are the options for adding it back in

WITHJPG=()

if [ -e "$JPGFILE" ]; then

IFS=$OLDIFS

WITHJPG=(-i "$JPGFILE" -metadata:s:v comment="Cover (front)")

## create the audiobook M4B file

## and write the metadata into the output file

ffmpeg -y -i "$OUTFILE" -i "$METADATA" "${WITHJPG[@]}" -map_metadata 1 -id3v2_version 3 -acodec aac -b:a "$BITRATE" -strict -2 -f mp4 "$M4BFILE"

## add the chapters every 10 minutes

mp4chaps -e 600 "$M4BFILE"

## tidy up

rm "$MP3METADATA"

if [ -e "$JPGFILE" ]; then

rm "$JPGFILE"

#rm "$OUTFILE"

echo "finished"

When encoding to the M4B using a re-combined file I saw a few of these errors from ffmpeg:

[mp3 @ 0x7fffc3bc9ea0] Header missing
Error while decoding stream #0:0: Invalid data found when processing input

1 2	[mp3 @ 0x7fffc3bc9ea0] Header missing Error while decoding stream #0:0: Invalid data found when processing input

These appear to be caused by the mp3splt program from when the original MP3 file was being split into 50 minute chunks, but I can’t hear any effect on the output.

Lots of information about the file can be gotten using mediainfo, to install in Debian/Ubuntu use: sudo apt-get install mediainfo, example use:

mediainfo --fullscan "big fat file.m4b"

1	mediainfo --fullscan "big fat file.m4b"

Links and References

ffmpeg, encode AAC: https://trac.ffmpeg.org/wiki/Encode/AAC
ArchLinux audiobook howto

g7smy.co.uk

Category Archives: Programming

Extracting GPS data from the GoPro 7

Installing the gopro-utils

Extracting the Data

Merging GPX files

Links and Sources

The Fridge Door is Open

Circuitry

Software

Links and Sources

Extracting MP3 audio from video files

Alternative Method

Creating an Audiobook

Links and References

COMPUTERS, ROBOTS AND PHOTOGRAPHY