Extracting GPS data from the GoPro 7

28th July 2019 Karl Leave a comment

While extracting the telemetry data from the GoPro is reasonably well documented I have found some gaps for getting the extracting utilities installed and when extracting and combining data from multiple files. These notes are for a Debian/Ubuntu installation in a BASH Shell.

Installing the gopro-utils

As I couldn’t find any straightforward instructions for installation, I’ll be going through everything I needed to do to get it working, you may have some of these packages installed already.

sudo apt update
sudo apt upgrade
sudo apt install ffmpeg golang gpsbabel git

sudo apt update

sudo apt upgrade

sudo apt install ffmpeg golang gpsbabel git

Now to get the gopro-utils and install them, I’m placing the source files into my Downloads directory, as well as the GPS data extractor we’ll be adding the other telemetry tools too, this is all a bit long winded.

cd ~/Downloads
git clone https://github.com/JuanIrache/gopro-utils
cd gopro-utils
go get github.com/JuanIrache/gopro-utils/telemetry
go get github.com/mlouielu/gpxgo/gpx
cd bin/gopro2gpx
go build gopro2gpx.go
sudo cp gopro2gpx /usr/local/bin/
cd ../bin/gopro2geojson/
go build gopro2geojson.go
sudo cp gopro2geojson /usr/local/bin/
cd ../gopro2json/
go build gopro2json.go
sudo cp gopro2json /usr/local/bin/
cd ../gpmd2csv/
go build gpmd2csv.go
sudo cp gpmd2csv /usr/local/bin/
cd ../gpmd2info/
go build gpmd2info.go
sudo cp gpmd2info /usr/local/bin/
cd ../gps2kml/
go build gps2kml.go
sudo cp gps2kml /usr/local/bin/

cd ~/Downloads

git clone https://github.com/JuanIrache/gopro-utils

cd gopro-utils

go get github.com/JuanIrache/gopro-utils/telemetry

go get github.com/mlouielu/gpxgo/gpx

cd bin/gopro2gpx

go build gopro2gpx.go

sudo cp gopro2gpx /usr/local/bin/

cd ../bin/gopro2geojson/

go build gopro2geojson.go

sudo cp gopro2geojson /usr/local/bin/

cd ../gopro2json/

go build gopro2json.go

sudo cp gopro2json /usr/local/bin/

cd ../gpmd2csv/

go build gpmd2csv.go

sudo cp gpmd2csv /usr/local/bin/

cd ../gpmd2info/

go build gpmd2info.go

sudo cp gpmd2info /usr/local/bin/

cd ../gps2kml/

go build gps2kml.go

sudo cp gps2kml /usr/local/bin/

Extracting the Data

You will need to find which stream in the video recording the data has been saved to, to find this use ffprobe to examine the recording and look for the stream that contains GoPro MET, for example:

$ ffprobe GX090155.MP4
[snip]
    Stream #0:3(eng): Data: none (gpmd / 0x646D7067), 40 kb/s (default)
    Metadata:
      creation_time   : 2019-07-28T11:09:25.000000Z
      handler_name    : GoPro MET
[snip]

$ ffprobe GX090155.MP4

[snip]

Stream #0:3(eng): Data: none (gpmd / 0x646D7067), 40 kb/s (default)

Metadata:

creation_time : 2019-07-28T11:09:25.000000Z

handler_name : GoPro MET

[snip]

You can see that what we are wanting is on stream 3, as far as I can tell this stays the same every time, I don’t know if it is different for other GoPro models.

This bash script extracts the GPS data in GPX format from all the GoPro GX recordings in the directory, other options have been commented out, if you are using Garmin VIRB edit there is also an option for use with that. The script creates two files, one that contains the raw data and another with the desired GPS data, the GPS output file has the same name as the recording, but in lowercase with a .gpx extension.

#!/bin/bash
FILES="GX*.MP4"
STREAM="0:3"
for MP4FILE in $FILES; do
   OUTFILE=$(basename "$MP4FILE" | cut -d. -f1)
   
   OUTFILE=$(echo "$OUTFILE" | tr '[:upper:]' '[:lower:]')
   BINFILE="$OUTFILE.bin"
   ffmpeg -y -i "$MP4FILE" -codec copy -map $STREAM -f rawvideo "$BINFILE"

   #gpmd2csv -i "$BINFILE" -o "$OUTFILE.csv"
   gopro2gpx -i "$BINFILE" -a 500 -f 2 -o "$OUTFILE.gpx"
   #gopro2gpx -i "$BINFILE" -a 200 -f 3 -o "$OUTFILE-virb.gpx"
   #gopro2json -i "$BINFILE" -o "$OUTFILE.json"
done

#!/bin/bash

FILES="GX*.MP4"

STREAM="0:3"

for MP4FILE in $FILES; do

OUTFILE=$(basename "$MP4FILE" | cut -d. -f1)

OUTFILE=$(echo "$OUTFILE" | tr '[:upper:]' '[:lower:]')

BINFILE="$OUTFILE.bin"

ffmpeg -y -i "$MP4FILE" -codec copy -map $STREAM -f rawvideo "$BINFILE"

#gpmd2csv -i "$BINFILE" -o "$OUTFILE.csv"

gopro2gpx -i "$BINFILE" -a 500 -f 2 -o "$OUTFILE.gpx"

#gopro2gpx -i "$BINFILE" -a 200 -f 3 -o "$OUTFILE-virb.gpx"

#gopro2json -i "$BINFILE" -o "$OUTFILE.json"

done

Merging GPX files

As the GoPro splits recordings into 4GB blocks, when extracting you will get a single GPX file for each recording. Many pages found by Google say that to create a single track from these all you need to do is append the files into one big file. This is wrong, what you end up with is a single file with many short tracks, when what you are after is one long track covering the entire journey. This bash script uses gpsbabel to create single merged file from the extracted GPX data, it creates a file called “gpsoutput.gpx”.

#!/bin/bash
FILES="gx*.gpx"
ff=""
for f in $FILES; do
    ff="$ff -f $f"
done
gpsbabel -t -i gpx $ff -o gpx -x track,merge,title="GOPRO COMBINED LOG"  -F gpsoutpt.gpx

#!/bin/bash

FILES="gx*.gpx"

ff=""

for f in $FILES; do

ff="$ff -f $f"

done

gpsbabel -t -i gpx $ff -o gpx -x track,merge,title="GOPRO COMBINED LOG" -F gpsoutpt.gpx

The next stage will be to write a script that combines all these and completes the job in one easy process.

Links and Sources

Web Page to extract GoPro data: https://tailorandwayne.com/gopro-telemetry-extractor/ Easy to use for individual files
https://github.com/JuanIrache/gopro-utils
GoPro Forum Extracting the metadata in a useful format
GPSBabel, Free software for GPS data conversion and transfer: https://www.gpsbabel.org/

CNC, Computer, Electronics, Hardware

Using Inkscape for CNC designs

18th May 2019 Karl Leave a comment

Inkscape is a free vector graphics editor for all major platforms, generally it is aimed at art and design users but it does have an option for generating G-Code for use in your favourite CNC software. While Inkscape doesn’t have many of the functions of proper CAD/CAM software it is an relatively easy place to start for creating basic designs, I have been using it to make boxes out of 3.5mm plywood.

These notes are based around my cheap CNC machine sold as an CNC3018 by a variety of Chinese manufacturers on Amazon and eBay, the included controller is a Woodpecker CNC board (Ardunio clone) I have upgraded to GRBL v1.1 and I am using version 0.92.4 (April 2019) of Inkscape with the included Gcodetools.

This post focuses on setting up Inkscape for the CNC machine and producing the g-code from your drawing, it is not intended to be an Inkscape tutorial.

Document Setup

With a new drawing set your Document Size, this should be the same as your CNC bed, in my case this is 300 x 180mm. From the Inkscape menu go to File > Properties and in the Page Tab set the Display Units (millimeters in my case), the Orientation to Landscape and Page Size width: 300 and height: 180. In the Grids Tab set the Grid Units to mm and the Spacing X and Spacing Y to 1.0. Back on your main page, turn the page grid on with: View > Page Grid.

By default Inkscape scales the stroke/line width when you resize a shape, to prevent this click the the fourth box from the right in the top icon bar “when scaling objects, scale the stroke width by the same proportion”

Turn Line Scaling off to prevent the line width changing when resizing

You can save this as a template, such as: CNC3018.svg or as the document default with: default.svg by saving the file to your templates directory:
On Linux and OS X: ~/.config/inkscape/templates/
On Windows: C:\Users\<username>\AppData\Roaming\inkscape\templates

The lines you draw will need to be the same width as the bit you are using in the CNC machine. Draw a rectangle, Right mouse click on the rectangle and select Fill and Stroke…. In the Fill Tab click the X – no paint box and on the Stroke Style tab set the width to that of the bit you are using – 1.5mm, subsequent rectangles will be in the same style, other shapes will need to be setup this way too. The colour of your lines should be black, there is some functionality for different colours to represent different depths but I have not yet worked out how to do this.

Layout Tips for G-Code Routing

Remember to check the dimensions of the cuts, with an outside cut such as the width and height of a box side you need to measure for the inside of your rectangle, for holes in your box measure to the outside edge, Inkscape sets distances to the outside edge.

For positioning holes for switches and the like, I add thin lines 0.1mm thick as guides and make use of the width/height settings as well as the Object > Align and Distribute options. A pair of digital vernier calipers are a great aid to discovering the required sizes. Remember to delete these before G-Code encoding.

When generating the G-Code each shape will be seen as an individual object, so lets say you want to have two sides of your box cut from a single sheet of plywood, this would be two rectangles abutting each other with a side to be cut overlapping. As it takes four passes to cut each shape 1mm at a time, this means it’ll take six passes down the centre. To fix this select both rectangles and then Path > Combine followed by Path > Difference to make a single object.

Outputting to G-Code

Now that you have completed your drawing, save your work then convert your objects to paths by selecting all objects then Path > Object to Path. You may also want to place your drawing near the bottom left of the document, as this is where the CNC router starts. Now using Gcodetools there are three things you need to do to produce the G-Code file. None of the Gcodetools windows close automatically when apply is clicked, you will need to do that yourself. From the Inkscape menu:

1. Extensions > Gcodetools > Tools Libary…
Select Tools Type: cylinder and click apply In the overlarge green box that appears you will need to set the tool diameter and feed speed.

This can be a bit fiddly as the text can become detached from the box and the settings lost, what seems to work most reliably for me is to change to Text Objects (F8) click on the numbers you want to change and once done go back to Select and Transform (F1). Resize the box afterwards to check that it is still working – if the green box moves but the text does not then Ctrl-Z a few times and try again.

Gcodetools Tools Library – Green background is detached from the text – this won’t work.

setting	default	used	description
Diameter	10	1.5	tool bit diameter in mm
Feed	400	300	speed while cutting through the material in mm/second
Penetration angle	90	90
Penetration feed	100	100	Plunge speed in the material in mm/second
Depth Step	1	1	Depth of cut on each pass in mm

2. Extensions > Gcodetools > Orientation Points
This tells the g-code where to start, normally bottom-left on the CNC Set the following:
– Orientation type: 2-points mode
– Z Surface: 0mm – this is the top of your surface
– Z Depth: -3.4mm – this is the thickness of material to cut, a negative number

3. Extensions > Gcodetools > Path to Gcode
This creates the G-code file, in the Preferences Tab set the following:
– File: output filename
– Directory: output directory
– Z safe height: 5mm – height above the work surface when moving between cuts
The filename once set doesn’t change, an incremental number is appended to the output filename. Click the Path to Gcode Tab before clicking apply (this appears to be a bug).

Your image will be updated to show the g-code routing, give this a visual check to ensure that all objects have been coded and that it looks right, the path to be taken should be in colour and contain arrows showing the direction of the router.

If there are too many arrows or if a line has arrows pointing in different directions then there may be an object underneath, check on your original artwork, in the image with the three circles below you see that A has not been converted to a path with Path > Object to Path, B has a duplicate object underneath and C is correct.

Gcode check; A and B have problems, C is correct.

The generated G-Code does not appear to include the Spindle Motor Start command – So remember to start the spindle manually in your CNC software before running the G-Code – its interesting how easily these bits break with a sideways load. Remember if you are cutting trough rather than engraving, don’t forget to put a layer to sacrifice between whatever you are making and the CNC’s bed, I use 5mm MDF/Fibreboard.

Links and Sources

Inkscape and Inkscape Tutorials
Gcodetools plug-in English support forum and on github
CNCjs
grbl
shapeoko BHSPitMonkey/Inkscape and Gcodetools Tutorial
Intro to G-Code in Inkscape for Laser Etching
raspberry-gpio-python

history

History: 1795 Map of Sheffield

25th April 2019 Karl Leave a comment

A map of Sheffield from 1795, originally published by William Fairbank and Son in September 1796. You can see all the familiar city place names as farms or villages.

The Parish of Sheffield in the County of York, by Wm. Fairbank and Son 1795

See more detail: https://www.g7smy.co.uk/history/sheffield1795/

history, Sheffield

History: Colliery Road is Weird

6th April 2019 Karl 2 Comments

Colliery Road in Sheffield Brightside is an odd throughway that passes under the Midland Mainline Railway providing a 290m (317 yards) route between the junction of Brightside Lane and Weedon Street to Holywell Road. It is a narrow single track road and the the Brightside Lane end has been cut into the hillside to accommodate the railway junction and sidings above. Nowadays access is limited to pedestrians and cyclists as at 2.32m (7′ 6″) high the bridges are far too low to be of any use for general traffic, the road was closed to motor vehicles in around 2016 and is now part of the Lower Don Valley Cycle Route¹. In this short essay I am looking a little into its history, the modifications that have changed the road to accommodate the railway and to speculate why it exists at all.

Two rail bridges run over the road, one for the Midland Mainline, originally the Sheffield to Rotherham Railway, constructed in 1838/39, this bridge is referred to in Drake’s Road Book published in 1840 where he says “We now cross a private road by a beautiful stone bridge – one which is infact the only ornamented bridge upon the line, and also the only one that is graced with parapets”². The other is now disused, it was part of Brightside Junction giving access to the various steel works as well as a line towards Catcliffe and the south as part of the Sheffield District Railway. As well as the two we now see, sometime between 1880 and 1955 there was also a third single track bridge on the north western side used for the gas and chemical works.

Colliery Road, Brightside [Ordnance Survey]

The 1854 map (Yorkshire 289) shows a footpath or track approximately following the route of the road today, leading from Brightside Lane under the railway and up to The Grange farm at the end of Wincobank Lane, on the 1892 map this is shown as a footpath, it looks to have been around a 620m (678 yards) walk from the farm to Brightside village.

By 1892 Brightside Junction had been added to the railway providing access to the steel works in that part of the Don Valley, at that time Colliery Road was moved to start from near near Naseby Street and run alongside the south-eastern side of the tracks for a short while before ducking under the line, this was to accommodate the embankment built for the junction and the goods line into Brightside Steel works. It wasn’t until 1896 when construction started on the Sheffield District Railway that the road was moved to go between the new railway and the existing goods line, this is when the roads route was fixed to that we see today.

Brightside 1854
Brightside 1892
Brightside 1905
Brightside 1906
Brightside 1924
Brightside 1934
Brightside 1947
Brightside 1949

Ordnance Survey Maps – 1854 to 1949 [National Library of Scotland]

In the 1930’s on the north-eastern side of the tracks Holywell Road was extended from Upwell Street alongside the Gas Works to connect to Limpsfield Road and replacing Burslem Street as part of an expansion of housing into the area. Before then this part of Wincobank was mostly farmland with some industry alongside the railway, most notably the 1906 map (CCLXXXIX.SW) shows the Grimesthorpe Gas Works taking up a large amount of land between Upwell Street and Colliery Road. Looking at the 1926 aerial photo below, the exit of Colliery Road from under the railway can be seen on the far left of the image. Today this land is used by a mail order distribution centre as well as some smaller wholesalers and trade suppliers.

The Grimesthorpe Gas Works, Sheffield, 1926 [Britain From Above]

Brightside Colliery was also situated on the north-eastern side, the mine was worked from 1855 to 1886 and the shafts were located at the end of Colliery Road, the colliery was owned by Unwin & Shaw Co. and the pit had two shafts, No. 1 from 1855 to 1871 and No.2 from 1868 to 1886, they worked the Parkgate seam³. From 1883 until around 1955 the site was occupied by the Brightside Chemical Works which was a part of the gas works dealing with sulphur produced from gas manufacture ^{[Sheffield History]}. Nowadays the land is used by a self storage facility and a motor vehicle repair company.

Extra width has been added to the mainline bridge over time

The purpose of Colliery Road?

The road must of had a high cost of construction, the changes that needed to be made to have it run under the railway including cutting it into the hillside at the Brightside Lane seems a lot of effort has gone into maintaining a private a road that gives such poor access.

A reason for this could be that the colliery supplied coal to the works and foundries on the south-eastern side of the railway, this would justify a bridge for the Sheffield to Rotherham Railway but by the time Brightside Junction was built the mine had been closed for around five years so why go to all the trouble to modify the road? Or is Colliery Road a legacy of the landowner insisting they have access from their farm at The Grange down to Brightside Lane? I can imagine a track existing before the railway and it being kept as a right of way as a route to Brightside village and the bridge across the Don which had been there in one form or another since before 1650⁴.

Comparing access for the colliery 1854 and 1892

By comparing the 1854 and 1892 maps above we can see similarities between the buildings being shown, the map also shows there is no roadway from the colliery site into Brightside as a field is in the way. The coming of the railway would have cut access off from Naseby Street and maybe the landowner did not want to sell part of his field, or set the price too high, to permit access to the nearest bridge over the line on Jenkin Road and forcing further development of Colliery Road, the bridge and its parapets.

Colliery Road
Graffiti on Colliery Road
Disused Rail Bridge - Sheffield District Railway
Station Road
At Brightside Junction
Colliery Road Rail Bridge
Colliery Road
Colliery Road
Colliery Road - Looking towards Brightside Lane
Colliery Road - Holywell Road end
Colliery Road Rail Bridge
Brightside Colliery Memorial

More of my Colliery Road photos can be found on flickr

Dates of Interest

1838/39	Sheffield to Rotherham Railway constructed (Midland Mainline)
1838	Brightside Station opens
1855-1871	Brightside Colliery – Shaft No.1 in operation
1868-1886	Brightside Colliery – Shaft No.2 in operation
c.1855-c.1955	Grimesthorpe Gas Works in operation
1883-c.1955	Brightside Chemical works – part of Grimesthorpe Gas Works
c.1890	Brightside Junction added – for the steel works
1896	Sheffield District Railway construction starts

Links and Sources

My photos of Colliery Road on flickr

Sheffield City Region Combined Authority, Transport Committee, 26 Sept 2016. p. 17
Drake’s Road Book of the Sheffield and Rotherham Railway, London 1840, p.33
The Coal Mining Industry of Sheffield and North Derbyshire, Ken Wain, 2014: ISBN-13: 978-1445639635
Hallamshire. The History and Topography of the Parish of Sheffield. Joseph Hunter, London 1819, p.227

About the Brightside Colliery memorial, regarding the deaths that occurred at the mine during its operations.
National Gas Archive – Sheffield and District Gas Company – Grimesthorpe Gas Works

Ordnance Survey maps at the National Library of Scotland:

Yorkshire 289 (includes: Rawmarsh; Rotherham.):
- Surveyed: 1850 to 1851 Published: 1854: https://maps.nls.uk/view/102345199
Yorkshire CCLXXXIX.13 (Rotherham; Sheffield):
- Surveyed: 1890 to 1891 Published: 1892: https://maps.nls.uk/view/125650762
- Revised: 1901 to 1902 Published: 1905: https://maps.nls.uk/view/125650765
Yorkshire CCLXXXIX.SW (includes: Rotherham; Sheffield.)
- Revised: 1901 to 1902 Published: 1906: https://maps.nls.uk/view/100949906
- Revised: 1921 Published: 1924: https://maps.nls.uk/view/100949903
- Revised: 1934 to 1935 Published: 1934: https://maps.nls.uk/view/100949900
- Revised: 1938 Published: 1947: https://maps.nls.uk/view/100949897
- Revised: 1948 Published: 1949: https://maps.nls.uk/view/100949894

Uncategorized

Epoxy Weight Ratio Calculator

7th February 2019 Karl Leave a comment

I have written this Weight Ratio Calculator, it splits a total weight by a ratio to give the weights needed to make up a mixture it can be used for any two-part solution such as epoxy resin or silicone rubber used for moulds.

Link: https://www.g7smy.co.uk/ratio

Electronics, Hardware

A footpedal control for my Dremel

26th January 2019 Karl Leave a comment

In this video I am adapting a sewing machine pedal to provide variable speed to my small Dremel drill. It’s not really suitable as the pedal is only rated to handle 80 Watts (0.42 Amps at 240 Volts) while the Dremel can take up to 125 Watts (0.52 Amps) of power, it should be alright for light work.

To control the power the footpedal has two stacks made of loose carbon disks, initially when the pedal is pressed power is passed through these and as more pressure is applied these compress to lower the resistance and allow more power through. When fully on, a metal disc engages across two contacts to allow all the current to pass. I think I would like to have a pedal that could handle more current, possibly something based around how a modern dimmer switch works.

Arduino, Electronics, Hardware

Rebuilding the Vacuum Fluorescent Display

23rd December 2018 Karl Leave a comment

Back in April 2016 I posted details on how a Vacuum Fluorescent Display (VFD) works and a method of driving it from an Arduino type microcontroller using transistors, while this worked well it made for a rather bulky clock. I have now rebuilt the display using proper VFD driver chips, added a rotary switch with LED’s, NeoPixel capability and used a WeMos D1 Mini microcontroller to give me internet access over WiFi so the time and date can be set using the Network Time Protocol (NTP).

This rebuild is much more practical than previously, it is now about 3cm deep and comprises of two boards back to back with pin header plugs/sockets as the interconnects. I have been running the clock for over a year without any problems, apart from having to switch the thing off and back on when Daylight Savings Time changes in the spring and autumn, but that’s a software (lazy programmer) problem.

Power Supply

Three different voltages are needed to operate this clock, 16 Volts for the VFD display anodes, 5 Volts for the microcontroller and NeoPixels and 3.3 Volts for the VFD cathode and other hardware.

From the 12 volt source, I split the power to a XL6009 boost module to supply 16V to the driver IC’s, the 5V power module can supply up to 5A which is enough for around 2 meters of NeoPixels. The 3.3V supply is created using a BA033T fixed voltage regulator running from the 5V module, I have used this to power the VFD cathode, the rotary encoder and VFD driver IC’s as this is the microcontrollers operating voltage.

Driving the Display

To drive the anodes on the VFD display I used three MAX6920 Tube Drivers, these have a four wire serial interface and can switch up to 76V on the display.

The physical connections from the driver IC’s to the VFD anodes (grids and elements, see the previous article for definition) are those that are most convenient for the PCB layout without having to use links or through-hole connections as the order of the pins for outputting the display can be mapped later in software.

In this case, the first anode pin on the display is on the right, but the serial data starts with the first MAX6920 chip on the left. I will explain in more detail in the software section below.

The Cathode is run from the 3.3 Volt supply, it has a 3.3V Zenner diode across it to protect against meltdown in case of over-voltage, the cathode is made from of strands of thin tungsten wires and shows as a short when testing in beep mode on the multimeter.

Rotary Switch

I have used the SparkFun RGB Illuminated Rotary Encoder, as well as the encoding it also includes three LED’s and a push button switch. As the WeMos only has limited number of pins available, I used an MCP23008 I²C I/O Expander. I have covered these encoders in a previous post and have used a similar method here.

In this project it turned out I don’t have much use for the rotary switch, the same result can be achieved with a push button and two LED’s, it goes red when setting the time over WiFi, goes green when the time has finished setting but is off when running normally, the push button displays the date when pushed. I expect the encoder could be used for setting an alarm should I ever have need to add one.

Software

The complete source code I am currently running in the clock can be found on my github repository, it has been written for the Arduino platform and can be considered unfinished and a bit of a mess. In almost all cases this clock will be a custom build with all the scavenged VFD displays being different I’ll be concentrating on how the display is being driven. Note that the software uses Latch and the MAX6920 driver chip uses Load.

To start, here are the physical connections for my VFD display, the display connections are shown looking from the back with pin one on the right, on the anode there are eleven grids and twenty-one elements, each grid illuminates a block of chosen elements. There are three MAX6920 driver chips, the first is on the left data is passed along the serial data line DIN/DOUT to the right:

Connecting the MAX6920 drivers to the display

While I’m not going to worry too much about how the data is sent to the display as this is being handled by the ShiftOutX library it may be useful to have an understanding of the byte order sent to the driver chips.

Breaking out the logic analyser we can see what is going on, in this first image below we can see that between the two markers one and two there are eleven data peaks to match the number of grids on the display the data is processed as a padded eleven character string: ‘ 212848 ‘ the first four contain nothing but the bit selecting grid to be used the next six are busy as they contain the time, and the last is blank. There also appear to be two blank clock cycles after these, this may be a programming error on my part.

the string is iterated so that each grid gets its own character with a very short delay between each this causes the persistence of vision effect and the time appears on the display.

In the next screen capture I have zoomed into more detail, it is showing the fifth grid and is displaying the number two. You can see that there are thirty-two clock cycles sent to the drivers while LOAD is high, this causes the driver chips to direct the data to the outputs. I am not entirely sure why the clock is being run while LOAD is low, I think it may be clearing the buffer before sending the required information.

You can see that 32 bits are sent to the display, but there are 36 pins available on the three MAX6920 drivers, you can also see that the outputs are arranged out of order. How do the drivers know which lines to switch for the display?

First off, an explanation of how the data is constructed then sent to the display. For this example I am sending the number five to the third grid, the data is constructed through the manipulation of a character array in three stages:

1.	blank char array:	00000000000000000000000000000000
2.	set the third grid:	00000000000000000000000000000100
3.	set the number:	00111000011000011100000000000100

Because of the way the bits are ordered when sent to the drivers I have a lookup array VFDlookup to make life easier, in this case grid 3 can be found at position 29 in the VFDdata character array. I start with a blank character array of 32 bits VFDblank and copy this to VFDdata, I then use VFDlookup to find the position of the bit to set: VFDlookup[3] = 29. For the elements the encoding for each character is set in VFDdigits the code for number 5 is 1110000110000111 the code was worked out by experimentation, seeing which element lit up after I had completed the hardware, I have ignored the extra symbols used for the DVD player so I only need to set pins 15-30. Once complete the character array is converted to a unsigned long integer and sent LSBFIRST – Least Significant Bit First via ShiftOut_32.

While that explains the elements encoding to display the correct digit, the grid selection however is a different kettle of fish. I built this clock over a year ago and have forgotten how it works. Grids 1 and 2 are connected to the output pins 1 and 0 on the third driver, pins 2-5 are then skipped, and the rest are connected to the grids, with three more being taken from chip 2. I can’t work out how it knows to skip those pins, or that grid 2 is connected to pin 1. But it is an awesome clock though.

Links and Sources

Github Repository: https://github.com/kazzle101/vfdclock6
WeMos D1 Mini
SparkFun RGB Illuminated Rotary Encoder – UK Supplier
Datasheet: MCP23008 – I/O Expander
Datasheet: MAX6920 – Display VFD Tube Driver
Datasheet: BA033T Fixed LDO Voltage Regulator – 3.3Volt output
Serial to Parallel Shifting-Out with a 74HC595

Computer, linux

FFMPEG for Video Conversion

13th November 2018 Karl Leave a comment

FFmpeg is a command line program to manipulate, convert, record and stream video and audio, it is available for Mac, Linux and Windows. Here is a handy list of commands for reference, these have been tested with version 3.1.12 in a Debian Linux environment. I expect this list to grow over time as needs arise.

• Rescale a 4K video to 1080p


ffmpeg -i inputFile.mp4 -c:v libx264 -crf 20 -preset slow -vf scale=1920:1080 outFile.mp4

ffmpeg -i inputFile.mp4 -c:v libx264 -crf 20 -preset slow -vf scale=1920:1080 outFile.mp4

• Convert to H.264 (AVC) codec for use on uploading to YouTube, Vimeo, etc:


ffmpeg -i inputFile.mov -f mp4 -vcodec libx264 outputFile.mp4

ffmpeg -i inputFile.mov -f mp4 -vcodec libx264 outputFile.mp4

Using this codec reduces the time it takes for the video to be available after upload, however YouTube converts the file again to the VP9 codec and unless you have a popular channel, 100 subscribers or more, then this can take a few days or weeks and in the meantime your video can appear quite poor and blocky even when watching at 1080p, especially when there is a lot of movement like in a car dash-cam video. You can use FFmpeg to encode to VP9 webm format with this bash script:

#!/bin/bash
INFILE="$1"

PASS1="-pass 1 -b:v 1000K -threads 8 -speed 4 -tile-columns 6 -frame-parallel 1"
PASS2="-pass 2 -b:v 1000K -threads 8 -speed 1 -tile-columns 6 -frame-parallel 1 -auto-alt-ref 1"

FILENAME=$(basename "$INFILE" | cut -d. -f1)
if [[ -z "${FILENAME// }" ]]; then
   echo "filename missing"
   exit 1
fi

OUTFILE="$FILENAME".webm

ffmpeg -y -i "$INFILE" -c:v libvpx-vp9 $PASS1 -g 9999 -aq-mode 0 -an -f webm /dev/null
ffmpeg -y -i "$INFILE" -c:v libvpx-vp9 $PASS2 -lag-in-frames 25 -g 9999 -aq-mode 0 -c:a libopus -b:a 64k -f webm "$OUTFILE"

#!/bin/bash

INFILE="$1"

PASS1="-pass 1 -b:v 1000K -threads 8 -speed 4 -tile-columns 6 -frame-parallel 1"

PASS2="-pass 2 -b:v 1000K -threads 8 -speed 1 -tile-columns 6 -frame-parallel 1 -auto-alt-ref 1"

FILENAME=$(basename "$INFILE" | cut -d. -f1)

if [[ -z "${FILENAME// }" ]]; then

echo "filename missing"

exit 1

OUTFILE="$FILENAME".webm

ffmpeg -y -i "$INFILE" -c:v libvpx-vp9 $PASS1 -g 9999 -aq-mode 0 -an -f webm /dev/null

ffmpeg -y -i "$INFILE" -c:v libvpx-vp9 $PASS2 -lag-in-frames 25 -g 9999 -aq-mode 0 -c:a libopus -b:a 64k -f webm "$OUTFILE"

This script is based on the encoding method shown in the WebM Wiki on my computer it is very slow and takes a quite a few hours to encode just nine minutes of video and the eventual results are so poor you’ll be wondering why you bothered.

• Convert to MP4 for use in Vegas Studio:


ffmpeg -i inputFile.mkv -codec copy outputFile.mp4

ffmpeg -i inputFile.mkv -codec copy outputFile.mp4

If you have a particularly old/odd video and get lots of pts has no value errors, then try this:


ffmpeg -fflags +genpts -i inputFile.mpg -codec copy outputFile.mp4

ffmpeg -fflags +genpts -i inputFile.mpg -codec copy outputFile.mp4

The -fflags +genpts option adds a Presentation Timestamp (PTS) to the frames, this must be before the -i as shown to work. Source.

• Set the video playback speed, this method adjusts the Presentation Timestamp (PTS) on each frame which may not work with older software. To slow down video divide the PTS by your required speed, this example slows the action by two times setpts=PTS/2.0. You can also reduce the number of dropped frames by increasing the frame-rate -r 50, in this case I went from 25fps to 50fps, but depending in the chosen speed frames may still be dropped.


ffmpeg -i inputFile.mp4 -r 50 -filter:v "setpts=PTS/2.0" outputfile.mp4

ffmpeg -i inputFile.mp4 -r 50 -filter:v "setpts=PTS/2.0" outputfile.mp4

Speed up your video by multiplying the PTS, in this case two times faster: setpts=PTS*2.0


ffmpeg -i inputFile.mp4 -filter:v "setpts=PTS*2.0" outputfile.mp4

ffmpeg -i inputFile.mp4 -filter:v "setpts=PTS*2.0" outputfile.mp4

• Convert file or extract audio from file into an MP3, the output is set to 128K constant bitrate


ffmpeg -i inputFile.mp4 -vn -acodec libmp3lame -b:a 128k outputFile.mp3

ffmpeg -i inputFile.mp4 -vn -acodec libmp3lame -b:a 128k outputFile.mp3

• Concatenate Video Files
This combines two video files, when using formats such as MP4 or MKV you will need to create intermediate files, otherwise only the first file will be included in the output:


ffmpeg -i file1.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts file1_inter.ts
ffmpeg -i file2.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts file2_inter.ts
ffmpeg -i "concat:file1_inter.ts|file2_inter.ts" -f mp4 -vcodec libx265 outputFile.mp4

ffmpeg -i file1.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts file1_inter.ts

ffmpeg -i file2.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts file2_inter.ts

ffmpeg -i "concat:file1_inter.ts|file2_inter.ts" -f mp4 -vcodec libx265 outputFile.mp4

• The opus not found error
When converting a file and you see an error like Could not find tag for codec opus in stream #1… you will need to state the output format


ffmpeg -i inputFile.mkv -f mp4 -vcodec libx264 outputFile.mp4

ffmpeg -i inputFile.mkv -f mp4 -vcodec libx264 outputFile.mp4

Links and Sources

https://www.ffmpeg.org/
FFmpeg Wiki: https://trac.ffmpeg.org/wiki

Arduino, Electronics, Hardware

Making a Printed Circuit Board

7th October 2018 Karl Leave a comment

Finally, I got around to videoing how I make a PCB. This one is for my Fridge Door Open alarm.

Arduino, Electronics, Hardware, Programming

The Fridge Door is Open

29th August 2018 Karl Leave a comment

My fridge door tends to rebound when closed staying open a smidgen and letting all the cold out. Rather than just checking that the door is properly shut, I thought it about time to have a microcontroller make a noise when the door has been left open too long.

This circuit uses an Arduino compatible Teensy LC for all the work, it has a phototransistor to sense the state of the fridge door light, a couple of LED’s one to indicate the power and another that comes on when the door is open. There is also Piezo buzzer to make an annoying noise after forty five seconds of door open time. The unit runs of a 3.7v rechargeable Lithium-ion battery and I have added a recharging circuit that takes power via the Teensy’s 5v USB port.

Circuitry

Note: These diagrams show 3.7v as the supply voltage. The Teensy LC can only tolerate a maximum 3.3V on the data pins, so these circuits are driven from the 3V output on the Teensy. They will all work without modification on the 5v Arduino Uno.

I have used a phototransistor to detect the fridge door light, there are two variants of this circuit light activated or dark activated, the 100k resistor can be replaced with a 100k variable if you need to adjust the sensitivity, the 330k resistor provides a weak pull-down on the output. The phototransistor is being used in switch mode to provide a logical output (rather than active mode which provides an output relative to the amount of light), so the output is connected to a digital input on the Arduino. The BC547 transistor is half of the darlington pair to provide extra gain on the output.

I chose the light activated switch, either will do but will provide different logical outputs to your controller. The circuit is enclosed in a small box inside the fridge and connected by ribbon cable to the controller, the ribbon cable is flat and does not upset the magnetic ‘seal’ on the fridge door.

To make some noise I used a piezo buzzer from an old computer, this is driven through a transistor as the Teensy does not provide enough current to drive it directly.

There is also a push button to provide a reset function if the buzzer is sounding while the door is open.

I have also added two LED’s, one to show power and anther that illuminates when the door is opened.

The final circuit if for recharging the battery, it connects to the 5V connection on the Teensy LC so charges the battery when the USB connection is in use. This has been copied from the MCP3831T datasheet.

Software

This uses an interrupt to listen for the light sensor, when the state changes, the door open pin is read to determine if the door is open or not. If it is then a timer is started, this gives you forty five seconds to complete your task before the alarm sounds. With the door closed the timer is stopped and set back to zero. If the sounder goes off while you are rummaging in the fridge the reset button can be pressed, this restarts the timer from zero again.

#define TEST_LED 13
#define FRIDGE_DOOR 6
#define DOOR_LED 8
#define POWER_LED 7
#define RESET_BTN 9
#define BUZZER 10

volatile boolean doorOpen = true;   // when the light is on
volatile boolean doorTrigger = false;
volatile unsigned long lastFridgeMillis = 0;
volatile unsigned long fridgeTimeoutMillis = 10;

boolean SerialDebug = false;
unsigned long doorOpenDelay = 45000; // 45 second timeout before buzzer sounds
unsigned long doorOpenTime = 0;
unsigned long startTime;
unsigned long previousToneMillis = 0;
unsigned long toneInterval = 20;
boolean soundTheAlarm = false;
boolean outputTone = false;
int toneMelody[5] = { 1600, 1900, 2000, 2100, 1800 };
const int toneMax = 5; 

unsigned long lastDoorOpenTime = 1;
char pout[80]; // for testing

// interrupt function
void fridgeDoor() {
  if ((millis() - lastFridgeMillis) > fridgeTimeoutMillis) {
    doorOpen = digitalRead(FRIDGE_DOOR); // goes high when dark
    lastFridgeMillis = millis();
  }
}

// see if the door is open or closed
void fridgeDoorOpen() {
  if (doorOpen == true && doorTrigger == false) {
    doorTrigger = true;
    startTime = millis();

    if (SerialDebug) {
      sprintf(pout, "door open: %d", doorOpen);
      Serial.println(pout);
    }
    digitalWrite(DOOR_LED, HIGH);
    return;
  }

  if (doorOpen == false && doorTrigger == true) {
    doorTrigger = false;

    if (SerialDebug) {
      sprintf(pout, "door closed: %d", doorOpen);
      Serial.println(pout);
    }
    startTime = 0;
    doorOpenTime = 0;
    soundTheAlarm = false;
    digitalWrite(DOOR_LED, LOW);
    return;
  }

}

// make some noise
void soundAlarm() {

  if (soundTheAlarm == false) {
    return;
  }

  int ii = 0;
  while (ii < 6) {
    if (soundTheAlarm == false) {
      break;
    }
    int i = 0;
    while (i < toneMax) {
      if (soundTheAlarm == false) {
        break;
      }

      unsigned long currentMillis = millis();
      if((currentMillis - previousToneMillis) > 50) {
        previousToneMillis = currentMillis;
        tone(BUZZER,toneMelody[i]-380,60);
        i++;
        }
    }
    ii++;
  }

}

// door open? then increment the timer - sound the alarm 
void fridgeDoorOpenTimer() {

  if (doorOpen == false && doorTrigger == false) {
    return;
  }

  if (digitalRead(RESET_BTN) == HIGH) {
    digitalWrite(POWER_LED, LOW);
    while (digitalRead(RESET_BTN) == HIGH) {
      delay(10);
    }

    if (SerialDebug) {
      Serial.println("reset");
    }
    startTime = millis();
    soundTheAlarm = false;
    digitalWrite(POWER_LED, HIGH);
  }

  // checking once a second
  doorOpenTime = millis() - startTime;
  if (doorOpenTime % 1000 == 0 && doorOpenTime != lastDoorOpenTime) {
    if (SerialDebug) {
      Serial.println(doorOpenTime);
    }

    if (doorOpenTime > doorOpenDelay) {
      if (SerialDebug) {
        Serial.println("Awwooga Awwooga - fridge door open!");
      }
      soundTheAlarm = true;
      soundAlarm();
    }

    lastDoorOpenTime = doorOpenTime;
  }

}


void setup() {

  pinMode(TEST_LED, OUTPUT);
  digitalWrite(TEST_LED, HIGH);

  if (SerialDebug) {
    Serial.begin(9600);
  }

  delay(2000);

  pinMode(DOOR_LED, OUTPUT);
  pinMode(POWER_LED, OUTPUT);
  pinMode(BUZZER, OUTPUT);

  if (SerialDebug) {
    Serial.println("ready");
  }
  digitalWrite(POWER_LED, HIGH);

  pinMode(FRIDGE_DOOR, INPUT);
  attachInterrupt(digitalPinToInterrupt(FRIDGE_DOOR), fridgeDoor, CHANGE);

  pinMode(RESET_BTN, INPUT);
  digitalWrite(TEST_LED, LOW);
}

void loop() {
  fridgeDoorOpen();
  fridgeDoorOpenTimer();
}

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

#define TEST_LED 13

#define FRIDGE_DOOR 6

#define DOOR_LED 8

#define POWER_LED 7

#define RESET_BTN 9

#define BUZZER 10

volatile boolean doorOpen = true; // when the light is on

volatile boolean doorTrigger = false;

volatile unsigned long lastFridgeMillis = 0;

volatile unsigned long fridgeTimeoutMillis = 10;

boolean SerialDebug = false;

unsigned long doorOpenDelay = 45000; // 45 second timeout before buzzer sounds

unsigned long doorOpenTime = 0;

unsigned long startTime;

unsigned long previousToneMillis = 0;

unsigned long toneInterval = 20;

boolean soundTheAlarm = false;

boolean outputTone = false;

int toneMelody[5] = { 1600, 1900, 2000, 2100, 1800 };

const int toneMax = 5;

unsigned long lastDoorOpenTime = 1;

char pout[80]; // for testing

// interrupt function

void fridgeDoor() {

if ((millis() - lastFridgeMillis) > fridgeTimeoutMillis) {

doorOpen = digitalRead(FRIDGE_DOOR); // goes high when dark

lastFridgeMillis = millis();

}

// see if the door is open or closed

void fridgeDoorOpen() {

if (doorOpen == true && doorTrigger == false) {

doorTrigger = true;

startTime = millis();

if (SerialDebug) {

sprintf(pout, "door open: %d", doorOpen);

Serial.println(pout);

}

digitalWrite(DOOR_LED, HIGH);

return;

}

if (doorOpen == false && doorTrigger == true) {

doorTrigger = false;

if (SerialDebug) {

sprintf(pout, "door closed: %d", doorOpen);

Serial.println(pout);

}

startTime = 0;

doorOpenTime = 0;

soundTheAlarm = false;

digitalWrite(DOOR_LED, LOW);

return;

}

// make some noise

void soundAlarm() {

if (soundTheAlarm == false) {

return;

}

int ii = 0;

while (ii < 6) {

if (soundTheAlarm == false) {

break;

}

int i = 0;

while (i < toneMax) {

if (soundTheAlarm == false) {

break;

}

unsigned long currentMillis = millis();

if((currentMillis - previousToneMillis) > 50) {

previousToneMillis = currentMillis;

tone(BUZZER,toneMelody[i]-380,60);

i++;

}

ii++;

}

// door open? then increment the timer - sound the alarm

void fridgeDoorOpenTimer() {

if (doorOpen == false && doorTrigger == false) {

return;

}

if (digitalRead(RESET_BTN) == HIGH) {

digitalWrite(POWER_LED, LOW);

while (digitalRead(RESET_BTN) == HIGH) {

delay(10);

}

if (SerialDebug) {

Serial.println("reset");

}

startTime = millis();

soundTheAlarm = false;

digitalWrite(POWER_LED, HIGH);

}

// checking once a second

doorOpenTime = millis() - startTime;

if (doorOpenTime % 1000 == 0 && doorOpenTime != lastDoorOpenTime) {

if (SerialDebug) {

Serial.println(doorOpenTime);

}

if (doorOpenTime > doorOpenDelay) {

if (SerialDebug) {

Serial.println("Awwooga Awwooga - fridge door open!");

}

soundTheAlarm = true;

soundAlarm();

}

lastDoorOpenTime = doorOpenTime;

}

void setup() {

pinMode(TEST_LED, OUTPUT);

digitalWrite(TEST_LED, HIGH);

if (SerialDebug) {

Serial.begin(9600);

}

delay(2000);

pinMode(DOOR_LED, OUTPUT);

pinMode(POWER_LED, OUTPUT);

pinMode(BUZZER, OUTPUT);

if (SerialDebug) {

Serial.println("ready");

}

digitalWrite(POWER_LED, HIGH);

pinMode(FRIDGE_DOOR, INPUT);

attachInterrupt(digitalPinToInterrupt(FRIDGE_DOOR), fridgeDoor, CHANGE);

pinMode(RESET_BTN, INPUT);

digitalWrite(TEST_LED, LOW);

}

void loop() {

fridgeDoorOpen();

fridgeDoorOpenTimer();

}

Links and Sources

Practical Electronics for Inventors, Third Edition. Paul Scherz and Simon Monk, pub. McGraw Hill, 2013. ISBN 978-0-07-177133-7
A pre-made USB LiPo Battery Charger – 3.7V Single Cell from Hobbytronics or from Sparkfun
Battery Charger MCP3831T Datasheet
TEPT4400 Phototransistor Datasheet

Installing the gopro-utils

Extracting the Data

Merging GPX files

Links and Sources

Document Setup

Layout Tips for G-Code Routing

Outputting to G-Code

Links and Sources

The purpose of Colliery Road?

Dates of Interest

Links and Sources

Power Supply

Driving the Display

Rotary Switch

Software

Links and Sources

Links and Sources

Circuitry

Software

Links and Sources

COMPUTERS, ROBOTS AND PHOTOGRAPHY