Computer, linux, Raspberry Pi

Copying and Resizing a Raspberry Pi SD Card

Leave a comment

The Micro SD Card I use in a Raspberry PI ran out of space so here is how I copied the contents of the 15GB drive to a new 64GB card and resized the partition. I used a separate computer running Debian and as the machine does not have a monitor or keyboard attached this will be being completed through the bash command line using SSH.

1. Making a copy of the SD Card.

Insert the old card into your computer, if the computer attempts to mount the drive then unmount it. We need to find which mount point has been used, do this with the lsblk command:

The device sde matches our SD card. So we will use that. The dd command is used to create the ISO image, I am creating the file in my home directory:

once complete, check that the records match:

Swap the SD cards and check the mount point again:

Now copy the iso image to the new card:

this can take a while, the write speed is rather slow:

and lsblk shows us the new partitions:

2. Resizing the partition

The copied partition is now the same size as the original. If you have space remaining, the new card can be put back in the Pi and use the raspi-config utility, and using the Expand Filesystem option in the Advanced Settings section. However if the drive is completely full you won’t be able to login as there won’t be enough space available for the temporary files created at login, to get round this you can use parted to resize, start with:

if you get the following message:

then quit from parted, if you are using a full size SD Card, check the Write Protect tab on the side of the card and try again, otherwise try:

If problem persists try formatting the new card in a camera, as these have a simple file system, and write the ISO image again.

We need to resize the larger partition /dev/sde2 with the ext4 file system, the smaller is used to boot the Pi and can be ignored. In parted, list the partitions with the print command:

Using resizepart set the new size, I set this to larger than the 15GB but smaller than the unallocated space, this to save me having to accurately work out the remaining space manually:

Now update the boundaries to grow and resize the partition into the freshly allocated space:

Now boot the Pi with the new card, login and use the raspi-config utility then in Advanced Options choose Expand Filesystem and follow the onscreen instructions. Once rebooted you should now be set to fill up your new card.

Links and Sources

Computer, linux, Programming

Extracting GPS data from the GoPro 7

2 Comments

While extracting the telemetry data from the GoPro is reasonably well documented I have found some gaps for getting the extracting utilities installed and when extracting and combining data from multiple files. These notes are for a Debian/Ubuntu installation in a BASH Shell.

Installing the gopro-utils

As I couldn’t find any straightforward instructions for installation, I’ll be going through everything I needed to do to get it working, you may have some of these packages installed already.

Now to get the gopro-utils and install them, I’m placing the source files into my Downloads directory, as well as the GPS data extractor we’ll be adding the other telemetry tools too, this is all a bit long winded.

Extracting the Data

You will need to find which stream in the video recording the data has been saved to, to find this use ffprobe to examine the recording and look for the stream that contains GoPro MET, for example:

You can see that what we are wanting is on stream 3, as far as I can tell this stays the same every time, I don’t know if it is different for other GoPro models.

This bash script extracts the GPS data in GPX format from all the GoPro GX recordings in the directory, other options have been commented out, if you are using Garmin VIRB edit there is also an option for use with that. The script creates two files, one that contains the raw data and another with the desired GPS data, the GPS output file has the same name as the recording, but in lowercase with a .gpx extension.

Merging GPX files

As the GoPro splits recordings into 4GB blocks, when extracting you will get a single GPX file for each recording. Many pages found by Google say that to create a single track from these all you need to do is append the files into one big file. This is wrong, what you end up with is a single file with many short tracks, when what you are after is one long track covering the entire journey. This bash script uses gpsbabel to create single merged file from the extracted GPX data, it creates a file called “gpsoutput.gpx”.

The next stage will be to write a script that combines all these and completes the job in one easy process.

Links and Sources

CNC, Computer, Electronics, Hardware

Using Inkscape for CNC designs

Inkscape is a free vector graphics editor for all major platforms, generally it is aimed at art and design users but it does have an option for generating G-Code for use in your favourite CNC software. While Inkscape doesn’t have many of the functions of proper CAD/CAM software it is an relatively easy place to start for creating basic designs, I have been using it to make boxes out of 3.5mm plywood.

These notes are based around my cheap CNC machine sold as an CNC3018 by a variety of Chinese manufacturers on Amazon and eBay, the included controller is a Woodpecker CNC board (Ardunio clone) I have upgraded to GRBL v1.1 and I am using version 0.92.4 (April 2019) of Inkscape with the included Gcodetools.

CNC Wood Cutting

This post focuses on setting up Inkscape for the CNC machine and producing the g-code from your drawing, it is not intended to be an Inkscape tutorial.

Document Setup

With a new drawing set your Document Size, this should be the same as your CNC bed, in my case this is 300 x 180mm. From the Inkscape menu go to File > Properties and in the Page Tab set the Display Units (millimeters in my case), the Orientation to Landscape and Page Size width: 300 and height: 180. In the Grids Tab set the Grid Units to mm and the Spacing X and Spacing Y to 1.0. Back on your main page, turn the page grid on with: View > Page Grid.

Inkscape Document Properties

By default Inkscape scales the stroke/line width when you resize a shape, to prevent this click the the fourth box from the right in the top icon bar “when scaling objects, scale the stroke width by the same proportion”

Turn Line Scaling off to prevent the line width changing when resizing

You can save this as a template, such as: CNC3018.svg or as the document default with: default.svg by saving the file to your templates directory:
On Linux and OS X: ~/.config/inkscape/templates/
On Windows: C:\Users\<username>\AppData\Roaming\inkscape\templates

The lines you draw will need to be the same width as the bit you are using in the CNC machine. Draw a rectangle, Right mouse click on the rectangle and select Fill and Stroke…. In the Fill Tab click the X – no paint box and on the Stroke Style tab set the width to that of the bit you are using – 1.5mm, subsequent rectangles will be in the same style, other shapes will need to be setup this way too. The colour of your lines should be black, there is some functionality for different colours to represent different depths but I have not yet worked out how to do this.

Setting the line width

Layout Tips for G-Code Routing

Remember to check the dimensions of the cuts, with an outside cut such as the width and height of a box side you need to measure for the inside of your rectangle, for holes in your box measure to the outside edge, Inkscape sets distances to the outside edge.

For positioning holes for switches and the like, I add thin lines 0.1mm thick as guides and make use of the width/height settings as well as the Object > Align and Distribute options. A pair of digital vernier calipers are a great aid to discovering the required sizes. Remember to delete these before G-Code encoding.

Using guidelines for accurate layout

When generating the G-Code each shape will be seen as an individual object, so lets say you want to have two sides of your box cut from a single sheet of plywood, this would be two rectangles abutting each other with a side to be cut overlapping. As it takes four passes to cut each shape 1mm at a time, this means it’ll take six passes down the centre. To fix this select both rectangles and then Path > Combine followed by Path > Difference to make a single object.

Outputting to G-Code

Now that you have completed your drawing, save your work then convert your objects to paths by selecting all objects then Path > Object to Path. You may also want to place your drawing near the bottom left of the document, as this is where the CNC router starts. Now using Gcodetools there are three things you need to do to produce the G-Code file. None of the Gcodetools windows close automatically when apply is clicked, you will need to do that yourself. From the Inkscape menu:

1. Extensions > Gcodetools > Tools Libary…
Select Tools Type: cylinder and click apply In the overlarge green box that appears you will need to set the tool diameter and feed speed.

Gcodetools Tools Library

This can be a bit fiddly as the text can become detached from the box and the settings lost, what seems to work most reliably for me is to change to Text Objects (F8) click on the numbers you want to change and once done go back to Select and Transform (F1). Resize the box afterwards to check that it is still working – if the green box moves but the text does not then Ctrl-Z a few times and try again.

Gcodetools Tools Library – Green background is detached from the text – this won’t work.
setting default used description
Diameter 10 1.5 tool bit diameter in mm
Feed 400 300 speed while cutting through the material in mm/second
Penetration angle 90 90
Penetration feed 100 100 Plunge speed in the material in mm/second
Depth Step 1 1 Depth of cut on each pass in mm

2. Extensions > Gcodetools > Orientation Points
This tells the g-code where to start, normally bottom-left on the CNC Set the following:
– Orientation type: 2-points mode
– Z Surface: 0mm – this is the top of your surface
– Z Depth: -3.4mm – this is the thickness of material to cut, a negative number

With the Orientation Points added.

3. Extensions > Gcodetools > Path to Gcode
This creates the G-code file, in the Preferences Tab set the following:
– File: output filename 
– Directory: output directory
– Z safe height: 5mm – height above the work surface when moving between cuts
The filename once set doesn’t change, an incremental number is appended to the output filename. Click the Path to Gcode Tab before clicking apply (this appears to be a bug).

Your image will be updated to show the g-code routing, give this a visual check to ensure that all objects have been coded and that it looks right, the path to be taken should be in colour and contain arrows showing the direction of the router.

If there are too many arrows or if a line has arrows pointing in different directions then there may be an object underneath, check on your original artwork, in the image with the three circles below you see that A has not been converted to a path with Path > Object to Path, B has a duplicate object underneath and C is correct.

Gcode check; A and B have problems, C is correct.

The generated G-Code does not appear to include the Spindle Motor Start command – So remember to start the spindle manually in your CNC software before running the G-Code – its interesting how easily these bits break with a sideways load. Remember if you are cutting trough rather than engraving, don’t forget to put a layer to sacrifice between whatever you are making and the CNC’s bed, I use 5mm MDF/Fibreboard.

Links and Sources

history, Sheffield

History: Colliery Road is Weird

2 Comments

Colliery Road in Sheffield Brightside is an odd throughway that passes under the Midland Mainline Railway providing a 290m (317 yards) route between the junction of Brightside Lane and Weedon Street to Holywell Road. It is a narrow single track road and the the Brightside Lane end has been cut into the hillside to accommodate the railway junction and sidings above. Nowadays access is limited to pedestrians and cyclists as at 2.32m (7′ 6″) high the bridges are far too low to be of any use for general traffic, the road was closed to motor vehicles in around 2016 and is now part of the Lower Don Valley Cycle Route1. In this short essay I am looking a little into its history, the modifications that have changed the road to accommodate the railway and to speculate why it exists at all.

The Bridges of Colliery Road

Two rail bridges run over the road, one for the Midland Mainline, originally the Sheffield to Rotherham Railway, constructed in 1838/39, this bridge is referred to in Drake’s Road Book published in 1840 where he says “We now cross a private road by a beautiful stone bridge – one which is infact the only ornamented bridge upon the line, and also the only one that is graced with parapets”2. The other is now disused, it was part of Brightside Junction giving access to the various steel works as well as a line towards Catcliffe and the south as part of the Sheffield District Railway. As well as the two we now see, sometime between 1880 and 1955 there was also a third single track bridge on the north western side used for the gas and chemical works.

Colliery Road, Brightside [Ordnance Survey]
The 1854 map (Yorkshire 289) shows a footpath or track approximately following the route of the road today, leading from Brightside Lane under the railway and up to The Grange farm at the end of Wincobank Lane, on the 1892 map this is shown as a footpath, it looks to have been around a 620m (678 yards) walk from the farm to Brightside village.

Colliery Road highlighted in yellow

By 1892 Brightside Junction had been added to the railway providing access to the steel works in that part of the Don Valley, at that time Colliery Road was moved to start from near near Naseby Street and run alongside the south-eastern side of the tracks for a short while before ducking under the line, this was to accommodate the embankment built for the junction and the goods line into Brightside Steel works. It wasn’t until 1896 when construction started on the Sheffield District Railway that the road was moved to go between the new railway and the existing goods line, this is when the roads route was fixed to that we see today.

Ordnance Survey Maps – 1854 to 1949 [National Library of Scotland]

In the 1930’s on the north-eastern side of the tracks Holywell Road was extended from Upwell Street alongside the Gas Works to connect to Limpsfield Road and replacing Burslem Street as part of an expansion of housing into the area. Before then this part of Wincobank was mostly farmland with some industry alongside the railway, most notably the 1906 map (CCLXXXIX.SW) shows the Grimesthorpe Gas Works taking up a large amount of land between Upwell Street and Colliery Road. Looking at the 1926 aerial photo below, the exit of Colliery Road from under the railway can be seen on the far left of the image. Today this land is used by a mail order distribution centre as well as some smaller wholesalers and trade suppliers.

The Grimesthorpe Gas Works, Sheffield, 1926 [Britain From Above]

Brightside Colliery was also situated on the north-eastern side, the mine was worked from 1855 to 1886 and the shafts were located at the end of Colliery Road, the colliery was owned by Unwin & Shaw Co. and the pit had two shafts, No. 1 from 1855 to 1871 and No.2 from 1868 to 1886, they worked the Parkgate seam3. From 1883 until around 1955 the site was occupied by the Brightside Chemical Works which was a part of the gas works dealing with sulphur produced from gas manufacture [Sheffield History]. Nowadays the land is used by a self storage facility and a motor vehicle repair company.

Extra width has been added to the mainline bridge over time

The purpose of Colliery Road?

The road must of had a high cost of construction, the changes that needed to be made to have it run under the railway including cutting it into the hillside at the Brightside Lane seems a lot of effort has gone into maintaining a private a road that gives such poor access.

A reason for this could be that the colliery supplied coal to the works and foundries on the south-eastern side of the railway, this would justify a bridge for the Sheffield to Rotherham Railway but by the time Brightside Junction was built the mine had been closed for around five years so why go to all the trouble to modify the road? Or is Colliery Road a legacy of the landowner insisting they have access from their farm at The Grange down to Brightside Lane? I can imagine a track existing before the railway and it being kept as a right of way as a route to Brightside village and the bridge across the Don which had been there in one form or another since before 16504.

Comparing access for the colliery 1854 and 1892

By comparing the 1854 and 1892 maps above we can see similarities between the buildings being shown, the map also shows there is no roadway from the colliery site into Brightside as a field is in the way. The coming of the railway would have cut access off from Naseby Street and maybe the landowner did not want to sell part of his field, or set the price too high, to permit access to the nearest bridge over the line on Jenkin Road and forcing further development of Colliery Road, the bridge and its parapets.

More of my Colliery Road photos can be found on flickr

Dates of Interest

1838/39 Sheffield to Rotherham Railway constructed (Midland Mainline)
1838 Brightside Station opens
1855-1871 Brightside Colliery – Shaft No.1 in operation
1868-1886 Brightside Colliery – Shaft No.2 in operation
c.1855-c.1955 Grimesthorpe Gas Works in operation
1883-c.1955 Brightside Chemical works – part of Grimesthorpe Gas Works
c.1890 Brightside Junction added – for the steel works
1896 Sheffield District Railway construction starts

Links and Sources

My photos of Colliery Road on flickr

  1. Sheffield City Region Combined Authority, Transport Committee, 26 Sept 2016. p. 17
  2. Drake’s Road Book of the Sheffield and Rotherham Railway, London 1840, p.33
  3. The Coal Mining Industry of Sheffield and North Derbyshire, Ken Wain, 2014: ISBN-13: 978-1445639635
  4. Hallamshire. The History and Topography of the Parish of Sheffield. Joseph Hunter, London 1819, p.227

Ordnance Survey maps at the National Library of Scotland:

Electronics, Hardware

A footpedal control for my Dremel

In this video I am adapting a sewing machine pedal to provide variable speed to my small Dremel drill. It’s not really suitable as the pedal is only rated to handle 80 Watts (0.42 Amps at 240 Volts) while the Dremel can take up to 125 Watts (0.52 Amps) of power, it should be alright for light work.

To control the power the footpedal has two stacks made of loose carbon disks, initially when the pedal is pressed power is passed through these and as more pressure is applied these compress to lower the resistance and allow more power through. When fully on, a metal disc engages across two contacts to allow all the current to pass. I think I would like to have a pedal that could handle more current, possibly something based around how a modern dimmer switch works.

Arduino, Electronics, Hardware

Rebuilding the Vacuum Fluorescent Display

Back in April 2016 I posted details on how a Vacuum Fluorescent Display (VFD) works and a method of driving it from an Arduino type microcontroller using transistors, while this worked well it made for a rather bulky clock. I have now rebuilt the display using proper VFD driver chips, added a rotary switch with LED’s, NeoPixel capability and used a WeMos D1 Mini microcontroller to give me internet access over WiFi so the time and date can be set using the Network Time Protocol (NTP).

VFD Clock

This rebuild is much more practical than previously, it is now about 3cm deep and comprises of two boards back to back with pin header plugs/sockets as the interconnects. I have been running the clock for over a year without any problems, apart from having to switch the thing off and back on when Daylight Savings Time changes in the spring and autumn, but that’s a software (lazy programmer) problem.

Power Supply

Three different voltages are needed to operate this clock, 16 Volts for the VFD display anodes, 5 Volts for the microcontroller and NeoPixels and 3.3 Volts for the VFD cathode and other hardware.

Power layout for the VFD clock

From the 12 volt source, I split the power to a XL6009 boost module to supply 16V to the driver IC’s, the 5V power module can supply up to 5A which is enough for around 2 meters of NeoPixels. The 3.3V supply is created using a BA033T fixed voltage regulator running from the 5V module, I have used this to power the VFD cathode, the rotary encoder and VFD driver IC’s as this is the microcontrollers operating voltage.

Back of the VFD Clock

Driving the Display

To drive the anodes on the VFD display I used three MAX6920 Tube Drivers, these have a four wire serial interface and can switch up to 76V on the display.

MAX6920 VFD Driver

The physical connections from the driver IC’s to the VFD anodes (grids and elements, see the previous article for definition) are those that are most convenient for the PCB layout without having to use links or through-hole connections as the order of the pins for outputting the display can be mapped later in software.

MAX6920 Anode Connections on the PCB

In this case, the first anode pin on the display is on the right, but the serial data starts with the first MAX6920 chip on the left. I will explain in more detail in the software section below.

The Cathode is run from the 3.3 Volt supply, it has a 3.3V Zenner diode across it to protect against meltdown in case of over-voltage, the cathode is made from of strands of thin tungsten wires and shows as a short when testing in beep mode on the multimeter.

Zenner Diode for the VFD Cathode

Rotary Switch

I have used the SparkFun RGB Illuminated Rotary Encoder, as well as the encoding it also includes three LED’s and a push button switch. As the WeMos only has limited number of pins available, I used an MCP23008 I2C I/O Expander. I have covered these encoders in a previous post and have used a similar method here.

Rotary Encoder on the I2C Bus

In this project it turned out I don’t have much use for the rotary switch, the same result can be achieved with a push button and two LED’s, it goes red when setting the time over WiFi, goes green when the time has finished setting but is off when running normally, the push button displays the date when pushed. I expect the encoder could be used for setting an alarm should I ever have need to add one.

Software

The complete source code I am currently running in the clock can be found on my github repository, it has been written for the Arduino platform and can be considered unfinished and a bit of a mess. In almost all cases this clock will be a custom build with all the scavenged VFD displays being different I’ll be concentrating on how the display is being driven. Note that the software uses Latch and the MAX6920 driver chip uses Load.

To start, here are the physical connections for my VFD display, the display connections are shown looking from the back with pin one on the right, on the anode there are eleven grids and twenty-one elements, each grid illuminates a block of chosen elements. There are three MAX6920 driver chips, the first is on the left data is passed along the serial data line DIN/DOUT to the right:

Connecting the MAX6920 drivers to the display

While I’m not going to worry too much about how the data is sent to the display as this is being handled by the ShiftOutX library it may be useful to have an understanding of the byte order sent to the driver chips.

Breaking out the logic analyser we can see what is going on, in this first image below we can see that between the two markers one and two there are eleven data peaks to match the number of grids on the display the data is processed as a padded eleven character string: ‘    212848 ‘ the first four contain nothing but the bit selecting grid to be used the next six are busy as they contain the time, and the last is blank. There also appear to be two blank clock cycles after these, this may be a programming error on my part.

Data being sent to the display

the string is iterated so that each grid gets its own character with a very short delay between each this causes the persistence of vision effect and the time appears on the display.

In the next screen capture I have zoomed into more detail, it is showing the fifth grid and is displaying the number two. You can see that there are thirty-two clock cycles sent to the drivers while LOAD is high, this causes the driver chips to direct the data to the outputs. I am not entirely sure why the clock is being run while LOAD is low, I think it may be clearing the buffer before sending the required information.

Data being sent to a grid

You can see that 32 bits are sent to the display, but there are 36 pins available on the three MAX6920 drivers, you can also see that the outputs are arranged out of order. How do the drivers know which lines to switch for the display?

First off, an explanation of how the data is constructed then sent to the display. For this example I am sending the number five to the third grid, the data is constructed through the manipulation of a character array in three stages:

1. blank char array: 00000000000000000000000000000000
2. set the third grid: 00000000000000000000000000000100
3. set the number: 00111000011000011100000000000100

Because of the way the bits are ordered when sent to the drivers I have a lookup array VFDlookup to make life easier, in this case grid 3 can be found at position 29 in the VFDdata character array. I start with a blank character array of 32 bits VFDblank and copy this to VFDdata, I then use VFDlookup to find the position of the bit to set: VFDlookup[3] = 29. For the elements the encoding for each character is set in VFDdigits the code for number 5 is 1110000110000111 the code was worked out by experimentation, seeing which element lit up after I had completed the hardware, I have ignored the extra symbols used for the DVD player so I only need to set pins 15-30. Once complete the character array is converted to a unsigned long integer and sent LSBFIRST – Least Significant Bit First via ShiftOut_32.

While that explains the elements encoding to display the correct digit, the grid selection however is a different kettle of fish. I built this clock over a year ago and have forgotten how it works. Grids 1 and 2 are connected to the output pins 1 and 0 on the third driver, pins 2-5 are then skipped, and the rest are connected to the grids, with three more being taken from chip 2. I can’t work out how it knows to skip those pins, or that grid 2 is connected to pin 1. But it is an awesome clock though.

Links and Sources

Computer, linux

FFMPEG for Video Conversion

FFmpeg is a command line program to manipulate, convert, record and stream video and audio, it is available for Mac, Linux and Windows. Here is a handy list of commands for reference, these have been tested with version 3.1.12 in a Debian Linux environment. I expect this list to grow over time as needs arise.

• Rescale a 4K video to 1080p

• Convert to H.264 (AVC) codec for use on uploading to YouTube, Vimeo, etc:

Using this codec reduces the time it takes for the video to be available after upload, however YouTube converts the file again to the VP9 codec and unless you have a popular channel, 100 subscribers or more, then this can take a few days or weeks and in the meantime your video can appear quite poor and blocky even when watching at 1080p, especially when there is a lot of movement like in a car dash-cam video. You can use FFmpeg to encode to VP9 webm format with this bash script:

This script is based on the encoding method shown in the WebM Wiki on my computer it is very slow and takes a quite a few hours to encode just nine minutes of video and the eventual results are so poor you’ll be wondering why you bothered.

• Convert to MP4 for use in Vegas Studio:

If you have a particularly old/odd video and get lots of pts has no value errors, then try this:

The -fflags +genpts option adds a Presentation Timestamp (PTS) to the frames, this must be before the -i as shown to work. Source.

• Set the video playback speed, this method adjusts the Presentation Timestamp (PTS) on each frame which may not work with older software. To slow down video divide the PTS by your required speed, this example slows the action by two times setpts=PTS/2.0. You can also reduce the number of dropped frames by increasing the frame-rate -r 50, in this case I went from 25fps to 50fps, but depending in the chosen speed frames may still be dropped.

Speed up your video by multiplying the PTS, in this case two times faster: setpts=PTS*2.0

• Convert file or extract audio from file into an MP3, the output is set to 128K constant bitrate

• Concatenate Video Files
This combines two video files, when using formats such as MP4 or MKV you will need to create intermediate files, otherwise only the first file will be included in the output:

• The opus not found error
When converting a file and you see an error like Could not find tag for codec opus in stream #1… you will need to state the output format

Links and Sources