Applied Mineral Exploration
Discussion and research relevant to mineral
exploration. Using the Columbus V990 (Visiontac) GPS logger
How to use an android tablet to map these
routes
This logger records gps locations at 1 second intervals to
memory on a micro SD memory card (2 Gbyte max, as it only
understands FAT16 filesystems). The data file format is CSV. It
also allows voice recording and waypoint marking which are useful
for logging sample locations as well as your (bicycle?) route.
The data files can be quite large as a trackpoint is recorded at
every 1 second interval. This recording interval cannot be
changed.
I have written this script ( in the AWK
language; yes I know I should use Perl, but I like awk! ) to
eliminate excess points in the raw CSV datafile by looking for
points of directional change and eliminating trackpoints which are
merely intermediate points on straight line segments.
On a 90 Km, 4 hour test route it reduced the file sizes by a factor of 10, either the CSV or subsequently converted KMZ, GPX format files. On this test route there were 14016 trackpoints in the raw logfile and after running the script, there were 1184 trackpoints. This is useful when uploading long routes to many internet mapsites as they often impose a maximum filesize for upload
The script requires that you have awk (or gawk)
installed on your system. All linux systems have this and there
are versions for windows, mac.
Run it from the command-line,
SYNTAX: awk -f
segments.awk {your-input-filename} >
{your-output-filename}
INDEX,TAG,DATE,TIME,LATITUDE N/S,LONGITUDE E/W,HEIGHT,SPEED,HEADING,VOX
921,T,130503,013653,12.518618S,131.066604E,73,27,358,
The script works with standard mode format and should also work with professional mode log format which includes extra variables
The script output is a CSV of identical format as the input file. Use your own mapping software to import it and convert it to GPX or KMZ etc. The program BT747 understands this format as do the java programs Time-album and Route-Converter that come with the logger. These programs can convert the CSV to other formats. Viking is a good mapping program for GPX data files and shows each trackpoint.
Download the file segments.awk here (3 KByte) (Sorry, this shows as text in the browser, copy and paste it to a file)
A perl version which optimizes and also converts the CSV to GPX format is also available and is discussed below.
This image (mapped in the Viking software) shows the trackpoints in the raw logfile as green squares, with a connecting route line. After processing the raw logfile, the black squares are the retained trackpoints, connected by a route line. The 2 tracks have been offset to allow comparison and it shows the difference in number of points by eliminating excess straight line trackpoints.
Although this example shows a route with due N-S and E-W roads, straight lines of any orientation are simplified.
Using this on an ANDROID tablet
(An android app to do this is now available here)
While traveling I carry only an android tablet (originally a
Flytouch 3, android 2.1, now Amicroe with android 4.1.1, jellybean
) rather than a heavy/cumbersome laptop computer. The usual
software tools to deal with Columbus CSV files do not work with
android. However, I have been able to resolve this so you can
transfer data from the logger, optimize it to a GPX format and use
a standard existing android app to show the route. Although my
tablet is "rooted", I do not think this is necessary to run any of
the software here.
The first part of the problem is to get data from the logger onto
the tablet. For most loggers a pseudo-serial connection over USB
is used and this is impractical on android as there are no drivers
to my knowledge. But with the Columbus logger you simply remove
the flash memory card and connect it to your tablet, possibly with
the USB-Flash adapter that comes with the logger. (Your tablet
merely needs to have a USB socket) Then use any file management
app (Blackmoon or Astro etc) to copy the native CSV track-file(s)
onto a location on the tablet SDcard.
Now you need to convert the native Columbus format CSV track-file
to a usable format and optimize it to remove unnecessary
trackpoints. To do this I converted the above AWK optimization
script into Perl, because awk has some limitations that make it
very difficult to use on android. The new Perl script, segx.pl, also converts the CSV
from the columbus logger into a GPX file which can be read by
(some) standard android apps and is widely used on internet
map-sites. It works with the Columbus logger in STANDARD
mode. It might also work with Professional mode CSV files, which
have more variables at each trackpoint, but this has not been
tested and the GPX output will lack the extra variables. This
script will also work on any computer with Perl such as Linux, or
even windows/mac if you install a perl interpreter.
It is possible to run Perl scripts on android. First you must install the SL4A (Scripting Layer for Android) application. Then you install the Perl for android application. Then you must run SL4A and install the Perl capability from its selection list. You need to have a live internet connection for some of these steps as these packages add additional live downloaded files. Now you can add Perl scripts in the /sdcard/sl4a/scripts directory.
To run the script, start the SL4A application icon. It will show
a list of available scripts, choose the segx.pl script I have written. (The script
name must end in .pl for SL4A to recognize it)
Select the "terminal" to run it. The script will prompt you for
the input filename and output filename. (They are simple command
line prompts, not pretty dialog boxes, but work OK) The default
location for the input and output files is
/sdcard/sl4a , so you will need to specify a full
pathname if you are using other directories. Upon completion you
will have a GPX format file (with your chosen name) of the logger
route in directory /sdcard/sl4a (unless
you specified an alternative location).
The android application "BackCountry Navigator PRO GPS
v4.4.0" can import this GPX trackfile and merge it with roadmaps
to show your route! Only a few android apps can import a GPX file
in this manner as most assume you have a map on the internet
rather than in a local file. This application can also merge maps
saved by "mobile atlas" in sqlitedb format, used by Rmaps. I use
the android Rmaps application with saved sqlitedb maps to plan my
routes and it is very convenient that these offline map databases
work in both applications without needing an internet connection.
Or you can upload the GPX file to many internet sites. Some
websites can convert the GPX to KMZ or KML formats for you and
these formats can be uploaded to your google "My Maps".
An android APP "Simplify Segments" to convert CSV to GPX while eliminating excess data points
In 2016 the above perl script was converted to Java and used to
make an android app. This
is available here. (csv2gpx.apk) It runs as "Simplify
Segments".
This is my first attempt at an android app and it is not very
polished, but works on android 4.1 and higher. I have not tested
it widely.
The app reads a CSV file and writes the converted data in GPX format to a new file, removing excess data points. You need to view this in another app such as Backcountry Navigator.
The app asks for the folder and name of the input CSV track file,
assuming Columbus format as discussed above. It then asks for the
folder name for the output file. It states that the output file
will be called "trackout.gpx". If you do not change this, the
output file will be the same name as your input file, but with
.CSV changed to .GPX . If you change the
name from trackout.gpx to your own filename, it will save to your
entered filename. Note that it never actually saves output to
"trackout.gpx". I warned you the app is un-polished!
This app saves only the Latitude, Longitude and elevation of each
point. It removes the time, which you need if you contribute
routes to Open Streetmaps.
No need to take a monster laptop while you travel. You can use the
Columbus logger and an android tablet to work out your route!