Add support for exporting Favorites / Flags / Want to go, etc...

.gitignore

@@ -0,0 +1,2 @@
+*.kml
+*.csv

README.md

@@ -1,19 +1,69 @@
-# Export Google Maps saved/starred locations to KML/CSV/Sygic
+# Google Maps Export
 
-This repository contain three [Python 3](https://www.python.org/downloads/) scripts that can be used to **export starred locations from Google Maps to other formats** which then can be imported by GPS navigation apps or other POI convertion tools. The three scrips are:
+This repository contain 4 [Python 3](https://www.python.org/downloads/) scripts that can be used to **export saved locations from Google Maps to other formats** which then can be imported by GPS navigation apps or other POI convertion tools. The three scrips are:
+
+* **csv2kml**: Export Favorites / Flagged Places / Want To Go / and custom lists to KML.
 
 * **json2kml**: this script converts the list of starred/saved places (a.k.a. POIs) from Google Maps into a **KML file** that can be imported into various GPS navigation applications (suchs as MAPS.ME).
 
 * **json2csv**: this script converts the list of starred/saved places (a.k.a. POIs) from Google Maps into a **CSV** (*Comma Separated Values*) file that can be imported into some POI convertion tools or edited directly in Excel.
 
 * **json2sygic**: this script converts the list of starred/saved places (a.k.a. POIs) from Google Maps into the internal format used by Sygic Android to save its favorites (**"items.dat"**) file.
 
-## How to export Google Maps saved/starred locations to a JSON file
+## Step 1: Google Takeout for JSON/CSV files
+
+1. Go to Google Takeout (https://takeout.google.com/settings/takeout). 
+2. Click “Select None”
+3. Select “Maps (your places)”.
+4. Select "Saved" - "collections of saved links (images, places, web pages, etc.) from Google Search and Maps"
+5. Google will export a ZIP file. Save this file to your local disk, open it and extract.
+
+Takeout contains several relevant files:
+
+- Maps (your places)/Saved Places.json
+- Saved/Favourite places.csv
+- Saved/Flag.csv
+- Saved/Want to go.csv
+
+## csv2kml
+
+First you need a Google API key
+
+1. If needed, greated a Google Cloud Project
+2. Enable Places API
+3. Goto <https://console.cloud.google.com/apis/credentials>
+4. Click: Create credentials
+5. Copy API key
+6. Run this and paste to save key
+
+```
+install -D /dev/stdin -m 600 ~/.private/google-api
+```
+
+This script depends on `mako` so:
+
+```
+pip3 install mako
+```
+
+Run it like this:
+
+```
+./csv2kml.py sample/takeout.csv > output.kml 
+```
+
+Checked `failed.csv` for entries we were not able to convert.
+
+Dev Notes
+- these files use FTID identifiers
+- https://stackoverflow.com/questions/47017387/decoding-the-google-maps-embedded-parameters
+- the undocumented API call is:
+    - `https://maps.googleapis.com/maps/api/place/details/json?key=KEY&ftid=0xd62377123a70817:0x85e89b65fcf7c648`
+- I was not able to find out what param1 is, but when a place (e. g., a store)
+  is selected, param2 is the Google Maps customer id (CID) in hexadecimal
+  encoding.
+- related: https://gpx2googlemaps.com/#/
 
-1.	Go to Google Takeout (https://takeout.google.com/settings/takeout). 
-2.	Click “Select None” and then select “Maps (your places)”. Make sure this is the only option selected.
-3.	Google will export a ZIP file. Save this file to your local disk, open it and extract the file “\Takeout\Maps (your places)\Saved Places.json” to a directory in your PC (do not change the file name).
-4.	Download one of the scripts below and save to the same directory where you saved "Saved Places.json".
 
 ## json2kml
 
@@ -49,3 +99,5 @@ After this, the following steps must be executed to generate the KML file from t
 * In Android devices, Sygic saves the favorites to file "items.dat" (it's a SQLite3 database). This file is located in folder _"/Sygic/Res/db/items.dat"_ if Sygic is configured to use internal storage or in folder _"/Android/data/com.sygic.aura/files/Res/db/items.dat"_ if Sygic is configured to use external SD card.
 * This script creates a new "items.dat" file with all saved places from Google. This file needs to be copied to one of the above foldres.
 * **IMPORTANT**: when overwriting "items.dat" files, **all current Sygic favorites _will be lost_**. Keep this in mind.
+
+

csv2kml.py

@@ -0,0 +1,87 @@
+#!/usr/bin/env python3
+"""
+Extract Google Place IDs from Google Takeout
+
+Usage:   extract.py favorites.csv > out.kml
+Author:  Devin Bayer <[email protected]>
+Repo:    https://github.com/akvadrako/json2kml
+License: GPL 3.0
+"""
+
+import sys, csv, re
+
+from mako.template import Template
+from dataclasses import dataclass
+from urllib.request import urlopen
+from os.path import expanduser
+from json import loads
+
+def log(code, *args):
+    sys.stderr.write('%8.8s | ' % code)
+    sys.stderr.write(' '.join(str(a) for a in args) + '\n')
+
+@dataclass
+class place:
+    name: str
+    note: str
+    url: str
+    comment: str
+    address = ''
+    lat = 0.0
+    long = 0.0
+
+path = sys.argv[1]
+places = []
+
+apikey = open(expanduser('~/.private/google-api')).read().strip()
+
+failed_fd = open('failed.csv', 'a')
+
+reader = csv.reader(open(path), delimiter=',', quotechar='"')
+for row in reader:
+    log('row', *row[:2])
+    if row == 'Title Note URL Comment'.split():
+        continue
+
+    p = place(*row)
+    p.lat = 0.0
+    p.long = 0.0
+    p.address = None
+
+    m = re.match(r'.*google.com/maps/place/.*/data=.*!1s(0x[0-9a-fx:]+)', p.url)
+    if not m:
+        log('nomatch', p.url)
+        csv.writer(failed_fd).writerow(row)
+        continue
+
+    ftid = m.group(1)
+    log('match', ftid)
+    resp = urlopen(f'https://maps.googleapis.com/maps/api/place/details/json?key={apikey}&ftid={ftid}')
+    data = loads(resp.read())
+
+    if data['status'] != 'OK':
+        log('error', data)
+        csv.writer(failed_fd).writerow(row + ['error: ' + data['status']])
+        continue
+
+    try:
+        r = data['result']
+        p.address = r['formatted_address']
+        p.lat = r['geometry']['location']['lat']
+        p.long = r['geometry']['location']['lng']
+        log('addr', p.address, p.lat, p.long)
+    except Exception as e:
+        log('error', e, data)
+        raise
+
+
+    places.append(p)
+
+temp = Template(filename='template.kml.mako')
+print(temp.render(
+    path=path,
+    places=places,
+))
+
+failed_fd.close()
+