Skip to content

An update for 2020 #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
josmithua wants to merge 9 commits into
base: master
Choose a base branch
from
Open

An update for 2020 #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
96c7b7c
An update for 2020:
josmithua Apr 15, 2020
d42eb8b
Fix leap year conditional, remove dead function code
josmithua Apr 15, 2020
b360781
Change old gmtoff col name to stdoff
josmithua Apr 17, 2020
b3ae3dc
Add type hints to generate_zones.py
josmithua Apr 17, 2020
1722157
Use f-strings, add some type hints, extract w_to_s func
josmithua Apr 17, 2020
06f9629
Minor refactorings:
josmithua Apr 22, 2020
c9b5150
FIX BUG: fix zone offset when it includes minute offset
josmithua Apr 22, 2020
32585e1
Add second example
josmithua Apr 22, 2020
9577138
Remove unneeded step from README
josmithua Apr 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,5 @@
build
*.o
example
.mypy_cache/
a.out
6 changes: 6 additions & 0 deletions .vscode/settings.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"python.pythonPath": "/usr/local/bin/python3",
"python.autoComplete.extraPaths": [
"./utils"
],
}
6 changes: 3 additions & 3 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ DEPS = zones.h
UTZ_DATA_DIR = vendor/tzdata
UTZ_REGIONS = africa,asia,australasia,backward,europe,northamerica,pacificnew,southamerica
UTZ_WHITELIST = whitelist.txt
UTZ_INCLUDES = majormetros
UTZ_INCLUDES = majorcities
export UTZ_DATA_DIR:=$(UTZ_DATA_DIR)
export UTZ_REGIONS:=$(UTZ_REGIONS)
export UTZ_INCLUDES:=$(UTZ_INCLUDES)
Expand All @@ -18,11 +18,11 @@ zones.h: $(UTZ_DATA_DIR) $(UTZ_INCLUDES) $(UTZ_WHITELIST) utils/generate_zones.p
./utils/generate_zones.py
zones.c: zones.h

whitelist.txt: vendor/android/timezones.xml majormetros utils/compile_whitelist.py
whitelist.txt: vendor/android/timezones.xml majorcities utils/compile_whitelist.py
./utils/compile_whitelist.py

# The IANA timezone database is missing zone links for many of the worlds largest metropolitan areas
majormetros:
majorcities:
./utils/compile_tzlinks.py

%.o: %.c $(DEPS)
Expand Down
19 changes: 16 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# μ time zone (library)

An embedded timezone library and ~3kB tzinfo database featuring nearly
all current timezones present in the IANA timezone database.
An embedded timezone library and ~3kB tzinfo database featuring nearly
all current timezones present in the IANA timezone database.
Designed for use on budget embedded systems with low program space.

The C header containing packed timezone information is generated from
Expand All @@ -23,7 +23,7 @@ formatting, etc to make efficient use of bit packs.
## Limitations

The current utility library does not support parsing /
packing all possible syntax of the source IANA tz database.
packing all possible syntax of the source IANA tz database.
Instead a subset corresponding to the what is needed to correctly parse
most zones is implemented.

Expand All @@ -32,3 +32,16 @@ most zones is implemented.
[zic man page and IANA tz database format documentation](https://linux.die.net/man/8/zic)

[vendored files](./vendor)

## Instructions to generate files (without Make)

1. Setup dev environment:
`python3 -m pip install -r requirements.txt`
2. Generate links based on major cities:
`python3 utils/compile_tzlinks.py`
3. Generate a list of timezones to include, based on major cities and timezones included in Android:
`python3 utils/compile_whitelist.py`
5. Generate `zones.h` and `zones.c`:
`python3 utils/generate_zones.py -d vendor/tzdata -r africa -r asia -r australasia -r backward -r europe -r northamerica -r pacificnew -r southamerica -w whitelist.txt -i majorcities`

Include different regions in step 5 based on your preferences
9 changes: 5 additions & 4 deletions examples/example.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,17 @@
* @author eV Quirk
*/

#include "utz.h"
#include "zones.h"
#include "../utz.h"
#include "../zones.h"

#include <stdio.h>
void main() {
void main()
{
printf("Total library db size: %d B\n", sizeof(zone_rules) + sizeof(zone_abrevs) + sizeof(zone_defns) + sizeof(zone_names));

udatetime_t dt = {0};
dt.date.year = 17;
dt.date.month = 9;
dt.date.month = 1;
dt.date.dayofmonth = 26;
dt.time.hour = 1;
dt.time.minute = 0;
Expand Down
64 changes: 64 additions & 0 deletions examples/example2.c
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
/** @file example2.c
* @brief Example of how to get local time from utc time and set TZ env
*
* @author Joshua Smith
*/

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>

#include "../utz.h"
#include "../zones.h"

static void copy_tm_to_udt(const struct tm *tm, udatetime_t *dt)
{
dt->date.year = tm->tm_year - 100;
dt->date.month = tm->tm_mon;
dt->date.dayofmonth = tm->tm_mday;
dt->date.dayofweek = tm->tm_wday;
dt->time.hour = tm->tm_hour;
dt->time.minute = tm->tm_min;
dt->time.second = tm->tm_sec;
}

static time_t uoffset_to_seconds(const uoffset_t *offset)
{
time_t seconds = offset->hours * 3600;
seconds += (offset->hours >= 0 ? offset->minutes : -offset->minutes) * 60;
return seconds;
}

int main()
{
// 1. Get seconds after epoch
time_t now = time(NULL);
printf("UTC TIME:\t\t%s", asctime(gmtime(&now)));

// 2. Get user selected zone
uzone_t active_zone;
get_zone_by_name("Halifax", &active_zone);

// 3. Calculate standard time, pretending it's the new UTC
time_t std_now = now + uoffset_to_seconds(&active_zone.offset);

// 4. Get tm struct and convert it to the proper udatetime_t type;
struct tm tm_std = *gmtime(&std_now);
udatetime_t dt;
copy_tm_to_udt(&tm_std, &dt);

// 5. Get offset based on standard time
uoffset_t offset;
char c = get_current_offset(&active_zone, &dt, &offset);

// Do what you want with the offset...

time_t true_now = now + uoffset_to_seconds(&offset);
printf("%s time:\t\t%s", active_zone.name, asctime(gmtime(&true_now)));
printf(active_zone.abrev_formatter, c);
printf("\n");

return 0;
}
33 changes: 33 additions & 0 deletions majorcities
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
Link Asia/Kolkata Asia/Delhi
Link Asia/Kolkata Asia/Mumbai
Link Asia/Shanghai Asia/Guangzhou
Link Asia/Shanghai Asia/Beijing
Link Asia/Tokyo Asia/Osaka
Link Asia/Shanghai Asia/Shenzhen
Link Asia/Shanghai Asia/Tianjin
Link Asia/Chongqing Asia/Chengdu
Link America/Sao_Paulo America/Rio_de_Janeiro
Link Asia/Karachi Asia/Lahore
Link Asia/Kolkata Asia/Bangalore
Link Asia/Kolkata Asia/Chennai
Link Asia/Tokyo Asia/Nagoya
Link Asia/Kolkata Asia/Hyderabad
Link Asia/Shanghai Asia/Wuhan
Link Asia/Shanghai Asia/Dongguan
Link Asia/Ho_Chi_Minh Asia/Hanoi
Link Africa/Lagos Africa/Onitsha
Link Asia/Kolkata Asia/Ahmedabad
Link America/New_York America/Washington,_D.C.
Link Asia/Chongqing Asia/Xi'an
Link America/New_York America/Boston
Link Asia/Shanghai Asia/Zhengzhou
Link Asia/Shanghai Asia/Shenyang
Link Asia/Shanghai Asia/Hangzhou
Link Europe/Berlin Europe/Dusseldorf
Link America/Chicago America/Dallas
Link Asia/Shanghai Asia/Quanzhou
Link Asia/Kolkata Asia/Surat
Link America/Chicago America/Houston
Link Asia/Kolkata Asia/Pune
Link Asia/Shanghai Asia/Nanjing
Link America/Los_Angeles America/San_Francisco
2 changes: 1 addition & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
BeautifulSoup
beautifulsoup4
click
geopy
tzwhere
40 changes: 21 additions & 19 deletions utils/compile_tzlinks.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,49 +1,51 @@
#!/usr/bin/env python
#!/usr/bin/env python3
""" Create tz database links for major metropolian areas that don't exist in the IANA db

eV Quirk
"""

import unicodedata

from BeautifulSoup import BeautifulSoup
from bs4 import BeautifulSoup
from geopy.geocoders import Nominatim
from tzwhere import tzwhere


def main():
geocoder = Nominatim()
geocoder = Nominatim(user_agent="utz", timeout=30)
tz = tzwhere.tzwhere()
links = []

with open('vendor/wikipedia/majormetros.html') as f:
soup = BeautifulSoup(f)
table = soup.find('table', {'class': "sortable wikitable"})
with open('vendor/wikipedia/majorcities.html') as f:
soup = BeautifulSoup(f, features="html.parser")
table = soup.find(
'table', {'class': "sortable wikitable mw-datatable"})
for row in table.findAll('tr'):
columns = row.findAll('td')
if columns:
metro = columns[1].find('a').text
country = columns[2].find('a').text
location = geocoder.geocode('%s, %s' % (metro, country))
if not location: # try just searching for just the metro
location = geocoder.geocode('%s' % metro)
city = columns[0].find('a').text
country = columns[1].findAll('a')[1].text
location = geocoder.geocode(f"{city}, {country}")
if not location: # try just searching for just the city
location = geocoder.geocode(city)
if location:
zone = tz.tzNameAt(location.latitude, location.longitude)
if zone:
metro = unicodedata.normalize('NFD', metro).encode('ascii', 'ignore')
metro = metro.replace(' ', '_')
if zone.split('/')[-1] not in metro:
city = unicodedata.normalize('NFD', city).encode(
'ascii', 'ignore').decode('ascii')
city = city.replace(' ', '_')
if zone.split('/')[-1] not in city:
alias = zone.split('/')[:-1]
alias.append(metro)
alias.append(city)
links.append(('Link', zone, '/'.join(alias)))
else:
print "%s, %s already present as %s" % (metro, country, zone)
print(f"{city}, {country} already present as {zone}")
else:
print "couldn't find zone for: %s, %s" % (metro, country)
print(f"couldn't find zone for: {city}, {country}")
else:
print "couldn't find location for: %s, %s" % (metro, country)
print(f"couldn't find location for: {city}, {country}")

with open('majormetros', 'w') as f:
with open('majorcities', 'w') as f:
f.write('\n'.join(['\t'.join(entry) for entry in links]))


Expand Down
2 changes: 1 addition & 1 deletion utils/compile_whitelist.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ def main():
tree = ElementTree.parse('vendor/android/timezones.xml')
zones.update([child.attrib['id'] for child in tree.getroot()])

with open('majormetros') as f:
with open('majorcities') as f:
for line in f:
zones.add(line.split('\t')[1].strip())
zones.add(line.split('\t')[2].strip())
Expand Down
2 changes: 1 addition & 1 deletion utils/example_strip_historical.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@


@click.command(context_settings=CONTEXT_SETTINGS)
@click.option('--dir', '-d', default=os.environ.get('TZ_DATA', 'tzdata'), help='Path to tzdata directory.')
@click.option('--dir', '-d', default=os.environ.get('TZ_DATA', 'vendor/tzdata'), help='Path to tzdata directory.')
@click.option('--build', '-b', default=os.environ.get('TZ_BUILD', 'build'), help='Path to build directory.')
def process(dir, build):
"""Sample script to strip historical timezones and rules"""
Expand Down
8 changes: 5 additions & 3 deletions utils/generate_zones.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env python
#!/usr/bin/env python3
""" Micro timezone generator

eV Quirk
Expand All @@ -7,13 +7,15 @@
import click
import os

from typing import List
from utz import TimeZoneDatabase

DEFAULT_REGIONS = "africa,asia,australasia,backward,europe,northamerica,pacificnew,southamerica"
CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])
H_NAME = 'zones.h'
C_NAME = 'zones.c'


@click.command(context_settings=CONTEXT_SETTINGS)
@click.option('--dir', '-d', default=os.environ.get('UTZ_DATA_DIR', 'tzdata'), help='Path to tzdata dir.')
@click.option('--region', '-r', default=os.environ.get('UTZ_REGIONS', DEFAULT_REGIONS).split(','),
Expand All @@ -23,7 +25,7 @@
@click.option('--whitelist', '-w',
default=os.environ.get('UTZ_WHITELIST', 'whitelist.txt'),
help='Zone whitelist.')
def process(dir, region, include, whitelist):
def process(dir: str, region: List[str], include: List[str], whitelist: str):
db = TimeZoneDatabase()

for r in region:
Expand All @@ -37,7 +39,7 @@ def process(dir, region, include, whitelist):

db.strip_historical()

included_zones = []
included_zones: List[str] = []
if whitelist:
with open(whitelist) as f:
for zone in f:
Expand Down
Loading