Index: infra_libs/time_functions/README.md |
diff --git a/infra_libs/time_functions/README.md b/infra_libs/time_functions/README.md |
deleted file mode 100644 |
index 4942e0c13972455d6cac6bed8e6d073d2de8d2d6..0000000000000000000000000000000000000000 |
--- a/infra_libs/time_functions/README.md |
+++ /dev/null |
@@ -1,341 +0,0 @@ |
-# A README ON TIME |
- |
-[TOC] |
- |
-## ABOUT THIS DOCUMENT |
- |
-This is a document intended to persuade you, in the course of your computer |
-doings, to use a particular representation of time called *stiptime*. Why a |
-particular representation of time, and why so staunch about it? The main reason |
-is that time is not handled well by libraries in any programming language, and |
-misuse leads to subtle bugs. Like memory allocators, representations of time are |
-not something application developers give much thought to. Unlike memory |
-allocators, you cannot expect your time libraries to "just work." This leads to |
-a wide class of bugs that are unintuitive, difficult to anticipate, and |
-difficult to test for. stiptime has been designed to avoid many of these |
-pitfalls. For a large number of time-related tasks, stiptime just works. |
- |
-If reading this document causes your head to spin, it has achieved its goal. Use |
-stiptime and go on with your day. |
- |
-## STIPTIME VS BIZARRO TIME |
- |
-### stiptime |
- |
-stiptime is a contemporary terrestrial time format meant to reduce the number of |
-time-related bugs in computer programs. It makes certain compromises to be |
-easier to implement on most operating systems and programming languages circa |
-2015-07-10T22:54:32.0Z. |
- |
-- stiptime is for absolute times: stiptime is meant to represent the absolute |
- (instead of relative) time an event happened. It is not intended for durations |
- or for local representations of time (it will not tell you where the sun is in |
- the sky). |
-- stiptime is terrestrial: it is not suitable for astronomical calculations or |
- events happening on Mars. It does not account for [relativistic time |
- dilation](https://en.wikipedia.org/wiki/Barycentric_Dynamical_Time) while |
- traveling through the solar system. It is not suitable for comparing clocks |
- across astronomically vast distances. |
-- stiptime is contemporary: it is not particularly useful for describing dates |
- in antiquity, such as anything before the invention of Greenwich Mean Time. |
-- stiptime is based on UTC, and uses UTC's concept of leap seconds. Many OSes |
- and date libraries [handle leap |
- seconds](https://tools.ietf.org/html/rfc7164#page-3) in a way that make |
- duration computations inaccurate across them. Unfortunately, continuous |
- timescales like [TAI](https://en.wikipedia.org/wiki/International_Atomic_Time) |
- are not easily available on modern OSes. In light of the difficulty of |
- obtaining TAI, stiptime compromises by being based on UTC and urges caution |
- when making duration computations. |
- |
-#### definition |
- |
-stiptime's format is UTC represented as follows: |
- |
- YYYY-MM-DDThh:mm:ssZ |
- |
-where |
- |
- YYYY: four digit year |
- MM: zero padded month |
- DD: zero padded day |
- hh: zero padded 24hr hour |
- mm: zero padded minute |
- ss: zero padded second, with a required fractional part |
- |
-You may notice that this is exactly the [ISO 8601 date |
-format](http://www.w3.org/TR/NOTE-datetime) with Z used to represent UTC. That's |
-because it is! Z is the [nautical |
-timezone](https://en.wikipedia.org/wiki/Nautical_time) for UTC. Since 'Zulu' is |
-the [NATO phonetic](https://en.wikipedia.org/wiki/NATO_phonetic_alphabet) |
-representation of Z, UTC (and by extension stiptime) can also be referred to as |
-Zulu time. Z is used instead of +00:00 as it unambiguously signals UTC and not |
-"put in any arbitrary timezone here." It is also short and compact. |
- |
-#### examples of stiptime |
- |
-- 2015-06-30T18:50:50.0Z *typical representation* |
-- 2015-06-30T18:50:50.123Z *fractional seconds* |
-- 2015-06-30T23:59:60.0Z *leap second* |
-- 2015-06-30T23:59:60.123Z *fractional leap second* |
- |
-#### lesser stiptime |
- |
-Lesser stiptime is to be used only when necessary. Lesser stiptime is Unix time |
-or POSIX time, "fractional seconds since the epoch, defined as |
-1970-01-01T00:00:00Z. Seconds are corrected such that days are exactly 86400 |
-seconds long." The reason stiptime is preferred over lesser stiptime is due to |
-that last correction. Since UTC occasionally contains days longer than 86400 |
-seconds, lesser stiptime cannot encode positive leap seconds unambiguously. The |
-consequences of this will be described in a later section. stiptime is preferred |
-over lesser stiptime, but lesser stiptime is definitely preferred over bizarro |
-time. |
- |
-### bizarro time |
- |
-Bizarro time is any time format that is used for absolute time that isn't |
-stiptime. Some of these may seem obvious and good, but a later section will show |
-their pitfalls. |
- |
-#### examples of bizarro time |
- |
-- Tomorrow, 3pm |
-- Tuesday 14, July 2015 4:38PM PST |
-- 2015-06-30T06:50:50.0PMZ *not 24hr time* |
-- 2015/06/30 18:50:50.0Z *incorrect separators* |
-- 2015-06-30T18:50:50.0 *no Z at the end, unknown timezone* |
-- 2015-06-30T18:50:50.0 PST *PST instead of Z* |
-- 2015-06-30T18:50:50.0 |
- [America/Los_Angeles](https://en.wikipedia.org/wiki/America/Los_Angeles) |
-- 2015-06-30T18:50:50.0+00:00 *using +00:00 instead of Z for UTC* |
-- 2015-06-30T18:50:50.0-07:00 *not using UTC* |
-- 2015-06-30T18:50:50Z *no fractional seconds* |
- |
-## STUCK IN TIME JAIL (THE PITFALLS OF BIZARRO TIME) |
- |
-Using bizarro time will eventually lead to "fun" and subtle bugs in your |
-programs. The inevitability of dealing with all of the contingencies of bizarro |
-time (and the libraries that handle it) leads to a profound frustration and |
-ennui — this is when you are stuck in *time jail*. The entrances to time jail |
-are many, but can be broadly classified into implementation-induced time jail |
-and timezone-induced time jail. |
- |
-### implementation-induced time jail |
- |
-Writing proper time-handling libraries is hard, which means that most time |
-libraries have quirks, unexpected behavior or outright bugs. Some of these even |
-occur at the operating system level. This section mostly describes the |
-[datetime](https://docs.python.org/2/library/datetime.html) module included in |
-the python standard library, but other quirks are documented as well. |
- |
-#### python 2.7 datetime |
- |
-- python 2.7 datetime lets you print a date that itself cannot parse |
- |
- |
- >>> import datetime |
- >>> from dateutil import tz |
- >>> offset = tz.tzoffset(None, -7*60*60) |
- >>> dt = datetime.datetime(2015, 1, 1, 0, 0, 0, tzinfo=offset) |
- >>> a = dt.strftime('%Y-%m-%dT%H:%M:%S %z') |
- >>> a |
- '2015-01-01T00:00:00 -0700' |
- >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S %z') |
- ValueError: 'z' is a bad directive in format '%Y-%m-%dT%H:%M:%S %z' |
- |
- |
-- it lets you print a date that it can parse most of the time, except that one |
- time when you have zero microseconds and then it can't |
- |
- See https://bugs.python.org/issue19475 for the problem, and see |
- [zulu.py](https://chromium.googlesource.com/infra/infra/+/master/infra_libs/time_functions/zulu.py) |
- for the 'solution.' |
- |
- |
- >>> import datetime |
- >>> a = datetime.datetime(2015, 1, 1, 0, 0, 0, 0).isoformat() |
- >>> b = datetime.datetime(2015, 1, 1, 0, 0, 0, 123).isoformat() |
- >>> a |
- '2015-01-01T00:00:00' |
- >>> b |
- '2015-01-01T00:00:00.000123' |
- >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S') |
- datetime.datetime(2015, 1, 1, 0, 0) |
- >>> datetime.datetime.strptime(b, '%Y-%m-%dT%H:%M:%S') |
- ValueError: unconverted data remains: .000123 |
- # okay, let's try with .%f at the end |
- >>> datetime.datetime.strptime(b, '%Y-%m-%dT%H:%M:%S.%f') |
- datetime.datetime(2015, 1, 1, 0, 0, 0, 123) |
- >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S.%f') |
- ValueError: time data '2015-01-01T00:00:00' does not match format '%Y-%m-%dT%H:%M:%S.%f' |
- |
-- it can't understand leap seconds |
- |
- |
- >>> import datetime |
- >>> datetime.datetime(2015, 6, 30, 23, 59, 60) |
- ValueError: second must be in 0..59 |
- |
-- it can't tell you what timezone datetime.now() is |
- |
- |
- >>> import datetime |
- >>> datetime.datetime.now().tzinfo is None |
- true |
- >>> datetime.datetime.now().utcoffset() is None |
- true |
- >>> datetime.datetime.utcnow().tzinfo is None |
- true |
- >>> datetime.datetime.utcnow().utcoffset() is None |
- true |
- >>> datetime.datetime.now().isoformat() |
- '2015-07-15T15:36:23.591431' |
- >>> datetime.datetime.utcnow().isoformat() |
- '2015-07-15T22:36:28.431225' |
- |
-- it can't mix timezone aware datetimes with naive datetimes (why have naive |
- datetimes to begin with?) |
- |
- |
- >>> import datetime |
- >>> from dateutil import tz |
- >>> a = datetime.datetime(2015, 1, 1, 0, 0, 0, tzinfo=tz.tzoffset(None, -7*60*60)) |
- >>> b = datetime.datetime(2015, 1, 1, 0, 0, 0) |
- >>> a == b |
- TypeError: can't compare offset-naive and offset-aware datetimes |
- |
-#### that's okay, I'll just use dateutil |
- |
-- [dateutil](http://labix.org/python-dateutil) is not part of the standard |
- library |
- |
- You'll have to venv or wheel it wherever you go. A (non-leap second aware) |
- stiptime parser is a single line of python and requires nothing more than the |
- standard library. A stiptime formatter is 4 lines, and also requires nothing |
- more than the standard library. |
- |
-- dateutil can't understand leap seconds |
- |
- |
- >>> import dateutil.parser |
- >>> dateutil.parser.parse('2015-06-30T23:59:60') |
- ValueError: second must be in 0..59 |
- |
-- parser.parse works great, except when it silently doesn't |
- |
- (note, running this example will yield different results depending on your |
- local timezone. lol.) |
- |
- |
- >>> import dateutil.parser |
- >>> a = dateutil.parser.parse('2015-01-01T00:00:00 PST') |
- >>> b = dateutil.parser.parse('2015-01-01T00:00:00 PDT') |
- >>> c = dateutil.parser.parse('2015-01-01T00:00:00 EST') |
- >>> a.isoformat() |
- '2015-01-01T00:00:00-08:00' # great! |
- >>> b.isoformat() |
- '2015-01-01T00:00:00-08:00' # uh... |
- >>> c.isoformat() |
- '2015-01-01T00:00:00' # uhhhhhhh |
- >>> c == a |
- TypeError: can't compare offset-naive and offset-aware datetimes |
- |
-#### python 2.7 time |
- |
-- time.time()'s definition is incorrect |
- |
- According to [the python docs](https://docs.python.org/2/library/time.html), |
- time.time() "[returns] the time in seconds since the epoch as a floating point |
- number." Except it doesn't, as (at least on unix) days are corrected to be |
- 86400 seconds long. Thus the true definition of time.time() should be "the |
- time in seconds since the epoch minus any UTC leap seconds added since the |
- epoch." |
- |
-### timezone-induced time jail |
- |
-These are a collection of gotchas that occur even if your timezone-handling |
-libraries are perfect. They arise purely out of not using UTC for internal |
-computations. |
- |
-- dates which represent the same moment in time can have different weekdays or |
- other attributes |
- |
- |
- >>> import dateutil.parser |
- >>> a = dateutil.parser.parse('2015-06-17T23:00:00 PDT') |
- >>> b = dateutil.parser.parse('2015-06-18T06:00:00 UTC') |
- >>> a == b |
- True |
- >>> a.weekday() |
- 2 |
- >>> b.weekday() |
- 3 |
- |
-- it's easy to write code thinking it's in one timezone when it's really in |
- another |
- |
- - pop quiz: what timezone does the AppEngine datastore store times in? |
- - pop quiz: what months does daylight savings take effect in Australia? North |
- America? |
- - pop quiz: what timezone does buildbot write twistd.log in? What timezone |
- does it write http.log in? |
- |
- |
-- you can have timezone un-aware code in a timezone that shifts (PST -> PDT). |
- |
- Now you have graphs wrapping back on themselves, systems restarting repeatedly |
- for an hour, or silent data corruption. This is the classic timezone bug. |
- |
-- ambiguous encoding |
- |
- If you're not careful, you can encode dates which refer to two instances in |
- time. The date '2015-11-01T01:30:00 Pacific' or '2015-11-01T01:30:00 |
- America/Los_Angeles' refers to *two* distinct times: |
- '2015-11-01T01:30:00-0800' and '2015-11-01T01:30:00-0700'. Imagine an alarm |
- clock or cron job which triggers on that. |
- |
-- illegal dates that aren't obviously illegal |
- |
- The opposite of ambiguous encoding: did you know that '2015-03-08T02:30:00 |
- Pacific' doesn't exist? It jumped from 2015-03-08T01:59:59 immediately to |
- 2015-03-08T03:00:00. |
- |
-- what does a configuration file look like where any timezone is allowed? |
- |
- You've now required everyone to convert every timezone into every timezone, |
- instead of every timezone into one (UTC): |
- |
- |
- ['2015-06-18T05:00:00+00:00' |
- '2015-06-17T23:00:00-07:00', |
- '2015-06-18T06:00:00-10:00', |
- ] |
- |
-### leap second time jail |
- |
-Finally, there is a small jail associated with errors occurring due to leap |
-seconds themselves. Unfortunately, stiptime is *not* immune to these. |
- |
-- ambiguous unix time |
- |
- |
- 2015-06-30T23:59:59.0Z -> 1435708799.0 |
- 2015-06-30T23:59:60.0Z -> 1435708799.0 |
- |
-- illegal unix time |
- |
- This hasn't happened yet, but if a negative leap second ever occurs, there |
- will be a floating point time which 'never happened.' |
- |
-- calculating durations using start/stop times |
- |
- Of course, calculating durations across leap seconds can cause slight errors, |
- mistriggers or even retriggers. Since they happen infrequently (and TAI is not |
- commonly available), stiptime has chosen to be susceptible. |
- |
-## CONCLUSION |
- |
-It is my hope that after reading this document, you will consider stiptime and, |
-occasionally, lesser stiptime to be the proper way to represent time in stored |
-formats. I firmly believe time should be displayed to humans in their local |
-format — but only in ephemeral displays. A local absolute time should never |
-touch a disk. Join me in using stiptime and get back some sanity in your life. |