OLD | NEW |
| (Empty) |
1 # A README ON TIME | |
2 | |
3 [TOC] | |
4 | |
5 ## ABOUT THIS DOCUMENT | |
6 | |
7 This is a document intended to persuade you, in the course of your computer | |
8 doings, to use a particular representation of time called *stiptime*. Why a | |
9 particular representation of time, and why so staunch about it? The main reason | |
10 is that time is not handled well by libraries in any programming language, and | |
11 misuse leads to subtle bugs. Like memory allocators, representations of time are | |
12 not something application developers give much thought to. Unlike memory | |
13 allocators, you cannot expect your time libraries to "just work." This leads to | |
14 a wide class of bugs that are unintuitive, difficult to anticipate, and | |
15 difficult to test for. stiptime has been designed to avoid many of these | |
16 pitfalls. For a large number of time-related tasks, stiptime just works. | |
17 | |
18 If reading this document causes your head to spin, it has achieved its goal. Use | |
19 stiptime and go on with your day. | |
20 | |
21 ## STIPTIME VS BIZARRO TIME | |
22 | |
23 ### stiptime | |
24 | |
25 stiptime is a contemporary terrestrial time format meant to reduce the number of | |
26 time-related bugs in computer programs. It makes certain compromises to be | |
27 easier to implement on most operating systems and programming languages circa | |
28 2015-07-10T22:54:32.0Z. | |
29 | |
30 - stiptime is for absolute times: stiptime is meant to represent the absolute | |
31 (instead of relative) time an event happened. It is not intended for durations | |
32 or for local representations of time (it will not tell you where the sun is in | |
33 the sky). | |
34 - stiptime is terrestrial: it is not suitable for astronomical calculations or | |
35 events happening on Mars. It does not account for [relativistic time | |
36 dilation](https://en.wikipedia.org/wiki/Barycentric_Dynamical_Time) while | |
37 traveling through the solar system. It is not suitable for comparing clocks | |
38 across astronomically vast distances. | |
39 - stiptime is contemporary: it is not particularly useful for describing dates | |
40 in antiquity, such as anything before the invention of Greenwich Mean Time. | |
41 - stiptime is based on UTC, and uses UTC's concept of leap seconds. Many OSes | |
42 and date libraries [handle leap | |
43 seconds](https://tools.ietf.org/html/rfc7164#page-3) in a way that make | |
44 duration computations inaccurate across them. Unfortunately, continuous | |
45 timescales like [TAI](https://en.wikipedia.org/wiki/International_Atomic_Time) | |
46 are not easily available on modern OSes. In light of the difficulty of | |
47 obtaining TAI, stiptime compromises by being based on UTC and urges caution | |
48 when making duration computations. | |
49 | |
50 #### definition | |
51 | |
52 stiptime's format is UTC represented as follows: | |
53 | |
54 YYYY-MM-DDThh:mm:ssZ | |
55 | |
56 where | |
57 | |
58 YYYY: four digit year | |
59 MM: zero padded month | |
60 DD: zero padded day | |
61 hh: zero padded 24hr hour | |
62 mm: zero padded minute | |
63 ss: zero padded second, with a required fractional part | |
64 | |
65 You may notice that this is exactly the [ISO 8601 date | |
66 format](http://www.w3.org/TR/NOTE-datetime) with Z used to represent UTC. That's | |
67 because it is! Z is the [nautical | |
68 timezone](https://en.wikipedia.org/wiki/Nautical_time) for UTC. Since 'Zulu' is | |
69 the [NATO phonetic](https://en.wikipedia.org/wiki/NATO_phonetic_alphabet) | |
70 representation of Z, UTC (and by extension stiptime) can also be referred to as | |
71 Zulu time. Z is used instead of +00:00 as it unambiguously signals UTC and not | |
72 "put in any arbitrary timezone here." It is also short and compact. | |
73 | |
74 #### examples of stiptime | |
75 | |
76 - 2015-06-30T18:50:50.0Z *typical representation* | |
77 - 2015-06-30T18:50:50.123Z *fractional seconds* | |
78 - 2015-06-30T23:59:60.0Z *leap second* | |
79 - 2015-06-30T23:59:60.123Z *fractional leap second* | |
80 | |
81 #### lesser stiptime | |
82 | |
83 Lesser stiptime is to be used only when necessary. Lesser stiptime is Unix time | |
84 or POSIX time, "fractional seconds since the epoch, defined as | |
85 1970-01-01T00:00:00Z. Seconds are corrected such that days are exactly 86400 | |
86 seconds long." The reason stiptime is preferred over lesser stiptime is due to | |
87 that last correction. Since UTC occasionally contains days longer than 86400 | |
88 seconds, lesser stiptime cannot encode positive leap seconds unambiguously. The | |
89 consequences of this will be described in a later section. stiptime is preferred | |
90 over lesser stiptime, but lesser stiptime is definitely preferred over bizarro | |
91 time. | |
92 | |
93 ### bizarro time | |
94 | |
95 Bizarro time is any time format that is used for absolute time that isn't | |
96 stiptime. Some of these may seem obvious and good, but a later section will show | |
97 their pitfalls. | |
98 | |
99 #### examples of bizarro time | |
100 | |
101 - Tomorrow, 3pm | |
102 - Tuesday 14, July 2015 4:38PM PST | |
103 - 2015-06-30T06:50:50.0PMZ *not 24hr time* | |
104 - 2015/06/30 18:50:50.0Z *incorrect separators* | |
105 - 2015-06-30T18:50:50.0 *no Z at the end, unknown timezone* | |
106 - 2015-06-30T18:50:50.0 PST *PST instead of Z* | |
107 - 2015-06-30T18:50:50.0 | |
108 [America/Los_Angeles](https://en.wikipedia.org/wiki/America/Los_Angeles) | |
109 - 2015-06-30T18:50:50.0+00:00 *using +00:00 instead of Z for UTC* | |
110 - 2015-06-30T18:50:50.0-07:00 *not using UTC* | |
111 - 2015-06-30T18:50:50Z *no fractional seconds* | |
112 | |
113 ## STUCK IN TIME JAIL (THE PITFALLS OF BIZARRO TIME) | |
114 | |
115 Using bizarro time will eventually lead to "fun" and subtle bugs in your | |
116 programs. The inevitability of dealing with all of the contingencies of bizarro | |
117 time (and the libraries that handle it) leads to a profound frustration and | |
118 ennui — this is when you are stuck in *time jail*. The entrances to time jail | |
119 are many, but can be broadly classified into implementation-induced time jail | |
120 and timezone-induced time jail. | |
121 | |
122 ### implementation-induced time jail | |
123 | |
124 Writing proper time-handling libraries is hard, which means that most time | |
125 libraries have quirks, unexpected behavior or outright bugs. Some of these even | |
126 occur at the operating system level. This section mostly describes the | |
127 [datetime](https://docs.python.org/2/library/datetime.html) module included in | |
128 the python standard library, but other quirks are documented as well. | |
129 | |
130 #### python 2.7 datetime | |
131 | |
132 - python 2.7 datetime lets you print a date that itself cannot parse | |
133 | |
134 | |
135 >>> import datetime | |
136 >>> from dateutil import tz | |
137 >>> offset = tz.tzoffset(None, -7*60*60) | |
138 >>> dt = datetime.datetime(2015, 1, 1, 0, 0, 0, tzinfo=offset) | |
139 >>> a = dt.strftime('%Y-%m-%dT%H:%M:%S %z') | |
140 >>> a | |
141 '2015-01-01T00:00:00 -0700' | |
142 >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S %z') | |
143 ValueError: 'z' is a bad directive in format '%Y-%m-%dT%H:%M:%S %z' | |
144 | |
145 | |
146 - it lets you print a date that it can parse most of the time, except that one | |
147 time when you have zero microseconds and then it can't | |
148 | |
149 See https://bugs.python.org/issue19475 for the problem, and see | |
150 [zulu.py](https://chromium.googlesource.com/infra/infra/+/master/infra_libs/ti
me_functions/zulu.py) | |
151 for the 'solution.' | |
152 | |
153 | |
154 >>> import datetime | |
155 >>> a = datetime.datetime(2015, 1, 1, 0, 0, 0, 0).isoformat() | |
156 >>> b = datetime.datetime(2015, 1, 1, 0, 0, 0, 123).isoformat() | |
157 >>> a | |
158 '2015-01-01T00:00:00' | |
159 >>> b | |
160 '2015-01-01T00:00:00.000123' | |
161 >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S') | |
162 datetime.datetime(2015, 1, 1, 0, 0) | |
163 >>> datetime.datetime.strptime(b, '%Y-%m-%dT%H:%M:%S') | |
164 ValueError: unconverted data remains: .000123 | |
165 # okay, let's try with .%f at the end | |
166 >>> datetime.datetime.strptime(b, '%Y-%m-%dT%H:%M:%S.%f') | |
167 datetime.datetime(2015, 1, 1, 0, 0, 0, 123) | |
168 >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S.%f') | |
169 ValueError: time data '2015-01-01T00:00:00' does not match format '%Y-%m
-%dT%H:%M:%S.%f' | |
170 | |
171 - it can't understand leap seconds | |
172 | |
173 | |
174 >>> import datetime | |
175 >>> datetime.datetime(2015, 6, 30, 23, 59, 60) | |
176 ValueError: second must be in 0..59 | |
177 | |
178 - it can't tell you what timezone datetime.now() is | |
179 | |
180 | |
181 >>> import datetime | |
182 >>> datetime.datetime.now().tzinfo is None | |
183 true | |
184 >>> datetime.datetime.now().utcoffset() is None | |
185 true | |
186 >>> datetime.datetime.utcnow().tzinfo is None | |
187 true | |
188 >>> datetime.datetime.utcnow().utcoffset() is None | |
189 true | |
190 >>> datetime.datetime.now().isoformat() | |
191 '2015-07-15T15:36:23.591431' | |
192 >>> datetime.datetime.utcnow().isoformat() | |
193 '2015-07-15T22:36:28.431225' | |
194 | |
195 - it can't mix timezone aware datetimes with naive datetimes (why have naive | |
196 datetimes to begin with?) | |
197 | |
198 | |
199 >>> import datetime | |
200 >>> from dateutil import tz | |
201 >>> a = datetime.datetime(2015, 1, 1, 0, 0, 0, tzinfo=tz.tzoffset(None,
-7*60*60)) | |
202 >>> b = datetime.datetime(2015, 1, 1, 0, 0, 0) | |
203 >>> a == b | |
204 TypeError: can't compare offset-naive and offset-aware datetimes | |
205 | |
206 #### that's okay, I'll just use dateutil | |
207 | |
208 - [dateutil](http://labix.org/python-dateutil) is not part of the standard | |
209 library | |
210 | |
211 You'll have to venv or wheel it wherever you go. A (non-leap second aware) | |
212 stiptime parser is a single line of python and requires nothing more than the | |
213 standard library. A stiptime formatter is 4 lines, and also requires nothing | |
214 more than the standard library. | |
215 | |
216 - dateutil can't understand leap seconds | |
217 | |
218 | |
219 >>> import dateutil.parser | |
220 >>> dateutil.parser.parse('2015-06-30T23:59:60') | |
221 ValueError: second must be in 0..59 | |
222 | |
223 - parser.parse works great, except when it silently doesn't | |
224 | |
225 (note, running this example will yield different results depending on your | |
226 local timezone. lol.) | |
227 | |
228 | |
229 >>> import dateutil.parser | |
230 >>> a = dateutil.parser.parse('2015-01-01T00:00:00 PST') | |
231 >>> b = dateutil.parser.parse('2015-01-01T00:00:00 PDT') | |
232 >>> c = dateutil.parser.parse('2015-01-01T00:00:00 EST') | |
233 >>> a.isoformat() | |
234 '2015-01-01T00:00:00-08:00' # great! | |
235 >>> b.isoformat() | |
236 '2015-01-01T00:00:00-08:00' # uh... | |
237 >>> c.isoformat() | |
238 '2015-01-01T00:00:00' # uhhhhhhh | |
239 >>> c == a | |
240 TypeError: can't compare offset-naive and offset-aware datetimes | |
241 | |
242 #### python 2.7 time | |
243 | |
244 - time.time()'s definition is incorrect | |
245 | |
246 According to [the python docs](https://docs.python.org/2/library/time.html), | |
247 time.time() "[returns] the time in seconds since the epoch as a floating point | |
248 number." Except it doesn't, as (at least on unix) days are corrected to be | |
249 86400 seconds long. Thus the true definition of time.time() should be "the | |
250 time in seconds since the epoch minus any UTC leap seconds added since the | |
251 epoch." | |
252 | |
253 ### timezone-induced time jail | |
254 | |
255 These are a collection of gotchas that occur even if your timezone-handling | |
256 libraries are perfect. They arise purely out of not using UTC for internal | |
257 computations. | |
258 | |
259 - dates which represent the same moment in time can have different weekdays or | |
260 other attributes | |
261 | |
262 | |
263 >>> import dateutil.parser | |
264 >>> a = dateutil.parser.parse('2015-06-17T23:00:00 PDT') | |
265 >>> b = dateutil.parser.parse('2015-06-18T06:00:00 UTC') | |
266 >>> a == b | |
267 True | |
268 >>> a.weekday() | |
269 2 | |
270 >>> b.weekday() | |
271 3 | |
272 | |
273 - it's easy to write code thinking it's in one timezone when it's really in | |
274 another | |
275 | |
276 - pop quiz: what timezone does the AppEngine datastore store times in? | |
277 - pop quiz: what months does daylight savings take effect in Australia? North | |
278 America? | |
279 - pop quiz: what timezone does buildbot write twistd.log in? What timezone | |
280 does it write http.log in? | |
281 | |
282 | |
283 - you can have timezone un-aware code in a timezone that shifts (PST -> PDT). | |
284 | |
285 Now you have graphs wrapping back on themselves, systems restarting repeatedly | |
286 for an hour, or silent data corruption. This is the classic timezone bug. | |
287 | |
288 - ambiguous encoding | |
289 | |
290 If you're not careful, you can encode dates which refer to two instances in | |
291 time. The date '2015-11-01T01:30:00 Pacific' or '2015-11-01T01:30:00 | |
292 America/Los_Angeles' refers to *two* distinct times: | |
293 '2015-11-01T01:30:00-0800' and '2015-11-01T01:30:00-0700'. Imagine an alarm | |
294 clock or cron job which triggers on that. | |
295 | |
296 - illegal dates that aren't obviously illegal | |
297 | |
298 The opposite of ambiguous encoding: did you know that '2015-03-08T02:30:00 | |
299 Pacific' doesn't exist? It jumped from 2015-03-08T01:59:59 immediately to | |
300 2015-03-08T03:00:00. | |
301 | |
302 - what does a configuration file look like where any timezone is allowed? | |
303 | |
304 You've now required everyone to convert every timezone into every timezone, | |
305 instead of every timezone into one (UTC): | |
306 | |
307 | |
308 ['2015-06-18T05:00:00+00:00' | |
309 '2015-06-17T23:00:00-07:00', | |
310 '2015-06-18T06:00:00-10:00', | |
311 ] | |
312 | |
313 ### leap second time jail | |
314 | |
315 Finally, there is a small jail associated with errors occurring due to leap | |
316 seconds themselves. Unfortunately, stiptime is *not* immune to these. | |
317 | |
318 - ambiguous unix time | |
319 | |
320 | |
321 2015-06-30T23:59:59.0Z -> 1435708799.0 | |
322 2015-06-30T23:59:60.0Z -> 1435708799.0 | |
323 | |
324 - illegal unix time | |
325 | |
326 This hasn't happened yet, but if a negative leap second ever occurs, there | |
327 will be a floating point time which 'never happened.' | |
328 | |
329 - calculating durations using start/stop times | |
330 | |
331 Of course, calculating durations across leap seconds can cause slight errors, | |
332 mistriggers or even retriggers. Since they happen infrequently (and TAI is not | |
333 commonly available), stiptime has chosen to be susceptible. | |
334 | |
335 ## CONCLUSION | |
336 | |
337 It is my hope that after reading this document, you will consider stiptime and, | |
338 occasionally, lesser stiptime to be the proper way to represent time in stored | |
339 formats. I firmly believe time should be displayed to humans in their local | |
340 format — but only in ephemeral displays. A local absolute time should never | |
341 touch a disk. Join me in using stiptime and get back some sanity in your life. | |
OLD | NEW |