Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(134)

Side by Side Diff: infra_libs/time_functions/README.md

Issue 2213143002: Add infra_libs as a bootstrap dependency. (Closed) Base URL: https://chromium.googlesource.com/infra/infra.git@master
Patch Set: Removed the ugly import hack Created 4 years, 4 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch
OLDNEW
(Empty)
1 # A README ON TIME
2
3 [TOC]
4
5 ## ABOUT THIS DOCUMENT
6
7 This is a document intended to persuade you, in the course of your computer
8 doings, to use a particular representation of time called *stiptime*. Why a
9 particular representation of time, and why so staunch about it? The main reason
10 is that time is not handled well by libraries in any programming language, and
11 misuse leads to subtle bugs. Like memory allocators, representations of time are
12 not something application developers give much thought to. Unlike memory
13 allocators, you cannot expect your time libraries to "just work." This leads to
14 a wide class of bugs that are unintuitive, difficult to anticipate, and
15 difficult to test for. stiptime has been designed to avoid many of these
16 pitfalls. For a large number of time-related tasks, stiptime just works.
17
18 If reading this document causes your head to spin, it has achieved its goal. Use
19 stiptime and go on with your day.
20
21 ## STIPTIME VS BIZARRO TIME
22
23 ### stiptime
24
25 stiptime is a contemporary terrestrial time format meant to reduce the number of
26 time-related bugs in computer programs. It makes certain compromises to be
27 easier to implement on most operating systems and programming languages circa
28 2015-07-10T22:54:32.0Z.
29
30 - stiptime is for absolute times: stiptime is meant to represent the absolute
31 (instead of relative) time an event happened. It is not intended for durations
32 or for local representations of time (it will not tell you where the sun is in
33 the sky).
34 - stiptime is terrestrial: it is not suitable for astronomical calculations or
35 events happening on Mars. It does not account for [relativistic time
36 dilation](https://en.wikipedia.org/wiki/Barycentric_Dynamical_Time) while
37 traveling through the solar system. It is not suitable for comparing clocks
38 across astronomically vast distances.
39 - stiptime is contemporary: it is not particularly useful for describing dates
40 in antiquity, such as anything before the invention of Greenwich Mean Time.
41 - stiptime is based on UTC, and uses UTC's concept of leap seconds. Many OSes
42 and date libraries [handle leap
43 seconds](https://tools.ietf.org/html/rfc7164#page-3) in a way that make
44 duration computations inaccurate across them. Unfortunately, continuous
45 timescales like [TAI](https://en.wikipedia.org/wiki/International_Atomic_Time)
46 are not easily available on modern OSes. In light of the difficulty of
47 obtaining TAI, stiptime compromises by being based on UTC and urges caution
48 when making duration computations.
49
50 #### definition
51
52 stiptime's format is UTC represented as follows:
53
54 YYYY-MM-DDThh:mm:ssZ
55
56 where
57
58 YYYY: four digit year
59 MM: zero padded month
60 DD: zero padded day
61 hh: zero padded 24hr hour
62 mm: zero padded minute
63 ss: zero padded second, with a required fractional part
64
65 You may notice that this is exactly the [ISO 8601 date
66 format](http://www.w3.org/TR/NOTE-datetime) with Z used to represent UTC. That's
67 because it is! Z is the [nautical
68 timezone](https://en.wikipedia.org/wiki/Nautical_time) for UTC. Since 'Zulu' is
69 the [NATO phonetic](https://en.wikipedia.org/wiki/NATO_phonetic_alphabet)
70 representation of Z, UTC (and by extension stiptime) can also be referred to as
71 Zulu time. Z is used instead of +00:00 as it unambiguously signals UTC and not
72 "put in any arbitrary timezone here." It is also short and compact.
73
74 #### examples of stiptime
75
76 - 2015-06-30T18:50:50.0Z *typical representation*
77 - 2015-06-30T18:50:50.123Z *fractional seconds*
78 - 2015-06-30T23:59:60.0Z *leap second*
79 - 2015-06-30T23:59:60.123Z *fractional leap second*
80
81 #### lesser stiptime
82
83 Lesser stiptime is to be used only when necessary. Lesser stiptime is Unix time
84 or POSIX time, "fractional seconds since the epoch, defined as
85 1970-01-01T00:00:00Z. Seconds are corrected such that days are exactly 86400
86 seconds long." The reason stiptime is preferred over lesser stiptime is due to
87 that last correction. Since UTC occasionally contains days longer than 86400
88 seconds, lesser stiptime cannot encode positive leap seconds unambiguously. The
89 consequences of this will be described in a later section. stiptime is preferred
90 over lesser stiptime, but lesser stiptime is definitely preferred over bizarro
91 time.
92
93 ### bizarro time
94
95 Bizarro time is any time format that is used for absolute time that isn't
96 stiptime. Some of these may seem obvious and good, but a later section will show
97 their pitfalls.
98
99 #### examples of bizarro time
100
101 - Tomorrow, 3pm
102 - Tuesday 14, July 2015 4:38PM PST
103 - 2015-06-30T06:50:50.0PMZ *not 24hr time*
104 - 2015/06/30 18:50:50.0Z *incorrect separators*
105 - 2015-06-30T18:50:50.0 *no Z at the end, unknown timezone*
106 - 2015-06-30T18:50:50.0 PST *PST instead of Z*
107 - 2015-06-30T18:50:50.0
108 [America/Los_Angeles](https://en.wikipedia.org/wiki/America/Los_Angeles)
109 - 2015-06-30T18:50:50.0+00:00 *using +00:00 instead of Z for UTC*
110 - 2015-06-30T18:50:50.0-07:00 *not using UTC*
111 - 2015-06-30T18:50:50Z *no fractional seconds*
112
113 ## STUCK IN TIME JAIL (THE PITFALLS OF BIZARRO TIME)
114
115 Using bizarro time will eventually lead to "fun" and subtle bugs in your
116 programs. The inevitability of dealing with all of the contingencies of bizarro
117 time (and the libraries that handle it) leads to a profound frustration and
118 ennui — this is when you are stuck in *time jail*. The entrances to time jail
119 are many, but can be broadly classified into implementation-induced time jail
120 and timezone-induced time jail.
121
122 ### implementation-induced time jail
123
124 Writing proper time-handling libraries is hard, which means that most time
125 libraries have quirks, unexpected behavior or outright bugs. Some of these even
126 occur at the operating system level. This section mostly describes the
127 [datetime](https://docs.python.org/2/library/datetime.html) module included in
128 the python standard library, but other quirks are documented as well.
129
130 #### python 2.7 datetime
131
132 - python 2.7 datetime lets you print a date that itself cannot parse
133
134
135 >>> import datetime
136 >>> from dateutil import tz
137 >>> offset = tz.tzoffset(None, -7*60*60)
138 >>> dt = datetime.datetime(2015, 1, 1, 0, 0, 0, tzinfo=offset)
139 >>> a = dt.strftime('%Y-%m-%dT%H:%M:%S %z')
140 >>> a
141 '2015-01-01T00:00:00 -0700'
142 >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S %z')
143 ValueError: 'z' is a bad directive in format '%Y-%m-%dT%H:%M:%S %z'
144
145
146 - it lets you print a date that it can parse most of the time, except that one
147 time when you have zero microseconds and then it can't
148
149 See https://bugs.python.org/issue19475 for the problem, and see
150 [zulu.py](https://chromium.googlesource.com/infra/infra/+/master/infra_libs/ti me_functions/zulu.py)
151 for the 'solution.'
152
153
154 >>> import datetime
155 >>> a = datetime.datetime(2015, 1, 1, 0, 0, 0, 0).isoformat()
156 >>> b = datetime.datetime(2015, 1, 1, 0, 0, 0, 123).isoformat()
157 >>> a
158 '2015-01-01T00:00:00'
159 >>> b
160 '2015-01-01T00:00:00.000123'
161 >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S')
162 datetime.datetime(2015, 1, 1, 0, 0)
163 >>> datetime.datetime.strptime(b, '%Y-%m-%dT%H:%M:%S')
164 ValueError: unconverted data remains: .000123
165 # okay, let's try with .%f at the end
166 >>> datetime.datetime.strptime(b, '%Y-%m-%dT%H:%M:%S.%f')
167 datetime.datetime(2015, 1, 1, 0, 0, 0, 123)
168 >>> datetime.datetime.strptime(a, '%Y-%m-%dT%H:%M:%S.%f')
169 ValueError: time data '2015-01-01T00:00:00' does not match format '%Y-%m -%dT%H:%M:%S.%f'
170
171 - it can't understand leap seconds
172
173
174 >>> import datetime
175 >>> datetime.datetime(2015, 6, 30, 23, 59, 60)
176 ValueError: second must be in 0..59
177
178 - it can't tell you what timezone datetime.now() is
179
180
181 >>> import datetime
182 >>> datetime.datetime.now().tzinfo is None
183 true
184 >>> datetime.datetime.now().utcoffset() is None
185 true
186 >>> datetime.datetime.utcnow().tzinfo is None
187 true
188 >>> datetime.datetime.utcnow().utcoffset() is None
189 true
190 >>> datetime.datetime.now().isoformat()
191 '2015-07-15T15:36:23.591431'
192 >>> datetime.datetime.utcnow().isoformat()
193 '2015-07-15T22:36:28.431225'
194
195 - it can't mix timezone aware datetimes with naive datetimes (why have naive
196 datetimes to begin with?)
197
198
199 >>> import datetime
200 >>> from dateutil import tz
201 >>> a = datetime.datetime(2015, 1, 1, 0, 0, 0, tzinfo=tz.tzoffset(None, -7*60*60))
202 >>> b = datetime.datetime(2015, 1, 1, 0, 0, 0)
203 >>> a == b
204 TypeError: can't compare offset-naive and offset-aware datetimes
205
206 #### that's okay, I'll just use dateutil
207
208 - [dateutil](http://labix.org/python-dateutil) is not part of the standard
209 library
210
211 You'll have to venv or wheel it wherever you go. A (non-leap second aware)
212 stiptime parser is a single line of python and requires nothing more than the
213 standard library. A stiptime formatter is 4 lines, and also requires nothing
214 more than the standard library.
215
216 - dateutil can't understand leap seconds
217
218
219 >>> import dateutil.parser
220 >>> dateutil.parser.parse('2015-06-30T23:59:60')
221 ValueError: second must be in 0..59
222
223 - parser.parse works great, except when it silently doesn't
224
225 (note, running this example will yield different results depending on your
226 local timezone. lol.)
227
228
229 >>> import dateutil.parser
230 >>> a = dateutil.parser.parse('2015-01-01T00:00:00 PST')
231 >>> b = dateutil.parser.parse('2015-01-01T00:00:00 PDT')
232 >>> c = dateutil.parser.parse('2015-01-01T00:00:00 EST')
233 >>> a.isoformat()
234 '2015-01-01T00:00:00-08:00' # great!
235 >>> b.isoformat()
236 '2015-01-01T00:00:00-08:00' # uh...
237 >>> c.isoformat()
238 '2015-01-01T00:00:00' # uhhhhhhh
239 >>> c == a
240 TypeError: can't compare offset-naive and offset-aware datetimes
241
242 #### python 2.7 time
243
244 - time.time()'s definition is incorrect
245
246 According to [the python docs](https://docs.python.org/2/library/time.html),
247 time.time() "[returns] the time in seconds since the epoch as a floating point
248 number." Except it doesn't, as (at least on unix) days are corrected to be
249 86400 seconds long. Thus the true definition of time.time() should be "the
250 time in seconds since the epoch minus any UTC leap seconds added since the
251 epoch."
252
253 ### timezone-induced time jail
254
255 These are a collection of gotchas that occur even if your timezone-handling
256 libraries are perfect. They arise purely out of not using UTC for internal
257 computations.
258
259 - dates which represent the same moment in time can have different weekdays or
260 other attributes
261
262
263 >>> import dateutil.parser
264 >>> a = dateutil.parser.parse('2015-06-17T23:00:00 PDT')
265 >>> b = dateutil.parser.parse('2015-06-18T06:00:00 UTC')
266 >>> a == b
267 True
268 >>> a.weekday()
269 2
270 >>> b.weekday()
271 3
272
273 - it's easy to write code thinking it's in one timezone when it's really in
274 another
275
276 - pop quiz: what timezone does the AppEngine datastore store times in?
277 - pop quiz: what months does daylight savings take effect in Australia? North
278 America?
279 - pop quiz: what timezone does buildbot write twistd.log in? What timezone
280 does it write http.log in?
281
282
283 - you can have timezone un-aware code in a timezone that shifts (PST -> PDT).
284
285 Now you have graphs wrapping back on themselves, systems restarting repeatedly
286 for an hour, or silent data corruption. This is the classic timezone bug.
287
288 - ambiguous encoding
289
290 If you're not careful, you can encode dates which refer to two instances in
291 time. The date '2015-11-01T01:30:00 Pacific' or '2015-11-01T01:30:00
292 America/Los_Angeles' refers to *two* distinct times:
293 '2015-11-01T01:30:00-0800' and '2015-11-01T01:30:00-0700'. Imagine an alarm
294 clock or cron job which triggers on that.
295
296 - illegal dates that aren't obviously illegal
297
298 The opposite of ambiguous encoding: did you know that '2015-03-08T02:30:00
299 Pacific' doesn't exist? It jumped from 2015-03-08T01:59:59 immediately to
300 2015-03-08T03:00:00.
301
302 - what does a configuration file look like where any timezone is allowed?
303
304 You've now required everyone to convert every timezone into every timezone,
305 instead of every timezone into one (UTC):
306
307
308 ['2015-06-18T05:00:00+00:00'
309 '2015-06-17T23:00:00-07:00',
310 '2015-06-18T06:00:00-10:00',
311 ]
312
313 ### leap second time jail
314
315 Finally, there is a small jail associated with errors occurring due to leap
316 seconds themselves. Unfortunately, stiptime is *not* immune to these.
317
318 - ambiguous unix time
319
320
321 2015-06-30T23:59:59.0Z -> 1435708799.0
322 2015-06-30T23:59:60.0Z -> 1435708799.0
323
324 - illegal unix time
325
326 This hasn't happened yet, but if a negative leap second ever occurs, there
327 will be a floating point time which 'never happened.'
328
329 - calculating durations using start/stop times
330
331 Of course, calculating durations across leap seconds can cause slight errors,
332 mistriggers or even retriggers. Since they happen infrequently (and TAI is not
333 commonly available), stiptime has chosen to be susceptible.
334
335 ## CONCLUSION
336
337 It is my hope that after reading this document, you will consider stiptime and,
338 occasionally, lesser stiptime to be the proper way to represent time in stored
339 formats. I firmly believe time should be displayed to humans in their local
340 format — but only in ephemeral displays. A local absolute time should never
341 touch a disk. Join me in using stiptime and get back some sanity in your life.
OLDNEW

Powered by Google App Engine
This is Rietveld 408576698