| OLD | NEW |
|---|
| 1 Datastore implementation internals | 1 Datastore implementation internals |
| 2 ================================== | 2 ================================== |
| 3 | 3 |
| 4 This document contains internal implementation details for this in-memory | 4 This document contains internal implementation details for this in-memory |
| 5 version of datastore. It's mostly helpful to understand what's going on in this | 5 version of datastore. It's mostly helpful to understand what's going on in this |
| 6 implementation, but it also can reveal some insight into how the real appengine | 6 implementation, but it also can reveal some insight into how the real appengine |
| 7 datastore works (though note that the specific encodings are different). | 7 datastore works (though note that the specific encodings are different). |
| 8 | 8 |
| 9 Additionally note that this implementation cheats by moving some of the Key | 9 Additionally note that this implementation cheats by moving some of the Key |
| 10 bytes into the table (collection) names (like the namespace, property name for | 10 bytes into the table (collection) names (like the namespace, property name for |
| 11 the builtin indexes, etc.). The real implementation contains these bytes in the | 11 the builtin indexes, etc.). The real implementation contains these bytes in the |
| 12 table row keys, I think. | 12 table row keys, I think. |
| 13 | 13 |
| 14 | 14 |
| 15 Internal datastore key/value collection schema | 15 Internal datastore key/value collection schema |
| 16 ---------------------------------------------- | 16 ---------------------------------------------- |
| 17 | 17 |
| 18 The datastore implementation here uses several different tables ('collections') | 18 The datastore implementation here uses several different tables ('collections') |
| 19 to manage state for the data. The schema for these tables is enumerated below | 19 to manage state for the data. The schema for these tables is enumerated below |
| 20 to make the code a bit easier to reason about. | 20 to make the code a bit easier to reason about. |
| 21 | 21 |
| 22 All datastore user objects (Keys, Properties, PropertyMaps, etc.) are serialized | 22 All datastore user objects (Keys, Properties, PropertyMaps, etc.) are serialized |
| 23 using `github.com/luci/gae/service/datastore/serialize`, which in turn uses the | 23 using `github.com/luci/gae/service/datastore/serialize`, which in turn uses the |
| 24 primitives available in `github.com/luci/luci-go/common/cmpbin`. The encodings | 24 primitives available in `github.com/luci/luci-go/common/cmpbin`. The encodings |
| 25 are important to understanding why the schemas below sort correctly when | 25 are important to understanding why the schemas below sort correctly when |
| 26 compared only using `bytes.Compare` (aka `memcmp`). This doc will assume that | 26 compared only using `bytes.Compare` (aka `memcmp`). This doc will assume that |
| 27 you're familiar with those encodings, but will point out where we diverge from | 27 you're familiar with those encodings, but will point out where we diverge from |
| 28 the stock encodings. | 28 the stock encodings. |
| 29 | 29 |
| 30 All encoded Property values used in gkvlite Keys (i.e. index rows) are | 30 All encoded Property values used in memory store Keys (i.e. index rows) are |
| 31 serialized using the settings `serialize.WithoutContext`, and | 31 serialized using the settings `serialize.WithoutContext`, and |
| 32 `datastore.ShouldIndex`. | 32 `datastore.ShouldIndex`. |
| 33 | 33 |
| 34 ### Primary table | 34 ### Primary table |
| 35 | 35 |
| 36 The primary table maps datastore keys to entities. | 36 The primary table maps datastore keys to entities. |
| 37 | 37 |
| 38 - Name: `"ents:" + namespace` | 38 - Name: `"ents:" + namespace` |
| 39 - Key: serialized datastore.Property containing the entity's datastore.Key | 39 - Key: serialized datastore.Property containing the entity's datastore.Key |
| 40 - Value: serialized datastore.PropertyMap | 40 - Value: serialized datastore.PropertyMap |
| (...skipping 330 matching lines...) Expand 10 before | Expand all | Expand 10 after Loading... |
| 371 of the suffix will always be the datastore key. The suffix is then used to call | 371 of the suffix will always be the datastore key. The suffix is then used to call |
| 372 back to the user, according to the query type: | 372 back to the user, according to the query type: |
| 373 | 373 |
| 374 * keys-only queries just directly return the Key | 374 * keys-only queries just directly return the Key |
| 375 * projection queries return the projected fields from the decoded suffix. | 375 * projection queries return the projected fields from the decoded suffix. |
| 376 Remember how we added all the projections after the orders? This is why. The | 376 Remember how we added all the projections after the orders? This is why. The |
| 377 projected values are pulled directly from the index, instead of going to the | 377 projected values are pulled directly from the index, instead of going to the |
| 378 main entity table. | 378 main entity table. |
| 379 * normal queries pull the decoded Key from the "ents" table, and return that | 379 * normal queries pull the decoded Key from the "ents" table, and return that |
| 380 entity to the user. | 380 entity to the user. |
| OLD | NEW |
|---|
Chromium Code Reviews
Issue 2604943002:
impl/memory: Replace gkvlite with "treapstore". (Closed)
