| OLD | NEW |
| 1 This directory contains a partial snapshot of the sqlite library from | 1 Instructions for importing a new release of sqlite from sqlite.org. |
| 2 http://sqlite.org. | |
| 3 | 2 |
| 4 Current version: 3.5.3, released 23-Nov-2007. | 3 First, you need to be on Linux. |
| 5 | 4 |
| 6 (This was built as 3.4.2 [13-Aug-2007] as described below, with the four | 5 Find the release you want at: |
| 7 intervening patches 4315, 4427, 4546, and 4556 applied by hand.) | 6 http://www.sqlite.org/cvstrac/timeline |
| 8 | 7 |
| 9 To import a new snapshot of sqlite: | 8 Search for "Milestone", and find the appropriate release. Click |
| 9 through, and snag the "Date" for use in DATE line below. |
| 10 Unfortunately, the actual displayed date string on that page will not |
| 11 work, you have to make it YYYY/MM/DD. [TODO(shess) Determine if the |
| 12 fact that sqlite.org thinks it's running under UTC is relevant.] |
| 10 | 13 |
| 11 - Visit http://sqlite.org/download.html and download the latest source | 14 DATE='2007/01/24 09:54:56' |
| 12 distribution. | |
| 13 - Unpack the source on a Linux machine. | |
| 14 - Change to the source directory and run: | |
| 15 $ ./configure --disable-tcl # build headers | |
| 16 $ make # build some generated files | |
| 17 - Copy the generated .c/.h files from the sqlite directory to this directory, | |
| 18 as well as those in src/ and ext/fts2/. Omit files which have been omitted | |
| 19 here. Here's an easy way to be sure you get everything: | |
| 20 $ cp /path/to/source/*.[ch] . # don't forget subdirs, too | |
| 21 $ gvn status | grep -v ^M # print the names of all new files | |
| 22 $ mkdir new; mv each new file to new | |
| 23 Then rebuild, and if any of the files in new/ are needed, move them | |
| 24 back into this directory, add them to the project, and "gvn add" them. | |
| 25 - Apply the preload-cache.diff (see below) | |
| 26 - Update this README to reflect the new version number. | |
| 27 | 15 |
| 28 Modifications for this release: | 16 # Get to the third_party/sqlite directory in your Chromium client. |
| 29 - I marked all changes I made with "evanm", so you can find them with | 17 cd ../third_party/sqlite |
| 30 "grep evanm *". | |
| 31 - Most files include sqlite3ext.h with SQLITE_CORE #defined, but two don't: | |
| 32 fts2_tokenizer.c and icu.c. Without this #define, the calls in | |
| 33 fts2_tokenizer.c try to go through some pointer to the sqlite API instead of | |
| 34 calling the functions directly (to work as a loadable module), but then crash | |
| 35 (because the other files never initialize that loadable module support). As | |
| 36 a hack I #defined it in these files, but it'd be nice to figure out what | |
| 37 really ought to happen here (perhaps this file is new and hasn't been tested | |
| 38 to verify it works right). | |
| 39 - shell_icu.c is a Chrome-specific file used to load our ICU data. shell.c | |
| 40 has been modifed to call into shell_icu.c. | |
| 41 - fts2_icu.c has a critical bug. U8_NEXT is used over a UTF-16 string. It's repl
aced | |
| 42 by U16_NEXT (jungshik) | |
| 43 | 18 |
| 44 Patch | 19 # Run the super awesome automatic merge tool (requires kdiff3). |
| 45 ----- | 20 # NOTE(shess): The following (which runs google_generate_preprocessed.sh) will |
| 46 The file preload-cache.diff patch must be applied to add a new function we use | 21 # produce different output on grhat versus goobuntu; I've been running it on |
| 47 to prime the database cache. It allows much faster performance by reading the | 22 # goobuntu. |
| 48 file in one contiguous operation rather than bringing it in organically, which | 23 ./google_update_sqlite.sh "$DATE" |
| 49 involves a lot of seeking. | |
| 50 | 24 |
| 51 FTS2 modification | 25 # Resolve any conflicts. Figure out if we've got everything we should |
| 52 ----------------- | 26 # have (see below), or if we can omit any changes we no longer need. |
| 53 In fts2.c, we added an additional check from fts3 to try to catch some | 27 |
| 54 additional problems. In buildTerms on line 3807, we replaced | 28 # TODO(shess) If we don't want the CVS dirs. |
| 55 if( iPosition<0 ){ | 29 # find sqlite_vendor -name CVS -execdir rm -rf {} + -prune |
| 56 with: | 30 |
| 57 if( iPosition<0 || pToken == NULL || nTokenBytes == 0 ){ | 31 # Find a sucker. Send review. |
| 58 It is from this change to sqlite: | 32 # TODO(shess) Describe an appropriate comment style. Seems like it |
| 59 http://www.sqlite.org/cvstrac/chngview?cn=4514 | 33 # should include the DATE line, and the sqlite version number. |
| 34 |
| 35 |
| 36 -------------------------------------------- |
| 37 |
| 38 Why all this convolution? The problem being solved is that we want to |
| 39 harvest changes from the sqlite CVS tree, and merge them with any |
| 40 local changes we make. In CVS, you would do this using a "vendor |
| 41 import", which is essentially a branch dedictated to the vendor |
| 42 version which is merged with local changes. |
| 43 |
| 44 third_party/sqlite_vendor/... is the CVS checkout for a particular |
| 45 build from sqlite.org. third_party/sqlite_google/... is the local |
| 46 version, with our local modifications. So we update the sqlite_vendor |
| 47 tree, then use perforce to downintegrate changes into our |
| 48 locally-modified tree. The downintegrate will call out any |
| 49 conflicting changes, but will otherwise just merge things together. |
| 50 Basically, sqlite_vendor is a gateway between CVS and perforce. |
| 51 |
| 52 Scott Hess <shess@google.com>, April 9, 2007. |
| 53 |
| 54 -------------------------------------------- |
| 55 |
| 56 How to run the SQLite tests for the Gears version of SQLite on Linux. |
| 57 |
| 58 cd ../third_party/sqlite_google/ |
| 59 mkdir build |
| 60 cd build |
| 61 make -f ../Makefile.linux-gcc testfixture |
| 62 make -f ../Makefile.linux-gcc test > /tmp/test.log |
| 63 egrep -v 'Ok$' /tmp/test.log |
| 64 # When run on a locally-mounted disk, my output ends with: |
| 65 # 0 errors out of 57887 tests |
| 66 |
| 67 Scott Hess <shess@google.com>, December 11, 2007 |
| 68 |
| 69 -------------------------------------------- |
| 70 |
| 71 As of September 12, 2008, these are our changes from sqlite_vendor: |
| 72 |
| 73 - fts2.c disables fts2_tokenizer(). |
| 74 - sqlite3Poison() in src/btree.c. |
| 75 - BEGIN defaults to BEGIN IMMEDIATE in parse.y. |
| 76 - Tweak to SQLITE_EXTENSION_INIT* in sqlite3ext.h. |
| 77 - That implied a change in src/test_autoext.c for testing. |
| 78 - Added fts.test and fts1.test in tests, modified quick.test. |
| 79 - src/os_symbian.cc. |
| 80 - Modifications to Makefile.linux-gcc and main.mk for compiling |
| 81 SQLite tests. |
| 82 - Compile warning fixed in func.c (check if this is still needed) |
| 83 - Fixed a typo bug in fts2_icu.c: "strlen(nInput)" (filed upstream as |
| 84 http://www.sqlite.org/cvstrac/tktview?tn=3543) |
| 85 |
| 86 Changes from Chrome: |
| 87 - I marked all changes I made with "evanm", so you can find them with |
| 88 "grep evanm *". |
| 89 - Most files include sqlite3ext.h with SQLITE_CORE #defined, but two don't: |
| 90 fts2_tokenizer.c and icu.c. Without this #define, the calls in |
| 91 fts2_tokenizer.c try to go through some pointer to the sqlite API instead |
| 92 of calling the functions directly (to work as a loadable module), but then |
| 93 crash (because the other files never initialize that loadable module |
| 94 support). As a hack I #defined it in these files, but it'd be nice to |
| 95 figure out what really ought to happen here (perhaps this file is new and |
| 96 hasn't been tested to verify it works right). Update: Seems this is an |
| 97 issue we get because we're using fts2 instead of fts3. |
| 98 - shell_icu.c is a Chrome-specific file used to load our ICU data. shell.c |
| 99 has been modifed to call into shell_icu.c. |
| 100 - fts2_icu.c has a critical bug. U8_NEXT is used over a UTF-16 string. It's |
| 101 rep$ by U16_NEXT (jungshik) |
| 102 - Added a new function sqlite3Preload we use to prime the database cache. It |
| 103 allows much faster performance by reading the file in one contiguous |
| 104 operation rather than bringing it in organically, which involves a lot of |
| 105 seeking. |
| OLD | NEW |