| OLD | NEW |
| (Empty) |
| 1 NOTE (2012-11-29): | |
| 2 | |
| 3 The functionality implemented by this extension has been superseded | |
| 4 by WAL-mode. This module is no longer supported or maintained. The | |
| 5 code is retained for historical reference only. | |
| 6 | |
| 7 ------------------------------------------------------------------------------ | |
| 8 | |
| 9 Normally, when SQLite writes to a database file, it waits until the write | |
| 10 operation is finished before returning control to the calling application. | |
| 11 Since writing to the file-system is usually very slow compared with CPU | |
| 12 bound operations, this can be a performance bottleneck. This directory | |
| 13 contains an extension that causes SQLite to perform all write requests | |
| 14 using a separate thread running in the background. Although this does not | |
| 15 reduce the overall system resources (CPU, disk bandwidth etc.) at all, it | |
| 16 allows SQLite to return control to the caller quickly even when writing to | |
| 17 the database, eliminating the bottleneck. | |
| 18 | |
| 19 1. Functionality | |
| 20 | |
| 21 1.1 How it Works | |
| 22 1.2 Limitations | |
| 23 1.3 Locking and Concurrency | |
| 24 | |
| 25 2. Compilation and Usage | |
| 26 | |
| 27 3. Porting | |
| 28 | |
| 29 | |
| 30 | |
| 31 1. FUNCTIONALITY | |
| 32 | |
| 33 With asynchronous I/O, write requests are handled by a separate thread | |
| 34 running in the background. This means that the thread that initiates | |
| 35 a database write does not have to wait for (sometimes slow) disk I/O | |
| 36 to occur. The write seems to happen very quickly, though in reality | |
| 37 it is happening at its usual slow pace in the background. | |
| 38 | |
| 39 Asynchronous I/O appears to give better responsiveness, but at a price. | |
| 40 You lose the Durable property. With the default I/O backend of SQLite, | |
| 41 once a write completes, you know that the information you wrote is | |
| 42 safely on disk. With the asynchronous I/O, this is not the case. If | |
| 43 your program crashes or if a power loss occurs after the database | |
| 44 write but before the asynchronous write thread has completed, then the | |
| 45 database change might never make it to disk and the next user of the | |
| 46 database might not see your change. | |
| 47 | |
| 48 You lose Durability with asynchronous I/O, but you still retain the | |
| 49 other parts of ACID: Atomic, Consistent, and Isolated. Many | |
| 50 appliations get along fine without the Durablity. | |
| 51 | |
| 52 1.1 How it Works | |
| 53 | |
| 54 Asynchronous I/O works by creating a special SQLite "vfs" structure | |
| 55 and registering it with sqlite3_vfs_register(). When files opened via | |
| 56 this vfs are written to (using the vfs xWrite() method), the data is not | |
| 57 written directly to disk, but is placed in the "write-queue" to be | |
| 58 handled by the background thread. | |
| 59 | |
| 60 When files opened with the asynchronous vfs are read from | |
| 61 (using the vfs xRead() method), the data is read from the file on | |
| 62 disk and the write-queue, so that from the point of view of | |
| 63 the vfs reader the xWrite() appears to have already completed. | |
| 64 | |
| 65 The special vfs is registered (and unregistered) by calls to the | |
| 66 API functions sqlite3async_initialize() and sqlite3async_shutdown(). | |
| 67 See section "Compilation and Usage" below for details. | |
| 68 | |
| 69 1.2 Limitations | |
| 70 | |
| 71 In order to gain experience with the main ideas surrounding asynchronous | |
| 72 IO, this implementation is deliberately kept simple. Additional | |
| 73 capabilities may be added in the future. | |
| 74 | |
| 75 For example, as currently implemented, if writes are happening at a | |
| 76 steady stream that exceeds the I/O capability of the background writer | |
| 77 thread, the queue of pending write operations will grow without bound. | |
| 78 If this goes on for long enough, the host system could run out of memory. | |
| 79 A more sophisticated module could to keep track of the quantity of | |
| 80 pending writes and stop accepting new write requests when the queue of | |
| 81 pending writes grows too large. | |
| 82 | |
| 83 1.3 Locking and Concurrency | |
| 84 | |
| 85 Multiple connections from within a single process that use this | |
| 86 implementation of asynchronous IO may access a single database | |
| 87 file concurrently. From the point of view of the user, if all | |
| 88 connections are from within a single process, there is no difference | |
| 89 between the concurrency offered by "normal" SQLite and SQLite | |
| 90 using the asynchronous backend. | |
| 91 | |
| 92 If file-locking is enabled (it is enabled by default), then connections | |
| 93 from multiple processes may also read and write the database file. | |
| 94 However concurrency is reduced as follows: | |
| 95 | |
| 96 * When a connection using asynchronous IO begins a database | |
| 97 transaction, the database is locked immediately. However the | |
| 98 lock is not released until after all relevant operations | |
| 99 in the write-queue have been flushed to disk. This means | |
| 100 (for example) that the database may remain locked for some | |
| 101 time after a "COMMIT" or "ROLLBACK" is issued. | |
| 102 | |
| 103 * If an application using asynchronous IO executes transactions | |
| 104 in quick succession, other database users may be effectively | |
| 105 locked out of the database. This is because when a BEGIN | |
| 106 is executed, a database lock is established immediately. But | |
| 107 when the corresponding COMMIT or ROLLBACK occurs, the lock | |
| 108 is not released until the relevant part of the write-queue | |
| 109 has been flushed through. As a result, if a COMMIT is followed | |
| 110 by a BEGIN before the write-queue is flushed through, the database | |
| 111 is never unlocked,preventing other processes from accessing | |
| 112 the database. | |
| 113 | |
| 114 File-locking may be disabled at runtime using the sqlite3async_control() | |
| 115 API (see below). This may improve performance when an NFS or other | |
| 116 network file-system, as the synchronous round-trips to the server be | |
| 117 required to establish file locks are avoided. However, if multiple | |
| 118 connections attempt to access the same database file when file-locking | |
| 119 is disabled, application crashes and database corruption is a likely | |
| 120 outcome. | |
| 121 | |
| 122 | |
| 123 2. COMPILATION AND USAGE | |
| 124 | |
| 125 The asynchronous IO extension consists of a single file of C code | |
| 126 (sqlite3async.c), and a header file (sqlite3async.h) that defines the | |
| 127 C API used by applications to activate and control the modules | |
| 128 functionality. | |
| 129 | |
| 130 To use the asynchronous IO extension, compile sqlite3async.c as | |
| 131 part of the application that uses SQLite. Then use the API defined | |
| 132 in sqlite3async.h to initialize and configure the module. | |
| 133 | |
| 134 The asynchronous IO VFS API is described in detail in comments in | |
| 135 sqlite3async.h. Using the API usually consists of the following steps: | |
| 136 | |
| 137 1. Register the asynchronous IO VFS with SQLite by calling the | |
| 138 sqlite3async_initialize() function. | |
| 139 | |
| 140 2. Create a background thread to perform write operations and call | |
| 141 sqlite3async_run(). | |
| 142 | |
| 143 3. Use the normal SQLite API to read and write to databases via | |
| 144 the asynchronous IO VFS. | |
| 145 | |
| 146 Refer to sqlite3async.h for details. | |
| 147 | |
| 148 | |
| 149 3. PORTING | |
| 150 | |
| 151 Currently the asynchronous IO extension is compatible with win32 systems | |
| 152 and systems that support the pthreads interface, including Mac OSX, Linux, | |
| 153 and other varieties of Unix. | |
| 154 | |
| 155 To port the asynchronous IO extension to another platform, the user must | |
| 156 implement mutex and condition variable primitives for the new platform. | |
| 157 Currently there is no externally available interface to allow this, but | |
| 158 modifying the code within sqlite3async.c to include the new platforms | |
| 159 concurrency primitives is relatively easy. Search within sqlite3async.c | |
| 160 for the comment string "PORTING FUNCTIONS" for details. Then implement | |
| 161 new versions of each of the following: | |
| 162 | |
| 163 static void async_mutex_enter(int eMutex); | |
| 164 static void async_mutex_leave(int eMutex); | |
| 165 static void async_cond_wait(int eCond, int eMutex); | |
| 166 static void async_cond_signal(int eCond); | |
| 167 static void async_sched_yield(void); | |
| 168 | |
| 169 The functionality required of each of the above functions is described | |
| 170 in comments in sqlite3async.c. | |
| OLD | NEW |