OLD | NEW |
| (Empty) |
1 NOTE (2012-11-29): | |
2 | |
3 The functionality implemented by this extension has been superseded | |
4 by WAL-mode. This module is no longer supported or maintained. The | |
5 code is retained for historical reference only. | |
6 | |
7 ------------------------------------------------------------------------------ | |
8 | |
9 Normally, when SQLite writes to a database file, it waits until the write | |
10 operation is finished before returning control to the calling application. | |
11 Since writing to the file-system is usually very slow compared with CPU | |
12 bound operations, this can be a performance bottleneck. This directory | |
13 contains an extension that causes SQLite to perform all write requests | |
14 using a separate thread running in the background. Although this does not | |
15 reduce the overall system resources (CPU, disk bandwidth etc.) at all, it | |
16 allows SQLite to return control to the caller quickly even when writing to | |
17 the database, eliminating the bottleneck. | |
18 | |
19 1. Functionality | |
20 | |
21 1.1 How it Works | |
22 1.2 Limitations | |
23 1.3 Locking and Concurrency | |
24 | |
25 2. Compilation and Usage | |
26 | |
27 3. Porting | |
28 | |
29 | |
30 | |
31 1. FUNCTIONALITY | |
32 | |
33 With asynchronous I/O, write requests are handled by a separate thread | |
34 running in the background. This means that the thread that initiates | |
35 a database write does not have to wait for (sometimes slow) disk I/O | |
36 to occur. The write seems to happen very quickly, though in reality | |
37 it is happening at its usual slow pace in the background. | |
38 | |
39 Asynchronous I/O appears to give better responsiveness, but at a price. | |
40 You lose the Durable property. With the default I/O backend of SQLite, | |
41 once a write completes, you know that the information you wrote is | |
42 safely on disk. With the asynchronous I/O, this is not the case. If | |
43 your program crashes or if a power loss occurs after the database | |
44 write but before the asynchronous write thread has completed, then the | |
45 database change might never make it to disk and the next user of the | |
46 database might not see your change. | |
47 | |
48 You lose Durability with asynchronous I/O, but you still retain the | |
49 other parts of ACID: Atomic, Consistent, and Isolated. Many | |
50 appliations get along fine without the Durablity. | |
51 | |
52 1.1 How it Works | |
53 | |
54 Asynchronous I/O works by creating a special SQLite "vfs" structure | |
55 and registering it with sqlite3_vfs_register(). When files opened via | |
56 this vfs are written to (using the vfs xWrite() method), the data is not | |
57 written directly to disk, but is placed in the "write-queue" to be | |
58 handled by the background thread. | |
59 | |
60 When files opened with the asynchronous vfs are read from | |
61 (using the vfs xRead() method), the data is read from the file on | |
62 disk and the write-queue, so that from the point of view of | |
63 the vfs reader the xWrite() appears to have already completed. | |
64 | |
65 The special vfs is registered (and unregistered) by calls to the | |
66 API functions sqlite3async_initialize() and sqlite3async_shutdown(). | |
67 See section "Compilation and Usage" below for details. | |
68 | |
69 1.2 Limitations | |
70 | |
71 In order to gain experience with the main ideas surrounding asynchronous | |
72 IO, this implementation is deliberately kept simple. Additional | |
73 capabilities may be added in the future. | |
74 | |
75 For example, as currently implemented, if writes are happening at a | |
76 steady stream that exceeds the I/O capability of the background writer | |
77 thread, the queue of pending write operations will grow without bound. | |
78 If this goes on for long enough, the host system could run out of memory. | |
79 A more sophisticated module could to keep track of the quantity of | |
80 pending writes and stop accepting new write requests when the queue of | |
81 pending writes grows too large. | |
82 | |
83 1.3 Locking and Concurrency | |
84 | |
85 Multiple connections from within a single process that use this | |
86 implementation of asynchronous IO may access a single database | |
87 file concurrently. From the point of view of the user, if all | |
88 connections are from within a single process, there is no difference | |
89 between the concurrency offered by "normal" SQLite and SQLite | |
90 using the asynchronous backend. | |
91 | |
92 If file-locking is enabled (it is enabled by default), then connections | |
93 from multiple processes may also read and write the database file. | |
94 However concurrency is reduced as follows: | |
95 | |
96 * When a connection using asynchronous IO begins a database | |
97 transaction, the database is locked immediately. However the | |
98 lock is not released until after all relevant operations | |
99 in the write-queue have been flushed to disk. This means | |
100 (for example) that the database may remain locked for some | |
101 time after a "COMMIT" or "ROLLBACK" is issued. | |
102 | |
103 * If an application using asynchronous IO executes transactions | |
104 in quick succession, other database users may be effectively | |
105 locked out of the database. This is because when a BEGIN | |
106 is executed, a database lock is established immediately. But | |
107 when the corresponding COMMIT or ROLLBACK occurs, the lock | |
108 is not released until the relevant part of the write-queue | |
109 has been flushed through. As a result, if a COMMIT is followed | |
110 by a BEGIN before the write-queue is flushed through, the database | |
111 is never unlocked,preventing other processes from accessing | |
112 the database. | |
113 | |
114 File-locking may be disabled at runtime using the sqlite3async_control() | |
115 API (see below). This may improve performance when an NFS or other | |
116 network file-system, as the synchronous round-trips to the server be | |
117 required to establish file locks are avoided. However, if multiple | |
118 connections attempt to access the same database file when file-locking | |
119 is disabled, application crashes and database corruption is a likely | |
120 outcome. | |
121 | |
122 | |
123 2. COMPILATION AND USAGE | |
124 | |
125 The asynchronous IO extension consists of a single file of C code | |
126 (sqlite3async.c), and a header file (sqlite3async.h) that defines the | |
127 C API used by applications to activate and control the modules | |
128 functionality. | |
129 | |
130 To use the asynchronous IO extension, compile sqlite3async.c as | |
131 part of the application that uses SQLite. Then use the API defined | |
132 in sqlite3async.h to initialize and configure the module. | |
133 | |
134 The asynchronous IO VFS API is described in detail in comments in | |
135 sqlite3async.h. Using the API usually consists of the following steps: | |
136 | |
137 1. Register the asynchronous IO VFS with SQLite by calling the | |
138 sqlite3async_initialize() function. | |
139 | |
140 2. Create a background thread to perform write operations and call | |
141 sqlite3async_run(). | |
142 | |
143 3. Use the normal SQLite API to read and write to databases via | |
144 the asynchronous IO VFS. | |
145 | |
146 Refer to sqlite3async.h for details. | |
147 | |
148 | |
149 3. PORTING | |
150 | |
151 Currently the asynchronous IO extension is compatible with win32 systems | |
152 and systems that support the pthreads interface, including Mac OSX, Linux, | |
153 and other varieties of Unix. | |
154 | |
155 To port the asynchronous IO extension to another platform, the user must | |
156 implement mutex and condition variable primitives for the new platform. | |
157 Currently there is no externally available interface to allow this, but | |
158 modifying the code within sqlite3async.c to include the new platforms | |
159 concurrency primitives is relatively easy. Search within sqlite3async.c | |
160 for the comment string "PORTING FUNCTIONS" for details. Then implement | |
161 new versions of each of the following: | |
162 | |
163 static void async_mutex_enter(int eMutex); | |
164 static void async_mutex_leave(int eMutex); | |
165 static void async_cond_wait(int eCond, int eMutex); | |
166 static void async_cond_signal(int eCond); | |
167 static void async_sched_yield(void); | |
168 | |
169 The functionality required of each of the above functions is described | |
170 in comments in sqlite3async.c. | |
OLD | NEW |