New premium version with web worker support upon customer demand (under consideration)

[brody4hire]

I received a couple inquiries for continued web worker support to help solve performance issues in two different cases:

1. Issue when populating with data from one or more external servers.
2. Issue when reading and processing binary/base64 image data from the database (ref: BLOB data support (MBTiles etc.) #10)

There is an existing version at https://github.com/litehelpers/Cordova-sqlite-evplus-legacy-workers-free but it has the following major problems:

• Extra communication overhead: response messages go through the main thread and there is library code to send each worker response messages to the correct worker thread.
• Standard Web SQL transaction API will not work with upcoming browser platform (ref: WIP: initial browser platform cordova-sqlite-storage#576).
• ADDITIONAL PROBLEM raised in Simultaneously Transactions with web workers and main app litehelpers/cordova-sqlite-evmax-ext-workers-legacy-build-free#3

The ideal solution would be to redesign the apps to move all processing and possibly XHR (HTTP requests) to web workers and reserve the main thread to handle DOM, coordination, and I/O tasks. (Note that this plugin already uses background threads to handle sqlite operations, with the exception of Windows which can easily be fixed.) Unfortunately this may not be a 100% solution since this plugin does have some internal pre/post processing in JavaScript.

The most general solution would be to offer the db.executeSql and db.sqlBatch functions within web workers. This solution would continue to have the extra communication overhead described above.

An alternative solution for case 1 would be to write the data from a worker and let the main thread listen for indications of data write success/failure results. This idea is inspired by existing ideas such as reactive programming and Flow Based Programming (FBP) as described by J. Paul Morrison.

For case 2 an alternative, recommended solution is to read the image data from the main thread and then send it to a web worker for further processing.

[kristfal]

@brodybits Thanks for taking a look at this.

Part of the performance issue when reading blobs is stemming from Cordova itself. If you look at this timeline, you'll notice that pollOnce and buildPayload accounts for almost half of the time spent on the operation (this was recorded on a Galaxy S3, so performance is expected to be poor).

In the example above, the rest of the thread lock is caused by inflating the binary payload, which is trivial to do in a worker.

Offering db.executeSql from a worker, and circumventing any communication via the main thread (including the cordova callback) before we manually transfer the processed data to the main thread would be preferable. If you're able to provide something for testing within the next couple of months, we would gladly evaluate it as discussed per e-mail.

[brody4hire]

Thanks @kristfal for the feedback and performance timeline. It does help make the case for web worker support. On the technical side:

I think the only way to achieve the desired functionality for @kristfal on Android (case 2) is to use a custom URI handler that handles the request and returns the result, maybe synchronously. I suspect it would be a blocking call on the worker but would have to do some experimentation to confirm. I would also have to do some experimentation to confirm whether or not it would be possible to achieve the same functionality on iOS. (It should be no problem to match this functionality on Windows.)

The other consequence of this scenario is that it could not be integrated with browser platform support (storesafe/cordova-sqlite-storage#576).

I hope it will be possible to support both case 1 (populate with data from external servers) and case 2, assuming it will work across the supported platforms Android, iOS, Windows, and possibly macOS. Any comments from @screendriver?

[brody4hire]

Also the idea to write data from a worker and listen for success/failure results in the main thread could be tricky on the Windows platform. I would have to do some testing to see if this would work or not.

[brody4hire]

Yet another issue with using database.transaction within web workers ("transaction within a transaction") was reported in litehelpers/cordova-sqlite-evmax-ext-workers-legacy-build-free#3.

[brody4hire]

To clarify: in case someone orders a new evplus version with web worker support I would like to completely remove the standard database.transaction support from that version.

[yinyinmyo]

You recommend that database.executeSql (for SELECT and single-statement changes) and database.sqlBatch (for multi-statement changes) instead of database.transaction.I try it.But i can't it.Because i dont know how to handle transaction using database.sqlBatch without using db.transaction.Please,show me the way expecially code sample.