Provisional 'subscribe to all' functionality #54

mamcx · 2023-07-03T22:45:54Z

Description of Changes

Add the ability to “send me everything” query.

It use the special

SELECT * FROM *

to trigger this.

This semantically doesn't seem right, or at least confusing from the POV of subscriptions: It means "send back what you have right now" instead of "keep streaming back all the changes".

This means this will break (ie changes to the database schema could fail in subsequent runs):

subscribe SELECT * FROM *
CREATE TABLE a
INSERT INTO a
DROP TABLE a
CREATE TABLE b

API

This is a breaking change to the module API
This is a breaking change to the ClientAPI

If the API is breaking, please state below what will break

* test * Fixed CI * Fixed tests * Fixed formatting * Fixed test error by removing chrono default features * Rename smoketests CI thing * Hopefully fix the testing issue * Fix typos * Fixed install git-hooks * Fixed formatting * Fixed compile error * fixed testing issues

Signed-off-by: Tyler Cloutier <[email protected]>

Co-authored-by: Tyler Cloutier <[email protected]>

* Clippy runs on default module * Fix clippy issue with default module --------- Co-authored-by: Boppy <[email protected]>

* I think somehow we are adding all tables as orphans, added log * Removed unneeded clone --------- Co-authored-by: Boppy <[email protected]>

) * Fix bug for subcription that was not executing or returning if empty * Clippy * Filter out deleted rows * Recover the op_type (DELETED, INSERTED) flag for the row in the subscription

* grandfathered docs improvements in bindings * improve bindings * address review comments * remove late bound lifetime * undo derive for Timestamp to bisect bug

* Forbid unsupported syntax in SELECT (like ORDER BY) & improve accuracy of sql test suite * Update crates/sqltest/src/main.rs Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]> --------- Signed-off-by: Mario Montoya <[email protected]> Co-authored-by: Mazdak Farrokhzad <[email protected]>

Signed-off-by: John Detter <[email protected]> Co-authored-by: Boppy <[email protected]>

* Working on improving commands that use identities * Fix lints * Reverted file that shouldn't have changed * Found and fixed all other todos * Addressed more CLI TODOs * Fixes for formatting issues * Set names of identities * Set name of identities + clippy * Small fix * Added the start of a doc comment, switching over to another PR * Fixed tests that needed to be updated * Addressed more feedback and fixed several clippy issues * Small fix * Apply suggestions from code review Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: John Detter <[email protected]> * Added more doc comments * Addressing more feedback * Fixed really old bug in SpacetimeDB * Tests to verify new functionality * Fix clippy lints * Email during identity creation is optional * Fix output so testsuite passes --------- Signed-off-by: John Detter <[email protected]> Co-authored-by: Boppy <[email protected]> Co-authored-by: Mazdak Farrokhzad <[email protected]>

* Some work * Getting smoketests working on mac * All tests are passing except known failing tests --------- Co-authored-by: Boppy <[email protected]>

…urn wether or not reducer has subscribers; Moved all ReducerEvent and ReducerArgs code into a single file (#10) Co-authored-by: Steve <[email protected]>

Co-authored-by: Boppy <[email protected]>

* Changed C# reducer signature; Change reducer invocation method to return wether or not reducer has subscribers; Moved all ReducerEvent and ReducerArgs code into a single file * Formatting, tests * Clippy * Removed union from C# codegen * Tests * Update crates/cli/src/subcommands/generate/csharp.rs Co-authored-by: Tyler Cloutier <[email protected]> Signed-off-by: John Detter <[email protected]> * Removed duplicate code * Lint * Clippy * CLIPPY * Generating custom indexes for unique / primary keys --------- Signed-off-by: John Detter <[email protected]> Co-authored-by: Steve <[email protected]> Co-authored-by: John Detter <[email protected]> Co-authored-by: Tyler Cloutier <[email protected]>

* Fix incomplete transaction bug * Clippy

Co-authored-by: Steve <[email protected]>

tokio runtimes cannot be nested, so we only create one if no ambient context can be found and pass around a runtime handle for spawning tasks.

* integrate jdetter's review * address phoebe's comments * improve errnos naming

Prior to this commit, callbacks in the Rust client SDK shared access to the global ClientCache while running asynchronously. This meant that a long-running or delayed callback could observe the ClientCache in a state later than the one that caused the callback's invocation, and had no way to access the specific state for which it was invoked. With this commit, each `Invoke` message to the callback worker includes an `Arc<ClientCache>` snapshot of the DB state when that callback was invoked. The callback worker stores that state in a `thread_local`, and methods that inspect tables (e.g. `TableType::iter`) read the state out of the `thread_local` when it is present. This allows callbacks to observe exactly the state which caused their invocation, never a later state, while maintaining the C#-like API where `ClientCache` access is based on free functions or static trait methods. --------- Co-authored-by: John Detter <[email protected]>

* Fix bug and added a test * Test updates * Fix tests --------- Co-authored-by: Boppy <[email protected]>

…ialization changes

#117)

spacetimedb.com/spacetimedb Co-authored-by: Boppy <[email protected]>

This commit renames the first argument to connect from host to spacetimedb_uri. It's not just a host(name), it's the URI of the SpacetimeDB instance.

* Fix bug in locating fields when a JOIN clause is present * Update crates/lib/src/relation.rs Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]> * Update crates/lib/src/relation.rs Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]> * Update crates/core/src/subscription/subscription.rs Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]> * Update crates/core/src/subscription/subscription.rs Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]> * Update crates/core/src/subscription/subscription.rs Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]> * Update crates/core/src/subscription/query.rs Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]> * Small doc nits * Fix test --------- Signed-off-by: Mario Montoya <[email protected]> Co-authored-by: Mazdak Farrokhzad <[email protected]>

This commit adds the `quickstart-chat` Rust module and client to the appropriate examples dirs.

Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]>

* Lint script * modernize: pnpm & prettier (#51) * Push * Push * Run prettier * Prettier stuff * modernize: Remove Husky & pre-commit (#53) * modernize: tsup (#54) * Push * jest * modernize: jest to vitest (#55) * modernize: pnpm workspace (#56) * Monorepo * Readme * Move tsconfig * Fix test * modernize: CRA -> Vite apps (#57) * Quickstart * test app too * Run pnpm * Remove eslint files * Format * modernize: ES2017 classes (#58) * modernize: in-house EventEmitter (#64) * Push * Undo min change * modernize: undici for WebSocket in Node (#59) * Use undici, remove other unneeded dependencies * pnpm install * Bundle everything * Move everything to devDependency * headers object * Bump version to 0.11.0 * Push * Hmm not working still * Add undici to peerDependencies * Measure size * Push * Fix formatting * Working on node!! --------- Co-authored-by: Zeke Foppa <github.com/bfops> Co-authored-by: Zeke Foppa <[email protected]> * modernize: Single output (#68) * modernize: quickstart, move server into client (#69) * Push * Run pnpm install * modernize: Engines field (#70) * modernize: Continuous Releases (#71) * Add pkg.pr.new * Fix directory * modernize: Changesets;provenance (#72) * Push * Push * pnpm install * Any update * modernize: Split lint and test GH actions (#73) * Push * Forgot to rename * We only care about Lint at commit level * modernize: pkg.pr.new compact mode (#75) * modernize: tweak prettier configuration (#74) * docs: Undici as peerDependency * modernize: webpackIgnore undici (#77) * Push * Webpackignore * modernize: Conditional browser build (#79) * Push * Remove webpackIgnore * fix: Remove obsolete comment * modernize: isolatedDeclarations;de-cyclic imports (#81) * Push * Fix test --------- Co-authored-by: Zeke Foppa <[email protected]>

* [bfops/docs]: C# fix * [bfops/docs]: empty --------- Co-authored-by: Zeke Foppa <github.com/bfops>

* One-off query support (#49) * SpacetimeDBURI argument in place of host and ssl enabled bool (#52) * SpacetimeDBURI argument in place of host and ssl enabled bool * Small cleanup --------- Co-authored-by: John Detter <[email protected]> * Add support for client address; re-run `spacetime generate` for quickstart client (#54) * README.md and License.txt updates (#53) * Update README.md * Add license.txt * Rename * Add LICENSE.txt with proper case --------- Co-authored-by: Derek Brinkmann <[email protected]> * Add support for client address; re-run `spacetime generate` for quickstart client Re: #299 --------- Co-authored-by: dbrinkmanncw <[email protected]> Co-authored-by: Derek Brinkmann <[email protected]> * Making Quickstart Work Again (#57) * Packaged nuget package, quickstart is now working again * Reset DBNAME --------- Co-authored-by: John Detter <[email protected]> * OnUnhandledReducerError (#31) Co-authored-by: Steve <[email protected]> Co-authored-by: Boppy <[email protected]> --------- Co-authored-by: james gilles <[email protected]> Co-authored-by: dbrinkmanncw <[email protected]> Co-authored-by: John Detter <[email protected]> Co-authored-by: Phoebe Goldman <[email protected]> Co-authored-by: Derek Brinkmann <[email protected]> Co-authored-by: SteveBoytsun <[email protected]> Co-authored-by: Steve <[email protected]>

…start client (#54) * README.md and License.txt updates (#53) * Update README.md * Add license.txt * Rename * Add LICENSE.txt with proper case --------- Co-authored-by: Derek Brinkmann <[email protected]> * Add support for client address; re-run `spacetime generate` for quickstart client Re: #299 --------- Co-authored-by: dbrinkmanncw <[email protected]> Co-authored-by: Derek Brinkmann <[email protected]>

* Lint script * modernize: pnpm & prettier (#51) * Push * Push * Run prettier * Prettier stuff * modernize: Remove Husky & pre-commit (#53) * modernize: tsup (#54) * Push * jest * modernize: jest to vitest (#55) * modernize: pnpm workspace (#56) * Monorepo * Readme * Move tsconfig * Fix test * modernize: CRA -> Vite apps (#57) * Quickstart * test app too * Run pnpm * Remove eslint files * Format * modernize: ES2017 classes (#58) * modernize: in-house EventEmitter (#64) * Push * Undo min change * modernize: undici for WebSocket in Node (#59) * Use undici, remove other unneeded dependencies * pnpm install * Bundle everything * Move everything to devDependency * headers object * Bump version to 0.11.0 * Push * Hmm not working still * Add undici to peerDependencies * Measure size * Push * Fix formatting * Working on node!! --------- Co-authored-by: Zeke Foppa <github.com/bfops> Co-authored-by: Zeke Foppa <[email protected]> * modernize: Single output (#68) * modernize: quickstart, move server into client (#69) * Push * Run pnpm install * modernize: Engines field (#70) * modernize: Continuous Releases (#71) * Add pkg.pr.new * Fix directory * modernize: Changesets;provenance (#72) * Push * Push * pnpm install * Any update * modernize: Split lint and test GH actions (#73) * Push * Forgot to rename * We only care about Lint at commit level * modernize: pkg.pr.new compact mode (#75) * modernize: tweak prettier configuration (#74) * docs: Undici as peerDependency * modernize: webpackIgnore undici (#77) * Push * Webpackignore * modernize: Conditional browser build (#79) * Push * Remove webpackIgnore * fix: Remove obsolete comment * modernize: isolatedDeclarations;de-cyclic imports (#81) * Push * Fix test --------- Co-authored-by: Zeke Foppa <[email protected]>

* One-off query support (#49) * SpacetimeDBURI argument in place of host and ssl enabled bool (#52) * SpacetimeDBURI argument in place of host and ssl enabled bool * Small cleanup --------- Co-authored-by: John Detter <[email protected]> * Add support for client address; re-run `spacetime generate` for quickstart client (#54) * README.md and License.txt updates (#53) * Update README.md * Add license.txt * Rename * Add LICENSE.txt with proper case --------- Co-authored-by: Derek Brinkmann <[email protected]> * Add support for client address; re-run `spacetime generate` for quickstart client Re: #299 --------- Co-authored-by: dbrinkmanncw <[email protected]> Co-authored-by: Derek Brinkmann <[email protected]> * Making Quickstart Work Again (#57) * Packaged nuget package, quickstart is now working again * Reset DBNAME --------- Co-authored-by: John Detter <[email protected]> * OnUnhandledReducerError (#31) Co-authored-by: Steve <[email protected]> Co-authored-by: Boppy <[email protected]> --------- Co-authored-by: james gilles <[email protected]> Co-authored-by: dbrinkmanncw <[email protected]> Co-authored-by: John Detter <[email protected]> Co-authored-by: Phoebe Goldman <[email protected]> Co-authored-by: Derek Brinkmann <[email protected]> Co-authored-by: SteveBoytsun <[email protected]> Co-authored-by: Steve <[email protected]>

cloutiertyler and others added 30 commits June 18, 2023 22:19

Initial commit

e5e3d78

Create LICENSE

bcb3d52

Rename LICENSE to LICENSE.txt

15a79df

Create LICENSE-SSPL.txt

ffb9e54

test

bd9f7f4

test (#1)

e8c0b53

* test * Fixed CI * Fixed tests * Fixed formatting * Fixed test error by removing chrono default features * Rename smoketests CI thing * Hopefully fix the testing issue * Fix typos * Fixed install git-hooks * Fixed formatting * Fixed compile error * fixed testing issues

Create README.md (#2)

9803c09

Signed-off-by: Tyler Cloutier <[email protected]>

Clean up repo and remove old files

ccf472f

Fixed network issue

97a8fa6

Refactor DbProgram to borrow transaction (#9)

addfddf

Co-authored-by: Tyler Cloutier <[email protected]>

Run clippy on default module (#7)

50e357a

* Clippy runs on default module * Fix clippy issue with default module --------- Co-authored-by: Boppy <[email protected]>

Added log for orphan tables (#13)

f02c047

* I think somehow we are adding all tables as orphans, added log * Removed unneeded clone --------- Co-authored-by: Boppy <[email protected]>

Fix bug for subcription that was not executing or returning if empty (#6

2c5dcd0

) * Fix bug for subcription that was not executing or returning if empty * Clippy * Filter out deleted rows * Recover the op_type (DELETED, INSERTED) flag for the row in the subscription

Refactor bindings a bit + document it (#8)

c7cd32b

* grandfathered docs improvements in bindings * improve bindings * address review comments * remove late bound lifetime * undo derive for Timestamp to bisect bug

Sendgrid environment variables can now be configured in dev (#4)

cf3215d

Signed-off-by: John Detter <[email protected]> Co-authored-by: Boppy <[email protected]>

Smoketests work on macos (#15)

0969e7e

* Some work * Getting smoketests working on mac * All tests are passing except known failing tests --------- Co-authored-by: Boppy <[email protected]>

Changed C# reducer signature; Change reducer invocation method to ret…

5eee895

…urn wether or not reducer has subscribers; Moved all ReducerEvent and ReducerArgs code into a single file (#10) Co-authored-by: Steve <[email protected]>

Added force flag so that remove can be more easily used in CI (#22)

13040aa

Co-authored-by: Boppy <[email protected]>

Fix incomplete transaction bug (#21)

3d9e58e

* Fix incomplete transaction bug * Clippy

Making C# reducer args classes partial (#28)

0ba0c8e

Co-authored-by: Steve <[email protected]>

SDK: avoid creating a nested tokio Runtime

efdd168

tokio runtimes cannot be nested, so we only create one if no ambient context can be found and pass around a runtime handle for spawning tasks.

Added an info API route

96e2d7a

[TypeScript] Add primary key annotation in TypeScript's autogen (#24)

d27d88c

bindings: address review comments (#20)

073889f

* integrate jdetter's review * address phoebe's comments * improve errnos naming

Add reducerEvent in callbacks (#23)

b221eec

Fix Spacetime Identity Remove (#33)

1a4a5ac

* Fix bug and added a test * Test updates * Fix tests --------- Co-authored-by: Boppy <[email protected]>

cloutiertyler and others added 12 commits July 29, 2023 17:37

Fixes to broken energy routes and stuff

7c44ea6

Fixed issues with identity token encoding caused by Identity type ser…

6cf7607

…ialization changes

Energy related updates

9e4a5cf

Fixed branding on the CLI

ff4e26b

cargo fmt

696bd49

Fix/improve incorrect serialization/deserialization of commit messages (

38e3692

#117)

sats: address george's review (#107)

2a43923

Add get primary key value/type functions to csharp autogen code (#118)

a4010ec

Default is now testnet.spacetimedb.com instead of (#95)

f545fd3

spacetimedb.com/spacetimedb Co-authored-by: Boppy <[email protected]>

Rename host argument in SDK's connect to spacetimedb_uri (#119)

34ce8ea

This commit renames the first argument to connect from host to spacetimedb_uri. It's not just a host(name), it's the URI of the SpacetimeDB instance.

Add quickstart-chat example module and client (#116)

1e7cf1e

This commit adds the `quickstart-chat` Rust module and client to the appropriate examples dirs.

mamcx force-pushed the mamcx/send_all branch from 6c5302c to 1d00e47 Compare July 31, 2023 18:40

Provisional 'subscribe to all' functionality

ee1eec8

mamcx force-pushed the mamcx/send_all branch from 1d00e47 to ee1eec8 Compare July 31, 2023 19:18

mamcx and others added 2 commits July 31, 2023 14:41

Update test for subscribe to thread the tx

5fa1566

Apply suggestions from code review

4519ef5

Co-authored-by: Mazdak Farrokhzad <[email protected]> Signed-off-by: Mario Montoya <[email protected]>

cloutiertyler force-pushed the master branch 3 times, most recently from 1ec98b1 to d3fd5ce Compare August 1, 2023 21:10

cloutiertyler closed this Aug 1, 2023

auto-merge was automatically disabled August 1, 2023 21:12
Pull request was closed

cloutiertyler force-pushed the master branch from d3fd5ce to 1d79d99 Compare August 1, 2023 21:12

cloutiertyler deleted the mamcx/send_all branch August 2, 2023 23:11

bfops added a commit that referenced this pull request Jul 17, 2025

Fix the C# module link in overview (#54)

f10dcf2

* [bfops/docs]: C# fix * [bfops/docs]: empty --------- Co-authored-by: Zeke Foppa <github.com/bfops>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Provisional 'subscribe to all' functionality #54

Provisional 'subscribe to all' functionality #54

Uh oh!

mamcx commented Jul 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

14 participants

Provisional 'subscribe to all' functionality #54

Provisional 'subscribe to all' functionality #54

Uh oh!

Conversation

mamcx commented Jul 3, 2023

Description of Changes

API

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

14 participants