Skip to content

Conversation

@coolreader18
Copy link
Collaborator

Description of Changes

Please describe your change, mention any related tickets, and so on here.

API and ABI breaking changes

If this is an API or ABI breaking change, please apply the
corresponding GitHub label.

Expected complexity level and risk

How complicated do you think these changes are? Grade on a scale from 1 to 5,
where 1 is a trivial change, and 5 is a deep-reaching and complex change.

This complexity rating applies not only to the complexity apparent in the diff,
but also to its interactions with existing and future code.

If you answered more than a 2, explain what is complex about the PR,
and what other components it interacts with in potentially concerning ways.

@coolreader18
Copy link
Collaborator Author

benchmarks please

@github-actions
Copy link
Contributor

github-actions bot commented Feb 12, 2024

Benchmark results

Benchmark Report

Legend:

  • load: number of rows pre-loaded into the database
  • count: number of rows touched by the transaction
  • index types:
    • unique: a single index on the id column
    • non_unique: no indexes
    • multi_index: non-unique index on every column
  • schemas:
    • person(id: u32, name: String, age: u64)
    • location(id: u32, x: u64, y: u64)

All throughputs are single-threaded.

Empty transaction

db on disk new latency old latency new throughput old throughput
sqlite 💿 - 436.8±4.16ns - -
sqlite 🧠 - 429.1±3.92ns - -
stdb_module 💿 20.0±1.56µs - - -
stdb_module 🧠 20.4±1.37µs - - -
stdb_raw 💿 937.9±1.83ns 949.7±3.62ns - -
stdb_raw 🧠 881.5±0.79ns 892.9±2.32ns - -

Single-row insertions

db on disk schema index type load new latency old latency new throughput old throughput
stdb_module 💿 location multi_index 0 53.3±4.09µs - 18.3 Ktx/sec -
stdb_module 💿 location multi_index 1000 94.9±4.67µs - 10.3 Ktx/sec -
stdb_module 💿 location non_unique 0 51.1±3.47µs - 19.1 Ktx/sec -
stdb_module 💿 location non_unique 1000 59.0±0.86µs - 16.6 Ktx/sec -
stdb_module 💿 location unique 0 51.1±4.25µs - 19.1 Ktx/sec -
stdb_module 💿 location unique 1000 87.2±4.86µs - 11.2 Ktx/sec -
stdb_module 💿 person multi_index 0 58.0±3.70µs - 16.8 Ktx/sec -
stdb_module 💿 person multi_index 1000 124.9±1.48µs - 7.8 Ktx/sec -
stdb_module 💿 person non_unique 0 54.7±3.27µs - 17.9 Ktx/sec -
stdb_module 💿 person non_unique 1000 129.2±6.99µs - 7.6 Ktx/sec -
stdb_module 💿 person unique 0 55.8±5.31µs - 17.5 Ktx/sec -
stdb_module 💿 person unique 1000 80.7±2.06µs - 12.1 Ktx/sec -
stdb_module 🧠 location multi_index 0 41.0±4.15µs - 23.8 Ktx/sec -
stdb_module 🧠 location multi_index 1000 47.7±0.20µs - 20.5 Ktx/sec -
stdb_module 🧠 location non_unique 0 37.6±2.59µs - 25.9 Ktx/sec -
stdb_module 🧠 location non_unique 1000 46.9±3.08µs - 20.8 Ktx/sec -
stdb_module 🧠 location unique 0 41.3±2.04µs - 23.6 Ktx/sec -
stdb_module 🧠 location unique 1000 68.6±1.11µs - 14.2 Ktx/sec -
stdb_module 🧠 person multi_index 0 47.5±3.06µs - 20.5 Ktx/sec -
stdb_module 🧠 person multi_index 1000 107.9±2.12µs - 9.1 Ktx/sec -
stdb_module 🧠 person non_unique 0 39.8±4.21µs - 24.5 Ktx/sec -
stdb_module 🧠 person non_unique 1000 99.5±3.20µs - 9.8 Ktx/sec -
stdb_module 🧠 person unique 0 39.2±4.62µs - 24.9 Ktx/sec -
stdb_module 🧠 person unique 1000 99.6±2.10µs - 9.8 Ktx/sec -
stdb_raw 💿 location multi_index 0 6.1±0.12µs - 159.1 Ktx/sec -
stdb_raw 💿 location multi_index 1000 9.1±0.08µs - 107.1 Ktx/sec -
stdb_raw 💿 location non_unique 0 5.4±0.02µs - 181.1 Ktx/sec -
stdb_raw 💿 location non_unique 1000 16.6±29.03µs - 58.8 Ktx/sec -
stdb_raw 💿 location unique 0 5.9±0.11µs - 164.5 Ktx/sec -
stdb_raw 💿 location unique 1000 18.7±31.84µs - 52.3 Ktx/sec -
stdb_raw 💿 person multi_index 0 7.6±0.33µs - 129.3 Ktx/sec -
stdb_raw 💿 person multi_index 1000 10.8±0.14µs - 90.1 Ktx/sec -
stdb_raw 💿 person non_unique 0 6.6±0.02µs - 148.0 Ktx/sec -
stdb_raw 💿 person non_unique 1000 8.9±0.10µs - 110.3 Ktx/sec -
stdb_raw 💿 person unique 0 7.1±0.02µs - 136.7 Ktx/sec -
stdb_raw 💿 person unique 1000 10.2±0.78µs - 95.7 Ktx/sec -
stdb_raw 🧠 location multi_index 0 3.4±0.00µs - 285.0 Ktx/sec -
stdb_raw 🧠 location multi_index 1000 5.1±0.01µs - 192.1 Ktx/sec -
stdb_raw 🧠 location non_unique 0 2.8±0.00µs - 343.4 Ktx/sec -
stdb_raw 🧠 location non_unique 1000 3.6±0.01µs - 269.7 Ktx/sec -
stdb_raw 🧠 location unique 0 3.3±0.00µs - 295.0 Ktx/sec -
stdb_raw 🧠 location unique 1000 4.6±0.02µs - 210.1 Ktx/sec -
stdb_raw 🧠 person multi_index 0 4.2±0.01µs - 231.3 Ktx/sec -
stdb_raw 🧠 person multi_index 1000 6.2±0.06µs - 158.7 Ktx/sec -
stdb_raw 🧠 person non_unique 0 3.5±0.02µs - 282.3 Ktx/sec -
stdb_raw 🧠 person non_unique 1000 4.5±0.04µs - 218.6 Ktx/sec -
stdb_raw 🧠 person unique 0 4.0±0.01µs - 245.0 Ktx/sec -
stdb_raw 🧠 person unique 1000 5.4±0.03µs - 179.3 Ktx/sec -

Multi-row insertions

db on disk schema index type load count new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str btree_each_column 1024 64 - 168.5±0.34µs - 5.8 Ktx/sec
sqlite 💿 u32_u64_str btree_each_column 128 64 - 109.2±2.95µs - 8.9 Ktx/sec
sqlite 💿 u32_u64_str no_index 1024 64 - 42.0±0.19µs - 23.3 Ktx/sec
sqlite 💿 u32_u64_str no_index 128 64 - 43.1±0.12µs - 22.7 Ktx/sec
sqlite 💿 u32_u64_str unique_0 1024 64 - 39.9±4.99µs - 24.5 Ktx/sec
sqlite 💿 u32_u64_str unique_0 128 64 - 43.2±1.30µs - 22.6 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 1024 64 - 145.2±15.46µs - 6.7 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 128 64 - 94.5±0.59µs - 10.3 Ktx/sec
sqlite 💿 u32_u64_u64 no_index 1024 64 - 31.8±0.13µs - 30.7 Ktx/sec
sqlite 💿 u32_u64_u64 no_index 128 64 - 29.8±1.62µs - 32.8 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 1024 64 - 34.2±0.16µs - 28.5 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 128 64 - 31.2±1.09µs - 31.3 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 1024 64 - 125.6±0.09µs - 7.8 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 128 64 - 84.9±0.13µs - 11.5 Ktx/sec
sqlite 🧠 u32_u64_str no_index 1024 64 - 29.8±0.14µs - 32.8 Ktx/sec
sqlite 🧠 u32_u64_str no_index 128 64 - 29.0±0.15µs - 33.6 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 1024 64 - 31.5±0.26µs - 31.0 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 128 64 - 31.8±0.08µs - 30.7 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 1024 64 - 112.2±0.22µs - 8.7 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 128 64 - 83.1±0.20µs - 11.7 Ktx/sec
sqlite 🧠 u32_u64_u64 no_index 1024 64 - 24.3±0.08µs - 40.3 Ktx/sec
sqlite 🧠 u32_u64_u64 no_index 128 64 - 23.3±0.15µs - 42.0 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 1024 64 - 26.8±0.15µs - 36.4 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 128 64 - 24.9±0.22µs - 39.3 Ktx/sec
stdb_module 💿 location multi_index 0 100 519.0±0.50µs - 1926 tx/sec -
stdb_module 💿 location multi_index 1000 100 554.5±2.27µs - 1803 tx/sec -
stdb_module 💿 location non_unique 0 100 433.8±0.29µs - 2.3 Ktx/sec -
stdb_module 💿 location non_unique 1000 100 455.6±1.72µs - 2.1 Ktx/sec -
stdb_module 💿 location unique 0 100 500.7±0.88µs - 1997 tx/sec -
stdb_module 💿 location unique 1000 100 551.4±10.25µs - 1813 tx/sec -
stdb_module 💿 person multi_index 0 100 842.2±9.59µs - 1187 tx/sec -
stdb_module 💿 person multi_index 1000 100 804.7±0.66µs - 1242 tx/sec -
stdb_module 💿 person non_unique 0 100 644.9±52.26µs - 1550 tx/sec -
stdb_module 💿 person non_unique 1000 100 752.8±1.24µs - 1328 tx/sec -
stdb_module 💿 person unique 0 100 751.7±0.56µs - 1330 tx/sec -
stdb_module 💿 person unique 1000 100 695.6±62.14µs - 1437 tx/sec -
stdb_module 🧠 location multi_index 0 100 257.3±5.98µs - 3.8 Ktx/sec -
stdb_module 🧠 location multi_index 1000 100 335.3±25.73µs - 2.9 Ktx/sec -
stdb_module 🧠 location non_unique 0 100 277.1±9.61µs - 3.5 Ktx/sec -
stdb_module 🧠 location non_unique 1000 100 248.2±28.39µs - 3.9 Ktx/sec -
stdb_module 🧠 location unique 0 100 258.7±17.60µs - 3.8 Ktx/sec -
stdb_module 🧠 location unique 1000 100 406.9±0.35µs - 2.4 Ktx/sec -
stdb_module 🧠 person multi_index 0 100 579.8±46.72µs - 1724 tx/sec -
stdb_module 🧠 person multi_index 1000 100 665.9±2.78µs - 1501 tx/sec -
stdb_module 🧠 person non_unique 0 100 452.9±23.78µs - 2.2 Ktx/sec -
stdb_module 🧠 person non_unique 1000 100 500.9±8.89µs - 1996 tx/sec -
stdb_module 🧠 person unique 0 100 560.3±2.57µs - 1784 tx/sec -
stdb_module 🧠 person unique 1000 100 563.3±53.64µs - 1775 tx/sec -
stdb_raw 💿 location multi_index 0 100 171.3±6.49µs - 5.7 Ktx/sec -
stdb_raw 💿 location multi_index 1000 100 184.0±0.65µs - 5.3 Ktx/sec -
stdb_raw 💿 location non_unique 0 100 128.1±0.16µs - 7.6 Ktx/sec -
stdb_raw 💿 location non_unique 1000 100 145.4±48.85µs - 6.7 Ktx/sec -
stdb_raw 💿 location unique 0 100 164.7±0.14µs - 5.9 Ktx/sec -
stdb_raw 💿 location unique 1000 100 172.5±0.50µs - 5.7 Ktx/sec -
stdb_raw 💿 person multi_index 0 100 284.5±0.46µs - 3.4 Ktx/sec -
stdb_raw 💿 person multi_index 1000 100 296.7±0.33µs - 3.3 Ktx/sec -
stdb_raw 💿 person non_unique 0 100 216.7±0.07µs - 4.5 Ktx/sec -
stdb_raw 💿 person non_unique 1000 100 218.8±0.26µs - 4.5 Ktx/sec -
stdb_raw 💿 person unique 0 100 256.8±3.29µs - 3.8 Ktx/sec -
stdb_raw 💿 person unique 1000 100 264.8±0.19µs - 3.7 Ktx/sec -
stdb_raw 💿 u32_u64_str btree_each_column 1024 64 - 196.1±0.17µs - 5.0 Ktx/sec
stdb_raw 💿 u32_u64_str btree_each_column 128 64 - 193.7±7.46µs - 5.0 Ktx/sec
stdb_raw 💿 u32_u64_str no_index 1024 64 - 148.4±0.12µs - 6.6 Ktx/sec
stdb_raw 💿 u32_u64_str no_index 128 64 - 150.4±0.21µs - 6.5 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 1024 64 - 175.3±0.18µs - 5.6 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 128 64 - 173.5±0.13µs - 5.6 Ktx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 1024 64 - 133.3±42.92µs - 7.3 Ktx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 128 64 - 113.2±0.18µs - 8.6 Ktx/sec
stdb_raw 💿 u32_u64_u64 no_index 1024 64 - 85.9±0.11µs - 11.4 Ktx/sec
stdb_raw 💿 u32_u64_u64 no_index 128 64 - 87.2±11.74µs - 11.2 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 1024 64 - 119.1±23.06µs - 8.2 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 128 64 - 106.7±0.10µs - 9.1 Ktx/sec
stdb_raw 🧠 location multi_index 0 100 119.6±0.10µs - 8.2 Ktx/sec -
stdb_raw 🧠 location multi_index 1000 100 134.0±0.12µs - 7.3 Ktx/sec -
stdb_raw 🧠 location non_unique 0 100 80.1±0.06µs - 12.2 Ktx/sec -
stdb_raw 🧠 location non_unique 1000 100 82.2±0.11µs - 11.9 Ktx/sec -
stdb_raw 🧠 location unique 0 100 117.1±0.24µs - 8.3 Ktx/sec -
stdb_raw 🧠 location unique 1000 100 124.1±0.55µs - 7.9 Ktx/sec -
stdb_raw 🧠 person multi_index 0 100 186.9±0.07µs - 5.2 Ktx/sec -
stdb_raw 🧠 person multi_index 1000 100 198.9±0.21µs - 4.9 Ktx/sec -
stdb_raw 🧠 person non_unique 0 100 122.8±0.08µs - 8.0 Ktx/sec -
stdb_raw 🧠 person non_unique 1000 100 124.3±0.07µs - 7.9 Ktx/sec -
stdb_raw 🧠 person unique 0 100 161.3±0.19µs - 6.1 Ktx/sec -
stdb_raw 🧠 person unique 1000 100 169.0±0.25µs - 5.8 Ktx/sec -
stdb_raw 🧠 u32_u64_str btree_each_column 1024 64 - 127.1±0.08µs - 7.7 Ktx/sec
stdb_raw 🧠 u32_u64_str btree_each_column 128 64 - 120.9±0.24µs - 8.1 Ktx/sec
stdb_raw 🧠 u32_u64_str no_index 1024 64 - 81.9±0.08µs - 11.9 Ktx/sec
stdb_raw 🧠 u32_u64_str no_index 128 64 - 83.2±0.04µs - 11.7 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 1024 64 - 108.6±0.08µs - 9.0 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 128 64 - 105.4±0.16µs - 9.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 1024 64 - 85.7±0.08µs - 11.4 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 128 64 - 79.3±0.11µs - 12.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 no_index 1024 64 - 53.4±0.07µs - 18.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 no_index 128 64 - 52.1±0.06µs - 18.7 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 1024 64 - 79.6±0.10µs - 12.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 128 64 - 74.6±0.08µs - 13.1 Ktx/sec

Full table iterate

db on disk schema index type new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str unique_0 - 9.4±0.02µs - 103.5 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 - 8.8±0.03µs - 110.7 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 - 8.3±0.04µs - 117.8 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 - 7.5±0.04µs - 130.3 Ktx/sec
stdb_module 💿 location unique 54.5±3.97µs - 17.9 Ktx/sec -
stdb_module 💿 person unique 66.0±2.44µs - 14.8 Ktx/sec -
stdb_module 🧠 location unique 55.7±4.99µs - 17.5 Ktx/sec -
stdb_module 🧠 person unique 63.1±2.74µs - 15.5 Ktx/sec -
stdb_raw 💿 location unique 12.5±0.00µs - 77.9 Ktx/sec -
stdb_raw 💿 person unique 13.4±0.00µs - 73.0 Ktx/sec -
stdb_raw 💿 u32_u64_str unique_0 - 13.4±0.01µs - 73.0 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 - 12.5±0.12µs - 78.0 Ktx/sec
stdb_raw 🧠 location unique 12.5±0.00µs - 78.3 Ktx/sec -
stdb_raw 🧠 person unique 13.3±0.01µs - 73.4 Ktx/sec -
stdb_raw 🧠 u32_u64_str unique_0 - 13.3±0.01µs - 73.4 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 - 12.4±0.01µs - 78.6 Ktx/sec

Find unique key

db on disk key type load new latency old latency new throughput old throughput
stdb_module 💿 u32 1000 29.3±2.59µs - 33.3 Ktx/sec -
stdb_module 🧠 u32 1000 28.3±1.29µs - 34.5 Ktx/sec -
stdb_raw 💿 u32 1000 1827.2±18.14ns - 534.5 Ktx/sec -
stdb_raw 🧠 u32 1000 1755.3±1.52ns - 556.4 Ktx/sec -

Filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string index 1000 10 - 5.5±0.01µs - 177.9 Ktx/sec
sqlite 💿 string no_index 1000 10 - 49.0±0.07µs - 19.9 Ktx/sec
sqlite 💿 u64 index 1000 10 - 5.2±0.02µs - 188.2 Ktx/sec
sqlite 💿 u64 no_index 1000 10 - 34.4±0.04µs - 28.4 Ktx/sec
sqlite 🧠 string index 1000 10 - 4.1±0.01µs - 238.6 Ktx/sec
sqlite 🧠 string no_index 1000 10 - 47.8±0.19µs - 20.4 Ktx/sec
sqlite 🧠 u64 index 1000 10 - 3.8±0.01µs - 257.3 Ktx/sec
sqlite 🧠 u64 no_index 1000 10 - 32.1±0.03µs - 30.4 Ktx/sec
stdb_module 💿 string indexed 1000 10 35.2±0.94µs - 27.8 Ktx/sec -
stdb_module 💿 string non_indexed 1000 10 215.2±0.25µs - 4.5 Ktx/sec -
stdb_module 💿 u64 indexed 1000 10 32.5±1.37µs - 30.1 Ktx/sec -
stdb_module 💿 u64 non_indexed 1000 10 185.3±2.55µs - 5.3 Ktx/sec -
stdb_module 🧠 string indexed 1000 10 35.6±2.02µs - 27.4 Ktx/sec -
stdb_module 🧠 string non_indexed 1000 10 217.4±2.73µs - 4.5 Ktx/sec -
stdb_module 🧠 u64 indexed 1000 10 30.8±1.82µs - 31.7 Ktx/sec -
stdb_module 🧠 u64 non_indexed 1000 10 187.4±4.76µs - 5.2 Ktx/sec -
stdb_raw 💿 string index 1000 10 - 2.6±0.01µs - 372.8 Ktx/sec
stdb_raw 💿 string indexed 1000 10 2.6±0.00µs - 378.2 Ktx/sec -
stdb_raw 💿 string no_index 1000 10 - 189.3±0.03µs - 5.2 Ktx/sec
stdb_raw 💿 string non_indexed 1000 10 205.8±0.82µs - 4.7 Ktx/sec -
stdb_raw 💿 u64 index 1000 10 - 2.5±0.01µs - 385.6 Ktx/sec
stdb_raw 💿 u64 indexed 1000 10 2.5±0.00µs - 392.2 Ktx/sec -
stdb_raw 💿 u64 no_index 1000 10 - 161.3±0.06µs - 6.1 Ktx/sec
stdb_raw 💿 u64 non_indexed 1000 10 173.0±0.05µs - 5.6 Ktx/sec -
stdb_raw 🧠 string index 1000 10 - 2.5±0.00µs - 385.8 Ktx/sec
stdb_raw 🧠 string indexed 1000 10 2.5±0.00µs - 392.2 Ktx/sec -
stdb_raw 🧠 string no_index 1000 10 - 189.5±0.53µs - 5.2 Ktx/sec
stdb_raw 🧠 string non_indexed 1000 10 205.6±0.06µs - 4.8 Ktx/sec -
stdb_raw 🧠 u64 index 1000 10 - 2.5±0.00µs - 398.0 Ktx/sec
stdb_raw 🧠 u64 indexed 1000 10 2.4±0.00µs - 403.0 Ktx/sec -
stdb_raw 🧠 u64 no_index 1000 10 - 161.1±0.04µs - 6.1 Ktx/sec
stdb_raw 🧠 u64 non_indexed 1000 10 173.0±0.07µs - 5.6 Ktx/sec -

Serialize

schema format count new latency old latency new throughput old throughput
location bsatn 100 1766.3±101.65ns - 54.0 Mtx/sec -
location json 100 3.4±0.09µs - 28.3 Mtx/sec -
location product_value 100 571.7±1.12ns - 166.8 Mtx/sec -
person bsatn 100 2.5±0.06µs - 38.7 Mtx/sec -
person json 100 4.7±0.06µs - 20.1 Mtx/sec -
person product_value 100 671.6±0.37ns - 142.0 Mtx/sec -
u32_u64_str bsatn 100 - 2.9±0.00µs - 32.7 Mtx/sec
u32_u64_str json 100 - 5.0±0.03µs - 19.2 Mtx/sec
u32_u64_str product_value 100 - 652.2±41.96ns - 146.2 Mtx/sec
u32_u64_u64 bsatn 100 - 1667.1±34.47ns - 57.2 Mtx/sec
u32_u64_u64 json 100 - 3.0±0.02µs - 31.5 Mtx/sec
u32_u64_u64 product_value 100 - 625.4±0.26ns - 152.5 Mtx/sec

Module: invoke with large arguments

arg size new latency old latency new throughput old throughput
64KiB - 89.8±7.16µs - -
64KiB 63.1±10.22µs - - -

Module: print bulk

line count new latency old latency new throughput old throughput
1 - 43.0±4.23µs - -
1 28.0±2.01µs - - -
100 - 217.7±1.41µs - -
100 199.6±0.67µs - - -
1000 - 1964.6±66.80µs - -
1000 1825.9±5.93µs - - -

Remaining benchmarks

name new latency old latency new throughput old throughput
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=1024/count=100 - 21.9±0.10µs - 44.7 Ktx/sec
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=128/count=64 - 17.3±0.12µs - 56.6 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=1024/count=100 - 20.6±0.05µs - 47.4 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=128/count=64 - 15.6±0.01µs - 62.6 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=1024/count=100 - 16.1±0.02µs - 60.7 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=128/count=64 - 11.2±0.06µs - 87.5 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=1024/count=100 - 14.8±0.01µs - 66.2 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=128/count=64 - 10.4±0.01µs - 94.0 Ktx/sec
stdb_module/💿/update_bulk/location/unique/load=1000/count=10 499.7±17.29µs - 2001 tx/sec -
stdb_module/💿/update_bulk/location/unique/load=1000/count=100 1244.8±10.73µs - 803 tx/sec -
stdb_module/💿/update_bulk/person/unique/load=1000/count=10 580.7±1.82µs - 1722 tx/sec -
stdb_module/💿/update_bulk/person/unique/load=1000/count=100 1003.3±0.73µs - 996 tx/sec -
stdb_module/🧠/update_bulk/location/unique/load=1000/count=10 297.3±15.05µs - 3.3 Ktx/sec -
stdb_module/🧠/update_bulk/location/unique/load=1000/count=100 685.6±25.30µs - 1458 tx/sec -
stdb_module/🧠/update_bulk/person/unique/load=1000/count=10 538.5±7.61µs - 1857 tx/sec -
stdb_module/🧠/update_bulk/person/unique/load=1000/count=100 825.3±0.84µs - 1211 tx/sec -
stdb_raw/💿/update_bulk/location/unique/load=1000/count=10 52.0±21.44µs - 18.8 Ktx/sec -
stdb_raw/💿/update_bulk/location/unique/load=1000/count=100 373.0±29.46µs - 2.6 Ktx/sec -
stdb_raw/💿/update_bulk/person/unique/load=1000/count=10 79.1±57.38µs - 12.4 Ktx/sec -
stdb_raw/💿/update_bulk/person/unique/load=1000/count=100 510.6±0.48µs - 1958 tx/sec -
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=1024/count=100 - 517.8±0.62µs - 1931 tx/sec
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=128/count=64 - 323.7±0.20µs - 3.0 Ktx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=1024/count=100 - 362.0±0.34µs - 2.7 Ktx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=128/count=64 - 234.1±0.06µs - 4.2 Ktx/sec
stdb_raw/🧠/update_bulk/location/unique/load=1000/count=10 31.8±0.04µs - 30.7 Ktx/sec -
stdb_raw/🧠/update_bulk/location/unique/load=1000/count=100 270.7±0.27µs - 3.6 Ktx/sec -
stdb_raw/🧠/update_bulk/person/unique/load=1000/count=10 42.3±0.07µs - 23.1 Ktx/sec -
stdb_raw/🧠/update_bulk/person/unique/load=1000/count=100 366.1±0.20µs - 2.7 Ktx/sec -
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=1024/count=100 - 364.3±0.43µs - 2.7 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=128/count=64 - 226.7±0.29µs - 4.3 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=1024/count=100 - 268.3±0.25µs - 3.6 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=128/count=64 - 173.5±0.15µs - 5.6 Ktx/sec

@github-actions
Copy link
Contributor

github-actions bot commented Feb 12, 2024

Benchmark results <title>502 Bad Gateway</title>

502 Bad Gateway


nginx

@github-actions
Copy link
Contributor

github-actions bot commented Feb 12, 2024

Benchmark results

Benchmark Report

Legend:

  • load: number of rows pre-loaded into the database
  • count: number of rows touched by the transaction
  • index types:
    • unique: a single index on the id column
    • non_unique: no indexes
    • multi_index: non-unique index on every column
  • schemas:
    • person(id: u32, name: String, age: u64)
    • location(id: u32, x: u64, y: u64)

All throughputs are single-threaded.

Empty transaction

db on disk new latency old latency new throughput old throughput
sqlite 💿 - 436.8±4.16ns - -
sqlite 🧠 - 429.1±3.92ns - -
stdb_module 💿 20.7±1.66µs - - -
stdb_module 🧠 20.6±1.44µs - - -
stdb_raw 💿 940.5±1.17ns 949.7±3.62ns - -
stdb_raw 🧠 883.7±1.56ns 892.9±2.32ns - -

Single-row insertions

db on disk schema index type load new latency old latency new throughput old throughput
stdb_module 💿 location multi_index 0 55.0±6.20µs - 17.8 Ktx/sec -
stdb_module 💿 location multi_index 1000 91.5±0.67µs - 10.7 Ktx/sec -
stdb_module 💿 location non_unique 0 52.4±3.21µs - 18.6 Ktx/sec -
stdb_module 💿 location non_unique 1000 57.5±3.59µs - 17.0 Ktx/sec -
stdb_module 💿 location unique 0 51.3±4.31µs - 19.0 Ktx/sec -
stdb_module 💿 location unique 1000 61.5±2.41µs - 15.9 Ktx/sec -
stdb_module 💿 person multi_index 0 60.8±5.12µs - 16.1 Ktx/sec -
stdb_module 💿 person multi_index 1000 125.5±1.51µs - 7.8 Ktx/sec -
stdb_module 💿 person non_unique 0 53.7±3.29µs - 18.2 Ktx/sec -
stdb_module 💿 person non_unique 1000 120.1±15.13µs - 8.1 Ktx/sec -
stdb_module 💿 person unique 0 56.9±3.73µs - 17.2 Ktx/sec -
stdb_module 💿 person unique 1000 107.6±5.51µs - 9.1 Ktx/sec -
stdb_module 🧠 location multi_index 0 40.5±2.17µs - 24.1 Ktx/sec -
stdb_module 🧠 location multi_index 1000 54.3±3.10µs - 18.0 Ktx/sec -
stdb_module 🧠 location non_unique 0 36.3±3.29µs - 26.9 Ktx/sec -
stdb_module 🧠 location non_unique 1000 76.0±0.74µs - 12.9 Ktx/sec -
stdb_module 🧠 location unique 0 39.9±3.20µs - 24.5 Ktx/sec -
stdb_module 🧠 location unique 1000 40.6±3.20µs - 24.0 Ktx/sec -
stdb_module 🧠 person multi_index 0 42.8±4.37µs - 22.8 Ktx/sec -
stdb_module 🧠 person multi_index 1000 106.1±1.57µs - 9.2 Ktx/sec -
stdb_module 🧠 person non_unique 0 42.0±5.26µs - 23.3 Ktx/sec -
stdb_module 🧠 person non_unique 1000 61.7±1.42µs - 15.8 Ktx/sec -
stdb_module 🧠 person unique 0 41.3±5.72µs - 23.6 Ktx/sec -
stdb_module 🧠 person unique 1000 58.5±8.52µs - 16.7 Ktx/sec -
stdb_raw 💿 location multi_index 0 6.2±0.01µs - 157.4 Ktx/sec -
stdb_raw 💿 location multi_index 1000 15.2±20.21µs - 64.3 Ktx/sec -
stdb_raw 💿 location non_unique 0 5.5±0.03µs - 179.1 Ktx/sec -
stdb_raw 💿 location non_unique 1000 7.1±0.07µs - 136.7 Ktx/sec -
stdb_raw 💿 location unique 0 6.0±0.00µs - 162.4 Ktx/sec -
stdb_raw 💿 location unique 1000 8.4±0.07µs - 116.5 Ktx/sec -
stdb_raw 💿 person multi_index 0 7.5±0.01µs - 129.6 Ktx/sec -
stdb_raw 💿 person multi_index 1000 11.0±0.13µs - 88.9 Ktx/sec -
stdb_raw 💿 person non_unique 0 6.7±0.03µs - 145.4 Ktx/sec -
stdb_raw 💿 person non_unique 1000 27.5±186.23µs - 35.5 Ktx/sec -
stdb_raw 💿 person unique 0 7.3±0.02µs - 134.6 Ktx/sec -
stdb_raw 💿 person unique 1000 10.1±0.14µs - 96.9 Ktx/sec -
stdb_raw 🧠 location multi_index 0 3.4±0.01µs - 285.7 Ktx/sec -
stdb_raw 🧠 location multi_index 1000 5.0±0.01µs - 195.6 Ktx/sec -
stdb_raw 🧠 location non_unique 0 2.8±0.01µs - 348.0 Ktx/sec -
stdb_raw 🧠 location non_unique 1000 3.6±0.01µs - 274.8 Ktx/sec -
stdb_raw 🧠 location unique 0 3.3±0.01µs - 299.6 Ktx/sec -
stdb_raw 🧠 location unique 1000 4.5±0.02µs - 214.7 Ktx/sec -
stdb_raw 🧠 person multi_index 0 4.2±0.01µs - 230.8 Ktx/sec -
stdb_raw 🧠 person multi_index 1000 6.1±0.06µs - 159.0 Ktx/sec -
stdb_raw 🧠 person non_unique 0 3.5±0.01µs - 279.7 Ktx/sec -
stdb_raw 🧠 person non_unique 1000 4.3±0.02µs - 224.8 Ktx/sec -
stdb_raw 🧠 person unique 0 4.0±0.01µs - 246.5 Ktx/sec -
stdb_raw 🧠 person unique 1000 5.5±0.13µs - 177.7 Ktx/sec -

Multi-row insertions

db on disk schema index type load count new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str btree_each_column 1024 64 - 168.5±0.34µs - 5.8 Ktx/sec
sqlite 💿 u32_u64_str btree_each_column 128 64 - 109.2±2.95µs - 8.9 Ktx/sec
sqlite 💿 u32_u64_str no_index 1024 64 - 42.0±0.19µs - 23.3 Ktx/sec
sqlite 💿 u32_u64_str no_index 128 64 - 43.1±0.12µs - 22.7 Ktx/sec
sqlite 💿 u32_u64_str unique_0 1024 64 - 39.9±4.99µs - 24.5 Ktx/sec
sqlite 💿 u32_u64_str unique_0 128 64 - 43.2±1.30µs - 22.6 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 1024 64 - 145.2±15.46µs - 6.7 Ktx/sec
sqlite 💿 u32_u64_u64 btree_each_column 128 64 - 94.5±0.59µs - 10.3 Ktx/sec
sqlite 💿 u32_u64_u64 no_index 1024 64 - 31.8±0.13µs - 30.7 Ktx/sec
sqlite 💿 u32_u64_u64 no_index 128 64 - 29.8±1.62µs - 32.8 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 1024 64 - 34.2±0.16µs - 28.5 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 128 64 - 31.2±1.09µs - 31.3 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 1024 64 - 125.6±0.09µs - 7.8 Ktx/sec
sqlite 🧠 u32_u64_str btree_each_column 128 64 - 84.9±0.13µs - 11.5 Ktx/sec
sqlite 🧠 u32_u64_str no_index 1024 64 - 29.8±0.14µs - 32.8 Ktx/sec
sqlite 🧠 u32_u64_str no_index 128 64 - 29.0±0.15µs - 33.6 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 1024 64 - 31.5±0.26µs - 31.0 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 128 64 - 31.8±0.08µs - 30.7 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 1024 64 - 112.2±0.22µs - 8.7 Ktx/sec
sqlite 🧠 u32_u64_u64 btree_each_column 128 64 - 83.1±0.20µs - 11.7 Ktx/sec
sqlite 🧠 u32_u64_u64 no_index 1024 64 - 24.3±0.08µs - 40.3 Ktx/sec
sqlite 🧠 u32_u64_u64 no_index 128 64 - 23.3±0.15µs - 42.0 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 1024 64 - 26.8±0.15µs - 36.4 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 128 64 - 24.9±0.22µs - 39.3 Ktx/sec
stdb_module 💿 location multi_index 0 100 477.7±11.79µs - 2.0 Ktx/sec -
stdb_module 💿 location multi_index 1000 100 515.2±9.49µs - 1940 tx/sec -
stdb_module 💿 location non_unique 0 100 405.8±9.45µs - 2.4 Ktx/sec -
stdb_module 💿 location non_unique 1000 100 413.9±1.90µs - 2.4 Ktx/sec -
stdb_module 💿 location unique 0 100 472.1±6.94µs - 2.1 Ktx/sec -
stdb_module 💿 location unique 1000 100 540.0±10.72µs - 1851 tx/sec -
stdb_module 💿 person multi_index 0 100 810.2±42.76µs - 1234 tx/sec -
stdb_module 💿 person multi_index 1000 100 778.3±50.43µs - 1284 tx/sec -
stdb_module 💿 person non_unique 0 100 677.0±0.45µs - 1477 tx/sec -
stdb_module 💿 person non_unique 1000 100 735.0±12.73µs - 1360 tx/sec -
stdb_module 💿 person unique 0 100 745.4±22.17µs - 1341 tx/sec -
stdb_module 💿 person unique 1000 100 754.0±10.97µs - 1326 tx/sec -
stdb_module 🧠 location multi_index 0 100 399.7±50.63µs - 2.4 Ktx/sec -
stdb_module 🧠 location multi_index 1000 100 433.4±10.54µs - 2.3 Ktx/sec -
stdb_module 🧠 location non_unique 0 100 274.4±42.14µs - 3.6 Ktx/sec -
stdb_module 🧠 location non_unique 1000 100 308.0±9.36µs - 3.2 Ktx/sec -
stdb_module 🧠 location unique 0 100 412.8±15.82µs - 2.4 Ktx/sec -
stdb_module 🧠 location unique 1000 100 409.2±54.68µs - 2.4 Ktx/sec -
stdb_module 🧠 person multi_index 0 100 606.4±0.55µs - 1649 tx/sec -
stdb_module 🧠 person multi_index 1000 100 643.7±1.02µs - 1553 tx/sec -
stdb_module 🧠 person non_unique 0 100 462.4±8.58µs - 2.1 Ktx/sec -
stdb_module 🧠 person non_unique 1000 100 503.5±0.88µs - 1986 tx/sec -
stdb_module 🧠 person unique 0 100 532.2±8.71µs - 1878 tx/sec -
stdb_module 🧠 person unique 1000 100 585.4±12.59µs - 1708 tx/sec -
stdb_raw 💿 location multi_index 0 100 167.2±0.11µs - 5.8 Ktx/sec -
stdb_raw 💿 location multi_index 1000 100 193.9±37.03µs - 5.0 Ktx/sec -
stdb_raw 💿 location non_unique 0 100 128.9±4.70µs - 7.6 Ktx/sec -
stdb_raw 💿 location non_unique 1000 100 129.3±0.53µs - 7.6 Ktx/sec -
stdb_raw 💿 location unique 0 100 163.4±1.90µs - 6.0 Ktx/sec -
stdb_raw 💿 location unique 1000 100 177.9±23.58µs - 5.5 Ktx/sec -
stdb_raw 💿 person multi_index 0 100 283.7±0.32µs - 3.4 Ktx/sec -
stdb_raw 💿 person multi_index 1000 100 323.6±83.06µs - 3.0 Ktx/sec -
stdb_raw 💿 person non_unique 0 100 218.3±0.13µs - 4.5 Ktx/sec -
stdb_raw 💿 person non_unique 1000 100 233.6±39.97µs - 4.2 Ktx/sec -
stdb_raw 💿 person unique 0 100 258.0±3.79µs - 3.8 Ktx/sec -
stdb_raw 💿 person unique 1000 100 266.3±0.25µs - 3.7 Ktx/sec -
stdb_raw 💿 u32_u64_str btree_each_column 1024 64 - 196.1±0.17µs - 5.0 Ktx/sec
stdb_raw 💿 u32_u64_str btree_each_column 128 64 - 193.7±7.46µs - 5.0 Ktx/sec
stdb_raw 💿 u32_u64_str no_index 1024 64 - 148.4±0.12µs - 6.6 Ktx/sec
stdb_raw 💿 u32_u64_str no_index 128 64 - 150.4±0.21µs - 6.5 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 1024 64 - 175.3±0.18µs - 5.6 Ktx/sec
stdb_raw 💿 u32_u64_str unique_0 128 64 - 173.5±0.13µs - 5.6 Ktx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 1024 64 - 133.3±42.92µs - 7.3 Ktx/sec
stdb_raw 💿 u32_u64_u64 btree_each_column 128 64 - 113.2±0.18µs - 8.6 Ktx/sec
stdb_raw 💿 u32_u64_u64 no_index 1024 64 - 85.9±0.11µs - 11.4 Ktx/sec
stdb_raw 💿 u32_u64_u64 no_index 128 64 - 87.2±11.74µs - 11.2 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 1024 64 - 119.1±23.06µs - 8.2 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 128 64 - 106.7±0.10µs - 9.1 Ktx/sec
stdb_raw 🧠 location multi_index 0 100 119.2±0.10µs - 8.2 Ktx/sec -
stdb_raw 🧠 location multi_index 1000 100 133.6±0.16µs - 7.3 Ktx/sec -
stdb_raw 🧠 location non_unique 0 100 80.3±0.04µs - 12.2 Ktx/sec -
stdb_raw 🧠 location non_unique 1000 100 82.2±0.08µs - 11.9 Ktx/sec -
stdb_raw 🧠 location unique 0 100 115.9±0.25µs - 8.4 Ktx/sec -
stdb_raw 🧠 location unique 1000 100 122.7±0.15µs - 8.0 Ktx/sec -
stdb_raw 🧠 person multi_index 0 100 186.9±0.10µs - 5.2 Ktx/sec -
stdb_raw 🧠 person multi_index 1000 100 199.9±0.21µs - 4.9 Ktx/sec -
stdb_raw 🧠 person non_unique 0 100 124.7±0.10µs - 7.8 Ktx/sec -
stdb_raw 🧠 person non_unique 1000 100 125.8±0.18µs - 7.8 Ktx/sec -
stdb_raw 🧠 person unique 0 100 161.8±0.19µs - 6.0 Ktx/sec -
stdb_raw 🧠 person unique 1000 100 170.5±0.30µs - 5.7 Ktx/sec -
stdb_raw 🧠 u32_u64_str btree_each_column 1024 64 - 127.1±0.08µs - 7.7 Ktx/sec
stdb_raw 🧠 u32_u64_str btree_each_column 128 64 - 120.9±0.24µs - 8.1 Ktx/sec
stdb_raw 🧠 u32_u64_str no_index 1024 64 - 81.9±0.08µs - 11.9 Ktx/sec
stdb_raw 🧠 u32_u64_str no_index 128 64 - 83.2±0.04µs - 11.7 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 1024 64 - 108.6±0.08µs - 9.0 Ktx/sec
stdb_raw 🧠 u32_u64_str unique_0 128 64 - 105.4±0.16µs - 9.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 1024 64 - 85.7±0.08µs - 11.4 Ktx/sec
stdb_raw 🧠 u32_u64_u64 btree_each_column 128 64 - 79.3±0.11µs - 12.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 no_index 1024 64 - 53.4±0.07µs - 18.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 no_index 128 64 - 52.1±0.06µs - 18.7 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 1024 64 - 79.6±0.10µs - 12.3 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 128 64 - 74.6±0.08µs - 13.1 Ktx/sec

Full table iterate

db on disk schema index type new latency old latency new throughput old throughput
sqlite 💿 u32_u64_str unique_0 - 9.4±0.02µs - 103.5 Ktx/sec
sqlite 💿 u32_u64_u64 unique_0 - 8.8±0.03µs - 110.7 Ktx/sec
sqlite 🧠 u32_u64_str unique_0 - 8.3±0.04µs - 117.8 Ktx/sec
sqlite 🧠 u32_u64_u64 unique_0 - 7.5±0.04µs - 130.3 Ktx/sec
stdb_module 💿 location unique 54.8±5.02µs - 17.8 Ktx/sec -
stdb_module 💿 person unique 64.6±4.29µs - 15.1 Ktx/sec -
stdb_module 🧠 location unique 55.3±6.50µs - 17.7 Ktx/sec -
stdb_module 🧠 person unique 63.4±2.14µs - 15.4 Ktx/sec -
stdb_raw 💿 location unique 12.5±0.00µs - 77.9 Ktx/sec -
stdb_raw 💿 person unique 13.4±0.00µs - 73.0 Ktx/sec -
stdb_raw 💿 u32_u64_str unique_0 - 13.4±0.01µs - 73.0 Ktx/sec
stdb_raw 💿 u32_u64_u64 unique_0 - 12.5±0.12µs - 78.0 Ktx/sec
stdb_raw 🧠 location unique 12.5±0.00µs - 78.3 Ktx/sec -
stdb_raw 🧠 person unique 13.3±0.00µs - 73.4 Ktx/sec -
stdb_raw 🧠 u32_u64_str unique_0 - 13.3±0.01µs - 73.4 Ktx/sec
stdb_raw 🧠 u32_u64_u64 unique_0 - 12.4±0.01µs - 78.6 Ktx/sec

Find unique key

db on disk key type load new latency old latency new throughput old throughput
stdb_module 💿 u32 1000 28.9±1.24µs - 33.8 Ktx/sec -
stdb_module 🧠 u32 1000 27.7±1.37µs - 35.2 Ktx/sec -
stdb_raw 💿 u32 1000 1826.5±3.68ns - 534.7 Ktx/sec -
stdb_raw 🧠 u32 1000 1762.5±0.90ns - 554.1 Ktx/sec -

Filter

db on disk key type index strategy load count new latency old latency new throughput old throughput
sqlite 💿 string index 1000 10 - 5.5±0.01µs - 177.9 Ktx/sec
sqlite 💿 string no_index 1000 10 - 49.0±0.07µs - 19.9 Ktx/sec
sqlite 💿 u64 index 1000 10 - 5.2±0.02µs - 188.2 Ktx/sec
sqlite 💿 u64 no_index 1000 10 - 34.4±0.04µs - 28.4 Ktx/sec
sqlite 🧠 string index 1000 10 - 4.1±0.01µs - 238.6 Ktx/sec
sqlite 🧠 string no_index 1000 10 - 47.8±0.19µs - 20.4 Ktx/sec
sqlite 🧠 u64 index 1000 10 - 3.8±0.01µs - 257.3 Ktx/sec
sqlite 🧠 u64 no_index 1000 10 - 32.1±0.03µs - 30.4 Ktx/sec
stdb_module 💿 string indexed 1000 10 36.0±1.19µs - 27.1 Ktx/sec -
stdb_module 💿 string non_indexed 1000 10 215.8±0.20µs - 4.5 Ktx/sec -
stdb_module 💿 u64 indexed 1000 10 31.4±1.18µs - 31.1 Ktx/sec -
stdb_module 💿 u64 non_indexed 1000 10 184.3±2.08µs - 5.3 Ktx/sec -
stdb_module 🧠 string indexed 1000 10 35.1±1.36µs - 27.8 Ktx/sec -
stdb_module 🧠 string non_indexed 1000 10 211.8±0.15µs - 4.6 Ktx/sec -
stdb_module 🧠 u64 indexed 1000 10 32.1±1.26µs - 30.4 Ktx/sec -
stdb_module 🧠 u64 non_indexed 1000 10 183.2±1.87µs - 5.3 Ktx/sec -
stdb_raw 💿 string index 1000 10 - 2.6±0.01µs - 372.8 Ktx/sec
stdb_raw 💿 string indexed 1000 10 2.6±0.00µs - 382.6 Ktx/sec -
stdb_raw 💿 string no_index 1000 10 - 189.3±0.03µs - 5.2 Ktx/sec
stdb_raw 💿 string non_indexed 1000 10 206.0±0.38µs - 4.7 Ktx/sec -
stdb_raw 💿 u64 index 1000 10 - 2.5±0.01µs - 385.6 Ktx/sec
stdb_raw 💿 u64 indexed 1000 10 2.5±0.00µs - 394.0 Ktx/sec -
stdb_raw 💿 u64 no_index 1000 10 - 161.3±0.06µs - 6.1 Ktx/sec
stdb_raw 💿 u64 non_indexed 1000 10 173.7±0.03µs - 5.6 Ktx/sec -
stdb_raw 🧠 string index 1000 10 - 2.5±0.00µs - 385.8 Ktx/sec
stdb_raw 🧠 string indexed 1000 10 2.5±0.01µs - 395.6 Ktx/sec -
stdb_raw 🧠 string no_index 1000 10 - 189.5±0.53µs - 5.2 Ktx/sec
stdb_raw 🧠 string non_indexed 1000 10 206.5±0.06µs - 4.7 Ktx/sec -
stdb_raw 🧠 u64 index 1000 10 - 2.5±0.00µs - 398.0 Ktx/sec
stdb_raw 🧠 u64 indexed 1000 10 2.4±0.00µs - 405.8 Ktx/sec -
stdb_raw 🧠 u64 no_index 1000 10 - 161.1±0.04µs - 6.1 Ktx/sec
stdb_raw 🧠 u64 non_indexed 1000 10 173.5±0.04µs - 5.6 Ktx/sec -

Serialize

schema format count new latency old latency new throughput old throughput
location bsatn 100 1755.8±32.97ns - 54.3 Mtx/sec -
location json 100 3.0±0.05µs - 31.3 Mtx/sec -
location product_value 100 562.2±0.60ns - 169.6 Mtx/sec -
person bsatn 100 2.6±0.00µs - 37.3 Mtx/sec -
person json 100 4.7±0.02µs - 20.2 Mtx/sec -
person product_value 100 671.5±0.30ns - 142.0 Mtx/sec -
u32_u64_str bsatn 100 - 2.9±0.00µs - 32.7 Mtx/sec
u32_u64_str json 100 - 5.0±0.03µs - 19.2 Mtx/sec
u32_u64_str product_value 100 - 652.2±41.96ns - 146.2 Mtx/sec
u32_u64_u64 bsatn 100 - 1667.1±34.47ns - 57.2 Mtx/sec
u32_u64_u64 json 100 - 3.0±0.02µs - 31.5 Mtx/sec
u32_u64_u64 product_value 100 - 625.4±0.26ns - 152.5 Mtx/sec

Module: invoke with large arguments

arg size new latency old latency new throughput old throughput
64KiB - 89.8±7.16µs - -
64KiB 64.9±9.38µs - - -

Module: print bulk

line count new latency old latency new throughput old throughput
1 - 43.0±4.23µs - -
1 31.0±2.82µs - - -
100 - 217.7±1.41µs - -
100 198.1±2.62µs - - -
1000 - 1964.6±66.80µs - -
1000 1899.4±607.98µs - - -

Remaining benchmarks

name new latency old latency new throughput old throughput
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=1024/count=100 - 21.9±0.10µs - 44.7 Ktx/sec
sqlite/💿/update_bulk/u32_u64_str/unique_0/load=128/count=64 - 17.3±0.12µs - 56.6 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=1024/count=100 - 20.6±0.05µs - 47.4 Ktx/sec
sqlite/💿/update_bulk/u32_u64_u64/unique_0/load=128/count=64 - 15.6±0.01µs - 62.6 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=1024/count=100 - 16.1±0.02µs - 60.7 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_str/unique_0/load=128/count=64 - 11.2±0.06µs - 87.5 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=1024/count=100 - 14.8±0.01µs - 66.2 Ktx/sec
sqlite/🧠/update_bulk/u32_u64_u64/unique_0/load=128/count=64 - 10.4±0.01µs - 94.0 Ktx/sec
stdb_module/💿/update_bulk/location/unique/load=1000/count=10 484.1±0.87µs - 2.0 Ktx/sec -
stdb_module/💿/update_bulk/location/unique/load=1000/count=100 945.4±235.32µs - 1057 tx/sec -
stdb_module/💿/update_bulk/person/unique/load=1000/count=10 563.0±2.96µs - 1776 tx/sec -
stdb_module/💿/update_bulk/person/unique/load=1000/count=100 1187.1±296.33µs - 842 tx/sec -
stdb_module/🧠/update_bulk/location/unique/load=1000/count=10 374.0±16.92µs - 2.6 Ktx/sec -
stdb_module/🧠/update_bulk/location/unique/load=1000/count=100 673.9±10.11µs - 1484 tx/sec -
stdb_module/🧠/update_bulk/person/unique/load=1000/count=10 510.8±0.74µs - 1957 tx/sec -
stdb_module/🧠/update_bulk/person/unique/load=1000/count=100 846.7±142.27µs - 1181 tx/sec -
stdb_raw/💿/update_bulk/location/unique/load=1000/count=10 44.8±0.34µs - 21.8 Ktx/sec -
stdb_raw/💿/update_bulk/location/unique/load=1000/count=100 371.6±30.77µs - 2.6 Ktx/sec -
stdb_raw/💿/update_bulk/person/unique/load=1000/count=10 60.5±0.16µs - 16.1 Ktx/sec -
stdb_raw/💿/update_bulk/person/unique/load=1000/count=100 508.6±0.34µs - 1966 tx/sec -
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=1024/count=100 - 517.8±0.62µs - 1931 tx/sec
stdb_raw/💿/update_bulk/u32_u64_str/unique_0/load=128/count=64 - 323.7±0.20µs - 3.0 Ktx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=1024/count=100 - 362.0±0.34µs - 2.7 Ktx/sec
stdb_raw/💿/update_bulk/u32_u64_u64/unique_0/load=128/count=64 - 234.1±0.06µs - 4.2 Ktx/sec
stdb_raw/🧠/update_bulk/location/unique/load=1000/count=10 31.9±0.04µs - 30.6 Ktx/sec -
stdb_raw/🧠/update_bulk/location/unique/load=1000/count=100 271.4±0.19µs - 3.6 Ktx/sec -
stdb_raw/🧠/update_bulk/person/unique/load=1000/count=10 42.1±0.03µs - 23.2 Ktx/sec -
stdb_raw/🧠/update_bulk/person/unique/load=1000/count=100 364.8±0.24µs - 2.7 Ktx/sec -
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=1024/count=100 - 364.3±0.43µs - 2.7 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_str/unique_0/load=128/count=64 - 226.7±0.29µs - 4.3 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=1024/count=100 - 268.3±0.25µs - 3.6 Ktx/sec
stdb_raw/🧠/update_bulk/u32_u64_u64/unique_0/load=128/count=64 - 173.5±0.15µs - 5.6 Ktx/sec

Copy link
Contributor

@cloutiertyler cloutiertyler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@coolreader18 coolreader18 added this pull request to the merge queue Feb 12, 2024
Merged via the queue into master with commit 295ae3d Feb 12, 2024
mamcx added a commit that referenced this pull request Feb 13, 2024
commit 8adad4c
Author: John Detter <[email protected]>
Date:   Tue Feb 13 11:29:27 2024 -0600

    Verify that the upgrade-version tool still works for every PR (#825)

    * Update upgrade-version tool

    * Verify that version upgrade still works for every PR

    * Add git diff to the end of CI

    ---------

    Co-authored-by: Noa <[email protected]>
    Co-authored-by: John Detter <[email protected]>
    Co-authored-by: Zeke Foppa <[email protected]>

commit 0260b09
Author: joshua-spacetime <[email protected]>
Date:   Tue Feb 13 08:23:40 2024 -0800

    test(829): Benchmark incremental update for table subscription (#830)

    Closes #829.

commit def2cae
Author: joshua-spacetime <[email protected]>
Date:   Tue Feb 13 08:21:47 2024 -0800

    chore(836): Remove tracing instrumentation from database iterators (#837)

    Closes #836.

    The tracing library does not fully remove all instrumentation at compile time.
    And since tracing is not zero cost,
    it must be removed from the hot path of query execution.

commit a32ef34
Author: Noa <[email protected]>
Date:   Tue Feb 13 09:16:09 2024 -0600

    Update upgrade-version tool (#826)

    * Update upgrade-version tool

    * Small cludge

    ---------

    Co-authored-by: John Detter <[email protected]>

commit d4bfb9a
Author: Phoebe Goldman <[email protected]>
Date:   Tue Feb 13 02:56:33 2024 -0500

    `ScanIterByColRange`: Avoid unnecessary `ProductValue` allocations (#804)

    Prior to this commit, `ScanIterByColRange::next` called `RowRef::to_product_value`
    and then `ProductValue::project_not_empty` on every row in the table
    to get a key, which it compared to the sought range.
    This always allocated at least a `ProductValue`,
    even when seeking a range of primitive type like `u64`.
    It also unnecessarily deserialized (and potentially allocated)
    columns not relevant to the search for rows which did not match the range.

    With this commit, we call `RowRef::project_not_empty` directly.
    When the sought range is a single column, this avoids allocating a `ProductValue`.
    Even when seeking a multi-column range, this avoids deserializing unrelated columns
    of non-matching rows.

    Non-indexed `IterByColEq` is always going to be slow, so this probably doesn't matter,
    but it's a trivial change.

commit faf8766
Author: Tyler Cloutier <[email protected]>
Date:   Mon Feb 12 13:42:29 2024 -0800

    This fixes replaying of the transaction log to no longer check constraints (#806)

    * This fixes replaying of the transaction log to no longer check constraints

    * Fixed based on Phoebe and Kim's comments

    * Cargo fmt

    * Cargo fmt

    * This fixes index updating for replaying deletes

    ---------

    Signed-off-by: Tyler Cloutier <[email protected]>

commit 295ae3d
Author: Noa <[email protected]>
Date:   Mon Feb 12 14:45:55 2024 -0600

    Fix benchmarks (#819)

commit 45ae7f7
Author: joshua-spacetime <[email protected]>
Date:   Mon Feb 12 11:57:56 2024 -0800

    chore(datastore): Remove debug instrumentation (#812)

commit 7491835
Author: Noa <[email protected]>
Date:   Mon Feb 12 12:28:42 2024 -0600

    ResultInspectExt is obsolete (#818)

commit 2de5bf0
Author: joshua-spacetime <[email protected]>
Date:   Mon Feb 12 10:13:36 2024 -0800

    perf(subscriptions): Add metric counter for subscribe calls (#817)

commit 4b41998
Author: joshua-spacetime <[email protected]>
Date:   Fri Feb 9 17:49:15 2024 -0800

    test: Add performance benchmark for incremental join query (#795)

commit b4f75d2
Author: Noa <[email protected]>
Date:   Fri Feb 9 13:40:03 2024 -0600

    Bump Rust to 1.76.0 (#811)

    * Bump Rust to 1.76.0

    * Fix new clippy lints

    * Use rust-toolchain in smoketests
@coolreader18 coolreader18 deleted the noa/try-fix-benchmarks branch February 15, 2024 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants