Firebase Alternative SDK Benchmarks

As requested by many, here are the methods I use to benchmark the performance of the libraries.

The benchmarks focus on Loading times, and evaluation times(parsing/compiling). Runtime performance is also much better on the alternative libraries, but in general runtime performance is not an issue in this kind of libraries because they only perform operations once in a while, usually when the user performs an explicit request of new data.

If you want to know more about why these metrics are important I highly recommend that you read Addy Osmani's JavaScript Start-up Optimization article.

If you want to know why performance is an issue with the Official Firebase SDK in the first place, please read the email I wrote to the Firebase team about it.

Sizes and Loading times

Library	Alternative SDK	Official SDK
firebase/app	N/A (not needed)	10.3KB
Authentication	2KB	50.8KB
Firestore	2.9KB	92KB
Storage	1.4KB	18.4KB
Total	6.3KB	171.5KB

As you can see, the difference is significant, if you were to use all of the specified above, you will end up with a bundle size 27 times smaller, that's 93% smaller in case you were wondering.

Evaluation performance(parsing/compiling)

Now let's take a look at the evaluation times, which represents the time it takes the browser to make all of the preparations needed for the script to be ready to run.

If you wanna know more about why this is such an important metric and how I made the tests please read this article: https://www.afasterweb.com/2017/09/06/script-evaluation-and-execution-costs/

In this test, we have two bundles. The first includes the official SDKs:

• firebase/app
• firebase/auth
• firebase/firestore
• firebase/storage

In the second we have the alternative SDKs:

• firebase-auth-lite
• firebase-firestore-lite
• firebase-storage-lite

Everything was bundled with parcel-bundler which has a leaner output than Webpack. No special settings, all code was minified. The code was loaded from a local server by using http-serve.

The times written below are specific main bundle script evaluation times, not the whole page. No code was run other than importing the libs specified in order to make the results as accurate as possible. And 3 warm-up rounds were made.

Chrome v80 on a Macbook Pro, Quad-Core Intel Core i5 2.4GHz.

Try #	Alternative SDK	Official SDK
#1	0.50ms	9.17ms
#2	0.59ms	8.46ms
#3	0.56ms	8.88ms
#4	0.50ms	8.94ms
#5	0.57ms	9.51ms
#6	0.58ms	8.44ms
#7	0.51ms	8.35ms
#8	0.47ms	9.19ms
#9	0.58ms	9.16ms
#10	0.52ms	8.25ms
Average	0.538ms	7.916ms

The alternative SDK is in average 93% faster than the official one on this test. Let's take a look at mobile.

Safari on an iPhone 7.

Try #	Alternative SDK	Official SDK
#1	4.134ms	62.33ms
#2	4.138ms	69.65ms
#3	4.098ms	45.69ms
#4	4.320ms	41.63ms
#5	5.358ms	42.46ms
#6	4.728ms	69.07ms
#7	4.207ms	53.58ms
#8	4.163ms	40.04ms
#9	3.289ms	60.65ms
#10	4.248ms	55.06ms
Average	4.367ms	54.016ms

On mobile, we see similar results, 92% faster this time.

Conclusion

At the time of writing, the alternative libs are not yet at feature parity. You can look at it as more of a "low level" solution. But the goal of this project is to get there, and I estimate that the bundle sizes will not increase by much at all. If you want numbers, I estimate an average of 30% worst-case scenario increase, which is still much much better than the official SDKs.

As you can see, the numbers speak for themselves. The difference is significant, and your users will feel it. Our libraries offer a solution that is 33 times faster on average.

If you wish to see different benchmarks, or if you did your own, please feel free to open an issue and share it or ask for help.