fix: ⚡ Double Digit Percentage Improvements. #169
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Nishant Arora <[email protected]>
SgtPooki
suggested changes
Jun 29, 2023
benchmarks/add-dir/src/helia.ts
Outdated
Comment on lines
63
to
69
| const rootCID = await unixFs.addDirectory({ path: parentDirectoryName }, unixFsAddOptions) | ||
|
|
||
| for (const dirent of dirents) { | ||
| await Promise.all(dirents.map(async dirent => { | ||
| const path = nodePath.join(dir, dirent.name); | ||
|
|
||
| if (dirent.isDirectory()) { | ||
| const cid: CID = await addDir(path); | ||
| rootCID = await unixFs.cp(cid, rootCID, dirent.name, { offline: true }) | ||
|
|
||
| } else { | ||
| const cid = await addFile(path); | ||
| rootCID = await unixFs.cp(cid, rootCID, dirent.name, { offline: true }) | ||
| } | ||
| } | ||
| const cid: CID = dirent.isDirectory() ? await addDir(path) : await addFile(path); | ||
| await unixFs.cp(cid, rootCID, dirent.name, { offline: true }) | ||
| })); |
Member
There was a problem hiding this comment.
@whizzzkid since you're not changing the rootCID, the final returned rootCID is an empty directory with only a single block.
Member
There was a problem hiding this comment.
The top is using the old code, buttom is using yours:
SgtPooki
reviewed
Jun 29, 2023
SgtPooki
reviewed
Jun 29, 2023
whizzzkid
changed the title
Member
|
pushed up some changes to this code. it does improve things, heres some output: testing /benchmarks/add-dir/node_modulesNew codeOld codeResultshelia-fs -- ms/op decrease of 48.48% testing /node_modules/ipfs-coreNew CodeOld CodeResultshelia-fs -- ms/op decrease of 17.49% |
SgtPooki
approved these changes
Jun 29, 2023
SgtPooki
added a commit
that referenced
this pull request
Jul 11, 2023
* test: create add-dir benchmark * chore(bench/add-dir): remove unused taskResult prop * test(bench/add-dir): add kubo-direct test * docs(bench/add-dir): ipfs-core node_modules output * chore(bench/add-dir): kubo-direct doesnt pin files when adding * chore(bench/add-dir): split helia benchmark by blockstore type (fs/mem) * fix: ⚡ Double Digit Percentage Improvements. (#169) * fix: ⚡ 2x improvement Signed-off-by: Nishant Arora <[email protected]> * fix: dag generation with parallelization at each level --------- Signed-off-by: Nishant Arora <[email protected]> Co-authored-by: Russell Dempsey <[email protected]> * test: use globSource in add-dir benchmark (#171) * fix: use glob source for importing directories - Use `globSource` from ipfs-utils for import - Make all impls return the same CID - Remove the dir size - if the CID is the same that's good enough * chore: fix linting * chore: remove benchmark from workspaces * test: align error and success output headers --------- Signed-off-by: Nishant Arora <[email protected]> Co-authored-by: Nishant Arora <[email protected]> Co-authored-by: Alex Potsides <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improves: #167
Parallel Children at Each Level.