Skip to content

optimized collect_dictionary_activations.py #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

optimized collect_dictionary_activations.py #8

wants to merge 1 commit into from

Conversation

@Idhant297
Copy link

@Idhant297 Idhant297 commented Aug 8, 2025 •
edited
Loading

was working with some custom models and datasets. faced some issues while working with collect_dictionary_activations.py which were as follows:

  • noticed that activation that were stored were not being offloaded from the GPU which led to OOM errors
    • fixed: now a checkpoint file is created inside the temp folder where the activations are to be stored.
  • also the dataset was working with didnt had BOS (delimiter) tokens so added a fixed lenght splitting logic too.

@Idhant297
added incremental processing and checkpointing for memory-efficient a…
47d2404 
…ctivation collection

- refactor get_positive_activations to process sequences incrementally
- add checkpoint system to resume interrupted jobs and prevent OOM errors
- optimize memory usage with immediate CPU and GPU offloading
- when delimiter tokens (BOS) are not found, fallsbacks to fixed length sequences
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Idhant297