Skip to content

Conversation

@rnwang04
Copy link
Contributor

@rnwang04 rnwang04 commented Oct 11, 2024

Description

1. Why the change?

  • Add timing logic to the generate function. When generate accepts do_print=True, print these timing information
  • When we set max_new_tokens=32, now actually output 33 tokens, fix this issue
  • transformer's generate actually return input_tokens + output_tokens, now we only return output_tokens, fix this issue

2. User API changes

No change.

3. Summary of the change

4. How to test?

  • Unit test: Please manually trigger the PR Validation here by inputting the PR number (e.g., 1234). And paste your action link here once it has been successfully finished.
  • Application test

@rnwang04 rnwang04 requested a review from jason-dai October 11, 2024 03:55
@rnwang04
Copy link
Contributor Author

@rnwang04 rnwang04 requested a review from plusbang October 11, 2024 09:36
@rnwang04 rnwang04 merged commit 310f18c into intel:main Oct 11, 2024
1 check passed
@rnwang04 rnwang04 deleted the updatenpu_pipeline branch October 11, 2024 09:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants