File tree Expand file tree Collapse file tree 1 file changed +2
-2
lines changed Expand file tree Collapse file tree 1 file changed +2
-2
lines changed Original file line number Diff line number Diff line change @@ -82,10 +82,10 @@ Then run the benchmarking script
8282# wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json
8383MODEL_NAME=" NousResearch/Hermes-3-Llama-3.1-8B"
8484NUM_PROMPTS=10
85- BACKEND=" openai-chat "
85+ BACKEND=" vllm "
8686DATASET_NAME=" sharegpt"
8787DATASET_PATH=" <your data path>/ShareGPT_V3_unfiltered_cleaned_split.json"
88- python3 vllm/benchmarks/benchmark_serving.py --backend ${BACKEND} --model ${MODEL_NAME} --endpoint /v1/chat/ completions --dataset-name ${DATASET_NAME} --dataset-path ${DATASET_PATH} --num-prompts ${NUM_PROMPTS}
88+ python3 vllm/benchmarks/benchmark_serving.py --backend ${BACKEND} --model ${MODEL_NAME} --endpoint /v1/completions --dataset-name ${DATASET_NAME} --dataset-path ${DATASET_PATH} --num-prompts ${NUM_PROMPTS}
8989```
9090
9191If successful, you will see the following output
You can’t perform that action at this time.
0 commit comments