vllm/README.md

安装环境

conda create -n vllm python=3.8
pip install -r requirements.txt

启动项目

conda activate vllm
bash run_api_nohup.sh
bash run_model.sh

测试

curl -H "Content-Type: application/json" -X POST -d '{"texts": "User: 你好\nAssistant:"}' http://192.168.31.149:12000/predict
curl -H "Content-Type: application/json" -X POST -d '{"id": "b412fc98-bdd7-11ee-8d23-d5e5c66dd02e"}' http://192.168.31.149:12000/search

返回"status_code"不出现 400 则调用成功