You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
majiahui@haimaqingfan.com
c5585be230
|
1 year ago | |
|---|---|---|
| .idea | 2 years ago | |
| .gitignore | 2 years ago | |
| README.md | 2 years ago | |
| articles_directory_predict.py | 2 years ago | |
| config_llama_api.py | 2 years ago | |
| flask_batch.py | 2 years ago | |
| flask_predict.py | 2 years ago | |
| flask_test.py | 2 years ago | |
| gen_paper.py | 2 years ago | |
| gunicorn_config.py | 2 years ago | |
| img.png | 2 years ago | |
| img_1.png | 2 years ago | |
| mistral_api.py | 1 year ago | |
| mistral_model_predict_vllm.py | 2 years ago | |
| predict.py | 2 years ago | |
| requirements.txt | 2 years ago | |
| run_api_gunicorn.sh | 2 years ago | |
| run_model.sh | 2 years ago | |
| run_model_predict.sh | 2 years ago | |
| tokenizer.py | 2 years ago | |
| vllm_predict_batch.py | 2 years ago | |
| yace.py | 2 years ago | |
| yace2.py | 2 years ago | |
| yuce3.py | 2 years ago | |
| 测试并发性能.py | 2 years ago | |
| 测试并发请求.py | 2 years ago | |
README.md
安装环境
conda create -n vllm python=3.8
pip install -r requirements.txt
启动项目
conda activate vllm
bash run_api_nohup.sh
bash run_model.sh
测试
curl -H "Content-Type: application/json" -X POST -d '{"texts": "User: 你好\nAssistant:"}' http://192.168.31.149:12000/predict
curl -H "Content-Type: application/json" -X POST -d '{"id": "b412fc98-bdd7-11ee-8d23-d5e5c66dd02e"}' http://192.168.31.149:12000/search
返回"status_code"不出现 400 则调用成功