e.g.
MODEL_NAME = "https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.2-GGUF/blob/main/mistral-7b-instruct-v0.2.Q5_K_M.gguf"
@asynccontextmanager
async def lifespan(_app: FastAPI):
ex_app.set_handlers(APP, enabled_handler, models_to_fetch={MODEL_NAME: {}})
yield
with automatic progress display in AppAPI UI and with ETag support probably to autoupdate model if file was changes on server?
we can check start of string for "https/http" to detect if this is direct link to file
Or something else(additional files?) will be needed for GGUF models and we need a more complex struct?
cc @kyteinsky ?
edited: ETags support added only for huggingface(they use sha256
of file and X-Linked-ETag
field), it should be enough.