Feature Request: (tools/server) Parameter for endpoint to run chat UI on when using --path #16184
Description
Prerequisites
- I am running the latest code. Mention the version if possible as well.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description
I would like a parameter for specifying the endpoint to run the default chat style interface for tools./server, e.g.
--default-ui-endpoint "chat"
This would run the default chat UI on /chat and /chat/ endpoints.
Note: I would appreciate help from a collaborator who has successfully created pull requests against this repo in the past. The code below works. I need to submit it the right way. -Brad
Motivation
I use the --path parameter with tools/server to specify a path to my completion UI. I would also like to include an endpoint /chat (or /chat/) to display the default chat UI.
Possible Implementation
These additions work, and are incorporated into my fork: https://github.com/BradHutchings/Mmojo-Server
common/arg.cpp (end of file):
add_opt(common_arg(
{"--default-ui-endpoint"}, "STRING",
"endpoint for accessing the default chat user interface",
[](common_params & params, const std::string & value) {
params.default_ui_endpoint = value;
// LOG_INF("Setting params.default_ui_endpoint: %s\n", params.default_ui_endpoint.c_str());
}
).set_examples({LLAMA_EXAMPLE_SERVER}));
common/common.h (insert before line 280):
std::string default_ui_endpoint = ""; // endpoint for chat UI)```
tools/server/server.cpp (insert after line 33):
// pre C++20 helpers.
bool starts_with (std::string const &fullString, std::string const &beginning);
bool ends_with (std::string const &fullString, std::string const &ending);
bool starts_with (std::string const &fullString, std::string const &beginning) {
if (fullString.length() >= beginning.length()) {
return (0 == fullString.compare (0, beginning.length(), beginning));
}
else {
return false;
}
}
bool ends_with (std::string const &fullString, std::string const &ending) {
if (fullString.length() >= ending.length()) {
return (0 == fullString.compare (fullString.length() - ending.length(), ending.length(), ending));
}
else {
return false;
}
}
tools/server/server.cpp (insert before // register API routes, ~5238):
if (params.default_ui_endpoint != "") {
std::string endpoint = params.default_ui_endpoint;
// gracefully handle leading and/or trailing /
if (!starts_with (endpoint, "/")) {
endpoint = "/" + endpoint;
}
while (ends_with(endpoint, "/")) {
endpoint = endpoint.substr(0, endpoint.length() - 1);
}
svr->Get(endpoint, [](const httplib::Request & req, httplib::Response & res) {
if (req.get_header_value("Accept-Encoding").find("gzip") == std::string::npos) {
res.set_content("Error: gzip is not supported by this browser", "text/plain");
} else {
res.set_header("Content-Encoding", "gzip");
// COEP and COOP headers, required by pyodide (python interpreter)
res.set_header("Cross-Origin-Embedder-Policy", "require-corp");
res.set_header("Cross-Origin-Opener-Policy", "same-origin");
res.set_content(reinterpret_cast<const char*>(index_html_gz), index_html_gz_len, "text/html; charset=utf-8");
}
return false;
});
svr->Get(endpoint + "/", [](const httplib::Request & req, httplib::Response & res) {
res.set_redirect(req.path.substr(0, req.path.length() - 1));
return false;
});
}