unworriedsafari
/
mill.py
1
0
Fork 0
Code Issues Pull Requests Releases 5 Activity

v1.2.0

This commit is contained in:
unworriedsafari 2024-02-29 07:54:35 +00:00
parent 134c33a151
commit bdb2cee332
7 changed files with 58 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
README.md
View File

@ -1,6 +1,6 @@
# README
`mill.py v1.1.2`
`mill.py v1.2.0`
Markdown interface for [llama.cpp](//github.com/ggerganov/llama.cpp).
@ -23,7 +23,8 @@ Developed and tested on Linux. I believe it could also work on Windows or Mac.
6. Streams output
7. Runs in a CLI environment as well as a CGI environment
8. Reads input document from `stdin`, writes output document to `stdout`
9. Lets you add support for any other language or LLM through Python modules
9. Lets you add support for any other language (i.e. other than Markdown) or
LLM engine through Python modules
## Example
@ -168,8 +169,8 @@ cat document.md | ssh <host> \
2>/dev/null
```
Use the command-line arguments to select a different language or LLM. You can
use `-h` for a usage description.
Use the command-line arguments to select a different language or LLM engine.
You can use `-h` for a usage description.
## CGI install + usage
@ -206,11 +207,13 @@ it to send your phone's clipboard directly to the CGI tool and copy the HTTP
response automatically back to the clipboard.
Use the `language` and `llm_engine` query-string parameters to select a
different language or LLM.
different language or LLM engine.
## Markdown tutorial
This section describes the Markdown language module of `mill.py`.
`mill.py` is controlled with variables embedded in the Markdown document.
In general, variables take the form
@ -230,7 +233,7 @@ sense. The value of a block with only a variable name is the empty string.
Variables are either syntax variables or LLM variables. The distinction is made
based on the variable type contained in the info string. Syntax variables have
type `mill` and are handled directly by `mill.py` while LLM variables have
other types and are passed on to the LLM module.
other types and are passed on to the LLM-engine module.
Syntax variables and LLM variables exist in two different namespaces. The
namespace is implied by the variable type. If the `reset` flag is given, then
@ -309,7 +312,7 @@ There are three different variable types for LLM variables:
The first type simply assigns the value to the name.
For some 'LLM engines' (like `llama.cpp`), it's useful to pass arguments via a
For some LLM engines (like `llama.cpp`), it's useful to pass arguments via a
file. This can be done using the second and third variable types. For example,
you can pass a grammar via either `--grammar` or `--grammar-file`. However,
grammars can contain tokens that `mill.py` does not know how to shell-escape.
@ -338,6 +341,9 @@ newline that ends the block is excluded.
## `llama.cpp` tutorial
This section describes the `llama.cpp` LLM-engine module of `mill.py`.
### LLM variables
`suppress eos`
@ -364,7 +370,7 @@ Using these variables results in an error.
### Environment variables
Apart from LLM variables, there are also a few environment variables that
influence the behavior of `mill.py`.
influence the behavior of the `llama.cpp` module.
`MILL_LLAMACPP_MAIN`
@ -414,10 +420,13 @@ To add support for another language:
6. When using the CGI interface, pass the `language=<language_id>` query-string
parameter.
If the environment variable `MILL_DEFAULT_LANGUAGE` is set to `<language_id>`,
`mill.py` uses the language by default.
## Adding support for other LLMs
Adding support for another LLM is similar to adding support for another
## Adding support for other LLM engines
Adding support for another LLM engine is similar to adding support for another
language:
1. Create a new Python module named `mill_llm_<llm_id>` where all
@ -430,6 +439,9 @@ language:
6. When using the CGI interface, pass the `llm_engine=<llm_id>` query-string
parameter.
If the environment variable `MILL_DEFAULT_LLM` is set to `<llm_id>`, `mill.py`
uses the LLM engine by default.
## Adding example documentation

20
mill.py
View File

@ -28,10 +28,13 @@ To add support for another language:
6. When using the CGI interface, pass the `language=<language_id>` query-string
parameter.
If the environment variable `MILL_DEFAULT_LANGUAGE` is set to `<language_id>`,
`mill.py` uses the language by default.
## Adding support for other LLMs
Adding support for another LLM is similar to adding support for another
## Adding support for other LLM engines
Adding support for another LLM engine is similar to adding support for another
language:
1. Create a new Python module named `mill_llm_<llm_id>` where all
@ -44,6 +47,9 @@ language:
6. When using the CGI interface, pass the `llm_engine=<llm_id>` query-string
parameter.
If the environment variable `MILL_DEFAULT_LLM` is set to `<llm_id>`, `mill.py`
uses the LLM engine by default.
## Adding example documentation
@ -61,7 +67,15 @@ the `runnable_example` variable is to have some text that can be executed by
`mill.py`. It should turn the README into an executable document.
"""
import importlib, re
import importlib, os, re
def default_language():
return os.environ.get('MILL_DEFAULT_LANGUAGE', 'markdown')
def default_llm_engine():
return os.environ.get('MILL_DEFAULT_LLM', 'llama.cpp')
def load_module(name):

6
mill_cgi.py
View File

@ -51,7 +51,7 @@ it to send your phone's clipboard directly to the CGI tool and copy the HTTP
response automatically back to the clipboard.
Use the `language` and `llm_engine` query-string parameters to select a
different language or LLM.
different language or LLM engine.
"""
import contextlib, io, mill, mill_readme, os, sys, urllib.parse
@ -66,8 +66,8 @@ if __name__ == '__main__':
args = urllib.parse.parse_qs(os.environ.get('QUERY_STRING',''))
language = args.get('language', 'markdown')
llm_engine = args.get('llm_engine', 'llama.cpp')
language = args.get('language', mill.default_language())
llm_engine = args.get('llm_engine', mill.default_llm_engine())
if os.environ['REQUEST_METHOD'].upper() == 'GET':
print('Content-type: text/markdown')

10
mill_cli.py
View File

@ -40,9 +40,8 @@ cat document.md | ssh <host> \
2>/dev/null
```
Use the command-line arguments to select a different language or LLM. You can
use `-h` for a usage description.
"""
Use the command-line arguments to select a different language or LLM engine.
You can use `-h` for a usage description. """
import argparse, mill, mill_readme, sys
@ -53,9 +52,10 @@ if __name__ == '__main__':
argparser.add_argument('-r', '--readme', action='store_true',
help='display the README')
argparser.add_argument('-l', '--language', default='markdown',
argparser.add_argument('-l', '--language', default=mill.default_language(),
help='language')
argparser.add_argument('-e', '--llm-engine', default='llama.cpp',
argparser.add_argument('-e', '--llm-engine',
default=mill.default_llm_engine(),
help='LLM engine')
args = argparser.parse_args()

6
mill_lang_markdown.py
View File

@ -17,6 +17,8 @@
"""
## Markdown tutorial
This section describes the Markdown language module of `mill.py`.
`mill.py` is controlled with variables embedded in the Markdown document.
In general, variables take the form
@ -36,7 +38,7 @@ sense. The value of a block with only a variable name is the empty string.
Variables are either syntax variables or LLM variables. The distinction is made
based on the variable type contained in the info string. Syntax variables have
type `mill` and are handled directly by `mill.py` while LLM variables have
other types and are passed on to the LLM module.
other types and are passed on to the LLM-engine module.
Syntax variables and LLM variables exist in two different namespaces. The
namespace is implied by the variable type. If the `reset` flag is given, then
@ -115,7 +117,7 @@ There are three different variable types for LLM variables:
The first type simply assigns the value to the name.
For some 'LLM engines' (like `llama.cpp`), it's useful to pass arguments via a
For some LLM engines (like `llama.cpp`), it's useful to pass arguments via a
file. This can be done using the second and third variable types. For example,
you can pass a grammar via either `--grammar` or `--grammar-file`. However,
grammars can contain tokens that `mill.py` does not know how to shell-escape.

5
mill_llm_llama_cpp.py
View File

@ -17,6 +17,9 @@
"""
## `llama.cpp` tutorial
This section describes the `llama.cpp` LLM-engine module of `mill.py`.
### LLM variables
`suppress eos`
@ -43,7 +46,7 @@ Using these variables results in an error.
### Environment variables
Apart from LLM variables, there are also a few environment variables that
influence the behavior of `mill.py`.
influence the behavior of the `llama.cpp` module.
`MILL_LLAMACPP_MAIN`

5
mill_readme.py
View File

@ -17,7 +17,7 @@
"""
# README
`mill.py v1.1.2`
`mill.py v1.2.0`
Markdown interface for [llama.cpp](//github.com/ggerganov/llama.cpp).
@ -40,7 +40,8 @@ Developed and tested on Linux. I believe it could also work on Windows or Mac.
6. Streams output
7. Runs in a CLI environment as well as a CGI environment
8. Reads input document from `stdin`, writes output document to `stdout`
9. Lets you add support for any other language or LLM through Python modules
9. Lets you add support for any other language (i.e. other than Markdown) or
LLM engine through Python modules
"""
import mill, mill_cgi, mill_cli