I’m interested in hosting something like this, and I’d like to know experiences regarding this topic.

The main reason to host this for privacy reasons and also to integrate my own PKM data (markdown files, mainly).

Feel free to recommend me videos, articles, other Lemmy communities, etc.

  • CubitOom@infosec.pub
    link
    fedilink
    English
    arrow-up
    11
    ·
    10 months ago

    Checkout ollama.

    There’s a lot of models you can pull from the official library.

    Using ollama, you can also run external gguf models found on places like huggingface if you use a modelfile with something as simple as

    echo "FROM ~/Documents/ollama/models/$model_filepath" >| ~/Documents/ollama/modelfiles/$model_name.modelfile