Repository navigation
windows compatibility? #2
Description
I'm a beginner , is this program compatible with windows? what are the neccessary steps. I have the alpaca.cpp already installed on my laptop
I'm a beginner , is this program compatible with windows? what are the neccessary steps. I have the alpaca.cpp already installed on my laptop
Activity
mudler commentedon Mar 21, 2023
Hi!
I don't have a windows machine to double check, but I don't think there should be anything specific preventing it to work there.
Just install docker and start in API mode https://github.com/go-skynet/llama-cli#advanced-usage, or I guess WSL should work too.
anm5704 commentedon Mar 21, 2023
Thank you, I ran it after installing docker
docker run -ti --rm quay.io/go-skynet/llama-cli:v0.1 --instruction "What's an alpaca?" --topk 10000I ran this command but it seems to start the process for 5-6 seconds and ends it without displaying any output,
the same thing happens in API, the localhost site doesnt even load and the process stops are 5-6 without output. Any idea what might be causing this
stasadance commentedon Mar 22, 2023
I was able to fix this by allocating more memory in Docker Desktop settings:
anm5704 commentedon Mar 23, 2023
hello! thanks, it seems in windows you have to create a .wslconfig in order to increase the limits. But its working now.
II'm confused tho, the API is for LLaMa right? and not the fine-tuned alpaca models (trained on gpt-3 data).
mudler commentedon Mar 23, 2023
the api should work for both, although I've tested only alpaca models. The new version
0.3has an--alpacaboolean flag too.mudler commentedon Apr 4, 2023
I guess we can close this now? Thanks @stasadance for pointing us in the right direction!
squashmerge rf-core-3 to fresh branch, attempt mudler#2 at fighting DCO
GeorgeS2019 commentedon Nov 2, 2024
we need easy windows installation documentation
feat: cloud-proxy backend + routing + middleware stack
feat: cloud-proxy backend + routing + middleware stack
feat: cloud-proxy backend + routing + middleware stack
feat: cloud-proxy backend + routing + middleware stack
feat(distributed): resumable file uploads via HTTP Content-Range
feat(distributed): resumable file uploads via HTTP Content-Range
feat(distributed): resumable file uploads via HTTP Content-Range
feat(distributed): resumable file uploads via HTTP Content-Range
feat(distributed): resumable file uploads via HTTP Content-Range (#10109
fix(llama-cpp): re-sync carry-patches to llama.cpp 7c158fbb
docs(paged): B-2 down_proj act-quant retune RESULT - negative (no hea…
docs(paged): scope porting the portable benefits to Metal/SYCL/Vulkan…
docs(distributed): move the active carrier label off the worker boxes