This is an example setup of a docker compose solution where a frontend (web) adds jobs to a queue and multiple workers (worker) pick jobs and executes them. I could not find a similar solution that ...

So far, running LLMs has required a large amount of computing resources, mainly GPUs. Running locally, a simple prompt with a typical LLM takes on an average Mac ...