• 9 Posts
  • 1.11K Comments
Joined 1 year ago
cake
Cake day: March 22nd, 2024

help-circle




  • First of all, these are private companies, not governments. They can technically do whatever TF they want, and we probably shouldn’t have ceded so much power to them.

    …Anyway, I think you have a point. Or at least part of one.

    It’s reasonable to draw red lines like “no nazism on our platform.” But at the end of the day Spotify and such can ban whatever they want, with no repercussions since it’s basically a network of defacto, legally shielded monopolies.

    So how would we feel if, say, they started banning podcasts a little too popular and too critical of the president?

    In other words, banning nazism as a policy is fine, but arbitrarily banning what looks bad to them is indeed going to be a problem.




  • Completely depends on your laptop hardware, but generally:

    • TabbyAPI (exllamav2/exllamav3)
    • ik_llama.cpp, and its openai server
    • kobold.cpp (or kobold.cpp rocm, or croco.cpp, depends)
    • An MLX host with one of the new distillation quantizations
    • Text-gen-web-ui (slow, but supports a lot of samplers and some exotic quantizations)
    • SGLang (extremely fast for parallel calls if thats what you want).
    • Aphrodite Engine (lots of samplers, and fast at the expense of some VRAM usage).

    I use text-gen-web-ui at the moment only because TabbyAPI is a little broken with exllamav3 (which is utterly awesome for Qwen3), otherwise I’d almost always stick to TabbyAPI.

    Tell me (vaguely) what your system has, and I can be more specific.