Sir Arthur V Quackington

  • 0 Posts
  • 5 Comments
Joined 2 years ago
cake
Cake day: June 30th, 2023

help-circle



  • Thing is, for your average user with no GPU and whp never thinks about RAM, running a local LLM is intimidating. But it shouldn’t be. Any system with an integrated GPU, and the more RAM the better, can run simple models locally.

    The not so dirty secret is that ChatGPT 3 vs 4 isn’t that big a difference, and neither are leaps and bounds ahead of the publically available models for about 99% of tasks. For that 1% people will ooh and aah over it, but 99% of use cases are only seeing marginal gains on 4o.

    And the simplified models that run “only” 95% as well? They can use 90% fewer resources give pretty much identical answers outside of hyperspecific use cases.

    Running a a “smol” model as some are called, gets you all the bang for none of the buck, and your data stays on your system and never leaves.

    I’ve been yelling from the rooftops to some stupid corporate types that once the model is trained, it’s trained. Unless you are training models yourself, there is no need for the massive AI clusters, just for the model. Run it local on your hardware at a fraction of the cost.


  • I honestly could give a shit. You think games don’t report telematics internally back to devs? Who share them with publishers anyways?

    Valve has so much telematicd data on you it’s insane. Same with other companies. Sony just isn’t good at burying the lead.

    At the end of the day it’s the same as a GOG Account, an Epic Login, or a Ubisoft Account. They all exist and we lost this fight when we allowed giant super publishers to merge and control distribution, and when no alternative to steam has ever got off the ground.

    Say what you will about Epic, but it is by f as R the closest anything is to breaking Valve’s iron grip, and it’s still a distant second place also ran.

    So yea, Sony requires a login. I wish they didn’t, but literally everything does besides GOG. They all use Denuvo and DRM and lock the exe to the client. Sony doing it to run their leaderboards at least makes some sense.