Nvidia's local, private AI chatbot is a high-profile step toward cloud independence.
See full article...
See full article...
Nvidia’s “Chat With RTX” is a ChatGPT-style app that runs on your own GPU
- Thread starter JournalBot
- Start date
I run Mixtral 8x7B on two laptops in my house - one is a 3080 with 16gb vram, and the other is a 4070 with only 8gb vram, each with 64gb system ram. I've been using both Faraday and LM Studio and both run well enough for me.
Upvote
3
(3
/
0)
benjedwards
Ars Centurion
I want to see more targeted deep dives in AI too. So far, circumstances haven't allowed me to do them very often. We are working on ways to get more staff on the AI beat, and I've been handing more and more AI stories to others reporters, and we'll probably try to get some freelancers to help. In the mean time, I'm just trying to keep up with the firehose as the "news guy." Features are a rare luxury. Just yesterday there were major AI announcements from Google, Meta, and OpenAI, and I only had time to cover one of them. It kills me. I'm trying to catch up today with what I missed, but fresh news keeps coming in as well. It's all about triage, and sadly stuff slips through the cracks.
BTW, I am planning out a roundup of local LLM options, so we'll see how that goes. Might take a while given all the distractions coming from the constant flow of news! Or if I can find someone to hand the project off to, I'll do that. Finding people knowledgeable about this stuff who can write well is very difficult at the moment. If you know anybody, feel free to email me benj.edwards at you-know-where! Put WOZNIAK in the subject line so it will stand out from the bajillions of PR emails that I cannot possibly read.
Upvote
10
(11
/
-1)
I remember a movie based on this idea. I had no idea it would end up being so popular a business model would emerge here in the really real world.
Upvote
0
(0
/
0)
Just run GPT4All from gpt4all.io ... No new hardware (nor OS) needed.I got the hardware to run this (30xx series GPU with 16GB Vram and 32GB ram). But I will need to upgrade to Windows 11, since thats stated as required at the download site.
Damn it, finally I may be forced to upgrade since I want to try this out ......
Upvote
0
(0
/
0)
The linked AI model isn't "uncensored". It's trained by a white supremacist who removed refusals for hate speech and for political topics he disagreed with. Please do not promote it. It is a crypto-fascist model. The chat agent's political alignement has been shifted, not just their etiical compass.
Upvote
0
(0
/
0)
OrangeCream
Ars Legatus Legionis
Mistral or LLaMa?
Mistral was founded by Arthur Mensch, Timothée Lacroix and Guillaume Lample and is a French company, so I’m assuming you’re talking about LLaMa? I can’t really find anything about his crypto-fascist tendencies, and really weird when you consider he is Jewish and his wife is Chinese/Vietnamese.
Upvote
-1
(1
/
-2)
I am referring to "Uncensored" llama models on Hugging Face. They are created by filtering instruct/chat datasets for refusals, only the people responsible don't just filter for refusals. They remove any mention of gay and trans people. They remove people of color. They remove the word "feminism" entirely. "Supremacist" and so on. By leaving these concept undefined they leave the harmful biases of the base model, Llama 2.
For example:
https://huggingface.co/datasets/cog...c52e0a39ae38c9ea28b77817/wizardlm_clean.py#L7
This is done to influence users of these language models without their knowledge or consent. Promoting them is dangerous. I don't know much about Mistral but I suspect the foundation model also has harmful biases that must be fined tuned out. Those who intentionally and selectively avoid doing so are acting out of malice.
Last edited:
Upvote
1
(1
/
0)
This is like saying black people like Kanye can't be racist. The idea itself is racist. Zuckerberg doesn't care. Why should he? He can survive the world ending and sleeps on a pile of money.
Upvote
2
(2
/
0)
OrangeCream
Ars Legatus Legionis
No, that’s not what I meant. The issue is where you have any proof, even if circumstantial, that he is racist?
Unfortunately his behavior can be pretty much summarized as amoral and driven by a desire for wealth and success more than anything else.
Upvote
-2
(1
/
-3)
I didn't say either way that I recall. I said the argument was stupid and racist, not that Zuckerberg is specifically.
I said Zuckerberg doesn't care. He knows this is the only way he can catch up to OpenAI and if Meta were to enforce their Llama2 COC they would alienate too many developers.
I said the creators of the "Uncensored" models, particularly Eric Hartford were either racists or willing to work with them, which is really the same thing. They don't just remove "as an AI language model" from fine tuning datasets. They remove Stonewall. They remove the civil rights movement. They make neo-Nazi chat bots and call them "Uncensored".
Upvote
0
(0
/
0)