I’m sorry, you are correct. The syntax and interface mirrors docker, and one can run ollama in Docker, so I’d thought that it was a thin wrapper around Docker, but I just went to check, and you are right — it’s not running in Docker by default. Sorry, folks! Guess now I’ve got one more thing to look into getting inside a container myself.

While I don’t think that llama.cpp is specifically a special risk, I think that running generative AI software in a container is probably a good idea. It’s a rapidly-moving field with a lot of people contributing a lot of code that very quickly gets run on a lot of systems by a lot of people. There’s been malware that’s shown up in extensions for (for example) ComfyUI. And the software really doesn’t need to poke around at outside data.

Also, because the software has to touch the GPU, it needs a certain amount of outside access. Containerizing that takes some extra effort.

https://old.reddit.com/r/comfyui/comments/1hjnf8s/psa_please_secure_your_comfyui_instance/

ComfyUI users has been hit time and time again with malware from custom nodes or their dependencies. If you’re just using the vanilla nodes, or nodes you’ve personally developed yourself or vet yourself every update, then you’re fine. But you’re probably using custom nodes. They’re the great thing about ComfyUI, but also its great security weakness.

Half a year ago the LLMVISION node was found to contain an info stealer. Just this month the ultralytics library, used in custom nodes like the Impact nodes, was compromised, and a cryptominer was shipped to thousands of users.

Granted, the developers have been doing their best to try to help all involved by spreading awareness of the malware and by setting up an automated scanner to inform users if they’ve been affected, but what’s better than knowing how to get rid of the malware is not getting the malware at all. ’

Why Containerization is a solution

So what can you do to secure ComfyUI, which has a main selling point of being able to use nodes with arbitrary code in them? I propose a band-aid solution that, I think, isn’t horribly difficult to implement that significantly reduces your attack surface for malicious nodes or their dependencies: containerization.

Ollama means sticking llama.cpp in a Docker container, and that is, I think, a positive thing.

If there were a close analog to ollama, like some software package that could take a given LLM model and run in podman or Docker or something, I think that that’d be great. But I think that putting the software in a container is probably a good move relative to running it uncontainerized.

The last time I used a commercial VPS, I’m pretty sure it used VNC to provide console access.

The VNC software I linked to above appears to support TLS. If TLS isn’t sufficient transport security, then most Internet-using software is going to be in trouble.

I’m not sure what you mean by subjective.

I haven’t looked at the VNC protocol for a while, but I don’t think that it imposes any terrible inefficiencies. A couple of decades back, I needed to implement something quick-and-dirty similar to VNC, and went with rendering window contents and handling dragging of windows locally, which I don’t believe that VNC can do (or didn’t then) but IIRC VNC has a tile cache, which, if intelligently used, should avoid most traffic. Dunno if it can deal well with efficiently rendering visual effects.

VNC is dead.

How so?

There are a number of software packages in Debian that implement VNC. To grab one random example, the last commit to their git repo was last month.

If OP is specifically wanting commercial support, they do mention two companies that provide services.

https://www.freerdp.com/support

FreeRDP itself is a community driven open source project. However there are some people and companies that do offer all sorts of commercial support around FreeRDP:

• David Fort - Hardening Consulting - Contact
• Thincast Technologies GmbH - Contact

FreeRDP doesn’t seem to be soliciting donations, though.

Mbin shows only a comm… magazine logo and its name, but instance URL is hidden.

Hmm.

That’s a little annoying, but if you have your mouse hover over the name, it looks like mbin has a pop-up showing the instance.

Thanks, fellas! I guess the first need would certainly be to fully archive the community in question, i.e.: https://lemm.ee/c/eurographicnovels.

Hmm. Yeah, if anyone’s posted images there to the pict-rs instance on lemm.ee, those will presumably be going down too.

checks

Yeah, like, you just posted this image yesterday. Like, post text federates, but other instances won’t have copies of the images.

EDIT: Normally, though, people are going to be posting to pict-rs on their home instance, so it’s just users on lemm.ee who are going to have the problem; images posted by people elsewhere should stay up.

I mean, at least tell them what the correct usage is.

OP, you probably want “software package” or “a piece of software”.

“Software” is a mass noun, like “butter”. You can’t have “a butter”. You can have “a pound of butter”.

In English, mass nouns are characterized by the impossibility of being directly modified by a numeral without specifying a unit of measurement and by the impossibility of being combined with an indefinite article (a or an). Thus, the mass noun “water” is quantified as “20 litres of water” while the count noun “chair” is quantified as “20 chairs”. However, both mass and count nouns can be quantified in relative terms without unit specification (e.g., “so much water”, “so many chairs”, though note the different quantifiers “much” and “many”).

https://en.wiktionary.org/wiki/software

Usage notes

Software is a mass noun (some software, a piece of software). By non-native speakers it is sometimes erroneously treated as a countable noun (a software, some softwares).

“A something” is only correct if the noun is a countable noun.

I have a ~400 Wh powerbank in my car. It charges off the cigarette lighter when it needs charging and the engine is running. That greatly increases my ability to run higher loads on a short term basis, and gives me wall power. I can also haul it to a power plug and charge it if need be. It also lets me power a laptop if I’m parked.

I use my phone for navigation, and a mount for when I’m on longer trips.

I think that a Pi might make sense if you need something that a phone can’t do, more-intensive compute, but if a phone can handle it, it might be preferable, since you’re probably going to sporadically upgrade your phone anyway and probably have it with you.

The phone crashing is going to be a problem even for non-satnav use, so it might be worth replacing.

One thing I noticed was that my phone could overheat—at least in its case, haven’t tried removing it—if it continuously ran OSMAnd for navigation. That’d make it reboot. A quick and easy way to avoid the problem is just to toggle off the OSMAnd display. The satnav still works, and you get verbal prompts, just need a double-tap on the hamburger button or whatever to bring it back. Probably it’d be better to have a feature to throttle OSMAnd screen updates (reduce battery usage too) since I don’t need super-rapid redraw on a screen that I’m rarely looking at. Dunno if that might be what affects you.

I have a JBOD SATA USB-C enclosure that can do eight drives and has a fan. I’ll follow up with the name in twenty minutes or so; not by it at the moment.

It took me a while to find it when I got it, because my previous JBOD USB-C enclosure — as with, apparently, most enclosures — didn’t have the ability to power back up on power loss without the power-on button being pushed. This has a mechanical button that locks in and doesn’t have that issue. If that’s something that would matter to you, I’d look for that when making a purchase.

It’s not a hardware RAID enclosure, but if you’re using it on a Linux system, you can set up RAID in software on that.

EDIT:

https://www.amazon.com/Syba-Swappable-Drive-External-Enclosure/dp/B0DCDDGHMJ

Also, follow-up point, but if you don’t have a backup already, I’d do that and then if you still want a RAID setup for data redundancy on top of that to reduce downtime in the event of a failure, do that then. RAID won’t guard against some issues that a backup will.

I mean, you can build a render farm on a single Raspberry Pi if you want, technically.

But the the requirements for a server that “does it all” remains a mystery to me.

“All” can include anything. I mean, you can include a home parallel compute render farm that will cost millions of dollars.

You’re going to have to narrow it a bit down. You can have people maybe suggest some of the things that they use their systems for. Maybe it’s hosting services for a cell phone that some people use cloud-based services for. Maybe it’s home automation. Maybe it’s a webserver. Maybe it’s AI image generation.

EDIT: To put it another way, a self-hosted server is just a computer, often without a monitor and keyboard directly attached, that you have in your physical possession. The range of things that that might be used for and capabilities it might have is really broad. It’s like saying “I want a vehicle. What is a vehicle that can do everything?” I mean, that might be a bicycle or a three-trailer road train, depending upon what you’re going for.

They’re all right, I suppose, but it wasn’t dissatisfaction with search results that caused me to want to use Kagi. Rather, that I wanted to use a search engine that has a sustainable business model that didn’t involve data-mining me or showing me ads.

If Google or whoever offered some kind of comparable commercial “private search” service with a no-log, no-data-mining, no-ad offering, I’d probably sit down and to compare the results, see what I think. I kind of wish Google would do that with YouTube, but alas, they don’t…

Kagi does have a feature where they will let you search the complete Threadiverse that I make use of, since I spend a lot of time here; there isn’t really a fantastic way to accomplish this on Google or another search engine that I’m aware of. They call that their “Fediverse Forums” search lens; that’s probably the Kagi-specific feature that I get the most use out of.

They have other features, like fiddling with the priorities of sites and stuff like that, but I don’t really use that stuff. They do let you customize the output and stuff. You can set up search aliases and stuff, but I can do most of that browser-side in Firefox.

They have the ability to run a variety of LLM models on their hardware, provide that as a service. I have the hardware to run those on my own hardware and have the software set up to do so, so I don’t use that functionality. If I didn’t, I’d probably find some commercial service like them that had a no-log, no-data-mining policy, as it’s more economical to share hardware that one is only using 1% of the time or whatever.

I dunno. They have some sort of free trial thing, if you want to see what their search results are like.

I want someone to prove his LLM can be as insightful and accurate as paid one.

I mean, you can train a model that’s domain-specific that some commercial provider doesn’t have a proprietary model to address. A model can only store so much information, and you can choose to weight that information towards training on what’s important to you. Or providers may just not offer a model in the field that you want to deal with at all.

But I don’t think that, for random individual user who just wants a general-purpose chatbot, he’s likely going to get better performance out of something self-hosted. Probably it’ll cost more for the hardware, since the local hardware isn’t likely to be saturated and probably will not have shared costs, though you don’t say that cost is something that you care about.

I think that the top reason for wanting to run an LLM model locally is the one you explicitly ruled out: privacy. You aren’t leaking information to someone’s computers.

Some other possible benefits of running locally:

• Because you can guarantee access to the computational hardware. If my Internet connection goes down, neither does whatever I’m doing with the LLM.

• Latency isn’t a factor, either from the network or shared computational systems. Right now, I don’t have anything that has much by way of real-time constraints, but I’m confident that applications will exist.

• A cloud LLM provider can change the terms of their service. I mean, sure, in theory you could set up some kind of contract that locks in a service (though the VMWare customers dealing with Broadcom right now may not feel that that’s the strongest of guarantees). But if I’m running something locally, I can keep it doing so as long as I want, and I know the costs. Lot of certainty there.

• I don’t have to worry about LLM behavior changing underfoot, either from the service provider fiddling with things or new regulations being passed.

I’m gonna bet that you’re going to get a much-more-economical return there by powdering whatever iron is used in building that thing and then dumping said iron powder into the ocean at an appropriate point.

kagis

https://liquidtrees.org/team

Nine Indians, four Spaniards, and a German, apparently.

You can bioengineer algae to do pretty much anything.

They are psychologically calming for people as well.

https://en.wikipedia.org/wiki/Dendrocnide_moroides

Dendrocnide moroides, commonly known in Australia as the stinging tree, stinging bush, or gympie-gympie, is a plant in the nettle family Urticaceae found in rainforest areas of Malesia and Australia.[3] It is notorious for its extremely painful and long-lasting sting.

Depends on the tree.

Let’s tie him up and sneeze on him.

As I recall, at least under US law, you can’t copyright genetically-engineered life, just get a twenty year biological patent. So I don’t think that FOSS status would be directly germane other than maybe in how some such licenses might deal with patent licensing.