I recently discovered that some popular federated instances have been using LLM-assisted moderation tooling that evaluates whether someone has said something bannable. They do this by running a script/app that sends the user’s comment history to OpenAI with the question “analyze this content for evidence of specific political ideology sentiment. Also identify any related political ideology tropes“. (The italic bits are where I’ve redacted the ideology they’re seeking).
OpenAI’s LLM (they’re using GPT-5.3-mini) then responds with something like:
and so on, hundreds of comments.
I have not named the instances or people involved, to give them time to consider the results of this discussion, make any corrective changes they want and disclose their practices at their own pace and in their own way. I have also redacted the evidence to avoid personal attacks and dogpiling. Let’s focus on the system, not the individuals involved. Today these instances and people are using it and maybe we’re ok with that because it’s being used by groups we agree with but what if people we strongly disagree with used it on their instances tomorrow?
The use and existence of this tooling raises a lot of other questions too.
What are the risks? Fedi moderators are often unsupervised, untrained volunteers and these are powerful tools.
What safeguards do we need?
Would asking a LLM “please evaluate this person’s political opinions” give different results than “find evidence we can use to ban them” (as used in the cases I’ve seen)?
What are our transparency expectations?
Is this acceptable and normal?
Should this tooling be disclosed? (it was not – should it have been?)
If you were given a choice, would you have opted out of it?
Can we opt out?
Are there GDPR implications? Privacy implications? Should these tools be described in a privacy policy?
Are private messages being scanned and sent to OpenAI?
How long should these assessments be retained and can we request to see it, or ask for it to be deleted?
Once the user’s comments are sent to OpenAI, is it used to train their models?
What will the effect be on our discourse and culture if people know they are being politically profiled?
Where are the lines between normal moderation assistance tools, political profiling and opaque 3rd-party data processing?
I hope that by chewing over these questions we can begin to establish some norms and expectations around this technology. The fediverse doesn’t have any centralized enforcement so we need discussions like this to develop an awareness of what people want in terms of disclosure, privacy, consent and acceptable use. Then people can make choices about which instances they join and which ones they interact with remotely.
And of course there are the other issues with LLMs relating to environmental sustainability, erosion of worker’s rights, increasing the cost of living and on and on. I can’t see PieFed adding any functionality like this anytime soon. But it’s happening out there anyway so now we need to talk about it.
What do you make of this?
I mean, you just did it. The OP post is not factually correct as he stated that it is at an instance level. It is not. It is at a moderator level.
Gotta say though, getting real tired of people telling me that I didn’t understand conversations I was a part of. No. It was not ‘debunked’. It was added under an opt-in toggle after everyone noticed it and called out his bullshit. You might be thinking of something else, but this is what I was talking about.
Piefed is infected with Rimu’s extreme opinionated garbage and he only backs down and puts them into a toggle after someone notices it. That’s not the behavior of a developer that I find personally trustworthy. So when he’s out here making a post that is outright false, claiming that instances are doing something that moderators are doing, I don’t trust him. Not to mention the extremely long conversations I had with Mr Kaplan about how Piefed.world needed to be un-rimu’d in order to work as LW wanted it to.
I personally prefer Rimu taking feedback into account and then making changes than Lemmy devs bluntly closing feature requests widely upvoted by the community because “reasons”
Let’s be honest here, it’s not like there are so many options we can choose from.
Also, the example you gave about shows more than anything else that Rimu did change his mind.
“All the settings in this screenshot are off by default”
He only changed his mind after someone discovered it. The thing I don’t like is his obfuscating his opinions that he’s injected into Lemmy. Having to manually discover them and then bug him about adding an opt out toggle is not exactly trustworthy behavior nor is it something I’m going to applaud. He still tried to sneak it in regardless and only backed down when called out. I’m not giving him props for that.
Rimu is like Microsoft Windows.
Adds in things the higher ups felt was needed. Users then discover it and say this is a bad thing to include in software.
Microsoft then says “right our bad, we should have thought twice on this”
Microsoft then does it again a few weeks later, repeat.
Comparing a single open source developer to one of the biggest tech corporations worldwide doesn’t really seem relevant.
Who are the higher ups in this case? There’s no Piefed board of management or investors to answer to.
All of that being said, I’m looking forward the day Pievolution can be run easily, then every instance can make their own choices, and these endless debates will end.
The code is open source, nobody bothers to read it. If that’s supposed to be obfuscation, then that’s not really effective.
The same arguments are made when an overwhelming amount of paperwork is dumped on someone during discovery in a trial. It’s not a good argument then and it’s not a good argument now. If you have to go through and check every single thing he’s done to find the hidden things he’s put in there without announcing then that is not transparent. You simply cannot change the fact that Rimu overwhelmingly puts his opinions into Piefed as a whole and then leaves it for everyone else to find the opinion and then tell him to add a toggle. He fights against it before eventually adding a toggle. Meanwhile, damage has already been done for however long with his garbage running rampant without anyone noticing it.
If he is not willing to point out what is opinion and what is code then I am not willing to fact-check every piece of his work to find what is code and what is crap. He’s being disingenous and underhanded.
I am not continuing this conversation.
I was referring to a different but similar case where someone intentionally spread mis-information about supposedly hardcoded things that turned out to be a complete nothingburger as all of it was behind an admin toggle. The same seems to be now true for this old issue you specifically pointed out here.
It is true that there is some experimental stuff in Piefed, which is part of the relatively rapid iteration of features, but looking at the code and also the explanations given by the Piefed development team I can really not see any malice in those settings. It is perfectly normal that things get overlooked or implemented partially and when someone reports a bug (like a missing admin configuration setting) it usually gets fixed quite quickly, and at least in my experience without much discussions.
But it isn’t. First off, you made an assumption and dismissed my initial complaint. Now you’re dismissing this one saying it’s basically the same thing when it isn’t. Having a long discussion with large admins saying “Hey. What the fuck is this stuff?” only for Rimu to constantly push back and saying how he wants to reshape everything is fucking concerning. The fact it took everyone pushing back against him to add it under a toggle even more so. You’d have a point with the whole “this happens” if this didn’t happen with every single major Rimu feature.
But, once again, Rimu is actively pushing misinformation and you have dodged the point that you are doing the same. This is not an admin or instance level problem. Moderators are doing this and claiming this is “instance level” is to be a liar.
And, as we all know, your experience is the only universal experience that everyone has at all times. I guess the month long conversation I had with Kaplan, head admin of Lemmy.world, about unfucking Piefed because Rimu filled it with his opinionated garbage didn’t happen. I guess the fork of Piefed being created that’s taking out all of his opinionated garbage didn’t happen. Not to mention his 4chan screenshot scanner (that can be bypassed immediately), the cm0002 filter he put onto piefed.social (that can be bypassed immediately), the blocking of any numbers of 88 put together (which can be bypassed immediately), the private votes that would prevent admins from locating vote brigaders (which can be opted, granted, but after a HEATED discussion in the piefed matrix).
Rimu is over opinionated to a fault and Piefed is the same.
You know nothing of which you speak.
But, once again, the only important point is that Rimu is actively spreading misinformation by claiming this is an instance problem when it is moderators that is doing it. Once again, with action after action, Rimu cannot be trusted.
So Kaplan knew about the “garbage”, but still wanted to go on and keep running Piefed.world?
Either Kaplan’s judgement is supposed to be relied on, or not, but that argument seems weird to say the least.
So it is opted. That people have to convince Rimu to make changes isn’t really an argument. People have to also convince the Lemmy devs to make changes on a regular basis, with less success.
Others are doing the same thing. This isn’t “weird” or bad judgment. I don’t like Kaplan but they did nothing wrong here.
Yes. If you skip over the entire problem, it does appear as if there is no problem. My issue is that he had to have his feet held to the fire over something that minor because he did not disclose it in the first place. If someone keeps trying to sneak shit past my face and then keeps backing down and going “Uwu sorry I add opt out” but then doesn’t add opt out for the other opinionated stuff that was not discovered yet, why would I applaud them? They’re still doing the bad thing.
What I’m saying is that even though admins know there were some opinions in the software, they still started Piefed instances. And that’s in a world where Lemmy exists.
In other words, every instance admins assesses whether the additional Piefed features make it worth it to accept the opinionated aspect of the software.
That’s the main value of Piefed here, and that’s why so many people prefer it compared to Lemmy. All the features people wanted, be it users, mods, admins, Rimu delivered them, and fast.
Doing something, even imperfectly, but listening to the users, still seem valuable to a lot of people. Some people obviously dislike it a lot, and expect the software to be written be a perfect person, but the devs behind Lemmy, Piefed and Mbin all have their flaws.
I did not state whether or not Piefed.world was created before or after it was found out that Piefed is littered with Rimu opinions. You are making a conclusion you simply cannot make.
He said, off of a false conclusion from a random assumption.
Brother, you are not listening to a word I have been fucking saying.
He’s not doing it imperfectly. HE’S BEING DISINGENOUS. If I were to talk to you and give you a ton of shit, fill all of it with some useful stuff but the rest is hyper opinionated, you’d expect me to tell you what was opinion and what isn’t. Rimu doesn’t. People have to discover it and then hold his fire to the flame. That is not listening to feedback over things that you knew was a problem. Especially when he knows it is opinionated and then puts it through anyway without informing anyone, without making a note of it, and expecting others to comb through his garbage and hold him accountable. And you are saying that is perfectly acceptable behavior?
Brother, you have lost essentially all of my respect here. As stated elsewhere, because you started two conversations with me at once, I am not continuing this with you.