This is basically the solution. Just give a few words to explain that different servers can have some rules differences and offer the easy join button.

Get people onboarded fast and easy! If they want to, they can learn more afterwards.

Why do you care about downvotes? You sharing your opinion and perspective is what makes the platform better. Other people clicking an arrow means nothing :)