Only thing I miss is proper support for some services I use. Minikube is afaik still a pain with podman, at least rootless. Gitlab runner still doesn’t support podman completely imho. But a plus to docker is that they still build packages for EL 7 while the podman version in EL 7 is pretty damn old. Besides from that I went podman all the way.
SELinux should not be an issue if you stick to common directories and use :Z flag after the mount path with docker, afaik podman uses the same mechanism. There’s even a tool for selinux container policies: https://github.com/containers/udica
Regarding firewall stuff, disable it on your machine and you are fine. Port forwarding in containers is necessary to connect to services, now way around.
Ah and read this: https://stopdisablingselinux.com/
It has a reason why it exists.