I've experienced this same issue when running 4.1.0-RC2 or 3, including AP not working well on subsequent power ups. I still have a copy of 4.0.0-RC4 and have downgraded to that for my 3A+ hotspots, and that works just fine. I still need 4.1 for my RPi 4B hotspot, though. Once in awhile, my 4B hotspot connects on the first power up. I have a Nextion screen attached via USB, so can monitor status via that. If it doesn't connect within a couple minutes, I just power it off and back on. Usually, it will connect on the second or third try, but it has taken as many as six power ups.
FYI, we're also discussing this issue on another thread:
viewtopic.php?f=3&t=2049