I've done rock climbing in both the five fingers and regular rock climbing shoes. Rock climbing shoes are definitely better 99% of the time, their rigidity helps you to maintain secure footholds.

There's the occasional exception though, I was once able to gain a foothold in the five fingers by sticking my big toe in a hole that fit it exactly.