That's not all that bad. The second time around you're not forced to fight them. Also, If you're really strong and going back to easier difficulties to get better rankings, you may not be able to get enough style points; so fighting enemies again helps.