Yeah, that's my only gripe with it. It takes a long time to build up and it only lasts a little while. Doesn't help that the Purple Orbs meant to be used to refill that gauge don't seem to show up very often, if at all.
Though I guess because it's very powerful in this game, it deserves to only last a little while.