Perhaps you should take a look at the latest libwiigui revision. There's a few important changes I've made to improve the efficiency of libwiigui that should help with this.
1) font rendering - freetypegx was clearing and repopulating the font glyph cache if you use mulitple font sizes
2) waitvsync was replaced with usleep (for a shorter wait period)
3) some logic changes - basically just avoid recreating text variables (eg: via sprintf) unless a value *actually* changed