In my personal opinion. I would say his absurd amount of "luck"(plot armour) should be held accountable for most of his circumstances. And I think he is responsible enough to manage his duchy, unlike "that guy"(can't remember if the word "brat" or lord was used to describe "him") from Eashen(Oda) that one of the Elite Four guys spoke of, before settling in Touya's country(Brunhild), for causing financial strife by raising the citizen's taxes, just so he can fill his own coffers. Unlike Touya, that lord was spoiled as whisical. And the citizens taxes; is nothing, but "chump change" to the self-sufficient Touya(his "cheat-like" abilities is also a contributor. Sorry Touya). As he barely even needs it, more than anyone.