This was more or less my impression, if the House has notable mentions in the books or are otherwise acknowledged in canon as controlling significant territory I don't see an issue.
Besides I think we've already made decent progress, we've now got banners for: Banefort, Belmore, Bolton, Caron...