Can one claim to live a good faith life while neglecting his family life?

When we look at Elkanah, a man who went up to Shiloh every year to worship and offer sacrifices to the Almighty God, it seems that he was a good believer.  However, when we consider that he especially loved Hannah, who had no children among his two wives, and that whenever he offered sacrifices, he gave the meat of the offerings to Peninnah and her children, but gave Hannah a double portion, it seems that he did not handle his family life well (1 Samuel 1:2-5, Korean Modern Bible).  Can one truly live a good faith life while neglecting his family life?