Yes, if there is some reason, then ask the Ulama that in these circumstances what is the law of Shariah e.g. some family member always causes harm to one like a scorpion or some irreligious member forces you to participate in evil,  forcing one to come there and watch T.V. If for the sake of Deen, the rights of the family members are not fulfilled, then this is absolutely correct. At this time, do not go to them. However, later on explain to them with softness. Take them to some Allahwala. Inshallah it will be beneficial. In extreme cases, severing all connections is permissible; however, consultation with the Ulama will still be necessary. If anyone faces such conditions, then he should consult with me privately. In the commentary of Mishkaat, Mirqat, there is an amazing discussion based on this. I will Inshallah give my opinion, which was very much liked by my Shaikh also. However, severing ties with close family members without the light of knowledge and merely following self-opinion without consulting the Ulama is not permissible.

Friends! A Hadith states that on the Day of Judgment, Allah Ta'ala will grant a tongue to blood ties which will say “O Allah! I am family ties. You have named me Rahm. Your name is Rahman (the most Merciful). I am derived from You i.e. I have emerged from You. Today, grant me my rights. Those who have joined me, You join them to Yourself and those who have cut me off, You cut them off from You.” Therefore, this is a very serious matter. Consult! Even though in extreme cases, due to certain difficulties, there are cases where severing of ties is permissible. However, as far as possible, try to accommodate. Do not break off family ties on insignificant and trivial matters.