We all have those who we may prefer to be around, or those we are more comfortable spending time with because we have more in common. We may spend more time with our physical family than we do other people. We may spend more time with families in our congregation with children the same age as ours. We may spend more time with those who have some of the same interests as ours. I do not believe this, in and of itself, is partiality. There are some scriptures in which we are commanded to show preference in some areas, which I believe do show that preferring to spend time with certain people (or groups of people) is not partiality.
For instance, in Galatians 6:10 we are told to “do good to everyone, and especially to those who are of the household of faith.” I believe this verse is showing that we need to go out of the way more in order to do good deeds for our brothers and sisters in Christ compared to those who are in the world. If I have a choice to spend time with or to do a good work for a nonChristian or a Christian in the same exact time slot, I believe a passage like this shows that I should choose the brother or sister in Christ.
Another example of a passage that shows we should show some judgment (or preference) in who we spend time with is 1 Corinthians 15:33 where Paul says, "Bad company ruins good morals." This passage shows the danger of surrounding ourselves with those who can impact us in negative ways spiritually. The context of this passage shows that it is primarily referring to false teachers, but I believe this can be applied more broadly to anyone who can have a negative impact on our spiritual well-being. We should prefer to spend more time with and to surround ourselves with those who can help us to get to Heaven.
There are more examples I could give, but these two go to show, I believe, that there are some instances where we should, and even must, show some kind of preference in regards to company we keep, and when we do so, we are not showing the partiality that is condemned in the Bible.
But we do need to be careful though that our preferences do not turn into partiality. If our preference to spend more time with certain people got to the point where we would be completely neglecting certain groups of people (ex. those who are not in our physical family or those who are not Christians), then it can get to the point where our “preference” becomes partiality. We need, as disciples of Christ, to be willing to spend time with those who are different from us so we can "love our neighbor" (all of our neighbors) as Jesus commands. Jesus’ command to love is not just for those who we prefer to spend time with. It is for everyone!