I can add this to the other prior answers (and it would have been my first answer anyway 
): there is a strong theme in the NT (and even, to a lesser extent, in the OT, where it expresses a different way), where the first people in line to be zorched by God are those who insist that God will not be saving those-other-people-over-there.
Even non-universalists recognize this judgment against a non-salvational attitude, with some frequency or at least to some degree. A Calvinist, for example, who doesn’t believe that God ever even intended to save the non-elect from their sins, might easily still think he himself, since he doesn’t know for sure God’s intentions in particular cases, ought to be cooperating with God by reaching out as an ambassador of the reconciliation to everyone in principle, even though in practice he would agree that it’s worse than pointless to reach out to those whom God doesn’t even intend to save from sin. (Or he might even agree there’s a point to reaching out to them, too, for the purpose of some kind of illustration.)
But even if a hardcore Calv didn’t want to work against God by accidentally reaching out to the non-elect (and so maybe for this reason among others, she worked at trying to figure out who isn’t of the elect so that she can treat them properly as “cigarette people” or whatever), she would at least agree that to refuse to pray for the salvation of God’s elect, even if “hyper-ogres” are included among the elect, whom she would prefer with strong emotions not to be saved and not to pray for, then she would be the one sinning against God, and against God’s saving long-suffering patience (God’s {makrothumia}). To insist on the non-salvation of God’s elect would even be evidence that she has not yet been regenerated by God, and worse that her own assurance of being of the elect was only a psychological lie she was telling herself as an unregenerate sinner! It is the non-elect, after all, who insist on God’s elect never being saved!!
I’m using an extreme example from among non-universalist believers here, to illustrate the point. Although presenting the same point from a different direction, using the Calvs again, they would have the first same answer in principle to the same challenge put to universalists (sometimes by inattentive and clumsy Calvinists, in my experience ): if God is going to surely save the elect anyway, then why bother praying for the elect to be saved? BECAUSE CO-OPERATING WITH GOD IS THE RIGHT THING TO DO! – if I refuse to cooperate with God, then I’m the one who is sinning. If God intends to save those people from their sins, out from doing injustice and into doing justice forevermore, then I had better be jumping along with that and meanwhile asking “how high would You have me be jumping, God?” 
Or else I’m the one insisting on final injustice.
(…which, cough, then becomes an indictment on Calv theology, even though not necessarily on Calvs personally, insisting on final injustice. 
But then I’m a Christian universalist, so not my problem, yay. 
)
That’s my primary answer in principle. In context of 1 Tim chapter 4, however, Paul has a more specific and practical application in view: stating this as a reminder of the importance of personal discipline by people who are already Christians, not to be doing this or that. For while bodily discipline, simply by itself, is of little profit, the spiritual discipline of godliness is profitable for all things since it holds the promise for the present life as well as for the future life to come. But for this reason we labor and strive: because we have fixed our hope on the living God Who is the Savior of all men, especially of those who believe.
This is basically the same as his statement before the love-hymn of 1 Corinthians, that it doesn’t matter how we suffer, or discipline ourselves, or whether we learn all knowledge or whatever: all that becomes useless and even worse than useless, if we do not have God’s true love, {agapê}. What is God’s true love? To be the savior of all persons. To always strive to save all persons, and to never give up or lose hope or lose faith in this love, because this love shall never fail.