"Just saying." As in, "Most people don't care what phrases irk you. Just saying." I do believe it is one of the most irksome phrases to enter the common vernacular in recent memory. Certainly, you can make a good argument for others, and people have. But in my humble opinion, this one takes the metaphorical cake for being at once redundant and irksome. It is redundant in that the speaker is, obviously, "saying" -- unless by that you mean 'speak', and they are singing, writing, miming, sending Morse code or otherwise communicating without speaking. In which case it is inaccurate. It is irksome in that it is consistently applied as a mitigating agent to snark (or, as Urban Dictionary puts it, “a term coined to be used at the end of something insulting or offensive to take the heat off you when you say it”). Which simply doesn't work. You don't get to jab someone in the eye in then take it back, sort of. Because you're "just saying". Either stand by what you say, and own your comments, or don't make them. Don't try to sneak between the two, by saying what you really mean and then observing that you're "just" saying. There's no "just" about it. You are saying, and saying what you mean. This is obvious, and no qualification is necessary; and, if the message was so harsh that you think that softening is in order, it's time to change the way you express yourself, rather than shrugging it off as no biggie because, like, I'm just saying, she soooo looks like a cow in that.
Now, for those to whom sarcasm is not a companion for rapier wit, wielded sparingly as appropriate, but the blunt instrument of boorish insult, a bludgeon with which to beat those unhappy enough to be on the receiving end, such a phrase is delightful. It is another means, albeit an intellectually dishonest one, by which to give offense while pretending to some measure of deniability. "Only an idiot would watch that show. Just saying." In those cases, it is, at least, not unexpected. What is beyond comprehension, though, is when this tiresome phrase passes from the lips of those who, in general, know better -- and not in the sentence, "'Just saying' is a sad attempt to say what you mean when you don't have the courage to stand by your words. Just saying." I allow a full exemption for such usage, as it illustrates perfectly the bothersome nature of the term whilst furthering a noble goal (the eradication of "just saying" from the common vernacular). Otherwise known as fighting fire with fire, beating them at their own game, etc. It is this instance that prompts my soul to weep -- when I witness otherwise upstanding and erudite intellects succumbing to this usage. (To be honest, after being subjected to its use enough times, there has been an instance or two where I, the self appointed crusader against the phrase, have found myself unthinkingly participating in such an abuse of language; but I prefer to push such dark days far from memory).
As with other creeping, speech borne language-bugs, such as the “like” pandemic that ravaged the English language during the 90's, this insidious phrase has crept its way onto the tongues of frequent language abusers as well as good citizens. Therefore it is my urgent plea, for the good of all humanity, or at least the English-speaking bits of it, that everyone, everywhere, drop the use of “just saying”. There's no “just” about it: you are saying. So own up to your words, or shut up. Just saying*.
* Exempted under the “fighting fire with fire” clause mentioned above. ;-)