fjm once mentioned at a con that she divides stories with magic into two kinds: "fantasy" and "fancy". I thought that was a useful distinction and immediately adopted it. Basically "Fantasy" stories obey "rules", both with regard to plot and character and also the magic. "Fancy", however, just throws in lots of cool stuff. So Lord of the Rings and Bujold's Curse of Chalion would be "fantasy" and Alice in Wonderland would be "fancy". Which you prefer is merely a matter of taste. There's nothing intrinsically better or worse in either and both can be done well or badly. ("Fancy" when done badly soon descends into either chaos or whimsy. "Fantasy" can end up reading like a badly transcribed RPG session.)
So far so good...
Then someone (who likes classifying things rather precisely) brought up the question of fairy stories. This led to more thought on my part and I came to the following conclusions.
The way I would split stories into "fantasies" and "fairy tales" is purely by considering the events and the characters. (I'm taking magic as a given in both, otherwise we're talking about something else completely.)
Fairy tales have a smaller scale and are about personal stories. They're about something happening to one or two people. e.g. a man was walking home late one night and saw some fairies dancing. A young girl was treated badly by her step-mother and made to sleep in the ashes. If the LoTR didn't exist, I would definitely put The Hobbit into the fairy tale category. As the LoTR does exist, then it has the effect of dragging The Hobbit into the fantasy category because the events there become part of a larger whole. I would definitely put Strange and Norrell into fairy tale because it's all about what happens to a small group of people and is about their relationships and tragedies. The interlude of the Napoleonic War is only relevant insofar as it alters the relationship between the two magicians. In fairy tales the protag gets him or herself into a mess and also gets him or herself out of it, sometimes with help, usually by their own efforts.
Fantasy, on the other hand is on a larger scale. Even if it is told through one person's eyes, it's about being swept up in larger events that effect lots of people, even whole countries and worlds. The protag may be one of the movers and shakers, but can also be the "little person" innocently minding their own business who gets dragged in regardless. So LoTR is definitely fantasy. So too, I think, is Emma Bull's War for the Oaks, despite it having faerie creatures in it, because it's about an ordinary human woman being swept up into an age-old battle. Though her story is told, it isn't only her story. Likewise stories like Narnia and Pullman's Lyra trilogy are thus fantasy.
Fancy is easy to spot because there doesn't have to be the underlying story logic. Fancy can be large scale or small scale. I don't tend to like fancy, but the Alice books are the classic example and I'd put Harry Potter right on the fantasy/fancy border. (As I said, I don't like rigid categories, I prefer the Venn diagram approach where you get overlapping circles.) Jasper Fforde's Eyre Affair and his recent The Big Over Easy are modern versions of fancy, where the whimsical is kept to the minimum, but they're neither fantasy nor fairy tale.
Of course fancy, fantasy and fairy tale can easily segue into the "weird shit" category, which is the only place I can put things like Malcolm Pryce's Aberystwyth noir thriller parodies, which are sufficiently odd that they can't be considered mainstream so I see them as huddling right next to the fantasy/horror border.
And now I've completely lost my train of thought after mentioning horror. I'm not even going to go there.
Where was I? Oh, yes...
As far as the writer is concerned, classification systems are only of use in order to help sort stories out when they've gone wrong. If you're writing a fairy tale type fantasy and the whole thing suddenly opens out into large vistas where the fate of the world is at stake, then it was fantasy all along and perhaps the opening should be tweaked, with hindsight, to put in the necessary hints up front, otherwise the reader who likes the small scale personal story might feel cheated and those who like the larger sweep of fantasy might be put off by the opening chapters.
Likewise, when the story is on the larger scale of fantasy, the characters shouldn't be spending a disproportionate amount of time on their personal problems except as they relate to the story as a whole. So their personal lives may have an impact on the greater story, but the personal life isn't the greater story. (I was guilty of this to a large degree in the early versions of The One About the War. I didn't, at that time, have a firm grip on what was important to the story and what was important to the characters, which is not at all the same thing. Confusion in this department leads to detailed scenes where the characters do lots of personal stuff which doesn't move the story forward. I call these "choosing the curtain material" scenes.[*] I now try to avoid them and remove them when spotted.)
However I'm not trying to put this forward as some kind of Universal Story Classification System. I've put it here for my own benefit as it clarified a way of thinking about stories that I think will be useful in the future when writing or story planning.
Take it or leave it, as you see fit.
[*] Similar to, but not quite exactly the same as, "dwarves drinking soup" scenes.