Frequentist vs Bayesian
Suppose I want to know the average height of people in the US. This average I will call hpop. To estimate hpop, I randomly choose 100 people and take their heights, and compute the average height. Call this sample average hsamp. For most people and I, saying "hpop is probably somewhere near hsamp" is good enough. But a statistician wants to know exactly how close hpop is to hsamp, and with what probability.
At this point, a frequentist (not to be confused with a Freakwentest), creates a "confidence interval." This involves estimating the variance of heights, and then applying some theoretical crap having to do with "normal" distributions to come up with a minimum number, say 5.5 ft, and a maximum number, say 5.7 feet, such that we can be "95% confident" that hpop is between 5.5 and 5.7 feet.
A bayesian, on the other hand, does something weird and ends up with a statement like "the probability that hpop is between 5.5 and 5.7 is approximately 0.95." This sounds a lot like being 95% confident that hpop is between 5.5 and 5.7, but it's not.
Someday I will have a better answer for the question, but until then I am an opportunist, picking between frequentist or bayesian methods on the basis of convenience instead of ideology.