The first step in any statistical analysis, be it frequentist or Bayesian, is the construction of a probabilistic model which describes, mathematically, the way the data under study has been generated. But the notion of probability differs in these two approaches to statistical inference.

The frequentist approach is based on the conventional notion of probability as frequency, commonly referred to as objective probability.

The (objective) probability of occurrence of an event, A, can be defined as the ratio of the number of ways in which A occurs compared with the total number of possible outcomes.

Nowadays, the common understanding is that some problems are better handled using frequentist methods while others with Bayesian methods.

In contrast, in Bayesian statistics the notion of probability, commonly known as subjective probability, is one of a measure of one’s degree of uncertainty about an event.

Under this approach, probability distributions are used to model knowledge about model parameters or hypotheses; this is done through the use of the famous Bayes’ theorem.

Another key difference is that in the frequentist approach, results are based solely on the data collected; Bayesian inference, through the use of prior distributions, incorporates any information about the unknown parameters which is external from the data collected, such as results from previous similar studies. Nowadays, the common understanding is that some problems are better handled using frequentist methods while others with Bayesian methods.