The hamming distance of two vectors $a, b \in \mathrm{F}$ is the number of elements of $a$ that differ from $b$.
In Python it can be calculated:
def hamming_distance(a, b):
if len(a) != len(b):
raise ValueError("a must be the same length than b")
return sum(x != y for x, y in zip(a, b))Supose $X: \Omega \to A \subseteq \R$ a discrete random variable, We call $P$ the mass function of $X$ defined by:
$$ P(x) = P[X = x] $$
The mass function is the analog of the density function $X$ were continuous random variable.
Entropy in information theory (also called Shanon Entropy) is a generalization of Thermodynamics Entropy (Boltzmann Entropy).
The entropy $H$ of a discrete random variable $X$ with possible values ${ x_{1}, x_{2}, \dots, x_{n} }$, with a probability mass function $P$ is defined by:
$$ H(X) = E(-\log(P(X))) $$
More explicitly:
$$ H(X) = - \sum_{i=1}^{n} P(x_{i}) \log (P(x_{i})) $$