The classical Bertini irreducibility theorem states that if $X$ is an irreducible projective variety of dimension at least 2 over an infinite field, then $X$ has an irreducible hyperplane section. The proof does not apply in arithmetic situations, where one wants to work over the integers or a finite fields. I will discuss how to amend the theorem in these cases (joint with Bjorn Poonen over finite fields).