Erdős Problem 486: Logarithmic density for sets avoiding modular subsets

For each $n \in \mathbb{N}$ choose some $X_n \subseteq \mathbb{Z}/n\mathbb{Z}$. Let $B = \{m \in \mathbb{N} : \forall n, m \not\equiv x \pmod{n} \text{ for all } x \in X_n\}$. Must $B$ have a logarithmic density?

Reference: erdosproblems.com/486

theorem Erdos486.erdos_486 : sorry ↔ ∀ (X : (n : ℕ) → Set (ZMod n)), ∃ (d : ℝ), {m : ℕ | ∀ (n : ℕ), ↑m ∉ X n}.HasLogDensity d

Erdős Problem 486

For each $n \in \mathbb{N}$ choose some $X_n \subseteq \mathbb{Z}/n\mathbb{Z}$. Let $B = \{m \in \mathbb{N} : \forall n, m \not\equiv x \pmod{n} \text{ for all } x \in X_n\}$. Must $B$ have a logarithmic density?