An effective way to counter their teleportation is to either not shoot at them, since they won't teleport unless shot at, or get a weapon powerful enough to kill them before they teleport. Meltaguns work well for this, as they can disintegrate a pack of Bloodletters at close range in a single and, at that range, inescapable blast. If they reach melee range, go on the offensive, since they can deal quite a bit of damage, but can't take as much. Note that some Renegade Militia from Chapter 15 onwards can summon Bloodletters on death.