Scalable evaluation of incoherent infidelity in quantum devices