Frustrated traditional owners have moved to speed up negotiations over protecting their sacred sites on a northern territory mine site.