Get a handle on the issue or it will destord the marriage. Do you have any contact for counciling
You could have another Misstress be his chastity key holder.