Some customers in Madison County will experience a temporary water outage for water system improvements on Wednesday.