Interest rates on loans to buy construction materials are down, but mortgages aren't. Labor costs are also a concern.