We sometimes use past tenses to describe things in the present or future that are imagined or unreal. It's time we went. We are using It's time... here to say that something is not happening ...