This is a great deal more than a new heart. This a heart filled with the Holy Ghost, the Divine Spirit, the power that causes us to walk in God's commandments.
This is the greatest crisis that comes to a Christian's life, when into the spirit that was renewed in conversion, God Himself comes to dwell and make it His abiding place, and hold it by His mighty power in holiness and righteousness.
Now, after this occurs, one would suppose that we would be lifted into a much more hopeful and exuberant spirit, but the prophet gives a very different picture. He says when this comes to pass we shall loathe ourselves in our own eyes.
The revelation of God gives a profound sense of our own nothingness and worthlessness, and lays us on our face in the dust in self-abnegation.
The incoming of the Holy Ghost displaces self and disgraces self forever, and the highest holiness is to walk in self-renunciation.