Happy the heart where graces reign. I. Watts. [Love to God.] First published in his Hymns & Sacred Songs, 1707 (2nd edition 1709, Book ii., No. 38), in 5 stanzas of 4 lines, and entitled, “Love to God." Of this hymn stanza iv. and the idea embodied in stanza v. had previously appeared in Watts's hymn, "'Tis pure delight without alloy," given in his Horae Lyrica, 1706, stanzas iii., iv. It is in extensive use in Great Britain and America.
--John Julian, Dictionary of Hymnology (1907)