There is a holy sacrifice. Charlotte Elliott. [The Contrite Heart.] This hymn is usually attributed to J. Montgomery on the grounds that its first appearance as far as yet traced was in the 1819 edition of Cotterill's Selection, to which Montgomery largely contributed; and that in the 2nd series of W. Oliphant & Son's Sacred Poetry, N. D. [circa 1839], pp. 291-2, it is attributed to Montgomery. In 1836 Miss Elliott published her Hours of Sorrow, with an Introduction "To the Reader" which begins:—
"Not for the gay and thoughtless do I weave
These plaintive strains;"
These words to our mind clearly intimate that the entire contents of the book were by Miss Elliott. At p. 10 this hymn is given in 5 stanzas of 3 lines, with the refrain "The contrite heart!" as in Cotterill's Selection, 1819, No. 341. Seeing that in 1819 Miss Elliott was 30 years of age, that it is in her Hours of Sorrow as above, and that in style and metre it is the same as a large number of her hymns, and that it is not in any known work by Montgomery, we have no hesitation in ascribing it to her. It is a sweet hymn for private use, and is found in several collections.
--John Julian, Dictionary of Hymnology (1907)