The song is based on the Biblical Psalm 137:1-4, a hymn expressing the lamentations of the Jewish people in exile following the Babylonian conquest of Jerusalem. The song also has words from Psalm 19:14...

'Let the words of my mouth, and the meditation of my heart, be acceptable in thy sight..."

It is one of a few pop songs whose lyrics come directly from the Bible (see also "Turn! Turn! Turn!" by Pete Seeger and The Byrds, "40" by U2, and "The Lord's Prayer" by Sister Janet Mead).

"Rivers of Babylon" was covered in 1978 by Germany-based disco band Boney M., with a version that was released as a single. Boney M.'s release stayed at the no. 1 position in the UK for five weeks and was also the group's only significant US chart entry, peaking at no. 30 in the Pop charts. Boney M.'s version of the song remains one of the top ten all-time best-selling singles in the UK, where it is one of only seven songs to have sold over 2 million copies.