The son of Captain Augustus and Margaretta Roberts, Hartley was born in New Plymouth in 1899. His ancestors may go back as far as the Eleutheran Adventurers, the earliest English settlers in The Bahamas.

As a young man he worked with his father in the family's general store and captained the schooner Albertine and the motor vessel Priscilla, both mail boats servicing Abaco and Eleuthera.

Hartley retired as a sea captain to devote more time to his business. He and his brother Roland were also partners in a general store and with Mr. Harold Lowe in the shark fishing business.

Hartley enjoyed agriculture and together with his wife, Mira Lowe, and his children, Mizaph, Noel and Minnie, they spent several weeks at a time on his farm on the mainland. He was active in community affairs and served the Church of God as a pastor and teacher.