A Wortley pedigree, by Hunter, reports that Nicholas de Wortley, son of Sir Nicholas de Wortley, married Joan, dau. of Ralph Musard and married second, Matilda de Dutton. Joan and Nicholas were the parents of Sir Nicholas de Wortley, husband of Horbury. 
Nicholas de Wortley married an heiress, Isabel Heron, who brought lands in Hoyland Swein. 
See also, a Wortley pedigree published by Foster. 
 Joseph Hunter, South Yorkshire, the History and Topography of the Deanery of Doncaster, Vol. 2 (London: 1831), 324.
 Alfred Gatty, Wortley & the Wortleys: A Lecture (1877), 8, [InternetArchive].
 Joseph Foster. Pedigrees of the County Families of Yorkshire, Vol. 2, West Riding (London: 1874), Wortley, [InternetArchive].