
The basic facts of the rift between Henry Wriothesley, 2nd Earl of Southampton and his wife Mary (daughter of Anthony Browne, 1st Viscount Montague) are pretty much as Gib’s uncle says.
Their marriage disintegrated in late 1579/early 1580 when the Earl heard that the Countess had been seen with a “common fellow” named Donesame or Dunsum at Dogmersfield, where the Earl was building an impressive house to add to his collection.
The Earl was already suspicious about his wife’s relationship with Donesame, and had forbidden her to see him.
Accounts of the rift are in the biographies of their son (Gib’s lord) by Charlotte Carmichael Stopes and G.P.V. Akrigg.
The contemporary source is an impassioned letter written by the Countess to her father, Viscount Montague. She protests her innocence, declares herself the victim of slander, and names one of the Earl’s attendants, Thomas Dymock, as her enemy.
The Earl had also told her that there was no point in her complaining of her treatment to the Queen. The Countess wanted to have her case heard by some of the Queen’s council.
Mrs Stopes’ transcription of the letter is printed in the Addenda to her biography. She accepts the Countess’s side of the story, and speculates that the Earl’s suspicions and hostility resulted from ill health. He died in October 1581.
G.P.V. Akrigg notes that Mrs Stopes’ transcription contains a number of misreadings. He doesn’t provide his own transcription, or say how significant her misreadings are, but points out that with so little evidence it’s impossible to reach any conclusion. However, his view of the Countess is consistently unfavourable, and he also misinterprets, to her detriment, later letters she wrote on entirely different topics.
Gib’s investigations are not much help. He read Bevis of Hampton, a medieval work of fiction, and was carried away by a few coincidences between the story of the young Bevis and that of his own young lord and master. However, he may have stumbled onto something when he later wondered if a gentleman had dropped a maggot in the Earl’s ear.
Thomas Dymock is one suspect. Another is the intriguing Charles Paget (c.1546–1612), the younger son of a recently ennobled Catholic family: his father was the 1st Baron Paget. Charles was about the same age as the 2nd Earl of Southampton, and a friend.
Both Mrs Stopes and G.P.V. Akrigg note a couple of contemporary references to him as the cause of the Southamptons’ separation.

One is from Robert Persons S.J. He blamed Paget and his associate Thomas Morgan, Mary Queen of Scots’ representative in Paris, for sowing dissension at every level within the community of English Catholics.
Father Persons’ memoir “A Storie of Domesticall Difficulties in the Englishe Catholicke Cause” is printed in the Catholic Record Society Miscellanea Vol 2 (1906). (The title of the memoir refers to Catholic infighting, not the state of any marriage.)
Sir Francis Walsingham, at the opposite end of the religious and political spectrum, also considered Charles Paget dangerous. But, to be fair, Sir Francis and Fr. Persons had very different reasons for their hostility towards him.

By 1581, when Charles Paget was named as a chief executor of the Earl’s will (along with Thomas Dymock), he was in France embarking on a career as one of Mary Queen of Scots’ agents.
In the late 1570s, did Charles Paget think that the Earl of Southampton would be more useful to the Catholic cause in England if he were estranged from his wife and father-in-law? Viscount Montague, although a prominent Catholic, seems to have been more outwardly conformist than his son-in-law was.

Alternatively, Paget may have thought that the more militant Jesuits (of which Fr. Persons was one) were endangering, rather than advancing, the Catholic cause in England, and may have earned Persons’ hostility by working against him.
Or perhaps he just liked playing games of deception, and the exercise of power and control over others?
Whatever, the rift between the 2nd Earl of Southampton and his Countess may have been less about adultery than it was about the political and religious climate of the time.