As well as the top job at Newmont, Goldberg has held several heavy-hitting roles, including senior positions at Rio Tinto from 2004-2011.
After announcing his intention to retire from Newmont earlier this year, he stepped down as CEO in October and was succeeded by Australian Tom Palmer.
"Given his extensive mining experience, Mr Goldberg will no doubt be viewed as a strong addition to BHP's board," said BMO analyst Edward Sterck.
Apart from his corporate roles, Goldberg was also director and vice-chair of the World Gold Council, director and treasurer of the International Council on Mining and Metals, co-chair of the World Economic Forum for mining and metals governors, and chairman and director for the US National Mining Association.
On joining the BHP board, Goldberg will become a member of the sustainability and remuneration committees.
Although he stepped down as Newmont Goldcorp CEO in October, Goldberg will still be involved with the company for a few more months, as he was appointed an executive adviser to Palmer until March 2020.