Gray joined Glasgow in August 2008 after a distinguished run with Scotland's age-grade teams. A former Under-18 and Under-19 cap, he later skippered Scotland Under-20 in 2009 after injury had ended his involvement in the 2008 IRB Junior World Championships.

In 2009 Gray signed a new two-year deal with the Warriors as they scrapped at the top of the Magners League and he was called up to the Scotland squad for the 2010 Six Nations by coach Andy Robinson off the back of some strong showings in the Magners and Heineken Cup.

Through the 2010 November internationals Gray grew as an international player, exhibiting grit in the tight as well as explosive play in the loose. During Scotland's disappointing Six Nations months later he was comfortably their standout performer, excelling against France in Paris and winning the Man of the Match honour in their sole win of the tournament, against Italy at Murrayfield.

During the World Cup, Gray started every match for Scotland though was unable to guide them through to the knockout stages of the tournament despite being one of their standout players.

In November 2011, Sale Sharks announced his signature with the lock signing a three-year deal with the Aviva Premiership club having reportedly turned down advances from Bath, Saracens and a host of Top 14 clubs. Gray joined the Edgeley Park side from June 2012.

Sale struggled in the English Premiership in the latter stages of 2012, as Scotland did on the international front. But Gray played an important role for the Scots in the 2013 Six Nations until injury ended his tournament and put his hopes of touring Australia with the Lions in doubt.

However his return to fitness and undoubted ability led to Warren Gatland's selecting him for the Lions touring party, despite a lack of match practice in the weeks running up to the selection.