James was educated at the Tudor House School in Moss Vale and at Cranbrook School. After obtaining the Higher School Certificate (HSC) at Cranbrook, Kerry decided it was time to toughen his son up to prepare him for the world of business. He was sent to his father's extensive Newcastle Waters cattle station in the Northern Territory, where he worked as a jackeroo. His father dismissed the idea of sending young James to university by saying "Why would he want to go there [to University]. To go learn to smoke marijuana?" Besides his father, his toughest mentor has been Albert J. Dunlap. Dunlap is known in American business circles for shedding massive numbers of company workforces to improve the finances of corporations; his tenure at Sunbeam is a specific example.

James Packer was worth AUD$7.25 billion in 2007, however due to poor investment decisions and falling profits, a television report on Thursday 21st June 2007 revealed that Mr Packer has lost in excess of $1 billion over the last six months. As a result, the 2008 BRW Rich 200 listed Packer as the third richest person in Australia with a personal wealth of AUD$6.1 Billion, behind Fortescue Metals Group chief executive Andrew Forrest and Westfield Group's Frank Lowy. This is the first time in 21 years that the Packer Family has not topped the list. He recently purchased a new Canadair Global Express for $40 million and spent a similar amount on the first ever Mangusta 167 motor cruiser. Whilst waiting on delivery of the Mangusta 167, he has been loaned a Mangusta 108

Packer is married to model and singer Erica Baxter, whom he wed in the equivalent of a civil ceremony on June 20, 2007 after dating on and off for four years. The wedding was at the Antibes town hall, and the second ceremony took place at Hotel du Cap - Eden Roc in Cap d'Antibes on the French Rivera. They are currently expecting their first child who will be born in June.

Packer's past girlfriends have included Tania Bryer and Joan Severance. After breaking off a two year engagement to Kate Fischer, in October 1999, he married swimsuit model Jodhi Meares with whom he separated in June 2002. He currently resides in the plush suburb of Bellevue Hill, in Sydney's luxurious Eastern Suburbs. He lives in the same estate (known as Packer Estate) that his late grandfather, Frank Packer, did. At the centre of this estate is the "Cairnton" mansion.

In 2002, Packer began attending the Church of Scientology in Australia, taking courses on the Church's Dundas business center. It was reported in the Australian media that Tom Cruise had urged Packer to join Scientology and begin auditing sessions. The Sydney Morning Herald said "the bond between the two men strengthened in the wake of the collapse of the telephone company One.Tel last year and his separation from wife Jodhi in June". During this time, Packer travelled extensively to the Scientology Celebrity Centre in Los Angeles. It has been reported by the Australian Daily Telegraph that Packer was an extra in Cruise's film The Last Samurai. According to The Sydney Morning Herald, Packer is no longer taking courses and has distanced himself from the Church of Scientology