The IOC executive board disqualified the entire team, the fourth gold and sixth overall medal stripped from that U.S. track contingent in the past eight months for doping.
Three gold and two bronze were previously removed after Marion Jones confessed to using performance-enhancing drugs.
Saturday's decision was almost a formality after Pettigrew gave up his gold medal in June. During a trial involving former track coach Trevor Graham, he admitted in May that he used EPO and human growth hormone from 1997 to 2003.
Five of Pettigrew's teammates also lose their medals: Michael Johnson and twins Alvin and Calvin Harrison ran in the final; Jerome Young and Angelo Taylor ran in the preliminaries.
It was Johnson's fifth gold medal of his stellar career. He has already said he was giving it back because he felt "cheated, betrayed and let down" by Pettigrew's testimony. Johnson still holds world records in the 200 and 400 meters.
Three of the four runners from the relay final have been tainted by drugs.
Alvin Harrison accepted a four-year ban in 2004 after admitting he used performance-enhancers. Calvin Harrison tested positive for a banned stimulant in 2003 and was suspended for two years. Young was banned for life for doping violations.
"We support the action taken today by the IOC," USOC spokesman Darryl Seibel said. "Athletes who make the unacceptable choice to cheat should recognize that there will be consequences. Those consequences can be severe including the loss of medals and results. We're in full support of this action. In other matters like this in the past we've worked with the IOC to make certain medals will be returned, and we'll do so again."
The IOC also disqualified Pettigrew from his seventh-place finish in the individual 400 meters in Sydney. And the committee banned him from attending the upcoming Beijing Games "in any capacity," including as a competitor, coach or technical official. Pettigrew has retired from competition, and the U.S. Olympic Committee said there were no plans for him to be in Beijing.
The IOC had previously tried to strip the relay team after it became known that Young tested positive before the Sydney Games. But a decision by the Court of Arbitration for Sport said the entire team should not be disqualified, and Pettigrew and the others were allowed to keep their medals.