The U.S. Anti-Doping Agency announced Hardy's decision, saying she did so "in the best interests of the team."
The 21-year-old from Long Beach, Calif., could have contested the drug test results before the American Arbitration Association and potentially filed an appeal with the Court of Arbitration for Sport, which would have kept her first Olympic berth in doubt until the eve of the games that open Aug. 8.
"While some might have chosen to exhaust their legal options to try to force their way into the games, Jessica instead chose to put her team's interests ahead of her own," said Travis Tygart, chief executive officer of USADA.
Hardy tested positive for a low level of clenbuterol, a prohibited anabolic agent, at the trials on July 4.
She didn't immediately return a cell phone message left Friday by The Associated Press, but Hardy told the AP in a July 25 interview that she was innocent and said she had no idea how she tested positive for clenbuterol.
"I had never even heard of this drug," she said. "I've never wanted to do anything remotely close to doping. It's never, ever crossed my mind. I've never been approached about doing it. It's never been an issue my entire career. It's the last thing I would ever do."
The U.S. swim team was training in Singapore on Friday and leaves Monday for Beijing.
Hardy qualified for the Olympics in two individual events - the 100-meter breaststroke and 50 freestyle
- and the 400 free relay. She was a strong contender to medal in her specialty, the breaststroke.
Hardy's withdrawal allows USA Swimming to place swimmers already on the team in her open events, according to executive director Chuck Wielgus.
Olympic rookie Rebecca Soni, who trained with Hardy at Southern California, will swim the 100 breast and Kara Lynn Joyce will swim the 50 free, he said.
It's the second lucky break Joyce has gotten since the trials.
The 22-year-old Georgia swimmer was initially added to the team as an alternate on the 400 free relay after Dara Torres withdrew from the 100 free.