"Mr. Warren compared same-sex couples to incest. I found that deeply offensive and unfair," Rep. Barney Frank, D-Mass., said in a broadcast interview.
"If he was inviting the Rev. Warren to participate in a forum and to make a speech, that would be a good thing," Frank said. "But being singled out to give the prayer at the inauguration is a high honor. It has traditionally given as a mark of great respect. And, yes, I think it was wrong to single him out for this mark of respect."
Warren, a best-selling author and leader of a Southern California megachurch, is a popular evangelical who stresses the need for action on social issues such as reducing poverty and protecting the environment, alongside traditional theological themes.
But gay rights advocates, who strongly supported Obama during the election, are angry over Warren's backing of a California ballot initiative banning gay marriage. That measure was approved by voters last month.
Although Warren has said that he has nothing personally against gays, he has condemned same-sex marriage.
"I have many gay friends. I've eaten dinner in gay homes. No church has probably done more for people with AIDS than Saddleback Church," he said in a recent interview with BeliefNet. But later in the interview, he compared the "redefinition of marriage" to include gay marriage to legitimizing incest, child abuse and polygamy.
Warren, in a speech on Saturday, said he took "enormous heat" three years ago for inviting Obama to speak at his church, even though the two men disagree on some issues. "Now he's invited me," Warren said.
Obama defended the selection of Warren last week, telling reporters that America needs to "come together," even when there's disagreement on social issues. "That dialogue is part of what my campaign is all about," he said.
Frank appeared on "Late Edition" on CNN.