Michael S. Gorbey, 38, was charged with being a felon in possession of a firearm, U.S. Capitol Police said.
The man also was wearing a tactical vest and had a bow in his car, Terrance W. Gainer, the Senate sergeant-at-arms, told The Washington Post.
Gorbey said he was going to the Supreme Court, Gainer said.
An officer first spotted the man near Union Station, about two blocks from the Capitol. The man's vehicle had wires coming out of the glove box and propane tanks inside, Gainer said.
Police used a robotic camera to see inside the vehicle, Schneider said, then destroyed the suspicious items inside with a powerful water hose.
