Feynman and the FBI had an extended encounter after the Bureau discovered he had been invited to speak at the USSR, which set off a flurry of investigations into his loyalty — even as he pestered the State Department for guidance on whether he should or shouldn't go, guidance they only gave belatedly.
Of particular interest to the FBI was his avid devotion to the art of lock picking, his high school membership in a socialism club (for social reasons, he swore), and the fact that he was a godless scientist who loved his bongo drums.
Original Documents are available:
One other element? A seven-page letter detailing a conspiracy theory that Feynman was a sleeper agent for enemies unknown, but probably communist ones.
Link to Original Source