President Barack Obama was not in the residence at the time.
In a statement, the Secret Service said the individual was seen flying the Parrot Bebop device in Lafayette Park at an altitude of about 100 feet (35 meters).
He was detained by uniformed Secret Service officers as the park — filled at lunch hour on a sunny spring day — was swiftly evacuated and the White House was briefly put on lockdown.
“As a precaution, the small UAV was swept and declared safe,” said the Secret Service, which released a photo of the red quadracopter — a popular consumer model with a video camera built in.
The operator was later identified by the United States Park Police as Ryan MacDonald, 39, of Vacaville, California.
That police agency said in a statement that the man was arrested “for violating federal law.” It did not elaborate other than to say MacDonald was cited and it is to appear in court.
Obama was meeting Gulf leaders at his rural Camp David retreat in Maryland at the time of the incident.
Flying drones of any size is prohibited within the US capital — a point that the Federal Aviation Administration is underscoring in a “no-drone zone” publicity campaign it launched earlier this week.
In January, a DJI Phantom drone — another consumer model — crashed onto the White House grounds after it slipped out of the control of the federal employee who was flying it in the early hours of the morning.
The incident was deemed to be an accident and no charges were filed.
A similar Parrot Bebop, weighing about one pound (500 grams), was flown indoors in January at the Capitol during a House of Representatives committee hearing on unmanned aircraft technology. (AFP)