Users of the Yurekuru disaster warning app were told that a magnitude 9.1 earthquake was about to hit the sprawling metropolis shortly after 5 pm (0800 GMT) with an alert reading “Earthquake! Earthquake!”.
But the expected shaking never came and the phones of nearby users of similar early warning apps remained silent.
Japan’s Meteorological Agency had sent and immediately withdrawn the alert, but the cancellation did not reach some app operators in time, a spokesman for the agency said.
“We are now investigating how the false information was sent out,” he added.
The agency’s website later said that it received the quake data from a seismometer east of Tokyo.
“When I saw the Yurekuru app screen, I prepared to die,” read a tweet in Japanese from one user.
“Yeah okay this was fake but it scared the hell out of me,” said another in English.
Yurekuru is a popular disaster-warning app in quake-prone Japan, and its operator says at least five million people have downloaded it.
Such apps became popular in the aftermath of a massive undersea earthquake that struck the country in March 2011.
The magnitude 9.0 quake sent a tsunami barrelling into Japan’s northeastern coast, leaving more than 18,000 people dead or missing, and sending three reactors into meltdown at the Fukushima nuclear plant.
A nationwide drill is held annually on September 1, known in Japan as Disaster Prevention Day, which comes on the anniversary of the 1923 Great Kanto Earthquake that killed more than 100,000 people and levelled Tokyo.
Japan sits at the junction of four tectonic plates, but most earthquakes, even moderately strong ones, do little major damage nowadays thanks to rigid building codes.
Due to the mistaken warning, which was also sent to some transportation operators, a small number of subway and commuter train lines in Tokyo were briefly stopped.