"I proposed to postpone for a year and [IOC] president Thomas Bach responded with 100% agreement," Japan's Prime Minister Shinzo Abe said.
The event will still be called Tokyo 2020 despite taking place in 2021.
In a joint statement, the organisers of Tokyo 2020 and the IOC said: "The unprecedented and unpredictable spread of the outbreak has seen the situation in the rest of the world deteriorating.
"On Monday, the director general of the World Health Organization, Tedros Adhanom Ghebreyesus, said that the COVID-19 pandemic is 'accelerating'.
"There are more than 375,000 cases now recorded worldwide and in nearly every country, and their number is growing by the hour.
"In the present circumstances and based on the information provided by the WHO today (Tuesday), the IOC president and the prime minister of Japan have concluded that the Games of the XXXII Olympiad in Tokyo must be rescheduled to a date beyond 2020 but not later than summer 2021, to safeguard the health of the athletes, everybody involved in the Olympic Games and the international community."
It is the latest major sporting event to be postponed or cancelled because of coronavirus, following Euro 2020, Masters golf and the French Open tennis.
The IOC had given itself a deadline of four weeks to consider delaying the Games but there had been mounting pressure from a host of Olympic committees and athletes demanding a quicker decision.
Canada became the first major country to withdraw from both events on Sunday, while USA Track and Field, athletics' US governing body, had also called for a postponement.
International Paralympic Committee president Andrew Parson said the postponement was "the only logical option".
He added: "The health and wellbeing of human life must always be our number-one priority and staging a sporting event of any kind during this pandemic is simply not possible.
"Sport is not the most important thing right now, preserving human life is. It is essential, therefore, that all steps are taken to try to limit the spread of this disease.
"By taking this decision now, everyone involved in the Paralympic movement, including all Para-athletes, can fully focus on their own health and wellbeing and staying safe during this unprecedented and difficult time."
The Olympics have never been delayed in their 124-year modern history, though they were cancelled altogether in 1916, 1940 and 1944 during World War One and World War Two.
Major Cold War boycotts disrupted the Moscow and Los Angeles summer Games in 1980 and 1984.
The Tokyo 2020-IOC statement continued: "The leaders agreed that the Olympic Games in Tokyo could stand as a beacon of hope to the world during these troubled times and that the Olympic flame could become the light at the end of the tunnel in which the world finds itself at present.
"Therefore, it was agreed that the Olympic flame will stay in Japan. It was also agreed that the Games will keep the name Olympic and Paralympic Games Tokyo 2020."
British Olympic Association chief executive Andy Anson said: "It would have been unthinkable for us to continue to prepare for an Olympic Games at a time the nation and the world no less is enduring great hardship. A postponement is the right decision."
British Paralympics Association chief executive Mike Sharrock said: "Stemming this global public health crisis and doing everything possible to safeguard the health and wellbeing of people should clearly take priority in these unprecedented times.
"We welcome the clarity this now gives Paralympic athletes throughout the world who have had their training and qualification plans severely disrupted."
On 22 January, Olympic qualifying events in boxing and women's football that were due to be held in Wuhan, China - the centre of the coronavirus outbreak - became the first to be moved or postponed.
There have now been almost 400,000 recorded cases of the virus worldwide, with the number of deaths approaching 17,000.
BBCLast modified on Tuesday, 24 March 2020