The head of the Tokyo Olympics organizing committee is stepping down after facing backlash over sexist remarks.

Yoshiro Mori, 83, said that he will announce his resignation on Friday.

His announcement comes more than a week after he said at an Olympics committee meeting that women talk too much in meetings. “If we increase the number of female board members, we have to make sure their speaking time is restricted somewhat, they have difficulty finishing, which is annoying,” he reportedly said.

Mori, who served as prime minister from 2000-2001, apologized the day after the news broke on Feb. 3, but said that he was not considering resigning.

But he faced a strong backlash both within Japan and internationally. The International Olympic Committee (IOC) at first accepted Mori’s apology, but later issued a statement saying his remarks were “absolutely inappropriate.” Yuriko Koike, the governor of Tokyo, said on Feb. 10 that she would skip a meeting with IOC president Thomas Bach, Mori and others next week.

Tennis player Naomi Osaka, who was born in Japan, called his comments “really uninformed” and “ignorant.” At least 500 Olympic volunteers quit, including three runners for the Olympic torch relay, according to Japanese media. A petition criticizing his comments as “prejudiced, narrow-minded and discriminatory” received almost 150,000 signatures. An editorial in a large daily newspaper called for him to resign.

Women’s rights activists say Mori’s remarks highlight the difficulty faced by women in Japan, which ranks 121 out of 153 countries in the World Economic Forum’s 2020 global gender gap report.

Read More: ‘Tokyo Is the Best-Ever Prepared Olympic City.’ IOC President Thomas Bach on Why the 2020 Games Will Go Ahead

Former Prime Minister Shinzo Abe committed to “make women shine.” He called for corporations to ensure that women had 30% of managerial positions by 2020. But they hold less than 15% of middle and senior management roles—one of the worst rates in the G20.

Abe appointed just one woman to his cabinet, and the current Prime Minister Yoshihide Suga appointed just two women to his 21-person cabinet prompting a prominent politician to call Japan a “democracy without women.”