Istanbul, which was bidding for the fifth time, beat Madrid in a head-to-head vote after the two cities were tied in the first round, reports Xinhua.
Tokyo received the most number of votes in the first round to advance.
Some supporters of Istanbul had initially thought the Turkish city had been awarded the Olympics before International Olympic President Jacques Rogge clarified that Istanbul had merely entered the second round.
The Japanese capital will be the first Asian city to host the Olympics twice, having also hosted the 1964 Games.
Tokyo was making its second straight bid after losing the 2016 race to Rio de Janeiro.