During his Tokyo visit, Justin Bieber stopped at Japan’s notorious Yasukini Shrine, which pays tribute to WWII criminals and is a breeding ground for political debate.

Clearly not keen to the history, the 20-year-old singer tweeted a photo showing him bowing his head in front of the shrine’s entrance with the caption, "Thank you for your blessings.”

The snapshot gained nearly 700,000 likes and multiple retweets before Bieber caught wind of the controversy behind the shrine.

Justin quickly removed the photo and replaced it with the Time cover of his visit, writing, “While in Japan I asked my driver to pull over for which I saw a beautiful shrine. I was mislead to think the Shrines were only a place of prayer. To anyone I have offended I am extremely sorry. I love you China and I love you Japan.”

The shrine honors Japan’s war dead, including former military leaders from the country’s imperial army who committed mass atrocities across Asia and the Pacific during World War II. It is still a very sensitive subject of conflict for Japanese and Chinese citizens.