The temple is the ancestral temple of the Xiage Branch of Tibetan Buddhism (also known as the Pudun Branch). Historical records indicate that Xialu Temple was built in 1087 during the Song Dynasty (960-1279) and later perished in an earthquake. The present temple was rebuilt in 1320.

The temple's main hall, called Xilulakang in Tibetan, is made of wood and stands in the south of the temple. With a three-storied front hall, two-storied back hall and wing halls, its lower and upper floors were built according to the Han (206BC-220 AD) and Tibetan styles respectively. The ground floor is a Tibetan-style scripture hall supported by 36 wooden columns in the middle and surrounded by rolling scripture corridors. The upper floor is a Han-style room with its front and south and north wing halls connected by winding corridors. All the halls have a gable and hip roof and are covered with green glazed tiles. Color drawings painted on the walls retain the architectural style of the Yuan Dynasty (1271-1368). Located above the front hall, in Budun Hall, is a sacrificial pagoda built to enshrine the founder, Budun. Along the walls of the rolling scripture corridors are various frescos that combine the artistic style of the Central Plains and Buddhist art from India and Nepal. The frescos are among the earliest specimens painted with colorful themes in Tibet. The temple also boasts a large number of cultural relics.