Beneath the upper church is a chapel which incorporates stone from ancient grottos inside its walls. Down a hole in the center of the sanctuary one can see caves that may have been part of the Byzantine shrine. These walls are engraved with crosses left by fifth-century Christians. On an even lower level there is a succession of caves from the Second Temple period. Since tradition places the palace of Caiaphas on this site, many believe that Jesus may have been imprisoned in one of these underground crypts after his arrest, however, these underground caves were normal in many Roman-era homes, and often served as cellars, water cisterns, and baths. On the north side of the church is an ancient staircase that leads down towards the Kidron Valley. This may have been a passage from the upper city to the lower city during the first temple period. Many Christians believe that Jesus followed this path down to Gethsemane the night of his arrest.