The roots of Sufism lie embedded in Islam itself. There are numerous passages in the Holy Quran which are of a mystical character. The Holy Prophet of Islam (peace be on him) himself displayed mystical inclinations and he very often retired to the cave of Hirah for the purpose of devotions, meditation and contemplation. The Holy Prophet was recipient of two types of revelations, one embodied in the Holy Quran, and the other that illuminated his heart. The former was meant for all, the latter for a selected few whose hearts could be illuminated with the Divine Light. The knowledge of the Holy Prophet was thus book knowledge (ilm-i-Safina), and heart knowledge (ilm-i-Sina). Ali got this heart knowledge from the Holy Prophet. It is related that after the Ascension, the Holy Prophet awarded a mantle to Ali which led to the illumination of his heart.

Hazrat Abu Hurayra, who was one of the Ashab-e-Sufa (The early Sufis), and narrated most of the Hadith. Once said to the Companions of the Holy Prophet that from the Holy Prophet he'd got two kind of knowledge. The first kind he had exposed to them (i.e. Knowledge of Hadith). The Companions asked that why he didn't tell them about the second kind of knowledge? Hazrat Abu Hurayra replied: "If I would expose that knowledge, then you people would cut my neck".