In cement composites, usually, reinforcement is provided to restrict the crack development and their further propagation under service conditions. Typically, reinforcements utilized in cementitious composites range from nanometer scale to millimeter scale by using nano-, micro-, and millimeter-sized fibers and particles. These reinforcements provide the crack arresting mechanisms at the nano/microscale and restrict the growth of the cracks under service loads, but usually, the synthesis of nano/microfibers, and afterward their dispersion in the cementitious materials, pose difficulty, thus limiting their vast application in the construction industry. Carbonaceous inerts are green materials since they are capable of capturing and storing carbon, thus limiting the emission of CO2 to the atmosphere. In the present study, a comprehensive review of the synthesis of low cost and environmentally friendly nano/micro carbonaceous inerts from pyrolysis of different agricultural/industrial wastes, and afterward, their application in the cementitious materials for producing high performance cementitious composites is presented, which have the potential to be used as nano/micro reinforcement in the cementitious matrix.