The innovative design of thin, multiphase flexible composite systems with good mechanical properties, low density and improved EMI shielding properties at low filler content has become a key area of research. In this work, we report the low temperature synthesis of three-dimensional ternary composites (PANI/NFO@CNTs and PANI/NFO@RGO) by oxidative chemical polymerization of aniline in the presence of two different binary composites, viz. NFO@CNTs and NFO@RGO. Enhanced impedance matching is achieved by varying the ratio of the carbon allotropes (CNTs and RGO) to the ferrite component. The synthesis of NFO, PANI/NFO@CNTs and PANI/NFO@RGO is validated by XRD and FTIR spectroscopy. Field emission scanning electron microscopy (FE-SEM) confirmed the synthesis of core-shell structures of PANI/NFO@CNTs and PANI/NFO@RGO, where the binary composites (NFO@CNTs and NFO@RGO) serve as a core onto which a tubular PANI layer was coated. Shielding effectiveness of 22.36 dB (99.41% attenuation) is exhibited by the ternary composite PANI/NFO@CNTs (8 : 1), while for PANI/NFO@RGO (20 : 1) a total shielding effectiveness of 31 dB equivalent to 99.92% attenuation was observed at a thickness of 2 mm. The ternary composite PANI/NFO@RGO (20 : 1) 4 mm showed a maximum SET of 43 dB corresponding to 99.996% attenuation of incident EM waves. The enhanced EMI shielding properties of the synthesized ternary composite systems are accredited to good impedance matching, effective dielectric and magnetic loss mechanisms and good conductivity, which facilitate multiple reflections and scattering of incident radiation.