Space variant beams are of great importance for a variety of applications that have emerged in recent years. As such, manipulation of their degrees of freedom is highly desired. Here, we study the general interaction of space variant beams with a magnetically influenced Rb medium exploiting the atoms versatile properties in terms of frequency and intensity dependent circular dichroism and circular birefringence. We present the particular cases of radially polarized and hybrid polarized beams where the control of the polarization states is demonstrated experimentally. Moreover, we show that such an atomic system can be used as a tunable analyzer for space variant beams. Finally, exploiting the non-linear properties of Rb vapor, we show that we can control the circular birefringence all optically, and thus modulate the polarization of the radial polarized beam.