Traditional stormwater management systems are designed to rapidly convey runoff to waterbodies or treatment facilities, but often fail to address challenges posed by urbanisation, climate change and extreme rain events. In response, sustainable urban stormwater management approaches, such as sponge city, emerged to replicate natural hydrological processes and provide multiple co-benefits, including pollutant removal, groundwater recharge, flood mitigation, biodiversity enhancement and urban greening. This study critically analyses the design, performance, and effectiveness of rain gardens as an essential component of the sponge city design. A qualitative research design was adopted through a systematic literature review of 67 peer-reviewed articles by following the PRISMA framework. The review examined three key themes: plant selection, filter media composition, and system effectiveness, alongside a comparative assessment of green and grey infrastructure performance. The findings revealed that the rain garden’s performance is primarily influenced by external factors such as climatic conditions, soil characteristics, and pollutants. And further optimal outcomes can be achieved when plant selection and engineering media design are tailored to the external factors. The filter media mainly control infiltration, storage, and pollutant attenuation, and the plants enhance nutrient uptake, evapotranspiration, biodiversity, and system resilience. The study also indicates that media amendments using biochar, zeolite, steel slag, and water treatment residuals further enhance the removal of pollutants and hydraulic stability. Rain gardens showed high effectiveness across studies, with over 90% suspended solids removal, 30–90% nutrient reduction, 50–90% runoff reduction, and notable heavy metal attenuation. Enhanced designs like internal water storage (IWS) and biphasic systems consistently outperformed conventional configurations. Compared to grey infrastructure, the rain gardens offer broader ecological, social, and climatic benefits. Integrating green-grey infrastructure presents a more resilient and sustainable approach to urban stormwater management under changing urban climatic contexts.