One of the most important aspects of a shower caddy is that it should not rust. This is why plastic is a great choice for a hanging caddy. Depending on its design, you may have to clean it more often than a stainless caddy however.
Plastic Hanging Shower Caddy:
Plastic Hanging Shower Caddy pictured: InterDesign Grand Arc Shower Caddy, Clear.
Zenith Shower Head Cady, White.