We live in Barcelona so I can tell you that in general the weather should be pleasantly warm and sunny during daylight hours, but gets chilly in the evenings. It's difficult to give you temps. as outside the summer months it can be changeable. But generally speaking the weather is better in the Autumn (fall) than it can be in the Spring.
I would say pack some t-shirts for sunny sheltered spots (great street cafe culture in Meditteranenan cities!) sweatshirts for cooler days and maybe a lightweight jacket for evenings.
Down in Madeira it will be noticeably warmer, but being an Island can get windy.
I have actually been able to swim in the Med in mid-October sometimes, hope you will be lucky too!
Try a search engine for year round weather predictions, or a tourist board site.
Anyway, have a great trip around the Med. Whereabouts are you from?