This should've been a comment as it only backs up Georg's answer, but it just got too long for that.
Plastic doesn't "leak", scientifically. However, it can go through the degradation process.
You don't provide enough info for anyone to decide, but if you're certain that was the "plastic" taste (I don't have much idea how you do) it's possible it may have gone through degradation, though I can tell you 95 percent of the time it isn't. And that's simply because if the plastics were too unstable to resist minor chemical or energetic "attacks" (read the wikipedia article for degradation to see what I mean), they wouldn't have produced it like that.
There is the urban hoax that doing stuff like pressurizing a mineral-water bottle by freezing the water inside releases dioxins that are carcinogenic and which cause the change of the taste of water, but this, is not true. (See here.)
Georg's answer pretty much sums it up, but I think it's better to add: It may also be possible that you (or anyone else, or mistakenly the company itself) put the "tasty" water in contact with soluble mineral salts. Thus, you drink hard water and swear for its taste!