I didn't partake much in high school until my senior year.  Interestingly, my senior year was by far by most successful academically although I suspect the most important factor was the discovery that you actually had to study to make good grades (a fact that had eluded me during the previous 3 years).  Later when I got to college and dropped back to "casual use" it likewise didn't have an obvious impact on my academic performance (summa [BAD WORD REMOVED] laude) but I don't know how I would have done if I hadn't partaken at all.  I can say I was always straight for lectures, tests, etc.  Clearly it impacts different people in different ways.

---