The key phrase for trips to Martha's Vineyard is, "chill out, man."  Whether you are staying for a couple days, or weeks at a time, the key is to travel with people who have no problem chilling out, enjoying the scenery, food, and most importantly, each other's company.