I don't make these myself, but here's a technique that looks like it would work:  http://gardening.about.com/od/craftsanddecor/ht/Herb_Oils.htm.  This source says to strain out the basil, and I think that's what I would do.  I think something woody, like rosemary, would probably be okay to leave in there, but I would think that basil would probably look nasty after a while.  This article says you can leave herbs in there, though:  http://homecooking.about.com/od/condimentrecipes/r/blcon3.htm. 

I wonder about food safety on these; when I was researching making garlic-flavored oil and ways to store garlic, I kept seeing warnings that you shouldn't store garlic in oil for more than a week (and to keep it in the fridge) because of the risk of botulism.  I wonder if that's also true for herb oil?