It seems that you may be having a case of FOLLICULITIS, pus collection below the hair follicle. THe best thing would be to take a course of oral antibiotic after review with your physician so that if needed the pus can be properly drained out.
Read more here
http://www.aocd.org/skin/dermatologic_diseases/folliculitis.html
http://www.nlm.nih.gov/medlineplus/ency/article/000823.htm
Regards
Dr. Gupta

