Your anatomy naturally gives you small bumps on both your labia majora (outer) and labia minora (inner). On the hair-covered labia majora, the hair follicles that contain hair and oil could look like the little bumps you are talking about. The labia minora have no hair but do contain oil (sebum) glands that could also resemble these bumps. Keep in mind, any bump that is new, painful or comes and goes should be assessed by your healthcare provider ASAP!
I'm not a medical expert, so I'm going to be anxious to read Sandy's response to this question. But from the little research I did, it looks like there could be a lot different possibilities for these bumps. WebMD has a good article that addresses this. It could just be a normal variation in your skin but it could also be something that needs medical attention. To be sure, I would strongly recommend you see your healthcare provider. Even though there's no discomfort with the bumps, they're a concern for you and it's always better to attend to those types of things.
Great question. It's possible that these are Fordyce Spots, which are superficial sebaceous (AKA oil producing) glands. They can appear on the labia, penis, and mouth. They are perfectly normal and common for many people, guys and gals! I even know a few girls that have them on the inner corners of their lips. That said, it's always a good idea to ask your healthcare provider about anything that seems a little out of the ordinary - they're a great resource. And don’t forget to check out what our Health Expert Sandy has to say. Hope this helps!