Yes, you hit the nail on the head, you need to sharpen your tools. "Fairly Sharp" isn't good enough. And you have to sharpen often. It sounds like you are new and don't have a sharpening system yet - a grinder should be your next purchase, but don't think of it as a grinder when you have your gouges near it, think of it as a sharpener, just touching up the cutting edge.
mike