When a special needle (ideally tipped with a diamond, for better sound, attached underneath the end of a kind of "arm" or lever) was placed by a person "in the groove" you could hear the music. If your hand was not steady and you missed the groove, you would hear scratchy sounds and possibly ruin the record.

Later they invented turntables that would drop the needle for you in the right place.

So... "getting in the groove" literally meant being able to hear the music, meaning you would be in a good mood also, as you had been successful at operating the equipment. These days, it is easy with CD's etc. You don't know how lucky you are! That's history for you!