Sperm Quality

My husband and I had no problems getting pregnant with our first child. Now, we're trying again and I'm wondering if the problems are on his end. Could it be possible that 2 years ago he had fine quality sperm and now not anymore? Or do men who have low quality sperm have it from the very beginning? He hasn't really changed a whole lot in terms of lifestyle (no drinking, smoking, pretty good diet), he just has been riding a bicycle about 45 minutes everyday for the past year. Any information about this is appreciated. Thanks.