The way you describe it disappearing and reappearing suggests it may be rating lag. Your rating is based on agreements, so after you’ve reviewed a nomination but before that nomination is decided, your rating will go down because the number of agreements hasn’t changed but the number of reviews has gone up, thus reducing the ratio of your agreements to reviews. The rating will then go back up if you were in agreement with the final decision.
Someone who hasn’t done many reviews is likely to see large swings in their ratings after doing a number of reviews in a short period of time. Someone who has done thousands of reviews is less likely to see big swings (unless there’s a Challenge, perhaps).
If this is indeed part of the issue, just taking a break of a day or two might see your rating improve. Of course, this depends on how well your decisions align with those of other reviewers, as others have noted. Reviewing the Criteria Clarification Collection category might be helpful.