There is a known issue where the cooldown timer for exhausted Skull Quests can sometimes disappear. These pending quests will seem as if they are ready to be played again even though they are still recharging.
When you try to start these quests during this time, you will see the pre-battle loading screen followed by being sent back to the team select screen. Thankfully, no Sands of Time are deducted.
Without the timer, it would be best to simply try again at a later time to see if the quest is ready to be played.