Saving and reloading seemed to fix it. But when it happened couldnt tap or click anything to remove, tried the bottom options and all over the screen
Weird, glad that reloading fixed it! No idea why it would happen randomly though :/