Deep learning algorithms often exploit spurious correlations among training data as shortcuts to make predictions. These shortcuts are misleading and unstable. They work for most training examples but do not generalize well to out-of-distribution samples. Recent works have attempted to address this issue using invariant learning methods which learn the (causally) invariant correlations across multiple training environments. Although these approaches can effectively remove biases from predictions, they require partitioning datasets into multiple environments that differ in how labels are spuriously correlated with the spurious attribute. However, such environment assignments are often unavailable at training time due to expensive annotations, privacy constraints, and difficulties in effectively grouping the dataset. This motivates us to develop a frustratingly easy algorithm for environment discovery.